Why would I do regression testing?

There was an evening event and late in the evening we were talking about testing. As usual in the agile circles, test automation creeped in to take the stage. This time though it was formulated perfectly for me to learn something essential.

A developer colleague took a sympathetic approach and told me he would feel bad for if he didn't create automation and in particular he feels he needs to work on automation so that I can focus on testing the new stuff and not repeating the same tests.

I immediately replied from my heart and experience. I have worked on my product for over two years, logged some thousands of issues but so far I have not repeated a single test. Even with the fact that the level of automation in my team isn't created by these aforable, sympathetic agilist developers who care for my well-being.

I realized that reply includes a core of how I deal with testing work. I'm active player in varying my approach. I might press the same buttons but I have different ideas racing through my mind. I don't use the same browser, I don't use the same user, the same data, the same story, the same combination. And I find problems the test automation will never find.

Don't get me wrong, I love having test automation around. I don't expect it to do any testing for me. Instead, I expect it to make my testing work less interrupted with plethora of simple problems automation can catch. Whenever I find a problem, it stops the testing I was doing. But more than automation, I love having active thinking developers around that think and use automation as their safety net, but don't go into relying only on automation. Thinking is the core of it all and sometimes some of that thinking gets packaged into automation.

Regression testing isn't what I do - at all. I'm not sure to what extent we should even talk about doing regression testing. I test, and regression is one of the risks I'm considering to motivate what I do. But there's no specific repeating same tests approach that I would call regression testing.

I've been teaching that tests have 'best before' dates with short expiration dates due to the changes that might cause the need of going back. It's really test results that expire, and the results are not the tests but the information about the system. I seek for same information again, but while at it, I also seek for new info based on the learning that other tests (and discussions, readings etc.) have given me. That's what a tester needs to set their mind to achieve.

Create with software, don't code - value of non-coders for code

My main aspirations this autumn are testing and agile software development in general (as always), improving my own coding attitudes and skills (can always try to work on stuff) and teaching kids to create with computers.

With the last one, I looked at a really nice presentation video from Linda Liukas last night:
http://www.bonniergrid.com/speakerevent.php?event=137

The video is pretty concise to look at, and on it Linda shares with clear enthusiasm her story of why she ended up being into code and inspires people to create similar experiences for all children for a better future - including both genders.

I look at Linda speaking with joy about code and I see similarities. I see similarities in the idea that when you talk about you work, you talk with love, affection and excitement. You pass the feeling on to whomever is listening to you, hoping that it might catch. And listening and experiencing that with Linda, I felt very blessed remembering feedback I've received over the years on my own presentations and trainings, that it oozes that I absolutely love what I get to do for work.

People who wholeheartedly love what they work on might have a chance of catching some of that passion and enthusiasm into the younger generation. At least my kids and their friends know that their mom has the best job ever, and it just keeps getting better with all the collaboration stuff we're working on in the teams under the "agile" flagship.

On the other hand, it saddens me that while there's so much in common, there's also a difference. From the stuff I'm excited about, I wouldn't teach kids coding.  I prefer not to code, and while there's people who prefer not to socialize, think critically, and think systems and value, I can easily work to bring my special skills to creation of code without writing a line of code. I would teach kids code. I would teach them testing and learning about systems someone else created, on levels that many current developers don't do in practice. I would teach them collaboration and valuing the different skills of others. I would teach them drawing and visual aspects as relevant part of what we're creating. And that it's just normal for someone to have an idea, others to run with the idea and all of us to improve the idea and make it practical with all the different skills we're bringing to the table.

Code is a tool that the future generations should learn not be afraid of. But as always, emphasizing the tool instead of the creation process just will not do. Linda was talking about "I made this" feeling. It's a feeling I recognize as tester - I was there to help for this other one to be actually able to use it. It's a feeling I recognize as designer - I was there to help get that important thing done with just two clicks. It's a feeling I recognize as a team manager - I was there to help people excel and reach their potential so that it actually works all together.

Let's teach kids creating with computers (as opposed to consuming). But coding? That too. But let them find roles in groups that are not labeled as strongly as ours are, but ones that bring out their special interests - in collaboration with the others with different skills and emphasis. 

Special words for programming concepts for tester

Every now and then I stop to wonder about the world of testing I live in. A recurring theme that stops me seems to be that of automation.

Working with selenium, I'm getting a hang of page objects. Page objects seem like a nice way of saying to create classes of concepts that neatly pack things up together, so that you wouldn't have elements from a page dispersed around your actual tests. A great idea that helps maintaining, and really obvious to developers except for giving the concept a new name.

There are two test automation specific terms that leave me pondering: data-driven testing and keyword-driven testing. I seem to actually have some problems with those words. For me, both of these terms are fancy ways of renaming existing programming ideas to test specific concepts, and as such, create an extra divide between the testers and developers. The way these concepts are presented to testers new to automation gives the impression that these two extremely old programming ideas are somehow the center of test automation without getting really into the world that belongs to a developer.

To me, data-driven testing is just a way of saying that you should use variables. It's a concept I've been practicing with my 5&6-year olds, and they seem to start getting a grasp of it. Why should we in testing say data-driven for any other reason than introducing the idea of variables. And why would we need a separate concept for that? And keyword-driven testing is just a way of saying that we should modularize code for reuse, right?

I think these two words in particular have their roots in the idea that tester's can't code. Well, many can't. But treating them worse than I treat my 5&6-year-olds in regards to their intelligence and capability seems wrong. People can take the challenge. People appreciate being supported. And people appreciate when they learn the stuff that started off as difficult.

I don't dislike automation work because coding is difficult. Some coding is difficult, but test automation isn't the hardest of programming challenges there is out there. I dislike it because it's more boring than the other options I can use my time for. And I appreciate my time on manipulating my team's developers to do the automation stuff over implementing it myself - for the benefit of the team in general. I dislike it because it steals potential great minds and turns them into machines if given without appropriate support of a balanced perspective of what it can and cannot do. I dislike it because it turns into a silver bullet that downplays the role of thinking tester by giving us keywords you can call to create scripts without knowing how to code, and excuses of not learning the next step that we're perfectly capable for. 

Is it just me? Most likely. :)

Acceptance testers as IT professionals - are there really that few ladies in IT?

I'm puzzled. There was an article posted today in a Finnish non-IT magazine pointing out that IT-industry needs more women (http://www.aamulehti.fi/Kotimaa/1194924416639/artikkeli/puheenaihe+italalle+tarvitaan+lisaa+naisia.html). But I keep wondering if there are really that few women in IT, really?

I know the stats show less than every fourth person "professional in IT" is a woman. But who are these "professionals" - who's in and who is not?

There's one study that defines a professional as a member of the data processing association (1). There's another study that doesn't really define what their sample is (2) but looks into ideas of IT as career before choosing what you study and after that. The after selection part seems to imply a proper schooling in IT.

I studies computer science at Helsinki University of Technology and Tampere University of Technology, and got to experience what it's like to be really in the minority. I'm not questioning that. I remember in particular the annoyance I caused at Tampere, refusing to be in the room to be counted as a new female student, as I knew there was a bet on our number as I knew the routines having spent time in Helsinki circles before that. There's few women in IT specific education programs, I take those numbers as facts.

What bothers me though is the experience from the industry and the message we seem to be sending out emphasizing how few women there are. What if there are more women in IT, but we just exclude them when we count?

Inspired by this idea, I asked around at the office today on how many people we have in IT roles and how many of those are female. I got a confirmation: out of the team of 15 developers, I'm the only woman. And some were not sure if I should be counted even if I can program because I'm a tester. Our product management team has many women, in significantly higher portions than in development. They were not included. Let alone many of the other roles that participate in development - they are all excluded.

I also looked back at time I was working at an insurance company, leading acceptance testing. I had teams with tens of people, significantly contributing to the success of software development as testers. There were other teams, even larger, working as business specialists defining how the systems would work. These teams were female dominated, to the extent that I can recall one man out of the tens of ladies. When I was there, I often asked the acceptance testers if they identified as testers. They would tell me no, they identify with the business stuff - concepts of domain. If someone asked them if they are "IT professionals", they would say no. But they are. They got their salaries from IT work. It was better to not emphasize that, because "IT professionals" get paid more. My ideas of what salaries I can get as an IT professional were ridiculously high in my colleagues eyes.

I work with software testing. Our of the people that I see in trainings and seminars, women are not a minority. Sometimes it feels quite the opposite, men are a minority. There's significant differences between industry sectors though. But I fear that many of testers, in particular acceptance testers, are excluded from the counts. In 2012, asking the members of Finnish Association of Software Testing (FAST) how many of them are members of the data processing association, sample in (1), half of them were. And FAST is a subgroup of Sytyke, which is a subgroup of the data processing association.

To know the real numbers, we would have to define what level of contribution counts you in as "working in IT". If 20 % of your working days go into IT but the other work you do feeds into that, should you be counted? Everyone uses software (IT) so what actions towards creation of software get you to be counted? Do you personally need to identify before you count?

Perhaps that's the problem. As a tester, I get to be actively excluded from small scale counts on a team level. I identify with IT, so I get counted. But the ones who contribute but don't identify should be counted too.

Excluding people who contribute isn't going to make this a more lucrative profession for the future ladies. Learning programming is another thing, but could we start by realizing that there could be more ladies in IT than we've given credit for due to how we count the numbers?

The numbers could be right, they could be wrong. But the point is that we should actively get the people who contribute recognized within the field as relevant contributors. It's not a women thing, but it's a roles thing. The acceptance testers deserve to be included, even if that drives us towards users as "IT professionals". 

Or perhaps we believe in the lesser value of those contributions, with the ideas of test automation and agile in our ideas of realistic future with less failure demand? 

The refences, articles in Finnish: 
1) http://t.co/RXXRWyOpTA
2) http://t.co/zIZROPHSp5

 

Work for free when you could get paid

As I've decided to leave the current employer that I work with to become a consultant again, there's been some talk around the office on how to find someone to replace me.

It's great to hear that the work I've done has been appreciated and that the vast numbers of fixed issues over the 2,5 years as well as the changes in how our teams work are appreciated. I've enjoyed my time here with all the puzzles and challenges, some of which I've blogged about. When I joined, I could build practices as I saw fit, and I saw things from agile development, exploratory testing and automated checking perspective. There was also the experience that all things "testing" are not the same in value: getting a developer to test (calling him tester) resulted as no improvement on product quality and throw-away test automation.

When I was recruited, I found this job by a lucky accident. My employer did not know of the right places to look for good candidates. We still haven't posted an ad for the position, but as I would like to help out, I've mentioned the job in passing at Finnish Association for Software Testing LinkedIn group and had three candidate contacts. We're requiring fluent Finnish, which already rules out two out of three.

As it seems my supervisor has not made much progress on posting the adds, I started looking into other routes to find him candidates. As the job isn't really something you need to be the best in testing in Finland for (modesty in my skills is not one of my traits...), it could also be a great opportunity for someone new with the right attitude for testing.  So I suggested to consider a recruiting program TestPro.

The idea with TestPro is that there's a 7 month period of training and practice with the recruiting companies, after which the companies decide on recruiting the candidate. There's 22 days of classroom training and full days of working at the recruiting company, and during which the candidate is unemployed and gets some social benefits, but isn't officially working as in getting paid for the work. This program is targeted for the unemployed or people under the threat of becoming unemployed, as in case you really are out of options it's a great opportunity to get into testing.

Some hours after suggesting this route to my supervisor, it hit me: I just made an awful mistake. I have two reasons to think I made a mistake:

1. There's a PAID job open right now, and I just postponed that for whoever would get recruited by seven months of slave labor. I should know how to do better.
2. The training the candidate receives is near to worthless from our perspective. It shows aptitude, interest and commitment (7 months of free work...) that you go into it, but the contents you will receive are not ones I would find valuable for the person to recruit. Unless the person is just right, the likelihood of ISTQB Foundation and ISTQB Agile Certificate courses to make them worse is high and they'd be better off in this job without them. The added focus on test automation attracts developers with no clue on how to perform testing with valuable results. 

There's still time to fix the mistake I'm making, I hope. Unless the temptation of "free labor" already turned some managers to the dark side. But I need to find the candidates that want to learn  by doing, take BBST and RST classes during the work hours and become a great tester within the context-driven ideas. I have little time to go look for candidates, but perhaps this blog post encourages the right one to get in touch. Even with the Finnish language skill required.

TestPro (http://www.testpro.fi) might be great, if you look for testers with the stuff they are training. I'm just not convinced they teach very much of the relevant stuff. Not only because I created the first TestPro contents years ago, but also because just few years back I was not welcomed when critiquing TestPro contents on agile testing that had very little resemblance to stuff that agile testing is in my experience. As I was told, I should not have had access to the course material that was shown to me because there was an NDA on not giving the material to non-participants.

If testing was taught more like a driver's licence

With the negative (and founded) feelings I have towards the ISTQB testing certifications, I've been recently trying very hard to focus on real alternatives, not just being against.

This post is again inspired by a tweet: 

@teemuvesala Certificate is like driver's license. It doesn't mean you are competent but it means you can be held accountable for your work.
— Esko Hannula (@eskohannula) August 20, 2014

Somehow, I hope certificate would be like a driver's licence. I also hope that courses leading up to certification (if such are even necessary for testing)  would be taught more like driver's courses.

When I teach exploratory testing, I often talk about experiences I had with driving school as analogy. It's been ages since I did my own and I still remember the nervousness that blocked me from focusing on all the aspects and how they were delivered, but I've had the fortune of following my younger siblings go through the process.

Not many years ago, my sister came back from her first driving lesson in driving school. She told the lesson started on an almost empty parking lot, where she was shown the basics: pedals, gears, steering wheel, mirrors and such, one at a time. She was driving around the parking lot, and all of a sudden the teacher tells to drive out, to the open road and towards the driving school just couple of blocks away. With a lot of focus into actions she is not used to putting together, the end result is her stating to our brother: "can you show me the basics again now that I'm not in panic mode, so that I can actually focus on the actions".

When I teach testing - the actual hands on testing - there's a lot of similarities. There's different actions that you - the tester - need to be able to balance in a way that fits your rhythm of learning. If you need to take a moment to actually read a log and not just rely on an automated tool to make sense of what you're experiencing, it's your right to do so. If you feel that taking notes while you test is difficult, you can structure your work so that you don't try to intertwine things quite so much, but take the time for the notes on regular intervals, focusing on just that. If coming up with test ideas while you test does not come naturally to you, nothing prevents you from taking a session just focused on brainstorming. You're in the driver's seat, in control.

When you start as a new tester and take a course, it's not the vocabulary that you need to make sense of to be useful. It's understanding what kinds of actions need to happen, and how you best fit those into your personality, skills and interests.

If testing on the courses was taught more like a driver's licence, we would be taught the basic actions we need to take from configuring the product to coming up with ideas of what to test, creating useful documentation for personal and team use and recognizing problems when they are right in front of us (and many more). We would be taught to focus on the actions first, and then allowed and encouraged to practice putting the actions together into series of actions.

Many times when I drive a car, I leave home and get where I was going, without a clue of what really happened on the route. After years of practice, the actions have morphed into something I don't pay attention to. But if anything happens on route, I'm surely reacting to it. It's hard to see the things that I had trouble with at early stages: changing gears and turning the wheel simultaneously. The act of testing is very much like that. We control the pace to make the flow perfect - for each tester.

Licence to drive allows you to practice with others. Licence to test that would be worthwhile does not yet exist. But if you look for courses that justify your usefulness as a tester, ISTQB Foundation is the idea furthest from useful. I should know, my name is on the creators of the syllabus and I'm sorry for being part of creating such a monster.

I think BBST (Black Box Software Testing) series - in particular the one Cem Kaner continues developing under Kaner Fiedler Associates Inc - is a real alternative. So is Rapid Software Testing by James Bach and Michael Bolton, and numerous independent trainer's (including myself) contents do a great job at delivering relevant contents. On trainings, more variety is usually better. Some messages sink in after repeating differently. And we just don't need to agree on a one true way to test, as there's so many different places and cases to do testing in. Context matters. Useful results matter.

Making a tester learn coding when not-coding is what I really need

Opening twitter inspired me this morning to write this blog post. Here's the specific inspirational piece:

There is no longer a question of whether testers should learn to write code. Smart, engaged testers are learning and succeeding with code.
— Marlena Compton (@marlenac) August 15, 2014

I feel very divided with this statement. I really agree with "smart, engaged testers are learning and succeeding with code", but I also disagree with it. I have evidence on smart, engaged testers who are learning and succeeding without code, and feel very strongly about not making those people feel inferior.

Here's a story I want to share on a smart, engaged tester succeeding with code. 

I work with a brilliant remote tester from Romania. She joined my projects two years ago with a junior tester title, quickly building and showing testing skills that many seniors can't match in my experience. She pays attention to details, she sees the big picture and is great at taking the product a flow at a time, motivated by the context of use. A lot of her time goes into telling about problems that everyone else (the developers) in her team misses - the information she provides surprises people, continuously. She's purpose-driven, polite and collaborative and drives improvement. And, she has the humility to see that as much as I respect her skills today, I respect her skills more tomorrow as every day is about learning more.

I did not contract her for coding skills. I contracted her for exploratory testing where developers will team up to provide code whenever useful. I soon learned she has a background in mathematics, and is smart to a level I can only admire. I'm still not sure how much of code background she has, as she does not emphasize things she cannot do, but things she can learn. And code would definitely be one of the things she can learn, no doubt about it.

About a year ago, we introduced the first selenium automation scripts into the product she is working on. Selenium stuff was set up as developers effort, and her role in the effort was more focused on helping with the ideas of what flows to automate, leaving the implementation with the developers with (more) coding experience. She introduced ToTest-tags as the team agreed for the ideas, but it turned out that the allocation of time into coding from the developers wasn't sufficient to make progress. As I encouraged her to take time away from the much needed hands-on testing work, she learned some basics of selenium and C# on project hours, she started adding automated checks.

The feedback from developers was quiet and clear. As they refactored the tests, they deleted all that she had created, silently. They did not replace them with ones that were working better, just deleted. Later, with me pushing for answers to understand what is going on, I learned that the code was ugly, it was clear that it wasn't done by a developer, and that the tests were brittle. Brittleness in execution was the main reason for deletion.

At first, I thought it was a better approach to just contribute the ToTest-tags and no code to be deleted. But as I looked at developers not making progress on selenium tests either, I realized something. I could never, as the test manager for this tester, accept the "fact" that she couldn't do the automation. It would be something in my power - our power - to change.

Just some months earlier, the tester needed to start working on a new feature area and I was told it would be a waste of her time as "the project will be over before she understands the area, the decision rules are so complex". We were talking of a person who would have no trouble hand-calculating all the decision rules with her math background, so I setting up for "let's show them how it goes" was a no-brainer. She not only understood the area faster, but also contributed to how the area was developed working well with the product manager, who was very thankful for her existence. Exercising the power of prioritizing into something that would take learning before results was successful.

So, I realized I had not thought the same way about her automation skills. I, as the manager, needed to exercise my power of prioritizing to make room for learning. I don't want her - anyone for that matter - to be the person who is referred to as "can't do it" even though she really wouldn't have time to do it. And of course she can do it, why couldn't she. Because the first minimal trial wasn't a brilliant success, should she give up? Of course not.

So as the summer approached, I suggested to take time for another round and we talked about the first round experiences. There was also an official reason, the no progress on automation work by the developers, but a big part of motivating realizations came from not accepting skills as they are - brilliant without coding. After spending some weeks on this while others were on vacation with support from her Altom colleagues, she came back telling she learned to choose "simpler flows" and focused on reliability of her scripts, comparing to previous rounds results - showing she learned.

There was again some frustration from the team developers with inexperience with version control systems (she wanted to only introduce a couple of scripts while she had added more), but this would all be better if she had more chances to practice continually. And now it seems that the scripts are included, running with the build and being a platform to continue from. The developers examples probably helped to find some aspects of acceptable style, so I would hope they don't end up deleted again. At least they seemed to survive a week of action so far.

Looking at the big picture for the team, I really would need her not doing automation. None of the others in the team see the problems she sees as she uses it. And she sees more, faster, when she isn't focused on automating. But while automating, she also sees problems. She mentioned two in two weeks, whereas her usual pace seems to be more of two in a day. Information-wise, I get a lot less from her as she automates. And the missing part just stays missing and we fix while in production, no one else contributes to that area for now, regardless of the efforts I've tried setting up for that as well.

I wholeheartedly agree with a friend stating "Test automation in general will take the tester's time right now, but will save them time in the future, when they don't have to do everything all over again and can really focus on things that others don't see". With this experience however, we have 10 people who can write the automation, and only one who knows the work the automation is supposed to help completing with results we expect. Simple flows the developers could implement - perhaps in this case again didn't. But they tell me the autumn will be different. 

With the "testers should code" trend, it would be selfish of me not to make her code. I need the other skills she has way more, but others will look down on her if I don't allow the coding time that we don't really need from her. And her being the real "full-stack developer" - the only one in the team for that matter - wouldn't be a bad goal either. For people who learn continuously, it's just about the choice of where to invest your time.