See more options

A test automation study group, and a round of collecting things people feel like studying. Easy problems, complicated problems. Something for us all, something to do just between two people. I bring out one thing I'd like us to do: review a fix to test automation script that we're now sharing between my team and another. All I knew of it was that there was a test that was failing too much, and there had been a Jira issue of it. A pull request was waiting. My internal lazy kicked in.

With the study group, we reviewed the pull request and just as we were about to accept it together one of the developers beat us to it. My main goals were fulfilled: the fix would improve the state of both of our automation information radiators (less red) and I never had to go through the Jira, the pull request and the code alone by myself being uncertain of the intent of the changes. Well, to be honest, I wouldn't have done that. I have an automation specialist in my team.

There was one more thing that came out of that study circle though. Reviewing the fix it became clear that the fix was a workaround. The test automation did magic to get around a product that did not behave in the way the script needed. I suggested we'd go for the developers to get a change into the product. It was clear that my suggestion was atypical, even if not unheard of. For me, it seems like a good idea to optimize overall development so that we could trust the feedback and not run circles around with creating workarounds, when in collaboration with the developers we could have a product change that makes things more straightforward. A nice discussion emerged.

Coming out of that study group, I summed my lesson up in a tweet. Susanne Sporys put in into a picture.

Sometimes we stop ourselves from asking because we think we know the answer. And yet, we often don't. Us telling ourselves something is possible comes from past experiences. The past is supposed to be past, and future is full of options. 

Slide incident, the long version

 I tweeted an image yesterday from James Bach's conference talk.

Let me be clear: I don't think it was ok to have this slide in the deck. It was less ok to spend a good ten minutes on it only to be interrupted by the audience half-way through explaining with "could we move to hearing about how the tester roles changes?" - the talk topic. If there was a critique of my morning keynote, the place for that was the time reserved for it after the keynote where there was a brilliant facilitator ready for that discussion. Or if this was indeed the place for this critique, I was there whole day available for a little chat to clarify facts.

[Edit: I object to James using his keynote to mischaracterize my person. He did not take my claims but his perceptions of me as a person. I'm calling for is James to focus on claims / content and fact-check what he presents. He did not. Please don't claim I'm saying you can't reference other people's work.]

The slide isn't quoting me. It's stating how James Bach differs from me. And surely, I prefer to be nice and kind, and care for safety of others in addition to mine. Right now I'm less worried about my feelings (I'm ok, just emphasizing this is _unacceptable_) and more worried of the impact attacks like this have on the speaker community so many of us work so hard to build.  It is not safe to have this behavior.

I've been second guessing my choice of sharing this, because if only people in the conference knew this happened, it wouldn't scare others. But it still happened, and being quiet about it isn't really a choice. It's like being abused, and not telling anyone. There's already comments out there saying I deserved it (because of speaking in public). And that I could avoid it (by not speaking in public).

Majority of voices recognize this as what it is. Not ok. Not acceptable. Not something to be referred to as "academic debate". Academic debates are references to what the other person is not just saying but thoughtfully writing. Not statements starting with "I" without the decency of fact-checking the "facts" the statements are supposedly based on.

It's kind of awful that I was really expecting this to happen. I was hoping there was a chance it wouldn't but instinctively knowing it would. It's just a continuation of a theme. A theme of wanting to tell what his keynote is about after I define mine. A theme of reminding me that "it must feel bad for me that he told me that I'm not a tester" and he'll "see if he can retract that after the peer workshop day" on Sunday. Didn't so couldn't. A theme of interrupting me five minutes into my talk with "ha, you just said you are not a tester" and refusing to not listen to the story first before jumping to conclusions about words I've chosen.

First awful awful man in the room moment talking over @maaretp already happened #TMAcad

— Remco van Bree (@blikkie) September 27, 2016

James ever says he knows I must feel awful by his actions. And he chooses them very deliberately, understanding how I feel. It doesn't make it better, but worse.

The worst part about these conference attacks is the feeling of alienation it leaves me with. It's like everyone is afraid to choose sides and I would just want us all to be on the side of "curious about good testing". I can best describe it as if my child died and everyone knew about it. No one would really know what to say. Some people would avoid discussion completely - we did not know before, we did not need to mingle now to address potentially a difficult topic. Others feel compelled to come and say something. They say they're sorry. But you feel they're somehow awkward and forced. Others are genuinely connecting and soon moving on to normal topics if I'm up for it, unsure if I am or not so a lot of probing is included.  Similarly, I feel like hiding. I don't want to be the party ruiner who talks about this. How about doing some testing or powerpoint karaoke instead?

In my talk, I talked about how safety is a prerequisite for learning. I talked about how testing is really about learning. And how we need to feel safe when we test and when we learn about testing.  I talked of deep love of testing and how a lot of the tacit knowledge is hard to transfer without sharing experiences.

I hope this experience stops happening to people. My heart aches for the person who let me know based on this that it happened before and someone left the industry for it. Same person: James Bach. Same pattern. Worse results.

This conference's organizer was aware of the risk and was there for me when it happened. She did not cause this. But she, like the other conference organizers are in power to stop this from happening more. I respect their choice on that, and realize that the middle ground of speaker facilitation could also provide a working solution.

With these thoughts, let me go back to testing as a non-tester and program as a non-programmer. Let's just do good stuff, learn a lot and enjoy the big portion work makes of our lives.

The way we teach through stories

We all have our lived lives and experiences acquired. We use those experiences to channel stories to illustrate the points we make in our presentations. My whole talk on a conference yesterday was about lessons I've picked up over the years on the features that would make me a better tester and could be of value to others. Working to become more kind and considerate, understanding that this is not about individuals but collaborative efforts are some of my key learnings that I feel transfer well over time.

There were absolutely wonderful sessions and discussions throughout the day. Richard Bradshaw was  articulate and delivered fun, experience-filled stories to illustrate his points about things he has included in his tester job to take things forward in projects he's been on recently. Anders Dinsen lead people on a thought-provoking journey to think of the really big problems that could take businesses down and our potential in missing those. Ash Coleman (in her debate with James Bach) gave tangible ideas on how we could find the future testers to the growing industry and I appreciated the insight the debate gave to understanding that experts can feel threatened by the new entries to the field. Bernie Berger shared testing insights illustrated with movie clips, and a test tools panel showed a very versatile round of experiences available in the room.

At a point of the day, I was kind of tuned out and thinking of what kinds of experiences, in particular but not limited to tools and automation I would find useful and I tweeted:

I’d like a testing conference where you can’t refer to experiences before 2010. Definitely not from the 90’s. The world is different.

— Maaret Pyhäjärvi (@maaretp) September 27, 2016

Surely there is historical value in remembering all the things that color our past, but also, we often talk to rooms of people who were not there 20 years ago. Who are not working with technologies from 20 years ago. Who are not in organizations stuck to where we were as industry 20 years ago.

Some of my personal experiences and stories can be entertaining lessons of history, but more from a folklore point of view than as helpful advice to current everyday life.

The conference yesterday was in the day. It was a New Testing Conference. Surely there was a glimpse of folklore here and there. And there were people from organizations that even today feel like they live 20 years in the future. But the glimpses of folklore made me aware of the amount of folklore I introduce when I talk. And I'd like to be aware of speaking (and choosing speakers) who will speak of recent experiences, reflecting against a relevant base of past experience to see the recent learnings and approaches deeply.

My developers haven't hated me for being a tester for almost 10 years. Why do I still keep mentioning that and transfer the expectation of bad relations to the new people? Do I need the old stories, or could I just source from the new stories of wonderful developer relations to set that as an example of what I expect to see?

The mechanisms I had to manage releases and release dates are completely different from the agile approaches to having continuous releases. How relevant is it to tell about the hard times of the past, when there is a way out of that hard time?

Conferences against the structural problems

I've been buzzing around all morning organizing all the things that need organizing before I jetset off to New York to do a keynote at the Test Master's Academy conference. That's kind of awesome. But simultaneously, it's not. It's a lot of (unpaid) work. Just like any other conferences where I speak. I have higher expectations for this one than many others, as I've been positioned to be the opening keynote and there are some internet friends I look forward to meeting. 

If you take a look at the conference lineup, you see plenty of men and women, and even people of color. There's people who are believers in exploratory approaches, and people who are keen to see automation rule to world of testing. And it's all amazing, just as it should be. 

But like I said, the speakers put a lot of work into conferences. There's the time for traveling (that isn't paid time even when the conference is). There's the time of organizing all the things that need organizing for being away. And there's the time for the conference. I have a great chance of actually enjoying this conference as I get to go first. Often times, I'm preoccupied with my preparations throughout the conference.

So I wonder, many times, if this all is worth the hassle. It's fun and all that, but is it really giving me as much as it is taking from me? 

Then I enter the twitterverse and see this:

20% of #testbash Brighton applicants were women. Come on ladies, you should all be submitting more if you want a better balance.

— Rosie Sherry (@rosiesherry) September 23, 2016

I retweeted it with my note: "There's a systemic problem under this, as the kindest and most encouraging of us see this. It's not up on women but the allies to change this". 

Allies is a word I don't really like, but I lack a better word. I mean men who care that the world changes to be a better way for their daughters, partners and friends in the underprivileged groups. 

The first step is to recognize there is such a thing as privilege, and I appreciate it might be hard. I have tons of it, even if I lack some. I'm a woman of a certain age (love the term and just learned that from Sandy Metz) which gives me courage I did not have when I was younger. I'm very white and from a society that has given me opportunities I wouldn't have had somewhere else. Privilege is a special right, advantage, or immunity granted or available only to a particular person or group. As someone who is underprivileged, it takes you more work to get the same done.

When talking of women, I hate talking of minority because where I come from, women are equal in numbers in conference attendance and work places in the field of software testing. Women are not a minority, but they are underprivileged. Even where I come from, often a majority of speakers in testing conferences are men. 

It's clear other conferences end up with a different result because they balance out by not just encouraging submissions but inviting specific topics and people. And that is justified because of the systemic forces that are against women.

• Less women are encouraged to speak for their companies in public.
• To get to be considered the one at your office who gets to go (or even submit), you need often years of work against the corporate cultures in IT favoring men.  To get to a speaking position, you might need to keep all your focus on keeping that position instead of donating time for free for conferences that are businesses. 
• Women tend to carry a bigger load of the emotional labor (organizing family life) than their partners and thus have less time available to use. 
• It can be harder to find time to be away from other duties, it can require direct financial investments and include a mental load from judgement on the choices you've made on leaving family behind. 
• When speaking, you're continuously facing "chosen for gender" allegations. 
• When speaking, you need to exert extra effort in presenting yourself in acceptable tone. 
• You feel you need to be good, not average to justify your existence. 
• You need to work against the stream, with lack of role models. 
• You need to accept that your feedback can be more harsh and personal just because of your gender. 
• It's common to ask (unpaid) diversity work from minority groups over offering to pay them for the work they do. 
• It can be harder to find the extra money to even loan for the conference on expenses if it comes out of your own pocket even if expensed later. You will know you chose a conference over something for your family and there's a gendered expectation on how the choice must go. 
• Coming back from a conference, it still feels women report on the harder treatment of the "stupid ideas" they pick up.

When the problems are systemic, it's not up to the underprivileged group to change it. It's up to all of us to change it. Conference organizers have a lot of power in the change if they choose not to be passive victims of the system. 

After a discussion with a friend, I managed to sum this up in just a few sentences. 

What the conference organizers can do:

• Pay for the work. All the work. Including submission work. When you stop thinking of submissions as your right to free labor, you start finding better ways of investing your conference budget than call for proposals. Good proposal is hard work. My talks tend to take me a week of work before 1st proposal. 
• Pay on time or early. Yes, I know it is hard because you don't have the money yet. Stop treating speakers of the world as a loan office. The underprivileged are less likely to submit when they know they have to admit to being challenged in this. 

Before numbers like 20 % women submitted are relevant, we need to consider what are the reasons that stop women from submitting and be open to the possibility that their reasons might differ from the reasons provided by men. And if 20 % of a 100 proposals is 20 talks for a conference that needs 10 talks in total, that might not be a bad situation at all. There's an actual potential for an all-female conference. 

Why would you? Because you believe that diversity means learning from people different to you makes the world a better place. Being representative would be important as different backgrounds bring different lessons. I do, do you?

Fighting the urge for Jira

I've spent a day deepening my personal understanding of end-to-end scenarios and reliability of all the test automation we have around here. I have not come to a conclusion about it yet, but started to slowly frame my role as an exploratory tester as someone who tests reliability of the test automation systems. And coming from my interests on clean code & reuse, I seem to be also taking an active role towards testing the sharing solutions on test automation make sense also or more in the future.

As I was testing end to end with an exploratory approach, I was bound to find some issues. I'm in an easy situation now in the sense that I have an old version that "works" to compare against, kind of like back when I was doing localization testing. If the comparison version was broken in the same way, we just mostly did not need to flag the problems.

All the issues I found ended up in a mindmap while testing. There was a color coding. Problems with the new not confirmed with the old. Problems with the new, confirmed not to be with the old. Problems with the old, vanished from the new. Problems with both.

As  the data was collected and was pretty convinced I knew enough for now, I stopped for a moment. Normally, this would be the moment when I, at latest, go to Jira and log some bugs. I had to fight the urge to do that.

I fight the urge, because I want to keep trying the fix-and-forget approach. Instead of taking these to Jira and moving on, I want to:

• Find the test automation that isn't catching  these (and pair up to make these caught)
• Find the developer who is contributing to these, to understand their work priorities on when my feedback on these (and others) would be most timely for not just randomly jumping around the product, but completing a feature / theme at a time
• If these are known issues, figure out a way to get and keep the main more release-ready

I believe I don't need to prove my worth in the number of issues in Jira. I find that in a new organization, the fear of someone coming to check my work based on Jira cases lurks in the background. And I fight the urge to go for Jira, which would be the easy route. 

Same same but different

I've been receiving tons of congratulations with my change of jobs. Some mention that my new work sounds awesome, some note that it looks like I went back to doing exactly what I left 8 years ago.

This is how I described my job out of a whim in LinkedIn:

A hands-on tester with enough seniority to figure out what is right for whatever goal assigned. Working on making the Corporate Security Client awesome through empirical evidence and smart testing approaches. 

With two weeks in the company and some hands-time with both production versions and upcoming versions of the product, I recognize many (most) things are the same. This lead me to ask myself the question: why is it that I see this as a way forward in my career? I clearly do.

On my previous round of F-Secure, I was in a test manager position - or at least, that is how I thought of it. There was a bunch of other testers I was helping to be more awesome at testing, and rarely when I felt there was time from all the meetings, I tested. Often mostly localizations and UI, perhaps because I was particularly adept for those tasks in relation to many others.

Test Management wasn't completely new to me, but it was new enough to hog a major part of my days. I relied more on information I could get from other people, than on information the software would give me.

This has changed. I lost a money-worthy argument with a significant vendor in a job after F-Secure because I had done what my manager told me: "you are too expensive / valuable to test hands-on, just guide the others". And I would have had better empirical evidence if I would have instead spent 2 days of the week locked up away from useless meetings, hands-on testing. I would have known things that I ended up speculating on. I could have shown what works and what doesn't better.

I notice that at F-Secure, I'm still fighting the old habit from coming back. I don't need to be part of all the discussions. I don't need to talk to every product manager and owner. I need to be selective, and I need to be able to trust people to tell me things - or that the software when tested tells me things they forgot to tell me. And I can do that now, with a few more years of experience under my belt.

There's another thing that is clearly very different. Now we have significant amounts of automation. And programmers who unit test! I'm delighted to have collaboration with my team's test automation specialist on end points we want to use when testing things, approaches to make our existing automation more versatile and smart, and things to add from what I learn through exploration.

Where the company is now and where I am now, all of this makes it different. And I believe being a better empirical technologist is a better step forward on my career than becoming a manager. Seniority gives me similar things than managers get from their role. But the practicality and impact through practicality - that's the thing I strive for.

Reasonable Expectations -exercise

So, you land into a project unknown. You collect a bit of info, organize the claims by sources you've heard them for, pay attention to differences between sources and decide who you're going to primarily believe, for now. And then you dig into testing.

A lot of what you're experiencing does not match any of the claims. And there's a bunch of claims you're making now that no one else did.

You realize there might be a usual problem going on, with "no-one's land". With enough small teams around, there's things where you just see things end to end that none who focus on their bits have. So what do you do?

I remember clearly a time, when I wrote clear and well-investigated bug reports about these, and then the ping-pong game starts. It often continues until there is either a developer with a system perspective or until I play personal relations on getting someone who wouldn't want to look at it to look at it to help pinpoint it further. I feel exhausted just thinking about this.

Today I run into something of this sort. And instead of writing a  bug report, I made a list of reasonable expectations I was planning to talk about. Reasonable expectations are my claims on what I find would be reasonable to expect and I let the developers tell me I'm wrong - or find out that I'm right and then discuss the bugs I did not want to log against the end-user perspective claims that are not true after all,  based on evidence. For now, until we've changed our software or our perceptions.

I realized  this isn't something new for me. I've had great success with this before, with a difficult project manager of the past. The mechanism is just something I've never before labeled. I play a lot with the dynamics of communication while I deliver the messages as a tester. The dynamic of making me the one who is proven wrong on the reasonable claims turns around the feeling of how we'd talk about the exact same thing through a bug report.

The Nameplay around Testers

I'm a tester by identity and have made a lot of my choices on what skills to focus on developing next based on what would be useful for testing. Testing is the quest for empirical evidence over speculation of plans, and the endless strive for better through learning with the application (or specification) as my external memory.

I care somewhat what I'm called, enough not to prefer to be called a developer. I'm a tester (and an occasional programmer) and being a tester is my way of finding community of like-minded peers who want to dig in as deep into testing as I do.

Today as I was testing in the corner of our team room, I was again reminded what it is that makes me different. I was laughing as the product misbehaved under my scrutiny, I felt immense joy in figuring out what needed to be fixed and getting someone to fix it so that I can forget about it. And I continued to the next puzzle the product presented me. I love my job!

But here they choose to not call me a tester. They call me a Quality Engineer. Like the title makes a difference, I'm still a tester and I'm still directed with the idea of empirical evidence. As a tester, I'm still not only "system testing", I'm testing. The variety of mechanisms is open to me. The only thing that limits me is my focus of filling gaps and my skills. And skills chance with deliberate practice.

As a tester, I test ideas. I test while the feature is being built. I test after it is "done" - and not nearly done enough too many times. I test it after we release it into production, finding gaps in what I can do (environment realism) and what I know to do (use cases). I'm never done, but every day I can dig in a little deeper or wider.

I also run again head first into the discussion of being more "DevOps" where everyone is a developer. I've had this discussion so many times that I could pay attention to the meta, the arguments I use time and time again:

• "Tester" is relevant to me as it helps me find my community of likeminded deep testing thinkers, the specialists
• No one is just a tester, specialty does not block you from doing other things if you have time
• Becoming good at something requires focus, and while half the industry is with less than five years of experience, skills like mine (tester skills) don't get built unless you get a fertile ground to build them
• Wanting to call me a tester is like wanting to call me a man. How about having everyone being craftswomen for a change. I'm still a woman and I'm still a tester, even if you think the other term magically includes the other. Let's just call everyone women and everyone testers. That is equally true. 
• Playing with words is boring. How about we mobbed on some testing instead? Pick an activity, any activity and I show you what a tester like me does.

 People in general are just to keen to find their boxes and stick in them. It's not the label that defines the box, it's the attitude and skills of the people.

Like I Never Was Away

My first week at a new job is behind me, and it has been one of the weirdest experiences of my life - in a good way.  I joined the company I used to work for 8 years ago, and with one week there, I'm starting to feel like I never was away.

I've changed a lot in the 8 years:

• I still prefer exploratory approach to testing over automating, but I'm now a tester and a programmer. I no longer live by imaginary limits of a role even though I strongly identify as a tester. 
• I've lived with systems bigger and more complicated than what we're developing. Things appear smaller than they used to. 
• I've learned that empirical evidence trumps speculation. That it is a better choice for my time to test hands-on than sit in a meeting where people tell their theories of the thing none used. 
• I've grown appreciation for (a lot of times idiotic) test automation as a means of fast feedback and a base we can build more on. 
• I value my freedom of choice even more than before. I love not getting tasks but visions to guide my work. I love not having a specific description of responsibilities other than being the most awesome professional I can  be. 

There's things where the company has changed and not changed. The elements of the products people talk about are much the same. Surely they've changes some three letter acronyms and come up with lots of new ones, but the domain concepts remain. There's a lot of same people, even if in different positions. It's fascinating to see which ones have advanced to management, which ones to deep technical roles and which ones appear to continue as I remember them back then. A big change is that there no longer is a big bureaucracy to get access to the code base.

For purposes of testing, there's one big change: there's more test automation than before and in a very healthy way.  I find in a week that I don't agree with a lot of the stuff on how automation is and my fingers itch for shared learning in mob format and some major refactoring. Also, I'm sensing a bit o my biggest automation fear around: the lack of great testing as focus on programming eats up too much of the intellectual bandwidth while learning. 

So with a week in, I feel like I never left. And I know what I want to pick up next. 

I want to spend time with the closest developers I have in my team and stop the programmer-tester boundary of unit vs. system tests. I'm betting we're mobbing on his unit tests in a few weeks because I already introduced ideas of what and how I want verified where his response was to realize some of that could be unit tests. 

I want to spend time exploring the biggest risk components with some tester colleagues who might settle for less than I would. And while learning what and how I would test, I want to use my closest automation tester colleague to turn some of my ideas into automation that helps us run things over time. 

I want to learn and share with everyone, starting with my tester (quality engineer) colleagues. We'll first do mob testing on some python test automation, as the automation study circle is an existing structure. To complement with my true interests, we've already agreed with one colleague we're starting an exploratory testing study circle. 

We'll see how this goes.  Let the fun times begin - there's lots of amazing people around. 

How I was interviewed as a tester

I've been a tester, test manager and again tester for a while. I think I know what I'm doing and I know I have a lot more to learn. I've been a restless soul telling all my employees so far that I will move after two years. Granlund expected me to last a year when I said two. And I stayed four and absolutely loved it.

As four years passed, I started thinking if I however should move. I've looked into two jobs, been offered the two jobs and ended up accepting one of them. The first I passed with a mantra "Quality of Life" - I had a lot of that with Granlund. No stress daily releases, great team and a supporting manager. 5 minute route to work. I concluded I would be a fool to give all of that up. And yet, I did.

For the job I passed, I was very close on accepting it. I went through the whole process of recruiting: meeting the recruiting manager, full day of evaluations with a psychologist, and even spend a full day training the extended team to figure out if I would like to work with them. The last thing was my suggestion, and we had a very nice and wonderful day of testing the company product in a mob format, finding bugs they were previously unaware of.

The reason I did not accept the job in the end was that I felt like the whole process was about finding reasons why I wouldn't be the right person or good enough. The recruiting manager knew of me from a shared past, and the questions I remember him asking were about my bad (past) habits of impatience. The psychologist was assessing how I would complement the team, but also pointed out that my "clock cycle" is fast and people may have hard time staying in my pace. The team training was about me seeing the team and the product, but when it came to the company framing the day, it was about "work sample from me". As if they were the only one making a decision when recruiting!

With the job I passed, I spent a lot of time thinking about what they could have done better. They could have compensated me for the time they pushed me around in recruiting process. Or they could have at least acknowledged that more (unpaid) phases to recruiting might protect them from bad hires, but also from really good ones. They could have made me feel they want me, not only that they could use me if I convince them on their points. They were not the only one needing convincing. And with senior software people, the employers might need to consider their approaches.

The job I accepted was opposite. They invited me for an interview very much the same way as the other, though networks. The first interview focused on figuring out how they can  build a job I would accept. They built the job for me, with a description that did not exist at that point.

I interviewed with the manager for the newly opened position, but our focus was on discussing ideas of what to do in the job. It felt more like an idea sharing session, and my focus was on assessing if I could learn to love this manager as much as I did my current one.

The HR interview seemed like an introduction to how awesome place the company is and how nice benefits they have. Sure, there were some questions on how I approach things and how I manage with the English language.

Then I interviewed with the team. It was a chat about stuff, with focus on "we just wanted to know you test and don't only manage". But I got to see that I would get along with them, just as much as the other way around before I needed to make my decision.

So I took the job. I started this Monday. I feel like I've come home again. I have amazing team, and interesting technical (and people) problems. I know I'm a piece to the puzzle that helps. On day 1, I got introduced to what we're doing. On day 2, I got the test environments up and running. And on day 3, I tested and found bugs; I shared ideas of how I'd want to test things we've created and got acceptance on testability changes; I got careful positive marks that we'd mob on different types of automation soon to bring together unit tests, system tests and exploratory testing because they just won't say no to things that make sense.

I'm feeling extremely grateful that there are companies who approach recruiting like they want to hire, instead of trying to figure out reasons why they wouldn't. The latter sounds a bit like testing: Look for opposing evidence. When there's none, we guess it's time to release.

I need to feel I'm wanted and needed. So for now, I'm just going to savor this. The energy of being wanted and welcome takes me a long way in doing the things I can help with. 

How I interviewed testers

My regular job responsibility is not to do recruitment. Recently, I've had two experiences of recruiting from completely different perspectives. First was one where two companies were considering to recruit me (or I was considering to be employed with them) and of the companies succeeded. The other was an experience to provide professional services to support recruiting. And that is what I wanted to write about.

The case was with a company that had interviewed several people and shortlisted two people that could be appropriate. The two people had gone through the regular interviews with non-testers. I offered to spend 30 minutes with each candidate they thought they would be interested in to pair test and give an assessment of how they approach testing.

My assessment report for the company included an hour of video of screen & us testing together, and a summary of their strong and weak spots as I saw them. We tested the company's application for a pre-selected feature, in a test environment.

I opened the session telling we'd work as a pair with the rule of "I have an idea, you take the keyboard". I had the idea first to introduce them into the application and most of the session I would be the hands, as their ideas mattered.

In 30 minutes, I got to see the testers flow on how they get started with something new. I got to see

• if they would take notes / model things while testing.
• if they see bugs. 
• how they model technologies and focus of testing. 
• if they were dependent on external answers or could provide theories of their own.
• if they at the end of the session has ideas for making the testing deeper or if they were ready to move on to other features. 

The interviewing non-testers had little ability to assess this without the guidance. Personally I think having the non-tester interviews first (which were longer than these sessions) was probably the wrong way to do the narrowing down the candidates.

The "cultural fit" people assess in interviews means we often look for people who look like us, talk like us and appear to fit in. But it leads us to choose people who might not fill in the gaps we have. 

And what if the best of testers in the lot never got to the finish line because they were dropped out by people who don't have the necessary knowledge of what testing by someone who knows how to test looks like? 

If I ever again move, I will suggest testing together in my next interview. With or without bugs, you see the way the person models the application and the system, and if nothing else, the recruiting managers would need to get better at recognizing skilled testers. 

Know Your Ground

There's this feeling when you join a new company that says you want to fit in. You want to please. You want to figure out what they do and how they do it, and not rock the boat too much.

It's not that you're afraid for your work. You're afraid for your ignorance. There's things you just can't know.

But then there comes these moments, where you need to not please but stand your ground. Bring in what you're about. I've had a couple of those moments already.

A senior team member assigned responsibility over the team suggests I could include all my ideas and comments and tasks in writing in Jira tasks. It very soon became obvious that my idea of testing (exploratory testing) and their idea of testing (test cases) are not an exact match. I suggested we experiment with what will end up in Jira. The contortion to the good testing I do to make it appear pre-planned (over continuously learning) and linear (over visual connections) just isn't something I volunteer to do even when asked.

A senior colleague advised on how to improve things and get to where I've taken my previous team: daily releases. I respectfully disagreed on the ideas, and suggested small changes regularly, almost continuously to assess if a direction we believe in would be the right way to go.

I look around, and I see too many people who step down and do what is asked. The ask was clear, why wouldn't the action be?

As a tester, I'm supposed to know how I do good testing. If something asked of me takes me away from that or makes me partially  go away from that, I shouldn't stand down without a good discussion.

I know my ground. I stand my ground. I negotiate and experiment, and move. But the starting point is that I know where I'm coming from and where I'm going. I believe this is a big part of why I love my work as much as I do. I'm not a victim, I'm an active player. 

Other lessons shadowed by Mob Testing

Reflecting what discussions took place in the last two days of Mob Testing on a course, I realized there is a pattern I think I'm seeing when teaching in this format:

Mob Testing overshadows the other testing lesson I'm trying to teach. 

Regardless of what my first exercise on testing is, people talk only on observations about Mobbing. And that is not a surprise: mobbing for majority of people is awesome new experience and they need to share both their love of the experience and concerns on "this wouldn't work at our office". And then there are sometimes some people who hate the format. I've now met one who describes the experience as causing anxiety "my brain melts and I can't think". But all in all, the focus with first exercise is in the mobbing.

During two days of mob format exercises, the discussion changes back away from the mechanism of how we learn to what we are learning about. So it feels that mobbing eats away the other learning for people unfamiliar with the style and I need to adjust my teaching to accommodate that.

Instead of having my first lesson in mob being a central one (like ability to create Selenium scripts or Exploratory Testing with Charters), I need the first one to be something people know already.

It's hard to pay attention to all the learning going on at once. Need to figure out leveling things appropriately. 

Mob Testing meets Automation

Some months ago, I received an email inviting me to do a public test automation course in Jyväskylä, Finland. This course would be my third in series with the same title over several years, last edition being in 2013. Thinking about it a while, I decided to go for it again.

Since 2013 I've learned a lot more on test automation. I've also learned how I rather teach things on courses, creating experiences where the whole group shares the experience. For that, I use Mob Testing (or Mob Programming).

I completely redesigned my 2-day Test Automation Course to be around five experiences in Mob Testing format.

1. Creating a basic Selenium Webdriver test (4 times...)
2. Refactoring a basic Selenium Webdriver test to Page Object Pattern
3. Test-Driven Development with JUnit
4. Test-After and Test-First ApprovalTests 
5. Changing Web Application for making Selenium tests easier 

These are all pretty simple first experiences, but as per feedback, people loved the format and enjoyed the experiences. I was particularly pleased with my improved skills on TDD, reflecting to the time when I did my first course as participant and escaped due to my extreme discomfort for lack of understanding. I loved seeing my "I never coded" testers work as part of the group and pick up what we were doing in this format in ways they wouldn't have without the group (or my) support. 

The redesign left 6 slides from my past course. The delivery left me with ideas of what to clarify / improve, and I definitely will do this again.

The Integration ApprovalTests

With my last day at Granlund, I want to share one of the proud moments we've had here with  my team with our test automation efforts.

Unit testing was never easy for us. With all the people in agile conferences talking about unit testing as if it is the most natural thing any developer could do, I found that either we're worse than everyone else (I doubt it) or the conference speaker circuit isn't representative. Knowing that many (even most) organizations struggle with unit testing helped.

We did try. I helped developers clear up time from schedules. I found us teachers to do workshops with us. And the project manager would follow some of the numbers very proudly, even reporting them as part of his monthly steering group reports (that never really made sense to me).

I asked about qualitative stuff: how did the developers feel creating and using them, was there examples from this week they could mention where a unit tests helped them and while there were some, it wasn't a very good experience. We addressed the feelings is not knowing what is worth testing, not being able to test things where architecture wasn't built with tests in mind and many more. Over a course of two years, we still kept hitting the same wall: perceived lack of value.

Seeing some agile developers test (TDD), rely and love their tests, it was clear that we were missing something. But TDD did not turn out to be something that could be easily introduced regardless of firm belief (on my part) of its value.

So we added first database checks and created a lot of value through data monitoring. I've written about this before. With architectural change to our printouts (the main thing our end users want out of the system we're producing), there was a chance of driving for API that would enable automated testing of some of the more complicated stuff to test manually.

We separated pushing the data into an excel with formatting functionalities into a layer of its own, and all the functionalities related to getting the right data right was separated. While conceptually the unit testing of all this was still too hard, the idea of testing against the API wasn't. And we used ApprovalTests to make it even more easy.

We created the scenario to test into the database manually using the application, and added an ApprovalTest for each scenario to keep track of getting the right contents back.

With ApprovalTest, the contents got automatically saved onto an .approved -file and formed a golden master - alert us if this changes.

We created 20 scenarios and started using them as part of our continuous integration. 

Over the course of a year we've had these, they have failed for us for reasons. They've saved us many times from side effects none could foresee.

We also have some unit tests with ApprovalTests, in particular with combinations approvals. It might take a moment to understand that your test is the code + the file, but when you do, this makes a lot of sense. And most importantly: it made practical sense for us in a situation where we were not ready for all the great and fancy ideas related to TDD.