Quality and Solo work

With Agile2016 conference, I had a chance on a yet another glimpse into how a group of programmers work together, and how that shows up in results. Llewellyn Falco wrote a blogpost on 5 ways to do decision trees in C#. I feel that nicely shows how you can have many different solutions, and how some of them have more limitations than others, but it doesn't show what I always find fascinating: how the deeper understanding of having the five options was born.

Each morning, there would be some group working on some coding kata. For the piece of code in question, the mob in the morning came up only with one solution. We often talk about mobs as things where bunch of great ideas emerge, but in this particular time-limited mob, they got to one solution.

What then created the others was that the limitations of the first solution left a nagging feel for one of the participants. And with a little sink-in time, solution 2 emerged. Another mob or pair came together to implement that, and then the magic started.

From two competing solutions, they quickly got to five competing solutions and a grounded discussion on which of the actual implementations they would consider the best.

The rule in mob that when you have two competing solutions, you do both just made even more sense. With two available, creativity of the group steps into play and produces more.

My lessons learned on watching this unfold:

• Actively try to find several ways to solve a problem
• Individuals with a nagging feeling are a powerful driving force for improvement

The niceness of the solution (an aspect of technical quality) improved because of a persistent individual. But as that idea of different solution was again fed into a group, the group got a lot better solutions out than the individual.

With mobbing, we keep repeating the idea that it is not about the most you can do, but about the best you can do. We need to recognize more often how much of an impact the passing idea around through different minds actually has on what ends up in the code. And individual is a trigger, but the group is needed to take that trigger to its full potential. Solo work misses a lot of this. Except of course for the wonderful people who keep talking about the five voices in their own heads. 

I paid to speak. Again.

Sometimes I think I have learning disabilities. I again submitted to Agile201x. I again got accepted. I again paid 1200 euros for the flights. I again gave up a week of my vacation to be here. Why do I do that?

I wonder this in particular with having 15 amazing, wonderful and brilliant people in my session. Out of 2500.

I've grown to realize that I'm seriously introverted, to a point where I experience social anxiety if I don't manage my situations well. I've just paid to live a week in what I could frame as a nightmare: 2500 people, mostly strangers and very few of my peers I would find easier to connect with. All moving around in a mass of people.

I usually speak at conferences, because the personal connections to people who I can learn with are valuable. Speaking enables people to approach me, to mention they share some of my interests (yay!) and after that all the connection problems vanish for me. I love talking about things I love, and that is probably why no one ever believes me being introverted.

With that said, I believe there is a big problem we have on conferences: a lot of them make the speakers pay to speak. This means that only those of us with privilege to afford paying all this and taking time away from work get our voices into the conferences have a chance of being heard.

I believe we are losing two main categories of voices:

• People who have done this enough to know that the not paying is just an excuse and you can choose the conferences you speak at more smartly. 
• People who can't take the financial burden, because their companies don't support them and they are not self-employed needing to keep themselves visible for sales purposes

As participants of conferences, you should care about this. It means your conference has selected to give you people who have something to sell. There are some pretty amazing consultants selling their knowledge to you out there so it might not be a problem. But it skews what we get to hear about and who we get to hear from.

The big conferences could afford to pay their speakers. I don't think they will, because this works for them. 

Here's the score from the session I did with Llewellyn Falco on Monday. 

Llewellyn is a consultant. He has incentive to sell his services. I'm not. I'm an amazing tester with real, daily long-term experience pairing and mobbing with people who are hesitant to all of this. For the session, our differences made it great.  At least I'd like to believe so. 

My company does not pay for my travel. My company gives me 5 days away from "real work" a year, and all the other conferences I'm on my own time. I *know* this is expensive. 

The #PayToSpeak model is erasing voices close to mine. Pretty much all with less insistence and privilege than I have. We have a lot to work to real diversity and representation of the industry. 

(Fortunately, the rest of my year is not pay to speak. I warmly recommend seeking speaking engagements with TestBash family of conferences or European Testing Conference. Agile Testing Days family of conferences is not bad either, but is likely to cap you're hotel stay inconveniently. Also, ask to be paid for expenses. There's still often the case of loaning the money to the conference, as they pay back after.)

Exploratory Testing: It's not what you think it is

I just finished my small session on "Exploratory Testing an API" at Agile2016 -conference. The session lead to two insights I feel compelled to share.

Imagine being handed google search box and being asked how you would test it. You come up all sorts of different inputs quickly,  Some of the very basic searches (character sets, number of words, types of search criteria) might lead to think about something more, like exact matches or searching for something that has a connected functionality like airline codes or currencies or vocabulary definitions. You know that's now all the testing for Google, and probably consider it is not the best way to test Google search all in all. But ideas start flowing quickly.

When presented an API, with a "fill in the blank" type of place, we freeze. The same things we could easily come up with while looking at a GUI, we don't and end up blankly staring at the IDE that is really just providing us a GUI to get using an API.

But the more relevant thing I started thinking is that how much of a disservice it is for all of us testers to let exploration examples always be guided to "fill in the blank" types of exercises. I had someone in the audience who clearly decided that was the purpose of the exploration, and surely, it can be one of the purposes of exploration.

But as a professional exploratory tester, I must say that I find that the examples we're presented at the conferences on quick tests poorly reflect my work. I look at an API, and the only reason for me to fill in the blanks is to acquire information - about the environment where this all sits in, about why would anyone want to use this, what is this for?

As my session gyrated this time to filling in the blanks and less on the actual exploration to learn, I feel more strongly about this. I need better ways of helping testers on the right track.

Because the testers who stay on the "fill in the blanks" track thinking just about inputs, they should have been out of job already a while ago.

It's not about finding any problems, it's about finding problems that matter.

I find it much more relevant to find out why a product isn't selling (well, it's free, so selling is not really what I mean here...) and how it could be easier for new people to get started with it, than figuring out that a combination "a§" ends up incorrectly saved in a text file.

A Quality Conundrum

This post is a goodbye to my latest addiction of the week: Pokemon Go. Like so many others, I've played it and enjoyed it.

I've had three typical scenarios of use to keep me engaged in the game.

1. Connection errors on all devices 
2. Crash on opening on iPhone 4s
3. Restarting the game with every sight of a pokemon and after every catch of a pokemon in iPad2

The first two days, I tried connecting every now and then, and never made it into the game. But the game was still interesting, and I kept coming back to it. And at some point, it actually allowed me to log in. 

My joy of it working was premature though, as I was naturally thinking I'd like to play it on my phone. It just seems it does not agree with my phone, and with every opening, first click in the actual game and it closes with a crash. 

In the last five days, I've been able to level up to 11, but all of this comes with a great cost. A minute of gameplay means at least a minute of wait time.

Thinking about this has lead me to understand that there is a quality conundrum. Clearly it hasn't mattered much to me that the game is completely of half unavailable. The only thing I recognize the problems had done to me is that it postponed my commitment to the game, as in paying real money for something. And now, bringing out the rational side of the that deletes the game as a time-waster. 

The game apparently has loads of users regardless of its problems. Released with problems, it has made its creators millions (missing a reference) that it wouldn't have yet made, if it was kept on development without releasing. People are clearly enjoying it. The millions they might be losing in addition to the millions they are already making seem irrelevant. 

My time did not cost them anything. It did not cost me enough to consider it relevant for the first week. But multiplied with the number of players, the cost has been high, yet distributed to come out of many pockets in small streams. 

I can only hope that acceptance of this level of "it works" has not come here to stay. But I fear that changing is already too late. We accept problems without compensation. Our lost time is not out of the creators pockets. And in some contexts like this, the lost revenue does not matter in relation to the revenue that is already flowing in. 

Contexts have never been as clearly different to me as they are today. And I worry about this becoming the best practice that starts defining the quality of my life. 

Calling automators good testers

I could have seen this post coming as I was writing my previous post - a longer explanation of why would I go about calling some automators good tester, as they show no understanding of what good manual (exploratory) testing actually does.

The core of it is that I believe there's two kinds of testing: testing as artifact creation (automation, documentation) and testing as performance (exploration). When you're good in one, you might not be good in both. But both are testing. The first focuses more on known knowns and known unknowns, whereas the latter focuses more on unknown unknowns and unknown knowns. 

I believe you can't spend decades automating a process (with good results) without learning to understand it and it's problems and solutions. While your understanding might not be perfect or complete, it can still be good. And looking at exploratory testers not actively going into the realm of automation (pairing with programmers qualify - friends with pickup trucks is a valid approach), that understanding is not perfect or complete either.

When test automators give the impression of believing in 100% automation, they often compare only things within testing as artifact creation. You learn stuff around creating those artifacts, and you include them. The manual testing they compare to is the manual testing exploratory testers loathe - the test case driven dumbed down commodity testing that drives down skill. 

I've come to think I understand this a little better these days through two friends. 

Pekka Klärck is a good friend of mine, and the creator of the Robot Framework. We've been members of the same local community long time, and in Finland we're lucky to get along even if we disagree heavily. Where I've been an exploratory tester most of my career,  he's been a test automator most of his career. We can easily get into the arguments around manual / automation, and respectfully agree to disagree. But with all of that, he has taught me to respect test automators as a different specialty, who are not any less of tester than likes of me (exploratory testers). 

Llewellyn Falco is another friend, and the creator of ApprovalTests. Working side by side with him, I've come to realize that there's generalist developers who are really good at testing, and who become better when exposed to good exploratory testing. I've also learned that the testers he's met over the years are nothing like what I perceive a skilled tester to be. 

Hanging out with people different than you can be exhausting and frustrating. They might walk over you, fail to hear you and you will need to try again. 

So we have four types of testers (at least):

• commodity testers
• exploratory testers
• test automators
• programmers becoming good at testing
• (test managers / architects - working in scale)

When we argue about what a tester must know and learn next, we should look at the balance. In organizations with loads of programmers becoming good at testing, you could use an exploratory tester. The ratio is more like 1:10 or 1:100 than 1:1. In organizations where programmers are bad at testing, even commodity testers provide value, but a mix of test automators and again a bit of exploratory tester skillset could be better.

Not everyone can hire the guru. Most organizations need to have homegrown gurus. The good homegrown gurus look around and learn a little more each day - regardless of what corner of testing / programming they end up starting with. 

I repeat this a lot but it's worth mentioning again. If this industry doubles in size every five years as uncle Bob has mentioned, half of the industry has less than five years of experience. Let's spread out the learning so that all relevant corners end up covered. More time is more learning, and there's no reason for any of us to stay in an assigned box, instead we should move as our interests guide us. 

Mind the style - sharing vs. correcting

Today I learned something that is useful to me as a context-driven tester. My responsibility is to teach myself and share, not to teach others. It's pull, not push for others too. Understanding other contexts and viewpoints starts with listening, not telling what I have to add.

Let me elaborate this a little.

There was a tweet.

@noahsussman Too much manual testing almost derailed the HealthCare dot gov rescue in late 2013. That's why I was brought in.

— Jason Huggins (@hugs) July 23, 2016

Seeing this, I felt the need to respond that while I agree that (good) automated testing was much needed there, they could have used any good testing.

I've come to understand that part of automation magic of people like Bret Pettichord, Noah Sussman and Jason Huggings is that they are strong testers and automators. There are a lot of people who are strong in one and weak in other, and to do the great things, you need strength in both. So far, I've come to know personally many amazing programmers who are also great at testing, but I've followed a lot less people I'd identify as programmers primarily in the area of test automation (automation testers).

So bringing in someone like Jason Huggins, you bring in both good testing and good automation. The statement saying "too much manual testing derailed" can include the idea that bad testing and lack of automation together derailed, and that fixing problems of bad testing and lack of automation can happen both at the same time.

As soon as I had commented on the tweet, I read more tweets in my twitter stream. I realized that Anna Royzman and Dan Ashby had also commented, and I felt a surge of empathy. Imagine I was saying good stuff, and the masses focus was on correcting me - how would I feel? I deleted the tweet that had existed for 5 minutes, and made a commitment to pay attention to my interaction on Twitter.

A good heuristic is that Twitter is for making new friends (Facebook is for keeping touch with the ones you have). Making new friends by correcting them is an awful approach in the online world.

If you're wondering why context-driven people come off as anti-automation even though we're not, this could be one of the reasons. We see we're adding new data points when we're correcting. The other person is not in a place to accept the information we're pushing. Focus on the good of automation. Feed what you want to see grow. 

Origin stories

I listened to the latest of the Let’s talk about Tests -podcast which was on the topic of origin stories. How did you end up as tester, was there a moment that defined your path and can you pinpoint it? Listening to the podcaster’s story, I felt compelled thinking about mine in writing.

I think there’s three defining key moments for me. The first is about getting started and the second is about finding freedom and third about making a commitment. 

Getting Started

It seems a lot of us kind of fall into testing. For me this happened through direct recruitment. Someone knew two details about me: I had been accepted to study computer science in Helsinki University of Technology (must mean some interest in computers and software) and I had studied the basics of the Greek language before university (must mean can recognize some Greek words). There was a localization project in 6 languages starting up in Helsinki back then, and Greek was one of the languages given to this location. 

I just went with the flow. I scheduled a test of my natural testing abilities and observation skills with the company (seeded bugs, testing localized version against an English reference) and got sucked in to part time work. 

The localization stuff was pretty routine, with Microsoft supporting many sites and projects. We had test cases and we had QA - the contractual idea of someone testing after us with same test cases and telling if we’re missing stuff that could be found around those cases. The feedback was great, although contractually quite intimidating at that point of career. 

Finding Freedom

I changed jobs, and did more localization testing with a Finnish product company with the experience I had. Then it became time to extend from localization to functional testing. I remember the moment when instead of test cases I was handed a (bad) specification with the responsibility to test a mail server on Solaris. Getting out of the box of someone else’s test cases that I would carefully tread through doing enough but not too much, I was now set free. I learned to love what I was doing. I did not know the name of it then, but exploratory testing made me an active learner back then. The difference was really just on how I perceived my responsibility area. 

Making a commitment

A while into the testing work, I came to the conclusion that everyone just hates testers and that I would be wasting my life sticking to it whether I like it or not. I had the youthful bloated ego thinking I could be so much more, and more meant being a developer. Developers command respect, right? I had already experienced that (junior) testers don’t get their voices heard and it frustrated me. 

I moved into a developer job and learned within that job that I could take a step forward and come up with five steps while testing my own stuff that would take me backwards. I learned that average developers don’t automatically command any more respect, and that there’s such an idea as programming as assembly line, where you’re just handed pieces to blindly implement. I did not last long, I came to the conclusion there was no reason for me to be unhappy in one job when I could be happy in another. And that some (many) developers commands just as little respect from the “important people” as testers do and that we could unite to make the world a better place.

I committed to being a great tester. I took a job that enabled me to read and think about testing, and teach testing, I started my journey on the road of learning every day, for the purposes of recognizing and filling relevant gaps in software (product) development. 

Eventful career

There’s been many moments of joy and frustration and life-changing insight for me. I think particularly fondly the moments that make me completely change my mind like realizing that I was teaching test cases while doing exploratory testing; like realizing continuous integration was actually a better idea than controlling change so that I could test “full builds”; like understanding that while smart manual testing can cover more ground that test automators give it credit for, I like automation too; like realizing that over planning and thinking things through, experimentation is giving a chance for “bad” things to turn out good in collaboration. 

I’ve changed jobs often, every 2-3 years. I’ve been a tester, test manager, project manager, developer, teacher and trainer, and a consultant. I’ve been given a great platform to learn, and I feel privileged having been allowed to share stuff throughout my career. I’ve had some amazing managers, and worked with mostly wonderful developers. The local Finnish testing community has been my lifeline, and I’ve learned a lot through published authors and more recently, blogs. 

I love being a tester and helping developers being more productive. But most of all, I love how we are allowed to learn and change, and have many forms.