Monday, January 25, 2021

Having Testers Makes Quality Worse

I'm a professional tester. I have been one for quarter of a century. I know by my results and my feedback that I have been useful for my organizations. But the longer I've worked in these types of roles, the more I understand that while a tester may be a solution, a tester may also be a problem in a people system. And whether we, testers, become problems has very little to do with us, and a lot to do with the other people around us. 

Today, I want to dig in a little deeper on an idea that I have been on for most of last year. 

First of all, let's start with what my career of quarter of a century is really about. While I am a professional tester, a significant part of my professional skill is knowing when to refuse to do testing alone and only do it while pairing to grow people around me. Another significant part is to walk away from teams that grow better when they have no testers, and join teams that are in places where they can grow having a fully contributing tester. My career is on *testing* not on *being a tester* and that difference is a significant one. I believe, paraphrasing Elizabeth Hendrickson, that testing is too important to be left for just testers. It belongs to everyone in software development. Sometimes it is useful to allow space for people to concentrate on it, and sometimes people who get to concentrate on it are called testers. 

Growing the Understanding Over Multiple Jobs 

With this blog post, I want to look at back over the last three jobs I have held. They all have one thing in common: I work in product development, with lovely, brilliant but imperfect software developers who have been soaked into culture describing what the organization expects of them with regards to testing. Yet, the situations in the teams I have worked in could not be much more different.

The first position of the three was one where I was hired as the first tester the organization had. They had 20+ years of software development experience and a very successful software business, and they had managed to do this without a tester. They were seeking one as they had identified, based on internal product management feedback, that there could be enough work for someone so that product management did not have to do all of their testing. Also, they had a metric on big visible error screens per logins, which hinted they really needed someone to tell them how to reproduce problems customers were seeing but not complaining about. 

For the years I spent with the organization, I could see they needed different things from me at different times. I was a hands-on tester for them for the first year, reporting all those cases the customers didn't complain about but were experiencing and the developers fixed them. They needed me to catch up with the understanding of what feedback they had been missing. But as they learned what they had not known, they learned to know it, and through pairing (and stealth pairing), the developers built rapport for their testing discipline.  Finally, we got to place where I did my best to avoid reporting bugs and work with developers to improve their understanding of how problems emerged. In the end as I was leaving, they did not need me - they needed a voodoo doll of me they could look at and go "you'd want me to test this, you'd want me to try different values, you'd want me to try more times". And while being the person who holds space for good work to happen, it was evident it was my time to see other teams. Since it was always one of me and 20 of them lovely developers, we worked out the idea of not expecting me to do all the testing from the start. 

The second position was one that I rejoined after being away from a decade. While I had been away, the team had found themselves into a position where there was a good test automation discipline and hired people they vaguely called testers, into writing system level automation that was a full time job to keep maintained and going forward. I found myself building bridges of ideas of tests I had from exploring to test automation, and building a culture of fix-and-forget with zero bugs on backlogs. The developers were soaking in every piece of feedback, and fixing things so effortlessly that in hindsight I feel like there weren't problems to address. The culture of managers staying out of the way and senior colleagues wanting to make space for great quality combined with documenting with test automation, extending every test idea over different environments could have been run with developers. The challenge I addressed mostly was one of communication - with our code being part of the system, understanding where what testing would take place wasn't always easy and straightforward. 

The third position is one that I hold now. I joined one team first to learn that while they could use me for a while, their long term would be better if I walked away and left them to try out their lessons now without a tester. I joined another team starting a few weeks ago, and see a similar pattern of needing full-time people on tending the test automation system, even when whole team pitches in. My work, again, appears to be addressing communication, and understanding where testing of various pieces and perspectives reside. I test to learn, to understand, and to distribute my lessons. I have now multiple teams without testers, and some of those do brilliant. The pattern emerging seems to be one of ownership, and that isn't always an easy one with systems of scale in multi-team setting. 

Was I getting somewhere?

I've been hired with organizations as a tester, and by choosing the pattern of how I operate with the idea that testing belongs to us all, I've come to be appreciated for results. I've been a tester, test manager and test coach depending on what the day needs me to be, and promptly addressed the risks of assuming tester = testing. I've seen most developer do well with testing, and I've seen many developers do better with testing than professional testers who focus on testing. I've seen how having a tester, and making space for a tester to do testing, makes quality worse. But I have also seen testers who truly bring added value, in understanding what we are building, understanding the customer and the domain, and thinking in terms of systems over components. And increasingly, I've seen tester/developers who specialize in programming test systems be critical in sharing a good baseline practice with their teams, and find new labels like SDET, eventually dropping the T while still continuing on the test side of systems. 

There is no one clear recipe for testers and developers. As a recruiting manager, you need to understand where your team is now, and where a tester could take them. But it always comes with a risk: having testers makes quality worse if developers let go of testing they are doing now. 



Sunday, January 3, 2021

Contemporary Exploratory Testing

We all know what testing looks like: it's when we hunt information, using the application, it's interfaces and all existing information as our external imagination to go even deeper in empirical understanding of what is true and what is an illusion. It involves a person or multiple people, using programming to get to places in time, but it is all framed in this quest of someone learning what more we could try knowing. 

Yet when organizations set up "testing", they ask for resources, pay limited attention to skills, focus on plans, covering requirements, writing and executing test cases, cutting down in testing when project schedules fall short. 

When we say "all testing is exploratory", we have the individual tester on their good day in mind. I call this exploratory testing, the verb. The act of testing is inherently exploratory when people are involved. No matter how strict a test case you gave, how much you told not to leave that path described, the human mind wonders and the human fingers make mistakes revealing more than the test intended. 

The organizational frame however makes a huge difference on how that inherently exploratory testing takes place, and how much of its wings are cut. I call this exploratory testing, the noun. Organizations often set up frames for testing that are far from exploratory, and get results that are far from what we would mean by results of exploratory testing. 

Exploratory testing (verb AND noun) is focus of my learning and teaching. I want to create excellent products with excellent testing, and I feel that the 36 year old coining of the term needs a revisit from its watered down interpretations. Thus I sometimes add still one more word to explain what I am aiming for: contemporary exploratory testing.

Contemporary is about today. I use that word to get away from the ideas of ISTQB and agile testing where exploratory testing was considered a technique, a thing you do for unknown unknowns on top of all the other testing recipes. For me, it has for the last 25 years of my career in it been an approach, a foundation of agency, learning and systems thinking that frames my choices of using all the other recipes. And I believe that is what it is at its roots, so calling my approach contemporary is saying I want to take a step away from some of the popular notions of it being the idea of just spending time with an application to find bugs. 

With this new year, I welcome you to join my exploration of how to understand and teach contemporary exploratory testing. I have many subproject on it:

  • Exploratory Testing Podcast. I just published my first episode yesterday, and will continue to do so on a monthly cadence. 
  • Exploratory Testing Academy. I work with  Parveen Khan (UK), Angela Riggs (US), Irja Straus (Croatia) and Mirja Pyhäjärvi (Finland) to create free testing video courses and a series of paid facilitated courses with focus on learning by doing testing under various constraints for various systems under test. 
  • Exploratory Testing Book. I finish the book I started so that you don't have to read all the individual articles and get out a timeline of where I am now compared to what I wrote earlier
  • Exploratory Testing Slack. I want to bring together people that are figuring out contemporary exploratory testing. I take it more as a forum of practitioners than a forum of consultants. 
  • Exploratory Testing Twitter account. I use this for promotions, because the one advice I was given on marketing (collect people's emails) is the one advice I don't want to use. I want pull over push, even if it is against all things marketing. 
I have a full time job at a company as a tester. My job pays well, better than average developers, and I have my company specific goals there. I do this all because I feel the one thing I should do better is scale. I make my work available for free to support scale. I have things I could use extra money on, but I leave the payment part for the community based on value. You can pay (but don't have to) for my book in progress. You can pay by buying me coffee. Since I am Finnish, you can never donate me money - only pay for my services. But I want that to be optional, and work against paywalls in my own little way. 

If out of this I create more great testers, more testing trainers, more love of testing in both professional testers and programmers - my aspirations are fulfilled.