Thursday, February 25, 2016

Testing isn't testing: A story of Skilled vs. Random Tester

You know, testing isn't testing. Sounds weird, right? So let me try to explain a discussion I seem to be having over and over again.

There's two people, and both of them could do testing on the exact same thing. Let's say one of them is a skilled tester - which is someone who is really paying attention to how you learn using a software product as external imagination - and the other is a random tester. The random tester might be good, but she is not the same as skilled tester, not unless focus is on learning to test great.

The random tester often is a developer, doing her end-of-development testing to the best of her abilities. And those abilities might be great, and the overall result well worth being called good enough.

When we then compare the "final testing" of a skilled tester and random tester, there seems to be this mysterious magical idea of skilled tester just being dropped in and doing their best performance. But, since the skilled tester learns in layers, the first time testing at the end of development isn't going to be the best performance. There's layer after layer to get deeper, and each layer provides useful results - just not the same.

Some managers still let skilled testers be around a little in the end, and confuse a good performance with the best performance. And many testers still confuse the Skilled Tester's good performance to the Random Tester's Best Performance, in particular when there's always an aspect of testing and learning even when you decide to all it "programming".

If you're serious about good quality, you'll want to enable the Skilled Tester's Best Performance. The Skilled Tester's good performance happens somewhere along the line, but the final show with the final results is the best you can get from her. The best performance happens at the end, after rounds of learning and practice.

So testing isn't testing. Performance and output from such a performance varies greatly. Random testers do a great job, and some of them are very close to skilled testers. It's a question of motivation and focus.

And if you wondered why this? I just had yet another encounter with someone who thinks they get to buy my best performance without practice. Because skilled must mean packaged, like a machine.


  1. I think your argument is only really true for bugs, which are not the only output of testing. Even with bugs a skilled tester aims to find important bugs as well as any bug where as a random tester looks for any bug. If you are skilled in any role you should be confident and be able to standout from someone random. If you compare any of the following there will more often than less be a difference between a skilled tester and a random tester. 1) test approach - a skilled tester should be able to know what approach to take to learn about the product and document their findings etc , random tester is often playing with the product looking for bugs 2) questions asked - random testers questions are based purely on their curiosity , a skilled testers questions are based on a lot more than just curiosity , understanding the context of the product, it's domains, it's users, SME's, technical questions etc 3) summary of testing - a skilled tester knows how to document the information gathered during their test session. Not just bugs but also those huh moments, potential risks, etc 4) how testing is conducted - a random tester might only functionally test a product throughout their session , where's a skilled tester can focus on one aspect and test it differently throughout a session... I think there's several difference between a random tester and a skilled tester.

    1. What I call "random", isn't random. It's someone who has looked into the application with purposes other than testing.

      And I would disagree on me claiming this is about bugs. It's about "final testing", that can include many kinds of things as a result, but the main result for me over bugs is information.

      My point in this article is that being skilled tester isn't magical, but that using that skill to do good testing requires time. Even a skilled tester does a crappy job if all they have is a day at the end. They are *people* and they learn in layers.