I spent a wonderful day Mob Testing with a bright group of people in Portugal today. They left me thinking deeply on two things:
- Importance of working in one's native language - the dynamic of seeing them work in English vs. local language was immense.
- Need for clear plans
I wanted to talk a bit about the latter.
I've been an exploratory tester for a long time. I've worked with projects with very detailed specifications, with the end result of having a system that worked as specified 100% but filled 0% of the real use cases the system was intended for. I've worked with projects that don't bother with specifications at all - all is based on discussions around whiteboards. And I've worked with projects where all specifications are executable, with the experience that to make it possible we often like to minimize the problem we're solving to something we can execute around.
The first exercise we do on my mob testing course involves an application that has very little documentation. And the little documentation it has, most people don't realize to go search for. The three sessions we did gave an interesting comparison.
First session was freeform exploration, and the group was (as usual) all over the place. They would click a bit here, move to somewhere completely different, check a few things there and make pretty much no notes other than bugs I strongly guide them to write down. The group reported as experience that they were "missing a plan".
Second session was constrained exploration by focusing on a particular area of functionality. The group was more focused, had hard time naming functionalities they saw and finishing testing of things they saw. Again the group reported as experience that they were "missing a plan" even if the box kept them more cohesive in the work they shared.
The third session was tailored specifically for this group, and I had not done that before. I allowed the group 30 minutes to generate a plan. They selected a feature with a web page claim after discussing what the unit of planning should be (use cases? user interface elements? claims on the web page?). Before spending any additional time hands on with the application on top of the two sessions earlier that had barely scratched the surface of the feature, they cleared up their plan.
The interesting outcome was that
- They found less bugs
- They were happier working against a recreated (low quality) checklist
- They missed more bugs they saw while testing and dismissed them as irrelevant.
- I saw some of the symptoms they saw as symptoms of something really significantly broken in the application, and having seen them test, I now know how I could isolate it. I suspect there are only a few people in that group who would know what needs more focus on.
I take this as a (trained) wish for clear assignments, clear answers and generally a world where we would have our tasks clear. I find myself thinking it is not the testing that I know, and that it is the testing a lot of my automator colleagues know. And that in getting out of that need of someone else placing us the "plan" and being active about making and changing our own plans as we learn is the core to good results.
We all come from different experiences. My experiences suggest that my active role as a software learner is crucial. Having documentation and plans is good, but having the mindset of expanding those for relevant feedback is better.
