Fundamental Disagreements

There are things in testing that never get old for me. The intellectual puzzle of uncovering the answer key to bugs that matter, and the dynamics of removing friction from uncovering bugs to such a level that the bugs never really existed. Figuring out communication that conveys things the people in this particular conversation would otherwise miss, with realisation that writing a specification is the wrong mindset on a learning problem. The results in efficiency and effectiveness, and building something of relevance that works. 

As a tester, while I center information about products and organizations, I'm also an active agent in the system that creates the products.  We're builders, building together.

I felt the need of saying this once again seeing Michael Bolton proclaim: 

So, for the thousandth time: testing does not make things better. Weighing yourself does not improve your health. Weighing yourself informs decisions about what you might do with regard to your health.

Me and my tester colleagues in projects are more than a measurement device like a scale. If we were like a scale, we would be commodity. We choose information we produce based on risks, and make decisions when we call out risks we focus on. We find information and choose to present and act on it. We have agency and a place at the table to do something with that information. 

Testing is about making things better. It is about selecting the information that would be valuable enough to make things better. And it is about timely application of that information, sometimes so early that we avoid making certain mistakes. To make things better, we need to be founded in empiricism and experimentation. We learn about the state of things and we actively change the state of things every single day at work. We don't decide alone but in collaboration with others. 

It's not enough to know, we have long since moved to the combination of knowing AND doing something about it. 

We want to know some things continuously and we create test automation systems to replenish that information in a world that keeps moving, that we want to move. 

We want to know some things deeply, and spend time thinking about them with product as our external imagination, without caring to capture all of that to test automation systems. 

We build better people, better products and better societal impacts by actively participating in those conversations. We consume as much information as we produce. 

We can decide and we should. 

While this is a fundamental disagreement, acknowledging it as such is what could move us forward.

Untested testing book for children

Many moons ago when I had small children (I now have teenagers), I saw an invite to a playtesting session by Linda Liukas. In case you don't know who she is, she is a superstar and one of the loveliest most approachable people I have had the pleasure of meeting. She has authored multiple children's books on adventures of Ruby and learning programming. I have all her books, I have purchased her books for my kids schools and taught lower grades programming with her brilliant exercises while I still had kids of that age. For years I have been working from the observation that kids and girls in particular, are social learners and best way to transform age groups is to take them all along for the ride. 

The playtesting experience - watching that as parent specializing in testing - was professional. From seeing kids try out the new exercises to the lessons instilled, my kids still remember pieces of computer that were the topic of the session, in addition to the fact that I have dragged them to fangirl Linda's work for years. 

So when I heard that now there is a children's book by another Finnish author that teaches testing for children, I was intrigued but worried. Matching Linda's work is a hard task. Linda, being a software tester of a past while also being a programmer advocate and a renowned author and speaker in this space, sets a high bar. So I have avoided the book "Dragons Out" by Kari Kakkonen until EuroSTAR now publicised they have the book available on their hub.

However, this experience really did not start on a good foot. 

First, while promotional materials lead me to think the book was available, what actually was available was an "ebook" meaning one chapter of the book and some marketing text. Not quite what I had understood. 

Second, I was annoyed with the fact that the children's book where pictures play such a strong role is not promoted with the name of the illustrator. Actually, illustrator is well hidden and Adrienn Szell's work does not get the attribution it deserves by a mention on the pages that people don't read. And excusing misattributing a gifted artists work by not allocating her as second author works against my sense of justice. 

So I jumped into the sample, to see what I get. 

I get to abstract to start with annoyance. It announces "male and female knights" and I wonder why do we have to have children's books where they could be just knights, or at least boys/girls or men/women over getting identified by their reproductive systems. Knights of all genders please, and I continue.

Getting into the book beyond the front page keeping Adrienn invisible, I find her mentioned. 

Ragons. Cand. Typos hit me next. Perhaps I am looking at an early version and these are not in the printed copy? 

Just starting with the story gives me concerns. Why would someone start chapter 1 of testing to children with *memory leaks*? Reading the first description of a village and commentary of it representing software while sheep are memory, I am already tracking in my head where the story can be heading. 

For a village being software, that village is definitely not going to exist in the sheep? I feel the curse of fuzzy metaphors hitting me and continue.

Second chapter makes me convinced that the book could use a translator. The sentences feel Finglish - a translation from Finnish to English. Gorge cannot really be what they meant? Or at least it has to be too elaborate for describing cracks into which sheep can vanish? Sentences like "sheep had stumbled into rock land" sounds almost google translated. The language is getting in the way. "Laura began to suspect that something else than dangerous gorges was now." leaves me totally puzzled on what this is trying to say.

Realising the language is going to be a problem, I move to give less time to the language, and just try to make sense of the points. First chapter introduces first dragon, and dragons are defects. This particular dragon causes loss of sheep which is loss of sheep. And dragons are killed by developers who are also testers and live elsewhere. 

We could discuss how to choose metaphors but they all are bad in some ways, so I can live with this metaphor. There are other things that annoy me though.

When a developer makes and error, she is a woman. That is, when we are introduced in explanation text dragons as defects, we read that "developer has made an error in her coding". Yet, as soon as we seek a developer to fix it, we load on a different gender with "he is known to be able to remove problems, so he is good enough". Talk about loading subliminal gender roles here. 

What really leaves me unhappy is that this chapter said *nothing* about testing. The testing done here (noticing systematically by counting sheep every day) was not done by the knights representing developer/testers. The book starts with a story that tells that dragons just emerge without us leaving anything undone, and presents us *unleashing the dragons* as saviors of the sheep instead of responsible for loss of the sheep in the first place. The book takes the effort of making the point that knights are not villagers, developers/testers are not users and yet leave all of testing for the villagers and only take debugging (which is NOT testing) on the developers/testers. 

If it is a book about testing, it is a book about bad testing. Let's have one developer set up fires, wait for users to notice it and have another developer extinguish the fire! Sounds testing?!?!? Not really. 

On the nature of this red dragon (memory leak), the simplifications made me cringe and I had to wonder: has the author ever been part of doing more than the villagers do (noting sheep missing) with regards to memory leaks? 

This is a testing book for children, untested or at least unfixed. Not recommended. 

Unlearning is harder than learning right things in the first place, so this one gets a no from me. If I cared of testing the book, setting up some playtesting sessions to see engagement and attrition of concepts is recommended. However, this recommendation comes late to the project. 

Testing, A Day At a Time

A new day at work, a new morning. What is the first thing you do? Do you have a routine on how you go about doing the testing work you frame your job description around? How do you balance fast feedback and thoughtful development of executable documentation with improvement, the end game, the important beginning of a new thing and being around all the time? Especially when there is many of them developers, and just one of you testers. 

What I expect is not that complicated, yet it seems to be just that complicated. 

I start and end my days with looking at two things: 

• pull requests - what is ready or about to be ready for change testing
• pipeline - what of the executable documentation we have tells us of being able to move forward

If pipeline fails, it needs to be fixed, and we work with the team on the idea of learning to expect a green pipeline with each change - with success rates measured over last 10 2-week iterations being 35 - 85 % and a trend that isn't to the right direction with an excuse of architectural changes. 

Pull requests are giving me a different thing that they give developers, it seems. For the they tell about absolute change control and reality of what is in test environment, and reviewing contents is secondary to designing change-based exploratory testing that may grow the executable documentation - or not. Where Jira tickets are the theory, the pull requests are the practice. And many of the features show up with many changes over time where discussion-based guidance of the order of changes helps test for significant risks more early on. 

A lot of times my work is nodding to the new unit tests, functional tests in integration of particular services, end to end tests and then giving the application a possibility to reveal more than what the tests already revealed - addressing the exploratory testing gap in results based on artifacts and artifacts enhanced with imagination. 

That's the small loop routine, on change.

In addition, there's a feature loop routine. Before a feature starts, I usually work with a product owner to "plan testing of the feature", except I don't really plan the testing of the feature. I clarify scope to a level where I could succeed with testing, and a lot of times that brings out the "NOT list" of things that we are not about to do even though someone might think they too will be included. I use a significant focus on scoping features, scoping what is in a release, what changes on feature level for the release, and what that means for testing on each change, each feature, and the system at hand. 

In the end of a feature loop, I track things the daily change testing identifies, and ensure I review the work of a team not only on each task, but with the lenses of change, feature and system. 

I tend to opt in to pick up some tasks the team owns on adding executable documentation; setting up new environments; fixing bugs and the amount of work in this space is always too much for one person but there is always something I can pitch into. 

That's the feature loop routine, from starting together with me, to finishing together with me. 

The third loop is on improvement. My personal approach to doing this is a continuous retrospective of collecting metrics, collecting observations, identifying experiments, and choosing which one I personally believe should be THE ONE I could pitch in just now for the team. I frame this work as "I don't only test products, I also test organizations creating those products". 

It all seems so easy, simple and straightforward. Yet it isn't. It has uncertainty. It has need of making decisions. It has dependencies to everyone else in the team and need of communicating. And overall, it works against that invisible task list of find some of what others have missed for resultful testing. 

Bugs, by definition, are behaviours we did not expect. What sets Exploratory Testing apart from the non-exploratory is that our reference of expectation is not an artifact but human imagination, supported by external imagination of the application and any and all artifacts.