How would you count bug reports you never get?

There's recently been a significant transformation on a piece of software we work on with my team. In the last month, we've done more releases of that piece than we managed to do in the year before it. There's a clear feeling of progress. And the progress has included the amount of code for the piece dropping to a quarter of where we started off with the transformation. The final frontier of siloed development has joined team ownership and our drive to clean up things to be understood.

Looking at this piece of software, I'm observing an interesting aura of mysticism on how users approach it. It's like I'm moving into the past where I four years ago surprised our product manager with the idea that software can work when he tries it out - something that should be close to obvious in my books.

I started to think about this seeing a tweet:

"We only had one or two bugs in production in 4 years of mob programming." @WoodyZuill. https://t.co/sKrEsC1VI3

— Cucumber (@cucumberbdd) April 26, 2016

The worst piece of code my team delivered probably had also just one or two bugs in production over the last year. But for us, the reason was that the users would seek workarounds rather than ask for fixes, or that the bugs just did not bug the users enough for them to speak up.

Since we moved the piece of software into the frequent release cycles, we've fixed a LOT of bugs users never complained about. A lot that users had not even run into. Looking at the  bugs we've fixed, I find it nearly impossible to believe that they could have been using the software all this time. But they did, they found their ways around everything that bugs us (and slows them down).

When you get little to none in bug reports from production, do you assume all is well? My sample says I shouldn't. Even asking the users directly might not give you a realistic picture.

And on the other areas where feedback is more frequent: if it bugs a user, it's a bug. And we have loads of things that bug the user, that we rather frame as feature requests, some of which we have no intention of acting upon. Counting them seems more of counterproductive. We could count how often we get interrupted with a quick request to change something in production that reorders our immediate priorities? But most of those wouldn't count as bugs in any traditional sense.

Emotions and trigger words in exploring

This post is inspired from two sessions of mob exploratory testing. I did one in Agile Serbia -conference on Saturday, and Llewellyn Falco did one on pre-Craftconf meetup in Hungary reusing my session description.

In discussing his experience of facilitating exploratory testing mob without me, it was interesting to notice what he had chosen to use from me and what he seemed to do differently. Here's what I learned:

• We both emphasize the private role of the mindmaps created while exploring. It's not a final deliverable and an external document, it's something intended to help you (your mob) while you're testing. 
• He emphasizes more the tracking of concepts on the mindmap than me. It seems I treat the product as my external imagination so that the mindmap is more of a secondary tool for me, whereas he treats the mindmap as a necessary model that needs to be built right from the start. I speculate that the different role we're seeing is related to me primarily thinking of all testing as performance (practice before making pretty much any documents) and he still has the strong developer background that might be driving him towards testing as artifact creation. More discussion on the differences in belief systems clearly needed to understand this.
• We both emphasize paying attention to the emotions while you test, but there is a clear difference in how we try to communicate that. He taught his group to pay attention to trigger words revealing negative emotions: "This is confusing", "I wasn't expecting that". Basically any negative sentiment, frustration and confusing being the most common ones. Trigger words should, as he explained, lead to making a note that drives a discussion about a possible bug with the developers. When I lead a teaching mob, I alert the groups on emotions in general. We both note that there's a lot of uncertainty around on what to speak up about among testers, leading teams to lose information they could have readily available. The encouragement to speak up through feelings is necessary.

I find Llewellyn makes an excellent case for developers picking up the thinking patterns around exploring. Learning from various mob exploratory sessions with testers around the world is probably the quickest way to get practical ideas from people's heads. You should try it too. Let me know if I can help! 

Spying on users - a new form of usability testing

There's all sorts of production monitoring tools we've been using, but I recently run into something different I've been looking into tonight. The tool is called Hotjar and a friend introduced it to me as a tool for usability testing. With the tool, you can see for each user in video format their mouse movements and clicks, and can build more fine-grained ways of analyzing when your users lose engagement on your pages.

How much of a spying tool this is became clear to me today as I went and checked for the first time the recorded uses of my personal landing page.

The red line traces the mouse movements. The red dots indicate clicks. I see what devices and browsers my visitors have used, and how long they've stayed.

Following what my users saw I can test different screen sizes and devices with eyes of my users, without setting the environments up myself. Doing this early on (and fixing), I could prune out problems through testing in production, annoying a limited number of users but coping with my limited ability to cover different combinations.

For now, I'm just blown away with this. And needed to share. I reserve my right to change my mind as always, but for now, I'm just excited. 

Roles and expected contributions

On a remote day of work, there was very little discussion. I was focused on the application over the team's Flowdock channel when a discussion started.

Four developers at the office had started to wonder about a user interface design element on something one of them was working on right now. The question on the channel was directed at the user interface specialist, with a picture: "what if it was this way instead?".

The discussion continued  on defining relationships of concepts to be selected: should you be able to select two at once out of the list or just one? Should there be a "no selection" option too?

The user interface specialist comes up with a conclusion on selection  between radio buttons and combo box, that the discussion boils down to: "There's no reason to hide the selections from the user".

A developer comes back with a consistency argument: there's another element conceptually just like this just right above this and it uses a combo box. Why shouldn't the two be the same?

I enter the discussion, repeating the consistency argument. But I also add a piece of data: in 90+ percent of the cases, the user does not want to make a selection of this. Selecting anything but "no selection" is a special case.

The conclusion changes with the added data. In this case it's natural to have a combo box over the radiobuttons. A design is agreed upon.

I share this story to emphasize that it does not matter whose role is what and what contribution you could expect based on the role. As a tester in the team, I have a lot of empirical data and a keen eye on the real use cases through listening to end users.

Seeing this through a discussion over addressing it through hands-on testing just made everyone's life a little easier. Hat tip to my developers on initiating a discussion that required them effort due to remoteness, and on persistency on caring how it would be. 

Cognitive Dissonance: How I learned to like programming

A remark in an online lean coffee caught my attention: "I don't write automation. I really don't like that work. And there's N developers and just one of me, so I leave the code for the developers".

It could have been me speaking. But it wasn't. It wasn't even someone who has been around me to learn to speak like me. But hearing something I recognized so strongly made me feel the urge I need to share my story of how things changed.

Rewind back a little over a year, and I would have sworn I will never be a programmer. "I really don't like that work" would have been exactly my words. Someone suggesting I could try that would be met with skepticism. You can probably find me saying that in public in this blog that is intended to keep me honest to myself - if you get something out of it that's a plus.

I started mobbing not because I wanted to learn programming, but because I wanted my team to work better together. I wanted to feel my social needs met at work. So in hindsight, little did I know about how the human mind works.

When thinking of changing jobs, I had an interview with a psychologist. Me mobbing with my team was one of the stories I shared with him, and he labelled what might have happened: Cognitive Dissonance.

Cognitive dissonance is a state in which our beliefs and actions are not in aligned. People have this habit of needing feel whole and consistent, and we do that to the extent of rewriting our history and perceptions.

If my foundational belief is that I don't really enjoy programming, I seek to not do it - after all, I believe I don't like it. Theories around cognitive dissonance seem to be saying that we change our beliefs from dissonant actions - not the other way around. When I do something that is against my belief system, it starts a process of rewriting parts of my belief system.

With mobbing, this happened slowly. After all, we were mobbing at most once a week in the office. The extra sessions to learn the teaching technique I organized with the local tech community added some. I always told myself I had other motives to join the mobs, I never really cared for the programming part of it. I wanted developers to learn testing. I wanted to see if I could think fast enough to spot problems without the product as my external imagination. I wanted to see if I could use mobbing to teach my exploratory testing skills to other testers. The coding was just something I had to endure.

But over time, my feelings changed. As I did more of the programming in the mob, it wasn't just that I became more confident with the things I already knew as technical non-programmer or that I picked up pieces while we were doing it. My attitudes started to change, impacting my belief system that I used to define my place and role in the system of creating software.

I went from "I don't write automation. I really don't like that work." to "Let's write some automation. But why focus only on test automation, we have a bigger problem to solve with thinking around code".

My claim is that convincing me on this in advance would have been close to impossible.

So think about it: how could you use cognitive dissonance in giving yourself a chance to change your fundamental beliefs by engaging in work you feel you don't do. Or even more: could you use cognitive dissonance in changing the mind of that difficult developer who never wants to create any unit tests?

Experiment. There's a lot of power there.

The stories you need to tell to product owners

"I could add this new reporting feature in less than a week", says the developer. "But that would not include the cleaning up and changing of components we should do in the same area", he continues. "We probably should just say the feature takes a month, because we have to be able to change the components too", he concludes.

Ever heard this monolog? The idea that you need to come up with a different story for your product owner, because they understand only features and not the technical maintenance work. The idea that all the technical maintenance work should be baked into the work, even if it transforms the scope of the work?

We ended up reframing this discussion. Let's be nice to the product owners, and enable them the fact that the feature is in the production sooner. It gets real feedback sooner, and it starts paying itself back sooner. And let's trust that it is ok to do the needed cleanup after, without baking it in.

Sometimes, the feature-orientation and control of the product owners causes development teams some peculiar behaviors. So this post is a start of my new thing to focus on: what could we do so that the relationship of the perspectives wouldn't be based on rules (who gets to decide what) but on trust and collaboration.

Approaching zero bugs

Zero bugs. Stop coding bugs. I can't help but smile a little whenever I hear this but often choose to step away from the argument. This was a topic, however, that Arlo Belshee addressed at Agile Alliance Technical Conference.

I don't really care much for zero. But I care for numbers becoming much smaller than what we're used to. And that is the perspective I listened into the discussion around the fancy marketing term.

The core idea I picked up was that when your code reads well, you make less mistakes. And that your ability to read code is more important than the executable tests keeping your code harnessed. Even more, it could be that the amount of code comments could be a negative indicator on the ability to code with less mistakes, as good code reads kind of as English.

I've been speaking about the fact that my team does continuous (daily) delivery without test automation around. It's nowadays so normal for me that I don't really put much effort anymore into explaining it to others. But when I still talked about it, I heard we're irresponsible. I heard it is not possible even if I've lived it for almost two years now. And it is possible with a team of 10 developers and one tester.

The zero bugs discussion gave me new tools to look at what we do and what might make us successful in what we do.

1. We focus on code readability.
   We're cleaning up and rewriting regularly. I encourage this behavior - even if it means I get to help test same things over and over again, as the amount of test automation is ridiculously narrow.
2. Developers do exploratory testing
   The developers test themselves quite extensively. This is actually the main reason I'm driving forward test automation in my team still, as I feel the manual testing must make us slower on this part. We're careful. But with care, we also succeed in introducing very little regression while delivering features with a steady pace.
3. Tester adds perspectives to exploratory testing
   When I test, I still find problems, so we're down to small numbers only after addressing the internal feedback. For the last year, the developer's testing skill has increased significantly, and a part of succeeding with this is my new discipline of avoiding writing bug reports (replacing them with discussions).
   
   There's a special approach on how I find my problems, and I still try to open it up better with my team. I read shape of code commits to know what I will cover. I will compare the shape I see to the shape I expect based on discussions. And shape of discussions is a form of both what is said a lot, a little and not at all. The shapes are models and I overlay a lot of different models to make up my mind of what to cover.

I see value in what I do to keep things great for production, but also love the fact that I'm increasingly less needed - I call that progress. But listening to the zero  bugs discussions, I get the idea that I might have been discounting the value of my other activity as a tester: making sure my developers are empowered and supported in their need of keeping the code readable, to their best current knowledge. I love how they pick up new things and ideas and hate their old stuff - that's how it is supposed to be.

The more we practice, the better we get. How often are we allowed to fix our past mistakes, made at times we knew the least? I would hope the answer is: every day.