Friday, January 3, 2014

When bugs feel too simple

I've spent today testing a feature reading its specification. I let the specification reading bring ideas into my head, and follow them - with the result of logging quite many bugs.

One bug in particular was a tipping point today on how I feel about my usefulness.

I was reading the specification mentioning copy feature, to realize that I've tried the feature but I have not really owned the data I'm copying, that is, intentionally creating it to be something that I would find relevant for this copy action. So I created the maximum data (filling in all the fields) with data that I could recognize (naming everything so that they tell about their location) and hit the copy button.

With some more tests, I could pinpoint the piece of data that was causing the problem, and the problem is not about the contents of the data I entered, just about me filling anything in to a particular field.

With  this bug, I could not help but to feel frustrated. A very simple way to see if a feature works is to use it. And if you will use it, why not use it with full data?

This example is from a team with me and six developers. I'm a part timer on the product, developers are full timers. And yet we, continuously, over and over again, regardless of all the discussions about developer testing end up with this: evidence of not having used the feature.

I went digging into the bugs I've logged in the last few weeks, and came to a very depressing conclusion. None of the bugs I've found required any more sophistication than this one. All it takes is a little time with the product.

Need to start doing something about this. Again and more. This is not a skills issue. This is an attitude issue. Developers can test this. For a reason I don't really get out of my team in discussions, their testing for these issues just does not happen often enough.


  1. Hei Maaret, kiitos blogistasi (and sorry if my Finnish is bad!)

    In my organisation I get developers to give me a demo of the feature before I agree to test it. I would ask them to first show me the form working with all the fields filled in (because it's always the complicated cases which show up all the bugs, as you note), and then to show me what happens when you don't fill in any fields, to test the validation. At that point I would go and do more complex testing myself (long inputs, special characters, cross-site scripting attacks, etc) but I get the developers to show me a couple of the basic cases like these. This doesn't take more than 10 mins and the developers are generally happy to show off their work.

    Then if an embarrassing bug like the one you described comes up during the demo, the developer is usually really embarrassed about it and it makes them think twice next time before they dare to demo something which isn't working to the tester!

    In this way you are pairing with the developers and mentoring them in how to do better testing. I'm not sure whether this would work for your organisation but it works well for me and will hopefully give you some ideas about how to try and get developers to go that one step further! Good luck :)

  2. Hi Jenny - nothing wrong with your Finnish there, nice! I actually do this quite often too, getting a demo. I'm still more at a point where I need to claim I need help (stealth pairing) to get someone to pair up with me, and on this particular feature the developer just went for a long vacation leaving the "done" feature for the rest of the team. Not intentionally, but it just went that way.

    Finding face time is also not always easy with my current place of work. This example is from a team where I feel all is possible, but my other team has people at the office only 2 days a week, and then always somewhere fully booked.

    I should definitely discuss first testing as a pair as an approach we could consistently do. Thanks for reminding me how useful that is.