Friday, April 13, 2012

If it looks too easy to be missed by developers...

On my first weeks at new job, I've had the pleasure of reporting bugs again. I find this particular result of testing to give me the feeling of achievement. The more relevant the problem, better.

There was one bug I reported on Monday, that just looked too easy to be missed by the developers in my team. As I originally reported it, the problem was that when logging in with one of our three main browsers, there's a highly visible error message. And that this seems to happen only with the recent builds, not in production version.

In the end of the week, I quickly asked in passing the developer whose component was failing, if a fix is available in the next weekly build. He seemed puzzled: what problem, what fix? I checked our Jira, and the issue had not been addressed - which is quite normal. He took a quick look at it, and came back with "I didn't change this for ages" with some details.

I started testing the issue more with the information from him. With fresh eyes, I realized I entered the program from a bookmarked link - something I hadn't mentioned in my original report. I also realized, that I had different addresses bookmarked in the other browsers. So I had missed a relevant bit of info I provided now.

Bottom line: if it looks too easy to be missed by developers, it may be that they didn't test, but in this case, I missed relevant factors that are needed to get the bug visible.  Talking sooner than later to the very busy developers is still a good idea.


  1. Hi, Maaret!

    Great post! This is not a rare case. As we all testers get in the flow we "think" we have all the information available. We might ignore the fact that the OS in x86 instead of x64 as we mentioned. We might have rebooted in the middle of installation as it prompted, but the report lacks the step.

    By accepting that we are fallible and allow ourselves to make errors, we might just be able to prevent most of them. So what if information is missing. Next time I'll do better and be sure to include the info. Just state the problem as soon as possible and with care. If there's something missing, then talk about it and and fix it. It's that easy.

    BR, Peksi

  2. Hi Maaret,

    Like your post. (Brought to my attention by Jari Laakso)

    One way to avoid this is to use the information in the bug report, perhaps after a few hours, and try and reproduce it yourself using only that information. If you cannot reproduce it then, why would you expect a developer to be able to.
    Another, and probably better, way is to use a method / tool to record your steps so that they are easily reproducible. As tools are not always available it is a good idea to teach yourself a kind of shorthand notation that lets you recreate steps if necessary. In any case you need to keep good focus, during your testing, of the fact that steps and action that you find natural are not so natural to others.


    Jean-Paul Varwijk

  3. In this case, the developer never read the bug report, so the steps in the bug report are not of much relevance. Instead, the relevant lesson I'm taking from this, is that there's a lot of dimensions you should pay attention to. The developer never told me "does not repro" but the just his puzzled appearance on not having touched the area told me I stopped my investigation of relevant dimensions too soon, which I filled in. There's a lot of sources of information, and if it seems too easy to be missed by others, there's a possibility it's not as easy as I thought.