Thursday, January 21, 2016

Users reporting bugs

I've had the pleasure of seeing how Llewellyn Falco deals with his open source project ApprovalTests. There's a lot for a tester to learn on the attitude you have towards free labour on your project: encourage it!

Even in closed source project, we have people who are not paid but will contribute to our project. I tend to think of the main group of them as users.

A recent tale of a user

He was playing a popular game, MineCraft, and run into a bug. He isolated the bug and was sure it was one, and decided to go over the lowest level of commitment someone unpaid can have on a project: reporting the problem. The bug wasn't critical for him, but it was not insignificant either. So he decided to contribute some of his time, to help a little.

He found the MineCraft Jira and run into quite a quandary. There were many possible projects the bug could belong to, and instructions disencouraged duplicates. Finding out if yours was in there was already quite a puzzle, but with the work done, he ended up realizing it most likely was not. Thinking he found the right Jira project, he logged an issue with all necessary details of steps to reproduce this.

In two hours, he got a response. Sounds great, right? To his disappointment though, the response was to point him to another project as he apparently had chosen the wrong project to report to.

Being an outspoken fellow, he responded with a kind email suggesting that the company / employee of the company might want to, in future, to consider the answers they give to their end users doing free work for them. Not to say "do more", but to show gratitude on work already done and drive things forward as paid people. That way the user might volunteer time again next time they run into something relevant. The response back was unappreciative, outlining that the employee already did the right thing guiding him to the next project.

As for how the tale ends, the bug got lost. The user never continued to the next project. And most likely won't bother again on spending time.

The contrast to ApprovalTests approach

I've been stunned to learn about how Llewellyn Falco deals with similar cases in his ApprovalTests project. When contacted on a bug, he asks the user for a pairing session. If it's nothing, he gets to briefly meet someone new somewhere. And if it is something, he pairs up with the user, fixing the problem during the session.

Unsurprisingly, he has made friends for life with this approach. He has users who seem to be coming back to report more.  His approach surprises people as it is so rare. But perhaps it should not be.

To get out software to improve, we really need to harness the opportunity and encourage free people to do more work for you. Paid people are in the position to treat the unpaid in a way that encourages their contributions.

I recognize that as professional tester, I hardly ever anymore volunteer reporting bugs without someone paying me for it. I've learned to report bugs as user with "reclamation" added to them, to get proper reaction and compensation for my time in cases of serious issues (2 times tried, 2 times got a discount of the service price I was paying).

Couldn't we treat the users as if we cared since we do? I care enough to take tasks from my internal users and to report back to them. Pairing for fixing with them would be awesome. Perhaps even something to aspire for next.

No comments:

Post a Comment