Mob testing at XP2015

Had a full-day workshop session at XP2015 pre-conference day, "Collaborative Exploratory and Unit testing". I learned many things. I learned that while I'm good with our joined title, I turn very uneasy whenever either one of us drops the definitive word. Unit testing as testing is testing as artifact generation. Exploratory testing as testing is testing as performance. And talking of either one of them as testing is very very confusing.

The main thing I learned though is how different the teaching dynamic for exploratory testing is when you're teaching in a mob format (one computer, everyone working on same problem) as opposed to teaching in paired format. In preparations for the workshop, I had imagined running this in pairs, with each pair working on their own computer. But when it turned out that there were just a handful of people, it just did not seem right to be pairing the small and uneven amount of people who would evidently move in and out of session during the day. So we organised them into a mob.

In preparations of the course, we had been pairing on the types of tasks we'd have the students do, and I was comfortable with the fact that my idea of what we'd do with the software in "identifying things to test" -session was a joined vision. Our paired session was a good example of collaboration, where I as the expert-navigator had been lead into additional insights from active pairing. But I was also very much aware that what I do when looking at software would be unlikely to happen by someone participating our workshop, unless they identified as exploratory testers. And to me it was a simple task: look at software, identify what you could test and write it down in some rational organisation in a mind map, so that you remember to get back on it or notice when you learn more about the software later on.

As the students started off in a mob, for the first round we enforced a rule of strict driver-navigator roles. In Finland in particular, this is a great way of making a rule to take both roles and practice them, so that later on the navigator role could be shared for the whole group. As there was so few people and two of us teaching, I suggested I'd join the mob as one of the students. Asking about backgrounds, I would also be the only one in this mob with experience on exploratory testing as out participants were primarily developer-oriented.

I was amazed at first how slow things were. It was hard to tell out loud things the driver should do and we needed to learn how to communicate about what to click, where to go and what to write instead of doing those ourselves. But we got better at that. Seeing things on the application was also hard, much harder than I would have imagined. Having to see what others were doing without being able to go and instruct was a healthy experience, that showed that the stuff I think is basic is hard for others. And I could only wonder how hard the things I considered hard in exploring would be. I was waiting for my turn, to give a few examples to drive us to a (what I considered) smarter direction, but when it was my time to navigate, My co-facilitator decided to announce that I had constraints. I could only restructure the map of what people had already seen and add things on the map that we had already been looking at but not yet marked down. My hands were tied, and I did what I can within those constraints, to realise that the additional challenge of the constraints was good for me in doing the exercise. There were some things they had seen but not marked down that I could use as examples of what to put in the map.

As the mob continued and everyone started adding perspectives, the things the group could see grew. I still observed the group working and couldn't help but making little "bug"-sounds whenever we'd see and miss a bug. We missed a lot of bugs I couldn't point out with the strict driver-navigator roles. If we would have been more in a mob format, we could have also logged those. There were issues I had not seen before even though I had been testing the feature, because people driving software their way made me see things differently.

On second round we removed the driver-navigator constraint, but also changed the exploration charter from identifying things to test to finding bugs. Finding bugs was something that the group had not been particularly successful with in the first round, but give the goal of focus on bugs instead of features, started quickly identifying things. At first there was a slight unclarity of what is a bug and what is a feature, but soon we got to a point where we'd make notes of "anything that might bug a user". Understanding of the software at hand grew.

I would have loved to continue on exploring in this format, but as we had planned, after finding bugs we were at a point to start mobbing to take the insights we had created and turn then into unit test code. So we spent a few more session on unit testing and ended our day with a summary.

With this experience, I will change my exploratory testing work course just slightly. I will teach first how to identify what there is to test, previously I've taught that in a later session but now I see from the mob experience that the experience is better (drives focus in bug-oriented testing) the other way around. And I will do the first session in a mob format, splitting people to pairs after they see what others are able to do and evened out a knowledge gap just a little by working all together. 

Wait times taught me to improve testing

As the world of software is right now and size of the industry doubling every five years, half of us are with less than five years of experience. This is a major revelation for me, reminding that some - many - of the experiences I've lived though over the 20 years might not even be real for the newer professionals. There's (lucky) individuals that have never experienced a project that was not agile and look at us talking about the change funnily, without being able to relate. With this idea in particular, I've come to realise that courses like ISTQB teaching testing folklore (of really old times!) is getting increasingly ridiculous and more of baggage (e.g. on what developers can and cannot do) than useful history.

There's an experience in the old times, that newcomers don't relate to in the same way: wait times. Still so much dysfunctional agile that this appears, but the emphasis is going down.

Some examples:

• Your developers are working on a feature and you can participate in designing, think through your strategies and tactics while waiting but there's a feeling of waiting until developer feels tester contribution is warranted. The smaller we learn to make the changes, the less this is an issue. 
• Your test environment is broken. While you could test, you could only report on the fact that the older web server version really isn't compatible with your software after the decision to support a newer infrastructure. Or the other project's feature just broke what you were supposed to focus on because the environment is shared. 
• Your build is broken, something went wrong with integrating the pieces together. Most likely it's not one of the pieces since pieces have owners, but whatever is in between the pieces and stops them from communicating, you work hard to find someone who even starts to look. After all, it could be somebody else's problem. 
• You've found so many bugs that your head is buzzing, and you can't focus in trying to find some more before some fixes arrive. 

I remember waiting a lot of the years. Waiting for the software. Waiting for the environment. Waiting for the fixes. 

In times before agile, I waited more and longer. The longer wait times were natural times to go study and learn some new skills. They were natural times to stop to think about what I'd like to improve in testing, and how I could make the change happen. They were natural times to wonder around the office, talk to other (testers) waiting and build on each other's ideas. 

When agile first happened to me, for a while I lost the improvement focus. Retrospectives were a ritual and timing wise short, so only the most pressing issues were addressed. As features to test were arriving continuously, I found myself to be continuously in hurry to deliver the fast feedback, and there were no natural times to stop to think and improve for days. I needed to learn to schedule for improvement.

Why am I writing about this? Why now? I was reminded by a discussion this week, that the courage to plan tasks of significant time on becoming better may not feel easy for those who are new. Those who don't have the comparison point of what it was like when you naturally had slack without organising for it. 

Nothing stops us from organising for slack. As thinking professionals, it might even be our responsibility. Improving and getting better as individuals, teams and organisations is part of our work. For each and evert one of us. 

Spreading your ideas without being there yourself

To cover up a cancellation on interactive session on development, I run a session today at BTD Conference. The session material wasn't mine, but I owned the delivery of it. The session was about group learning, doing Approval Tests Koans (programming puzzles) with regular reflection on what we were learning. Llewellyn Falco did not only make his puzzles available, but shared all the material on how to run this session.

I asked for permission to use his materials primarily because I find the added power of asserts in unit testing that Approvals bring very nice and powerful and thus the content worth spreading. But in advance I had many second thoughts on how it would be to teach on someone else's material. And not just teach, but do a session in a conference, where my personal view was that all the sessions would somehow be original material by their authors.

With the experience, I look at this differently. Llewellyn would not have been here today but I was. If I did not use his materials, I wouldn't have taken the time to create something similar but original - I would have just skipped doing this. And if that  happened, the people who were at the session, would not have had the lessons they had from it.

This leaves me thinking, that we in general need to be more open about sharing our ideas, materials and even courses. In protectionism, we lose the possibility to reach new audiences, on the perceived benefit of being able to cash in on personal delivery.

I've had all my material creative commons Attribution -licenced forever. The reasons were twofold: openness protects my right to access the material when I change companies and the contents were often so much based on my experiences, that direct reuse would not anyway be possible. But to use things I've created as a stepping stone to make creating something better, to enable not reinventing the wheel was something I deemed relevant.

With Llewellyn's openness towards his very reusable and nicely packaged material, I need to rethink my approaches to be even more open. We need to learn to share better. 

Caring for developers

Over the years, there's two main themes on my work on quality software. The main track for a long time has been providing information - actionable information - through means of testing. But the longer I work on software and the more comfortable I am being myself, the more I notice I focus on the other: showing caring and love for the wonderful developers I get to work with.

I've read from numerous remarks that the work I do on that side is not testing, but it is something that complements my testing, it is something that works towards the same goal: valuable, quality software. A happy, skilled developer delivers better results. Being allowed to think, being supported in growing in skills and hearing that you're doing well and improving, they all create a positive cycle.

As a tester, I pay attention to what I get to see in software. Sometimes I see software that doesn't talk to the other pieces of the system - just like the people doing the pieces don't talk. Instead of just reporting the symptoms, I may gently push for a bit of discussions there. Sometimes things barely work, and break more with every attempt to fix. Instead of leaving the developer alone with that, I have the habit of bringing in help they did not request but could use. Sometimes I suspect the code isn't created with a change in mind, and I go the extra mile convincing managers to allow time for team work to change something that works now to something that will work in long term.

Developers seem to appreciate being allowed to refactor - and sometimes what I say makes a big difference on them being allowed to do what they should, or them taking the initiative to do what should be done. And when puzzled on being allowed to go to trainings, I can just choose to say the supporting words out loud, helping people get to trainings they haven't been in for years.

We're in this together. It's just not what my role is, but it's also a lot about who I am as a person. And I'm someone who wants to see the world get better and people be happy. 

Hindsight at the office

Today my team learned a major lesson about what our product is supposed to be. I'm ecstatic, having contributed to a world-changing realisation by asking the right thing at the right time. Even if the "world" is just out little product development sandbox. The realisation leads to a major technology/architecture direction change that results from us understanding (or thinking we do - a few more tests required!) our customers actual needs better.

As the lesson sunk in, we spent a bit more time discussing it with the team in a great, open to possibilities spirit in general. I was feeding off the energy. Until someone stated it: if we had known that we need configurable reporting when we created reporting, we could have taken it into account when building it.

I find that a lot of people don't notice what is wrong with that statement: it's hindsight. If we had known - but we did not. If we could redo our decisions, there's a lot of things we'd be wiser about. Learning is wonderful. But it means that before we learned, there was really nothing else we could have done. It would be wonderful if like in games, we would have a save point and three opportunities to do-over, but we don't. That's games, not software development projects. Or life in general.

I find that hindsight is a symptom of not quite yet understanding what Agile means: embracing change. If you always wish you were smarter in the past, are you really open for change and opportunities to learn?

For me, this has been one of the difficult lessons as a tester. I've actively needed to learn to look forward, see that what we make now is the best we can make now, and it's not about my individual contribution where we end up now. But where ever we are, we can always go forward. Agile is a journey, not a destination. Product quality is not a destination either. We continuously evolve our product, with our understanding.

As a tester, this has meant that I resist my urge to say that a change shouldn't happen just before the release. The best way is forward. Make mistakes, fix what you broke and learn. The products you deliver when you don't stop the change because "software is now frozen" tends to be better. And it being better relies on people actively, in collaboration, trying to make it better. 

Honorariums and conference organising

This is my year of international conference talks as well as continued conference organising. I've recently spent a lot more time than ever before thinking about conference (event in general) organising, after I realised that I'm on the verge of professional in that. You become professional with practice, and the non-profits I've contributed in have given me a LOT of practice.

My motivation for organising events so far has been motivated around my personal learning. I get to meet people, talk to people and pick their brains. Usually longer and deeper, as I'm organising.

My own speaking started as a way of introducing myself, what I'd like to talk about. Even as an extrovert who loves talking to people, I'm very shy talking one-on-one to strangers unless I know we share a common interest. Speaking to an audience gets people to talk to me. And I love talking back.

Recently, I've been getting a little less on the side of learning when organising and speaking. Many people I talk to are more junior than me, and while they're wonderful and help me structure my thoughts, the mind-changing conversations tend to happen with people who have been around for a while.

With my personal learning motivation, I might organise peer conferences (international, the brains I want to pick don't seem to be available in Finland or I've used what I can get from those) and open spaces like coaching camps, to emphasise people who teach (seniority) as participants.

But I love organising the other stuff too, and I'm turning that more towards a business. I'm just now deciding, that I want to work on getting paid for the time I use on organising. Eventually.

It's very much similar to the consideration about getting paid when you speak at an event. I'd like to see, as an organiser that the conferences and events I do would be moving towards a model where we could afford to pay a honorarium (speaking fee). But while my events are free, cheap and non-profit, there's a lot of work to manage the risk of promising that up front.

Most conferences make the profit in the last two weeks. If I paid an honorarium of 2000 euros for every speaker (plus travel+logding), I would need more trust in my paid audience than I have right now. So, I'm thinking about a conditional honorarium model: we'd budget for honorarium and pay that if we make the target on audience. We need to work towards more ethical conference organising.

I really want to bring in people who don't only deliver a talk, but stay after the talk to hang around and to network. I realise this is sometimes very limiting on who I should invite.

There's five categories of people who speak at conferences:

• Speakers who are local. When travel expenses are insignificant, all you invest is your time. And if you meet good people and get to listen to the other talks for free, perhaps its worth it. Or you show up just to deliver your talk to promote your own existence or your company. 
• Speakers who are willing to pay to talk to lower the personal total cost of conference. The fact that you get to participate for free is enough of compensation. Sometimes the conference is just awesome and you want to be there at least once. 
• Speakers who are willing to pay to talk for promotion reasons. It could be that you're promoting yourself (need a job, want to be perceived as a speaking professional) or that you're promoting your company (we sell products / services, awareness of those between the lines). The amount you're willing to pay depends. Oversees travel is more expensive, different conferences are more of promotional value than others. 
• Speakers who are unwilling to pay directly to talk. These people will contribute time for free, but expect the direct costs of travel and lodging to be covered. 
• Speakers who are unwilling to lose money for talking. It's a group of people who often have enough requests and paid work to consider opportunity cost: time used on a conference is time not used on a paid customer. And the promotional value isn't enough to not get direct income from this gig. And these are people who think conferences need to shape up their act, and vote with their feet on not participating if the finances are unfair. (For an article on how much and why a honorarium should be, see http://www.thenerdary.net/post/84544230452/a-formula-for-speaking-fees) 

Many big, commercial and successful conferences are based on getting people in the first three categories. The speakers pay to speak. Keynotes tend to be invited and financially compensated. These conferences would have a choice to do things other way, to attract more suggestions and presumably raise the bar on what goes through into the program. Why would they, if what they have this way is good enough for their purposes and helps them make more profit? 

Many small, local non-profit conferences and events also rely on the first three groups, as they have financially no other choice. 

If you want you call of proposals to attract the full talent pool, you need to also promote the honorarium. With every "pay to speak" component, you are likely to lose a group of people. And even that is not enough to reach all: sometimes the most important thing is that the person feels she/he is wanted in that particular conference. Nothing beats the personal touch.