Slow progress and patience - getting devs to test more

Torn between my two completely different teams, this is a story from the one that makes me impatient. When I joined - already 1,5 years ago - I negotiated on "test-fix-finalize" week where the whole team would do 'testing' - except we would not do testing, we would do code changes to make automated tests possible. We would fix bugs that we really would not want to document in our tests,. And we would add automated tests, unit-level first.

The allocation to "test-fix-finalize" has stayed, leaving a small window of exploring how the system works before it goes live. And we do need that window since we have not managed to get very good at small incremental changes or test automation on any level. Or, some might still argue that 2-3 weeks of developer work is small.

Quite recently we have moved into introducing Selenium tests. After the first examples and the discussion about "can't we just outsource this to some testers somewhere", the team has been adding those. There's been workshops, first one day and this month two days, where the whole team sits together and codes tests together. We don't do this for anything else except tests, so I'm kind of happy to contribute to that.

When adding tests, we've also been changing the product. And that again, to me, in some small scale, is a plus on the list of why I wanted the work to stay inhouse, with this team, to not get yet another round of tests that no one maintains, runs or feels any ownership of. Yes, we did that once or twice, before I joined.

Having people sit together has also other positive impacts. Like this week, while they were doing the tests and I was spending time on my other project, they started talking about how they want to do things. I get an email telling about their "decisions", where the most relevant to me was this: they decided to plan their effort so that it includes testing - well, their idea of testing - and do trade with other developers on tasks to get fresh pair of eyes on their own code. Kind of nice step, I just wish still the system testers would be part of that cycle too. But this did not exist before so its definitely progress. 

I was thinking about what actually brought this change: visibility and sharing the pain. A month ago we moved the system testing backlog into the same Jira-project where all developer work is tracked, because the time was right. And since then, the team had to see how the single tester was getting piled with three times the work that could be completed. And being the nice people they are, they are actively thinking how to help.

I wish it would help immediately, but I guess I need to work on my patience. Step at a time, never give up. If things don't feel right, there's a way to change them. Sometimes finding the way is just taking quite some time...

Reading specifications with thought

There's a rock I've hit hard too many times recently and the pain I feel is clearly telling me there's a need of changing. As part of me processing this, I thought I'd blog. After all, it's been too long.

Starting with an example, simplest out of the selection. We've been adding a feature to our product, which is about printouts. I've seen the spec, read it as thoughtfully as I did in that time, and waited a little to get the feature implemented to do some testing.

It turns out that the feature was nice in isolation of other surrounding similar features. It turns out it was nice on picture and text, but when put together with live scenarios, there's more than first meets the eye.

So I log a bug on that. We eagerly discuss it with the team and decide on a change to what we had thought previously. Implement and again comes testing...

It turns out that we missed yet another scenario. While the first issue was that printouts of two sorts worked differently on main view, the second one is that there's more than the main view and again inconsistencies that would lead us to a different design.

I'm frustrated, as I did not see these coming looking at the spec. Then again, anyone else did not either. But failing twice in a row, with very similar pattern, is just painful. And we keep failing with a similar pattern.

So thinking about what to change, I realize I'm puzzled. I could take more time to test the specifications, since I seem to have some touch to seeing the problems with the live product. Then again, someone else could do that too.

Positive side: no one in the team is initiation discussion whether these are bugs / should be fixed. Just frustrated with the inability to remove the root cause that keeps us stomping on similar issues again and again.

For anyone handling tester's issue reports - there's usually more than the one described symptom. Removing a symptom without addressing the underlying problem isn't taking us where we'd want to be.

Off to reading the next specification, with scenarios I just defined to support my thinking. Perhaps these will help the others see problems too, time will tell. Reading with thought is quite different from just reading. So many things between the lines that make us waste everyone's effort.

Types of events in Agile Finland community

I seem to be taking a habit of replying blog posts with blog posts, and here's the source of current inspiration to write: http://learninggamedev.wordpress.com/2013/08/01/join-me-in-re-imagining-the-agile-finland-coaching-circle/

I've been setting up a session to reimagine things in Agile Finland in a larger scale than just coaching circles, with 24 participant limit this time. Thus I've tried to also positioning the coaching circle in a bigger scheme of activity. The fact has been that for quite some time, coaching circles were pretty much the only activity, muddling them into "all things agile, from beginner to advanced". But it does not have to be so yet it can be.

With discussions around, I gathered the types of events we seem to bounce between when organizing:

• circles (like coaching circle) are for practice and deep knowledge though shared insights
• dojos and retreats and camps (like coding/testing) are for practice
• unfacilitated meetups are for meeting people interested in similar themes, but finding the right conversation is up to you
• facilitated meetups are for talking about a topic together in a group / subgroup, making finding the right conversation a little easier
• peer conferences (unconferences) are for sharing real experiences and debating them for insights
• conferences are for listening to ideas and experiences with possibility to discuss or gain experiences through exercises
• webinars are for listening to ideas and experiences in cases where you can't be onsite
• working parties are for creating something together. 
• courses are for teaching a topic with exercises to practice

I placed some of these types of events on a scale of Content vs. Group size to show how different event types are usually intended for different audiences.



For Agile Finland, the Agile Drinks/Dinners have been unfacilitated, with nice groups of people meeting up, sometimes with a specific theme and sometimes not. The Dinners we've organized within Finnish Association for Software Testing have been facilitated, meaning that there is one discussion ongoing for the whole group instead of everyone grouping up to a different theme.

We have the unfacilitated / facilitated Dinners. We have the dojos. We have the coaching circles. And we have the conferences. What we've been missing is something for a beginner/intermediate audience that allows a larger crowd and that happens more regularly - a mini-conference with a facilitated discussion twist. That's what Agile Breakfast will be about, and the first one will most likely be 6.9.2013 in Ruoholahti - first Friday morning of the month. Location still needs confirming.

For the events status, I collected this: https://docs.google.com/spreadsheet/ccc?key=0Ak7e3JZGe_hadDBscGF4NEFtZm5JdnVMZEE1ZXF0NUE&usp=sharing - just to see what there could be.

Choices, choices ... on how to use time

For about half-a-year, I've been thinking and preparing a webinar series on testing, with a few main motivations.

• Finland is not Helsinki. To do things with Finnish Association of Software Testing or Agile Finland, we can't do all sessions in Helsinki.  Some of the most inspiring Finns I want to hang out with are located in Oulu, Tampere and Turku. And most likely there are inspiring people I haven't had a chance of meeting, since they just don't come to sessions in Helsinki.
• Finland is not Europe or the whole world. I have a strong feeling that we need to be more connected in Europe as well as internationally, and doing EuroStar made me realize how few of our field's big names are from Europe outside UK. Just looking around has taught me about several people I'd like to know better, but I lack the chance of traveling like a consultant. Thus the webinar series would give people like me access to people and ideas that are not otherwise available. 
• Great people and great presentations are not online yet. Knowing from personal experiences of listening to a lot of presentations by Finnish colleagues, a lot of this stuff is not online. I can't give a link to listen to amazing presentations that others should hear, and I suspect my local community isn't only local community like this. Tapping into a wider range of presenters would be great. I'd love to show off my colleagues from Finland, and I'd love to see people I haven't seen before. 

I wrote a description document on this quite some time ago on this.

Since then, I've had chats on this with people. From someone running a lot of online content, I learned that webinars might be really dull way to deliver the contents. From another one running webinars I learned that the numbers on a scheduled session are surprisingly small. From our own technical trials, I learned that while it is possible, a lot of participants actually fade away during the webinar. For presenters, I learned that some people feel off talking all by themselves, and others find the talking to an invisible crowd comforting without the stress they get from live presentations.  From discussions, I learned that videotaping may be a key to what I want to achieve - new content over repeating the same old stuff. And that while we have these conferences where we actually just listen to the presentations, people find the breaks and the face-to-face connections and the forced presence of being there to listen really good. 

With the limited time available to organize something, all this talking is taking me to a conclusion different than the one I had half a year ago. I want to use my time slots on seeing people face to face, but videotaping the great contents to give others the chance of tapping into what we talk about. For the non-Finnish presenters, I want to help those who already have the webinars set up find the people and topics they should show.

Thus, I won't be organizing a series of webinars, but a series of events that happen in face-to-face settings, with a streaming possibility. And if anyone has contents they want to make available, I'm volunteering as happy to help to get that stuff out - in front of an audience or just a camera.

Refining agile to see what and why for Agile Finland

I'm organizing an aquarium discussion for August on finding a purpose for Agile Finland. I took the idea from someone at latest Agile Dinner / Drinks, and it should be fun hearing what people have to say on what we should be and do.

With the small group I talked with on the Agile Drinks session, we seemed to be quite well in synch with what we'd like. We'd like to shift focus from the adjective 'agile' into words that seem more meaningful like productive, results-oriented and happy. The things we value when we seek agile as opposed to not-agile.

Back with the purpose: nothing special, just save Finland, tweeted one of the discussion participants. The idea that valuable software is a key asset for the future here, we need to seek ways to collaborate and to do it better - for all roles in development and in touch with development.

I know I no longer want to waste my life into planning for an acceptance test for a year and a half, only to learn that either the plan fails (even though it was updated throughout the wait time) or that when the plan is successfully executed, the testing done is of no value really. I want to plan continuously, and do continuously. And learn from both our successes and failures.

Milestone for a test resistant team

Whenever someone ask me if we're "agile", I notice I start to feel the need to apologize. Sure, we're on that journey with some criteria, but not there on so many others.

For example, my team is somehow trying to have everyone do "testing", but their idea of testing easily could be "development buffer", "try it once" and "go for coffee, you can relax now". When they manage to find the motivation to pair up with another developer, it's a little better, but the enthusiasm never last long for repeating anything.

While agile teams could be test infected, this one is long on the route to be test resistant and the probabilities of a sudden infection are not getting better yet.

The team has rounds of "let's automate tests in some way" behind them. The most recent one on unit tests went the furthest, with some of the team managing to refactor the code to add tests (one out of nine) and others came up with all sorts of reasons why legacy code made things too difficult for them. The round before that was to take a summer intern and ask him to record tests based on test cases the team had done. Those tests were never run by anyone other than the intern.

For the unit testing effort, I negotiated 1/4 of the teams time over a 6 month period, so unlike usually, the external schedule pressure was really to the problem. We created structures where we could do relevant refactoring, but too  little of that happened. And we had an external coach to help us with the tests just a little, with the end result of us realizing the vastness of refactoring / architectural change that would be needed to get where we want to be.

With  the unit test automation gone and the problems of not enough testing happening, we agreed to do some group exploratory testing sessions. With a few of them under our belt, they seem to do the job for now, but with the late feedback and the repeating nature of checking similar situations are wearing out the developers. This will not last, not for this type of information need.

In one of our group exploratory testing sessions, I drifted off to look at my team mates and think about the next experiment in the efforts to make it better for us. We had failed with UI automation and none was interested in touching it. No one wanted to spend their summer on automating tests that would not be helpful. So I called a consulting colleague in testing, and we talked about the smallest possible thing and timeframe we could set up to do a selenium proof of concept. I could do that myself, if it did not include learning that I did not have time for right now. But I knew enough to explain to my boss why we'd invest on an external to do something like this right now.

The proof of concept was delivered this week, in a meeting with the whole team. The proof of concept showed developers a style of tests they would accept as part of their work. This time, the external did not set up extensive suite of fragile scripts to throw to the developers to maintain, but a few structured examples and a useful explanation of what doing these include for our software. The external did not introduce new tools or languages, but just a driver for the browser interface that could be called from the code as we know it now. Owning small scope from someone else created positive buzz, and if the time frame of setting up was longer, we'd have more implemented but less sense of ownership.

The tests were integrated right away into the local code repository the consultant had no access to. And they ended up being run with the build cycle to find a problem on their first day of existence, a side effect of common services change that did not work in this area quite as the idea was with a big visible error message - just the type of information we usually look for in the release testing. And the problem was quickly fixed to get the tests back running again.

For the first time, I think there is an actual chance this will fly. And next I'll need to find ways of showing what else there is in the lovely world of testing.

A tale of a tester that is tester no more

I listened in to a story today about a traditional tester in a scrum team, where the team helped the tester out of the "I'll wait for features to complete, let you show me a demo and then test what you showed me" and introduced this fellow to programming tasks and full team membership. End result: the fellow quit his job, and changed careers out of software development.

Thinking about this makes me feel sad and upset, because it shows so many things that are so wrong to me.

I have no idea who this fellow is, but the story is either a slightly changing urban legend starting from one person, or this type of thing is getting increasingly common. I don't know if the fellow hated his job in the first place and is truly happier with change of career - perhaps. So, I'll speculate based on what I am, what I do, and what I get to hear regularly about my work and profession: testing.

Someone external might describe my work as "waiting for the features to complete to ask a demo then". Someone external might be puzzled why the 1-hour test for a small change may take me days, as we don't share a model of what "testing" is. I find that it's up to me to explain more of my thinking and where the time goes, but I also respect that some people might not be inclined to do all that, in an "agile" environment that is supposingly based on trust.

Testing that I do is not a wave of a magic wand on top of all things done already to confirm all things we know are true. I find the testing that I do to be somewhat complex learning process, with focus on a lot of different models. And while I work with my team early on to try to make sure we know what we're aiming at developing and delivering, given a few days for the information to sink in and learning to happen, I find new ideas, risks, things to consider. And the implemented feature, in the product context, gives me even more ideas.

I seek to provide information about the product that may be defects in the sense that it doesn't do what we promised, but I also provide information about how the choices we've made make the product in the long run potentially less valuable to its stakeholders, and all too often information about the fact that the value just might not be there with the way we've decided to go about solving the problem. I provide the information by learning myself. If I'd have the answers to begin with, I'd pass them on. But saying what we should have known is so much easier in hindsight.

While I appear to be "waiting", I might actually be working on another feature, that we released earlier and I tested then, but since that time the info has again sunken in deeper, and I have a new idea that I need to see about. Or, I might be building a model, that will enable me to do a better job at testing the feature, with data in the context of the product / system it's part of. The model might be a coverage map, it might be something to help me make sense of my assumptions to allow me to change as I learn while I test, or it might be a model of risks, usage scenarios and value. Or I might just be simply going through a 50+ page list of "so you think you're done, did you think of these" -examples. Or knowing the customer promises of tight schedules with compromises on some aspects of quality, I might be just as pressed with schedule and work on some other feature in a shallow way while waiting for the other feature to complete. And when my team mates show me a demo of what they completed, they need to really think down of me if they consider that I would test the things they showed, instead of just using that as one more practical way of learning more into what I should focus on to do valuable things.


While I nowadays again automate some tests, I hate the idea of dumbing down the work I do - most of it is still thinking and trying things out by hand. Squeeze the timeframe enough, and the testing I do starts to look a whole lot like what I was demoed as the thinking gets lost. And filling my days with programming tasks to be equal with the other team members is squeezing the timeframe. But that's not the intent I had while empowered to consider testing and I'm responsible defending that, but in an environment that trust me to learn from my mistakes on allocating too much time on learning something that did not turn out to be relevant.

To add a piece of my current context: if our team doesn't test, the customers don't seem to mind / complain. They mention occasional problems in passing, and we end up using our time on recompleting the same things again, after considerable wait time. It makes us unhappy. 

If I would happen to work in an environment, who show little respect for my skills and my profession, I too would rather go work in a fast food chain. Staying sane is important. Care more and try to believe the people whose work you have not done - testers or product managers or programmer-developers or user experience specialists - actually might have a point invisible to your eyes until you dig deep enough.