Tuesday, October 11, 2016

The three ways to solve 'Our Test Automation Sucks' in Scrum

Scrum - the idea of working in shorter increments. The time frame could be a month and when you struggle with a month, you'll try two weeks. Or even one week. But still there's the idea of plan, do, and retrospect.

When we work in short increments, a common understanding is that moving fast can make us break things. And when things could break, we should test. And with the short cycles, we're relying on automation - like it was our lifeline. But what if our test automation sucks, is there no hope?

Option 1. Make it not suck.

I would love this option. Fix the automation. Make it worthwhile. Make it work.

Or like someone advised when I hinted on  troubles with automation: hire someone better. Hire a superstar.

No matter what you need to do to make it not suck, do it. And with a lot of things to test, there's a lot of work on fixing if a lot of it sucks. And what sucks might just be the testability of the application. So don't expect an overnight change.

Also, don't give up. This is the direction you will go to. But it might not be quick enough to save you.

Option 2. Freeze like crazy.

This option seems to be one that people resolve to, and it is really an antipattern. It feels like the worst of two worlds. You slow down your development to make time for your testing. You test, you fix, you despair. And you repeat this again and again, since while the main is "frozen", some work gets bottled up somewhere just to cause a big mess when unfreezing takes place.

Freezing brings in the idea that chance is bad now that we need to fix it. Hey, maybe chance in a way that breaks things is the bad thing, and making developers wait for improving things isn't exactly helping.

Let go. We're not the gatekeepers, remember. Freezing is, a lot of times, gatekeeping. Would there be a safe to fail way to get to the lesson of letting go?

Option 3. Do continuous releases with exploratory testing

I've worked with options 1 and 2 long enough to know that while we work for option 1 to be reality, there's a feasible option too. What if we would only put things in main that can be released now?

What if, instead of thinking of the programming as the only manual tasks, we'd realize the testing is too. Couldn't we find a way not only to program but also to test before we merge our changes into the mainline.

I've lived with option 3 for a few years (with gradually less sucking automation), and I'm having hard time seeing why would anyone choose to work any other way. This basically says: stop doing scrum. Do a feature at a time, and make your features small. Deliver them all the way through the pipeline.

Continuous Delivery without Automation is awesome. With automation, it gets even better. But the exploratory part (the 'manual' thinking work, just like programming the chances) isn't going away any time soon.