Tuesday, February 2, 2016

Thanks, but I still have test automation

There's another big buzz in the world of testing, as the allowed and acceptable words are being refined. This time, the word is automation.

I know testing cannot be automated for now. In its full features, it's a process of collecting empirical feedback on things from we know to expect to things we had no idea could exist. It's full of all kinds of wonderful activities, where a thinking human is in critical position. It's hard to talk about testing because your testing and my testing won't be exactly the same anyway. Why would it be such a bad thing when people talk about test automation meaning the parts they believe they can automate.

I also like to think of it this way. If programmers job in general is to automate things, to get started on that does not mean everything has to be automated. A robot that does heavy lifting for people is valuable, even if it only operates in the warehouse and does not deliver things at your front door.

Merriam-Webster defines automation as "the process of putting an apparation, operation or system under control of mechanical or electronic device". When we put parts of testing under control of electronic device (the tool that we need to attend to differently than if we did not have it), I'm doing test automation.

Having more words to explain the details would be helpful. What type of test automation are we talking about? Is it execution? Monitoring and reporting? Setting up data and environments? Which specific problem in the domain of testing we're trying to control removing humans at least partially? 

I've seen my share of wasteful test automation. I've seen it replace thinking where thinking is due. But I've seen that fixed by introducing experimentation culture and empirically assessing what worked for us. It can include failing with automation on the value it provides. We already have words to discuss that problem: value, opportunity cost, short-term and long-term. Many of us are having that conversation continuously.

A main principle I always remember of context-driven testing is that people are the most important aspect of any context. How about believing that people will look for solutions, and figure out what does good enough job for them without redefining anyone else's namespace. If using the word test automation makes me less of a context-driven tester, so be it. I just want to ensure the organizations I'm working with have the best chances of doing a good job. And good job looks different depending on organization and the day we're looking at it. Everyone is learning every day, including those of us who think they've already learned a thing or two.

** as for calling automators technical testers, no. There's technical testers who don't program but are intertwined into the system / environment technical aspects. Like saying people in IT departments wouldn't be technical.