When in 2023 four people including myself got together in a panel to decide if test automation is friend or foe, I chose it was a foe. Panels where we all agree aren't fun, and in search of really understanding something more, we need to find more than the places we agree on. I remembered that one as particular panel, so I went back to rewatch it to the points we were making. The panelists were Ben Dowen, Beth Marshall, Brijesh Deb and myself. I remembered that one as particular panel, so I went back to rewatch it to the points we were making.
What is test automation?
- Code doing something useful for testing
- Support for exploring and coded checks in testing
- Not silver bullet
- Way to add efficiency through code
- Human led testing could include writing automation
- Definition of manual or automation is a myth, it's both
- Roadmap to turn manual to automation, not a journey of learning how to do real work for you
- Overanalysing is not route to success - try test suite canvas by Ashley Hunsberger
What if I figure that test case becomes more complicated to automate than I thought originally?
- Start with feasibility study
- Try doing it and figure out that your skills and knowledge aren't enough? Ask around when you hit a roadblock? Step through and start learning.
- Too much studying not enough of real work that stays around, fail to learn a little
- Collaboration up front, talk with developers - failing early is always an option
- Learning from people around you on the ideas of what to try first is not feasibility study, it's collaboration and whole team work on important problems
What is the biggest challenge to test automation and how to overcome it?
- They are about people, culture and communication
- Misunderstanding about automation not needing maintenance
- Manager or colleagues who say let's automate everything
- Getting system into the right state to start the actions has a lot to setup, testability interfaces, knowledge from various teams
- Success turning to failure overnight, dualism for the organizations, surviving generations of testers and the move to whole team automation
Does every tester need to be coder in a future?
- More than now, industry doubles and we need different ways of starting
- Some testers don't know they are testers, and some of them are paid less than us testers
- Future makes every coder a tester, and already more of them are than we give them credit for - code aware and not 100% if anyone's job
- Overfocus is exclusionary and stops otherwise fantastic people from starting at this industry - no need to code in binary, not necessary to always code in java
- Coding is an entry level job - I want both, and teaching basic programming is simple
- Advancing in companies this works well for all kinds of testers, but it's broken in recruitment
- We ask for the wrong things in the recruitment, recruiters need to understand testing not just automation
- Coding in test jobs is wrong kind of tasks, like a bubble sort Is test automation is ultimately a way of cutting costs?
- Augments what you're doing - cutting costs in one place (maintenance changes) to have more of it elsewhere (new value)
- 50% of jobs changed, only 5% replaced by automation
- In isolation it does not save time, cost or improve quality, must be supported by other activities
- Operating test automation in a silo away from the product development does not produce greatest results
How can you balance need for speed and efficiency and maintaining high level of test coverage?
- Everything is code nowadays, now you see changes somewhere with IaC and I don't need to run a test if there is no change, slow down the time, control the change
- Taking control of the world around me as part of automation efforts
- If you think test automation is the goal not a tool that adds more effective testing
- Testing isn't the goal, it's a goal we use. Even software development is not the goal. It's the value we are creating for customers.
- 1250 tests, 20 times a day, learning after having already invested in small scale feedback programmatic tests, keeping us on the right track.
- Misselling stream of information as silver bullets of these techniques exists
If we focus on test automation, are we risking that we don't focus on other things?
- Yes, and. Not either or.
- Testing plays an important part in improving quality, but it's not the whole picture.
- Red-Yellow-Green call. Red lead to punishing testers.
- Watermelon tests, testing your tests
- Had multiple levels of teams of testers. Effort but needing explainer.
Is test automation friend or foe?
- Friend. High maintenance friend. A needy one.
- Friend. Work on a mutually beneficial with boundaries or you will fall out.
- Foe. Respectful even if you don't like it. Collaborate with the enemy when stakes are high.
- Friend. Friend who likes to be pampered. Can turn into a foe.
