Saturday, October 8, 2016

Two testers, testing of same feature

Two testers, working in same team with very different assignments as their starting point. Both same age, in years of life and career, both women, but very different backgrounds. Both starting in the same organization almost at the same time.

The first tester is hired as as automation engineer. She has spent her career with code and used to identify as programmer. Her previous company moved her from programmer to tester, still programming the tests on a highly technical interface-based system. She approaches testing an automation idea at a time, ticking off test by test to add to her test suite.

The second tester is hired as an exploratory tester. She reads code regularly, but will instinctively approach any testing as a wide scale learning effort, where detail of automation is best used when there first is a good understanding of what we have and how it works.

In a week, the first tester automates one test. The test fails on upcoming weeks as other teams intentionally break functionalities in order to introduce a new feature. The second tester creates one mind map and has lots of conversations that change her understanding of what the state of the feature is supposed to be now, and which of the observations are meaningful.

While the first tester takes tools as they're given, the second tester questions everything. The second tester insists on manually testing in pieces to tweak inputs more easily, and finds ways of breaking down the long chain to run thousands of little tests finding out what kinds of things are relevant. The first tester automates the long chain, until discussing with the second adding the idea of automating in shorter bits, just to realize that it's not as straightforward with automation as you might hope for.

I suspect that over time, you could get to good testing from both of these angles, but it is also evident that there is hardly enough time for the first tester's approach to grow to coverage. The exploratory approach brings long term benefits not through the available test automation, but the improved code and programmer knowledge.

We're lucky to have both. I've been lucky to have the first as something developers do, while the latter has been my assignment.

I'm looking forward to playing more with time boxing my time to be more like tester one and seeing if that makes me more super than I already am with focus on exploration. It is clear that those are, for me, conflicting learning goals and my head just isn't ready on intertwining the two yet. Perhaps in a year.

And the most interesting question of them all: when tester one and two collaborate, from their different starting points, will they become the same? Both are *testers* after all. 

4 comments:

  1. I'd suggest that Tester One has a poor grasp of automated UI test design, and also has a poor grasp of how to do this work with an agile-ish software development team. I say this as someone who founded the cross-browser UI automated test projects for both Wikipedia and Salesforce.org.

    I've said before that automated UI test design is a discipline unique enough that doing it can be honorable full time work. There are also many excellent resources from which to learn about good automated UI test design. (I've written some of them.)

    Tester One should consider a more thoughtful approach to the work. Or their team should hire someone with more experience than Tester One seems to have.

    ReplyDelete
    Replies
    1. Oh, I agree. There's two things on the way of this, though. The automated UI testing on windows applications isn't as straightforward, and there's a lot of legacy tester one or two wasn't around to build. When you join an organization where a lot has been done, changing it takes a bit of effort. And I'm sure the two testers will make a change together - with the developers changing the application to make life a whole lot easier.

      Delete
  2. The way I see it:
    - "tester one", hired under the label "automation developer", checks
    - "tester two", hired under the label "exploratory tester", tests

    ReplyDelete
    Replies
    1. The way I see it, both test. One focuses on testing as creating artifacts (which is hard in this particular context) and the other focuses on testing as performance (which can go around any obstacles computer programs get stuck with). I dislike saying checking. There's different kinds of testing.

      Delete