Wednesday, May 21, 2014

A tester crisis with shift-left work

There's been a long-time discussion about involving testers earlier, but the term "shift-left" is a new concept for me, saying basically the same thing: the need to involve testers early not to find as many problems later in the lifecycle.

I've done a share of shift-left work in the role of a tester, commenting and pointing out flaws. But as I continued working on design stuff, I started noticing that I moved from my tester role ("here's a problem, could you think of a way of fixing it") into a designer role, suggesting solutions. I find the difference essential, as in the tester role I tend to remain somehow disconnected with the eventual solution I did my best for, whereas in the designer role the mistakes we make are mine as well.

Just yesterday the first feature designed by me got into testing - to be done by me. I felt different. I felt quilty for liking the appearance of simplicity of the feature. I felt it must be incorrect. I felt I must have missed something really relevant. I felt insecure.

We did a pair testing effort on the feature, with quite limited time in relation to the complexities related to that. About half-way through our paired session, I felt the urge to mention that the design / fit for use aspects needs to be attacked very critically, as I fear for my bias. I noticed I selected my own test approach for the session based on the concern of bias - comparing with a version of similar feature in a previous version of the product, with different technologies. I paid particular attention to things the old one allows the user to do, and making sure those could be done with the new. And I felt even more strongly that if the previous wizard had 9 steps and the new only 5 steps, there just has to be something wrong. And I kept looking for evidence. I've logged a bunch of bugs (13 from our session yesterday) but none of those bugs yet has what I look for - signs of this design not working for its purpose.

With something I've created, I feel the need of doing my worst to break it is more urgent. I will try. I will invite others to try. And monitor the feedback just as I do with any other features.

I realized I've seen similar shifts of perspectives turn bad earlier on in my career. I've seen decent testers turn into confirmatory mindset by participating in design, missing relevant issues while testing. So I feel that it's my duty to do the extra work needed to try to tackle the concern. If I fail in my attempts to show it does not work, it just might work.

Then again, its no extra duty. It's what I always do testing features. The extra is just the nagging feeling that my bias needs attention. Will see if it did in just few weeks.



  1. That is an interesting problem to consider. As we move testing earlier in the process, with potentially greater, earlier input on the final design, do we possibly damage or inhibit the value of our testing later in the process?

    I certainly think that is possible. I wonder what habits or actions we can take that will help us to protect against or to break through some of the bias that we might build up from building involved earlier in the process.

  2. This is not unlike the problem of whitebox vs blackbox vs greybox testing. When you understand the code you have developed, you 'know' for example that it should handle 1 item just as well as 100 items because you used a linked list. Testing that a linked list from a library works isn't useful, YET there might be an issue when you have 100 items that code you didn't touch wouldn't handle those 100 items (say you worked on a cart page but didn't know how checkout worked). Knowing the insides has value, but it can bias you. I don't know if there is a solution. :/

    - JCD

  3. I think you are missing the point of "shift left testing". When I push to include testing earlier in the SDLC it isn't so I can do other peoples work. The reason is to TEST the requirements to ensure they are complete, reflect what the business wants, and (most importantly) are they even testable. Same with any design documents. At no point do I ever become a BA, or an Architect. I am still, and always will be, a tester. By applying testing rigor to requirements and design you shift the defects left; thus reducing the cost of those defects and (in some cases) significantly reducing the number of downstream defects.