Not testing? Not a tester?

I met up with a few colleagues last night, and after we had the official business mostly taken care of, we chatted briefly about what's going on at work. We don't work in the same companies, so the stuff we do tend to vary a lot.

I told I'm working, in addition to learning the product and finding problems on the side (=testing), with specifications, namely specification by example -style specifications. It just happens that for a product with a lifecycle, it's not a particularly good idea to NOT have a specification, and I refuse to write test cases that are separate since I've kind of bought in to the idea of living specifications. As this is work done for features that already exist, I do the spec based on what I learn when testing and do workshops with the product managers to go through if they're balanced examples instead of tests that cover all.

One of the colleagues pointed out, that what I'm doing isn't testing - I go into someone elses territory. In the next sentence he pointed out that actually what I'm doing is like creating test cases for someone else to execute, since most of the under-the-hood automation I create with the coding work done by my team's developers.

About a year ago another one of my colleages who I also met yesterday told me I'm no longer a 'normal tester'. Apparently that means, that I've earned my place and get invited to board meetings to help understand the information testing provides. And they actually listen.

I still think I'm a tester - at heart. I've put loads of hours in understanding what I could do, what I could have others do, and how to get just a little better at explaining what I'm doing and why. It's funny how upset I get when I hear that what I do is not 'normal' or that I'm not a tester. It just shows how important the craft is to me personally.

Not that difficult - or is it?

I tested a little new feature today. It did not take many hours to implement, but ended up taking more time test. The feature itself seemed to work nicely - in isolation, where it will never be. Investigating the feature in use within the product revealed that while it worked, other more relevant didn't after you had used this one. All in all, a galore of bugs in just about every aspect of the product context I could vary. Not only for the developer of this feature, but for the others - as this little addition brought out weakenesses in existing implementation that seemed fine without this.

There wasn't anything difficult I did today. I'm still half-headed from a long vacation and wanted to start off easy. Which leaves me wondering about what we're missing to miss out all the relevant bugs that just take a bit of patience to look for.

The testing I did today did not require much testing skill, but it required patience. It required me to try things that are somewhat similar but still different in some dimensions. I need to work on training patience to my fellow developers. Find motivating for that somewhat hard, as it seems our business has survived long enough without it - you could always, so far, fix the problem when someone complains about it within a day. The hard part is to find out the problems are there and leaving the hard parts for others - product managers, customers or maybe a tester - feels so tempting.

An evening eye-opener

For Saturday fun, I'm reading Gojko Adzic's book Spefication by Example. Not for the first time, but I thought this time I'd try read it in order intended, instead of the usual jumping to the most relevant first.

On the very beginning of the book, describing key benefits of Specification by Example, there's a quote. Paraphrasing: In other organizations in the end testers find something wrong with the product and developers need to continue, and in ours they don't. The issues that the team have are about making a test (automated) go green, not later feedback surprises.

I got stuck on this particular out-of-context quote, namely because it, unintentionally, resonates with something I'm experiencing. Not specification by example and hitting the right target, but being in an organization where the issues used to be "tests not becoming green" and now there's definitely more churn - going back and forth.

To illustrate my story, I'm sharing a picture from our Jira that I've been looking at recently, trying to figure out what to do next.

Something happened a while back. Something that makes our software development look a lot less effective. They got their first tester that focuses on testing. They had at some point before a tester who focused on test automation, and with that, the wrong kind of test automation, so much of what I'm showing come as surprises.

It's not that you couldn't use the product. Apparently you can, since there's customers using it. And apparently they can't be very unhappy or they would complain in ways our people would understand. I'm not quite sure what's happening there.

But I know what is happening testing-wise. I started testing and logging my issues a bit before out latest release. And the stuff I'm finding for addressing is more than the fixing / development work of our whole team.  The problems are not about missing requirements per se, but about surprising side effects of two supposedly separate things. They are problems of shortsightedness about things that can go wrong and believing you'd be luckier than the others. And not knowing hides it all.

I had organization, who could have no exploratory testing and no churn. No real information either. Having e.g. me work on anything like specification by example & test automation now would seem like the wrong choice, as they need quick eyes open effectively reported -testing first. But I still kind of hope to get to that too, to turn the trend around for better.

Why would we improve the way we work?

I had a short, but inspiring discussion at work today with some of my team's developers. Since it's apparent someone for some reason wants things to improve, they were asking for set goals and better understanding of where (and why) we're heading. Apparent for the reason that they just recruited me, but still not so clear since while they may now invest a little on "tester", they still haven't invested visibly into fixing.

I gave a short answer as I see it: there's a tradeoff in the way they develop. When we deliver "more short-term business value", we take "quality/technical debt". We're approaching a point where the debt and the interest of the debt are threatening our ability to deliver features, and we are unwilling to scale the team size but look for other solutions. Our challenge is not about us wanting to do things in ways that make sense, but about the effort and timeframe we need to invest to get where we want to be.

Another one of the developers who wasn't part of the discussion as it started, suggested from his cubicle that this could actually be what we'll clarify and discuss with the product management team. Didn't test that yet - will soon. This is a very simple and basic approach, but today reminded me how many times it has been effective. And most likely, I will learn more layers on what we actually find relevant for our product to become a better tester for it when talking about this.
 

If it looks too easy to be missed by developers...

On my first weeks at new job, I've had the pleasure of reporting bugs again. I find this particular result of testing to give me the feeling of achievement. The more relevant the problem, better.

There was one bug I reported on Monday, that just looked too easy to be missed by the developers in my team. As I originally reported it, the problem was that when logging in with one of our three main browsers, there's a highly visible error message. And that this seems to happen only with the recent builds, not in production version.

In the end of the week, I quickly asked in passing the developer whose component was failing, if a fix is available in the next weekly build. He seemed puzzled: what problem, what fix? I checked our Jira, and the issue had not been addressed - which is quite normal. He took a quick look at it, and came back with "I didn't change this for ages" with some details.

I started testing the issue more with the information from him. With fresh eyes, I realized I entered the program from a bookmarked link - something I hadn't mentioned in my original report. I also realized, that I had different addresses bookmarked in the other browsers. So I had missed a relevant bit of info I provided now.

Bottom line: if it looks too easy to be missed by developers, it may be that they didn't test, but in this case, I missed relevant factors that are needed to get the bug visible.  Talking sooner than later to the very busy developers is still a good idea.

New product, new team, new practices

For a bit over a week now, I've been wondering where I ended up in my quest for hands-on testing work. With hard choices on the way, I'm now working for Granlund, a civil engineering company, with a product that handles building-related data. The domain is something I have little idea as of before, and I'm looking forward to learning a lot on that in addition to tuning and changing whatever I can with my testing skills. We have a small team of less than 10 people, and I'm the first and only tester. Most of my colleagues in development seem to work remotely, but within a week, I've had a change of learning they're just as much fun to work with as I expected.

I start off with a redesigned version of a product that has been around for quite a while. The redesigned version is also out in production, new versions of it going out once a month. With customers actually paying for the product, they must be doing something right, even if they never had testers around.

After reading up on what the product is about with a shallow scan of its documentation, I've worked on:

• Setting up test management based on sessions with Rapid Reporter and csv-note scanning tool to show metrics I will create - as I won't create test case counts
• Learning the product's capabilities (and quality) with doing exploratory testing on its main feature areas
• Existing test suites and redesign of test documentation
• Redesigning a consultant-suggested testing methodology that I just can't believe would provide added value (unless faking testing is considered that to someone I did not yet meet there)

There's two strong first impressions:

1. I've got a likely case of "asking for testing, when you should ask for fixing" ahead of me
   I find it somewhat funny that so many people equate testing (quality-related information service) with fixing (making the quality right) and don't think of the dynamics that the added information will bring in. Then again, understanding the business impact and meaning of the existing technically oriented issues is a service I think I can help with. 
2. As there's not enough rational testing examples around, it's easy to take the basic book on what's a test case and try replicating it without thinking enough
   I've enjoyed reading the attempts to design tests in per-feature-area test suites of varying sizes, but all with step-by-step test cases repeating most of the steps again and again. I took on of these documents, 39 pages with 46 documented test cases and read it through in detail to make a mindmap of mentioned features (I do need a feature list to support my testing). While reading and using the product for learning it in practice (a couple of 1,5 hour sessions), I came up with one-page mindmap mentioning 88 things to test and four dimensions that cause repetition on significant amount of testing that should happen, like different browsers, user rights, and such. Out of the 39 pages, came out 3 things I could not directly deduct from the user interface with little information on the actual product. While doing this stuff, I marked down some (quite many) issues I would write bug reports on - if it wasn't the area we're about to rework on in a significant manner right about now. 

Looking forward to all this - and the chances it provides for writing stuff and providing examples of something that is doable to "just a tester".

Looking for a new job

I feel excited and puzzled - I'm actively looking for a new job. I've spent last 2,5 years at Ilmarinen, and enjoyed the lessons learned there greatly. We've succeeded, with my projects and teams, to get projects, where the contractor does their share on quality-related work and acceptance testing phase is actually just that: testing, not fixing. We have great business specialists in acceptance testing, they've learned something relevant about testing with me and work with contractors is less cumbersome than before.

So why I'm changing: because I watch out contractors to get to do the stuff I'd want to do with the limited time I have for work. They actually create the quality, their testers enhance the productivity of the teams they work in. They do testing for finding problems so that customers like us don't have to. And I want to be doing that: work close to development, actively making a difference in quality and productivity for the company I work for, and the end users of the products our teams are creating.

So I'm looking for a senior testing specialist position, that would enable me to do hands-on testing work. I'd love to take a spin at test automation, from the extending exploratory (not manual, but brain-engaged!) testing point of view, doing more and using ideas that are not possible without tools. With test automation, however, I know I'd be stronger with a team where I could provide practical ideas of what we'd test, instead of trying to do that by myself alone. The reason is that I have developed a practical way of developing and grouping ideas, and focusing on automation alone may prove to be difficult as the list of stuff that would need to be covered may take more time than first assumed.

My ideal position would be:

• working with a system or product with relevance: either successfull business-wise or relevant purpose, like with Ilmarinen: there's no pension paid without the success of the chain of systems that make it happen
• working with a development organization: either product company or company providing software development projects on contract. Essentially, working with people who create the software. 
• enabling me to do hands-on exploratory testing, develop exploratory testing related practices on smaller and larger scale.

I'm considering both work in Finland and relocating somewhere in the world. But, since life is short and I really want to do more of hands-on testing work than I can in my current position, I would prefer to move quickly. I can work either as a self-employed contractor or as employee. If you happen to know anyone I should talk to, give me a hint.

A tester at heart, always learning - that's what testing is about. So I think I've gotten pretty good at learning to be useful quickly.