Tuesday, May 24, 2016

Ways to get to continuous delivery

I listened in on a talk today about Test Automation and Continuous Delivery. Paraphrasing what I think the speaker said is that:

  • None she knows of does daily deliveries without test automation (and stays sane - we heard of an other example like that though where "expecting normal workdays would just not be appropriate") 
  • We need to stop coming up with excuses on avoiding the test automation
In the end of the talk, I offered a counter-experience to understand her ideas a little better. I work with a  team that has, for two years now, done daily deliveries and with ridiculously little test automation. I feel we are still very much sane, we deliver quite safely. We are working - slowly - on adding some test automation, but for a long time, there was none. It took us long to build up skills for the right kind of automation we would find useful.

What I believe I learned is that we had an interesting discussion about our beliefs in what to invest in.

In her case, the problem she was solving was that testers were very frustrated with finding simple bugs in manual testing. That is a problem we sort of shared.

In her case, her team sought for solutions in the testing space and went for test automation. She told the audience that their automated tests are valuable, and find bugs pretty much on a daily basis. 

In my case, my team sought for solutions in development and testing space, jointly. We had a limited amount of things we could do at a time, and conversations with my developers lead us to decide on investing primarily in cleaning up the code to make it more readable. 

I asked the speaker today how their code was, and she confirmed theirs was messy.

So my train of thought is: you can choose to invest in making things better for your testers. If you feel you can invest over silo limits (not within the the testing silo), these two might be your options. You could get to same or in my experience, a better place investing into clean code. But most people seem to go for test automation as the primary investment, even with unclean code. 

When your code is clean, your devs can have discussions around it easier. Devs share it easier, ask for feedback more eagerly. Devs make less mistakes as you go about changing it. With less mistakes, there's less of the simple bugs to be found in manual testing.

And when you top that with smart, adaptive exploratory testing like my team does, it is possible to do continuous delivery without automation. Automation would make us faster, but not necessarily safer. The cleanliness of code is what seems to have a connection in how we experience safety in customer feedback. 

I don't think this is an excuse to avoid test automation for us. The excuse is that there has been a lot of skills to build up, and we chose to build those skills clean code first, unit testing with asserts and approvals and selenium tests later. And we're just still taking baby steps in the automation front, but doing great on the clean code front, reaping the results in our ability to deliver without breaking production. 



Wednesday, May 18, 2016

Technical Assets and the difference for testing

This Monday, we released something we've been working for way too long. To get to this point, we've deleted massive amounts of code and refactored (with the definition of it can be refactoring without unit tests with R# in heavy use).

For a period of time, there was a separate code-base for every configuration of the environment. I think we were up to five by time of cleanup.It was a tester's nightmare. If you'd find a problem (and there were plenty, more than anywhere else in our product), the problem was twofold. First of all, the problem was likely to be elsewhere, and it was likely the developer would forget one or more of those places when fixing - not to speak about how slow fixing was. Second, whenever something got fixed, something else somewhere broke. It felt like a house of cards.

Seeing the difference in how this felt was easy to see in my team, since every other part of our application has a different feel to it. When there's a fix, I can count with one hand's fingers the amount of side effects. Fixing feels quick. Also, the fix gets very often applied to different places at once. Surely, sometimes there are side effects but rarely.

The feel of the first and the latter is like night and day. With code like the first, one of me isn't enough for one developer. For code like the second, one of me is plenty for 5 developers.

I've looked at the ongoing discussion long enough to see where the difference I'm experiencing first hand comes from. It comes from skilled developers who build smart technical assets. They actively build and share components to have things done in one place. But the biggest difference comes from communication: they find the components, because they talk.

Mob Programming has increased the amount of practice level talking, and I expect that the continued mobbing will lead to more technical assets - one line of code to do a thing that takes hundreds when reinvented. And the impact on how we test is profound. It's the way it should have always been, but wasn't, not for all parts.

We all do better when we all do better

To take a step back to defocus from a complex and intensive testing task I need to criticize myself on, I'll blog about something old. Something that has stuck with me all these years, and that has amplified over the decade I've known it.

The Finnish Testing Community Scene, when we started bringing it together 10+ years ago, coined in many ways to one great man taking initiative: Erkki Pöyhönen. Eki is still around for testing, very much so, but nowadays so much hands-in-dirt in the projects that I get to see a little less of him. Eki is an extraordinary person, a connector of people, and I've learned a lot from him and with him.

The grand wisdom that made me exclaim how much I appreciate him, is that a long time ago, he told me (one of his) philosophy: Promoting others promotes you.

At the time, that was an important lesson for me to learn. As an individual contributor, I was working hard on the idea that I need to make something out of myself, working on my brain, my knowledge, my skills and abilities. And everything I did, Eki would notice, mention and speak about, showing appreciation, promoting. I loved him for what he did, and so did everyone else. He finds the good stuff, everywhere.

We all do better when we all do better. Today is a good day to remember to appreciate all the great stuff that happens without pointing to all the not so great stuff. Eki's advice did not sink in for me in one go, but was a long process of personal experimentation with the idea, seeing the comparative results.

Thank you Eki for making me a little bit better person (for myself). And happy names day.


Tuesday, May 17, 2016

A day in the life of an agile tester

Yesterday, at the end of the working day, our product owner approached our team area, at a point where I was the only one at the office. He goes to explain a piece of feedback from a user, and we pair to understand the feedback as an example with the product.

We think back to what we had agreed on the feature, and together we learn that while it works as specified it does not work as we intend the customer experience to be. The missing feature is kind of relevant, even blocking. And for us, with that joint learning, the feature request becomes a bug.


We don't really care about the label, we just care about the customer. And we need a change.

Appropriately, the morning after there's a regular meeting for us, the team and the product manager, to improve the world as we know it. The previous evening, I had already alerted my team on this, and we had just the right people around the table in solving it. The developer brings in the deep understanding of current implementation and we learn this will not be a minor change.  However, the developer states "I can have it done by tomorrow".

I smile with the idea that he forgot to ask if I'm available for testing it with this schedule, especially knowing it will be an evening thing even if we can do bits and pieces throughout the day. The final experience, the product that speaks to me, will not be fully available before the changes have been done. But I'm not the gatekeeper here. I know that there are days after tomorrow, but I don't mind him taking a stretch goal. I know our product owner well enough to know he has flexibility when we talk about days, not this month's release vs. next month's release.

After the meeting, the dev messages me, unprompted. He tells me he realized that he might have been optimistic on schedule because we had pre-agreed collaboration with another department  and a weekly meeting to interrupt the day. And that he's sorry for not thinking about my schedule. We agree not to worry about estimates (we are very much in the #NoEstimates idea) but focus on what helps us forward, without haste. Thinking over deadline.

Throughout the day, the developer mentions things he's changing, kind of as a checklist for me to see what to cover. But for me, that is a checklist of things he already covers. I give him as much ideas as I have, help him work through the minimum scope and risks.

Late in the afternoon, I know that it will be a long night before it's done. Working on it has revealed a bunch of dependencies and connections. Instead of telling  the product manager that the developer was overly optimistic, I mention that I will not be available for quite as late as this would require. And it's better for the two of us to take a look at it without rush. No disagreement, work continues.

4:23 pm the developer mentions he just checked in the code in a branch he worked from and asks for help. I test, and I find 7 issues, out of which 3 have been introduced with the change. He fixes all but two. One is related to a complicated set of rules and we agree to take more time on pairing on that. The other is an inconsistency that has been there before, that I just hadn't paid attention to before. I know that addressing the one left behind is only a matter of days anyway, and we agree it can wait now to keep our eyes on the customer need.

There's a lot we could improve here. But there's a lot of good here too. And it's just a day in the life of this project. And my hat tip to having great developer colleagues.

Experimenting and the agile world


If James Bach's 1st slide made me explain, that reinventing the tester role needs, in my experience, be much more than defining the role, this post is different. This post is about the ideas attributed to the AGILE WORLD. Matt Heusser was quick with his response.


I find that tweeting this slide is not an endorsement. Reacting to it is not giving visibility to bad things. These bad things exist, and we need to speak about them, to work our way through them just like we've worked (and are working) our way through the ideas that testers and agile don't belong together.

This slide is awful. Let me start with the most awful thing on it. The attack on Lisa Crispin.

It's time the 15 years of bullying Lisa Crispin to stop. I would have broken down into pieces if I was attacked as much as James Bach is attacking Lisa. I wanted so much for James Bach to like me that I looked at this going on for years. I'm ashamed of my behavior.

Even if it was true that Lisa's  (and Janet's)  book doesn't talk about testing with some weird definition of what qualifies, none of that justifies the attack that goes way beyond content debate and resembles holding a grudge on something that I can mostly interpret as being angry over voicing out the fact that the world is changing. That interpretation is mine, and probably very incorrect. I don't really care why this is going on, I care that it should stop already.

I know many testing specialists who cite Lisa's book. That claim on the slide "not cited or thought of as a testing book" is ridiculous.

This leads me to the second item I have to write on. Testing is testing is testing. We speak past each other. This has become more clear to me, as I started having long, deep discussions with a friend who identifies as a developer and very much into testing. Whenever he says "testing" and talks about what testing gives you (spec, feedback, regression, granularity), I would make a wry face. When he said testing, he talked about a very specific type of testing, which might as well be the only type of testing he knew (I doubt it, although I recognize that his exploratory testing awareness and ability has greatly increased since our collaboration and he represents a professional tester quite well nowadays).

It helped me realize that testing as he understood it is a majority view. To change the majority view, it isn't helpful for me to say that you use the words wrong and what you actually do is what we call checking. Instead, I would talk about the part that he was missing, calling it exploratory testing. Even if all testing is exploratory. Even if their testing included some exploratory aspects, it was more constrained than what my style would add as long as it was centered around creating an artifact, automation.

His kind, the test-infected programmers, have done a lot of good for the world of testing. It saddens me that there are still too few with those skills. The devs I work with are amazing in many other ways, but not test-infected. They believe in code being readable and understandable helping them avoid bugs, and that is working well for us while we find ways of building our unit testing skills.

Phrases like “Testing is boring but coding is fun" are statements around the other definition of testing. We agree, I think, that the mundane going through same moves is boring. And the test automation movement is about finding ways of turning that problem domain into coding and maintainable structures. The tooling has gone forward a lot with agile and developers coming into the problem because it was made "coding".

Devs that are test-infected don't deserve to be hearing that what they do does not qualify as testing if it does not qualify as all the testing. And it qualifies for a much bigger part than the tester's community, in its defensiveness, seems to be giving it credit for.

(yes, I know there are bad developers out there. I'm just lucky not to work with them. I'm not cleaning up after bad developers as major part of my testing.)

When we learn more about meanings of words in common use, we add labels to them, not redefine other's words. There used to be one kind of guitar. With the invention of an electric guitar, the old guitar became acoustic. That's what I feel should happen to testing. Unit testing is the electric guitar, and exploratory testing is the newer invention for the majority of the world. I don't need to reclaim my words, I need to learn to communicate.

The old grudge argument is also very much one that I observe the first claim with. "Agile was not created by testers or with testers. It's programmer's utopian vision." Agile, as I experience it, is reinvented daily. It's not created by programmers, but by people. And it's about people. I've been a major part in defining what agile (and its success) looks at my places of work. And I see people meeting other people in conferences and work places refine the ideas and ways we speak about agile. Agile is evolving, and looking at it as it was 15 years ago seems off. After all, it's all about learning - many of us have learned a lot in 15 years.  

Agile is about getting people together with a shared responsibility, not on testing but on the product we're creating - and testing as a part of it. Some people are more difficult that others. Some people are always right, they argue. And agile has found a way of dealing with this, experiments. Let's try everyone's way. Instead of debating for the merits without the hands-on experience, let's experience it together, try to make it work instead of proving it's awful and then see the results. Failing with an experiment and learning is good. But this gives us a mindset of openness to solutions that are outside what the experts might have thought of. And it makes every agile project different from one another.

I said different things about AGILE WORLD before I lived through years of day-to-day life in agile projects. Some of them struggle more than others.Ones I've been in have made magnificent end-user-experience improvements on product quality (as value of the product)  and quality of life. I love being a testing specialist in agile projects, as it enables me to be good at testing in a productive way (continuous impact on what comes out).

Your mileage may vary, but it does not take away the fact that I'm experiencing something good.

The question should be: if the bad agile is common, what is it that we who get it to work are doing differently? We don't know, could some of the protectionist energy be used to help understand that better? Because, it seems to be working for a lot of us. I think it might be just about kindness, consideration and respect. Bullying others for disagreements of opinion is not accepted.

Need to reinvent testers?

James and Jon Bach are delivering their course on Reinventing Testers as of now, and as I'm not there, I can only rely on the glimpses twitter has to offer with #MakeTestingGreatAgain. 

One of the first things that caught my eye is a slide on Why is there any need to reinvent testers?


Point 1: "Because I am a tester and I need to improve myself"

Ok, so I am a tester. I'm not a tester just by role, I'm a tester by profession. I'm a tester by identity. I've been a tester for 20 years and it's (professionally) all I know how to be. I feel uncomfortable when I was last Friday referred to by a customer representative as being "the programmers".

There's two ways for me to work on this.

Option 1. I can decide that what I am is what I am and that is not going to change. I can (and have) apply job crafting to reinvent tester to be whatever I want and am able to do. So far I've crafted my job as a tester enough to have other testers (and in particular researchers of testing) tell me that I'm not a tester.

Option 2. I can work on changing my identity. I've already been working on confronting my love of the tester identity by representing myself as something else. I've joined hackathons representing as a programmer. I've deliberately joined discussions with groups that don't know me representing a business person (easy to fool people because as a tester, that is a core to what makes me so good), a UX specialist (not that hard either, I've always cared about design and it's an area of testing feedback), a programmer (needs interest in technologies and some knowledge, but surprisingly many programmers know very little as well) and a project manager (like a business person, but simpler as the world of opportunities is narrower). No matter what I represent, I'm still me. And all this experimenting has given me new found respect for what I am and that it is useful. I don't have to be a tester to be a tester.

I believe that a deliberate focus on things we label "testing" is what made me the tester I am. Earlier in my career, I could have ended up developing different traits than what I find useful as a tester. I could have chose not to stare at the screen while testing, listening to my inner voices telling me what I observe and deduct. I could have been tempted to take the easy route and just manage testing when I chose to dig in deeper in doing it. I could have fallen in love with code and let it take me as it has taken my colleagues. I saw people like me model after other people than I chose to model, and end up not so great at testing. It takes deliberate practice. And hours are limited, until you look at the hours on a long enough timeframe.

I've started to see that with Option 1, I'm like a fireman at the time when sprinkler systems changed the world. I'm ready to become an arsonist or encourage people to being arsonists just so that I would be needed.

Testing is important skill, but it's a skill that no longer belongs to just testers. We have found ways of making the need and pacing of testing very different (agile & the business environment change). I find it necessary to challenge the status quo now, but rewriting things back to the "good old days" isn't my choice as of now.

I agree, I need to improve myself. But what the improvement means lies my disagreement.

Point 2: Problems: Craft, Companies, Programming, Expectance of low quality

All these problems are problems in the industry, with a large enough sample. Within the sample, there are examples of places where these are and have been addressed. Could it be that the ones within the realms of agile actually have found ways of doing things better and actually reinvented testers in a way that the masses are just not ready to accept.

I wish I would have been blogging longer, and I could show you in detail how my perspectives have changed. I've lived through things in my own project at work. I haven't just analyzed it from afar, but taken change of it as my personal responsibility - with my team and my organization.

In a way, I don't care about the industry. I've seen that within the industry, there are companies that do it better. How about opening up the channels to truly listen to those who work with this, and stop telling them that their experiences must be incorrect because of someone else's experiences.

It was supposed to be context-driven testing, what happened to context when defining and discussing this stuff? We lose context, and muddle the waters by combining the worst of the industry as a motivation on how we, as the profession of tester need to improve.

No thanks.

(Note: I'm not open for a debate on this. Debate, as it happens in the testing world is a form of bullying and I claim my right to not engage in needing to defend myself. Instead, I'm open to a dialog. If you really care about why I feel the way I do and what are my experiences that make me feel the way I do, I'd love to work on those. Or if you want to explore on why you feel differently and how your experiences differ from mine. I want both of us to respect our varying experiences that define what we see and emphasize.)

Monday, May 16, 2016

Explore a world without roles

I find that there might be a connection of the loud "role of a tester" discussion and messages like this that make us realize we are not as irreplaceable as we'd like to think.
It is very much same as the discussion I had after my session at AATC with one participant. High-level managers are looking at their organizations quality-related problems and coming to the conclusion that the highest priority change is to fix the source of badness, and testers as sin-eaters (a phrase I picked up from Jesse Alford) are part of a problem.

The tester-profession, on the other hand, has been trying to figure out value and role, to a point where it feels like rallying a campaign that has long ago lost it's focus and gotten lost on talking about role, not the problem we're trying to solve with a role.

I believe that in the greater scale of things, removing entire testing departments is good. One of the reasons I think this is good that looking at the ISTQB number of 500 000 testers, I've already seen my fair share of testers who just provide no value. There's also a fair share that do provide value, but there's more of testing value that developers can deliver than testers seem to give them credit for.

Removing the department of testing in the first phase wakes up the developers. They find ways. They will improve. I find it is often easy to become better than what you were with the separation with weak testers and weak collaboration.

But also, it gives the room for the new kind of testing to emerge. There will be things that the teams feel challenged with. And they will find people with exploratory testing mindset to fill in some of those gaps. They might want, primarily, that people with the mindset will also be able to code. Some get what they want and end up never using the programming ability to directly.

I'm growing tired of the focus on the tester profession as sub-optimization we're selling. I see the real-life problems with sub-optimizing testing, focusing on testing metrics over great products.

So I just want to explore this further: could there be other questions and approaches that would serve us better in creating the perfect world of software, where the special skills and abilities could come together than sticking to roles?