Polarities of test data

I've recently been testing a complete redo of our reporting functionalities, and all in all surprised in how is it possible for a pair of developers to think it works when it does not. Even when there is a clear oracle in form of an existing previous implementation you can test against.

Testing this redo as if it is new functionality but with a simpler oracle has lead me to create simpler test data. I first handle just individual pieces and then move on further into combinations. The main principle guiding me is control: me in control of the data, understanding the basic before going into the complex and complicated.

My testing of the reporting functionalities was interrupted by a release with just a few little updates. All the updates were related to upgrading components, both 3rd party and our apps shared components. These usually cause specific types of problems, so I run a set of explorations around basic scenarios but this time, did not pay attention to data much. I used the data I had created for the reports, simple and controllable.

And a bug escaped us: there's a grid component that we for purposes of one view overloaded for height calculation, ending up with a problem that in other places, scrolling would fail. A classical (for us) mistake of one developer working with the component in one place tweaking it to be for that purpose, and it having trouble then when used elsewhere.

For me the interesting lesson was on data. If I had been on my typical data, I could not have avoided to see the bug. But since I was in a narrow and limited, controllable set of data, it hid the problem.

With continuous delivery though, the problem was shortlived. But it lead me to create two specific sets of data to reuse as part of my checklists. There's never one, but I can try doing smart selections of what I keep available. 

Value in contributing or learning - when to automate tests

I listened to a podcast with Llewellyn Falco today. There's many things of interest, but one worth an inspiration for this blog post.

In one point of the discussion, there is a discussion about failures. The first story Llewellyn tells about failure is having five developers work for a week on programming when the end result of that task is equivalent to 6 hours of manual one-time work. The story is about asking of value in advance. Good lesson.

I have had the privilege of many discussions about unit testing and test automation with Llewellyn. The story reminded me of my perception: with test automation, I often bring forth the discussion about real need of repeating and value of spending hours and hours in automating something I don't care to repeat but could do in seconds. But while the value seems like a relevant driver for me, the point that I feel we end up with is doing things anyway just to learn if (and how) it can be done. Value of learning (and solving a challenge) goes over the value of using time on testing.

There's a rule for unit testing Llewellyn cites a lot: Test until bored. As long as you are contributing to the overall value that testing could provide immediately or in long term, you are not bored. As long as you are learning, you are not bored.

I'm puzzled with this: I can always name many things I intentionally don't test, as there is never enough time for testing it all. I'm painfully aware of the opportunity cost of writing documentation or creating test automation over testing just a little more from the endless list of ideas.

I see my teams developers approach unit testing avoidance with similar arguments that I use on selecting what is worthy of automation on system or feature level. It makes me appreciate the approach test-oriented programmers like Llewellyn take: the challenge of learning is worth the effort.

But I still can't help but wonder: why the same rule of overall value wouldn't hold in the domain of testing as for other problems we solve with programming?

Perhaps because: 

• adding the first test enables us to add more tests like that with a cheaper cost - repeatability is the solving a particular problem in a particular way in addition to repeating the same test by means of automation
• if we don't automate because of cost-value discussion (with unknown cost), it's just another trick in the bag of excuses for us
• we don't really know to estimate the cost before we have a solution, until then we can only discuss if the problem is worth solving at all 
• the discussion about the unknown cost can take more effort than just doing it

The core difference in the story in the podcast and our common experiences of automating tests are in knowing how the problem can be solved. Perhaps the rule of "contributing or learning" applies in these examples. 

Birds and statues - or phrases to keep to yourself

There's a conflict emerging - on merge conflicts and how to deal with them.

Sometimes I wonder why I care - I'm a tester after all. But I do care, since the quality of the code matters to what kind of results we can achieve as a team with help of testing. I care enough to nurture an atmosphere where refactoring is not only allowed, it is encouraged. We're not trying to stop change, we're trying to flow with change. And we've been relatively good at that, doing continuous delivery without test automation. Clean code is easier to work with. And in most cases, we have pretty similar idea what is clean code.

It seems there's an exception to every rule. My exception comes in the form of different beliefs. There's one (with 20+ years of experience) who believes that the things I've taught to believe about team work and clean code are incorrect. That we should find ways of working with code by one individual only to avoid merge conflicts - and human conflicts that result in changing the status quo.

When I suggest we wouldn't have a conflict if we paired or mobbed, I'm not being helpful. So I try not to mention that while I still think of that - a lot.

When I suggest we would have less conflict if we had smaller methods and that refactoring would be a good idea, I'm told we should stop refactoring all in all and always just write new code. We can just add fresh branches, leaving the old that worked there as is, still working. And that code style and cleanliness is just an opinion anyway.

When I suggest doing smaller increments that can be released to contain the conflicts, I get a shrug. And I get a bunch of others saying how good a strategy that is, but also remarks on how this area is just different.

When I ask what we could do, I hear we could just work on different areas completely, in isolation. To avoid merge conflicts - and human conflicts. It's worked for decades, what would be different now?

There's a phrase that I managed to keep to myself that I've mentioned before this all became urgent and pressing: be the bird, not the statue. I heard this at Agile 2015 from Arlo Belshee. The one who stays put is the one who gets hurt in merge conflicts in the modern software development. But saying that right now might be again not helpful.  But I think of that - a lot. And I admire the conflict-averse other developers, who increase their birdlike features in all three dimensions of how they deal with the shared code, leaving one statue there to realize implications later.

One experience of teaching kids creating with computers and programming

I've been slightly struggling with time management issues this autumn, having so many things to do and so little time. Today was an end of one experiment I wanted to do, and it turned out great even if different than what I had in mind when I got started.

I wanted to try out teaching 1st graders about creating with computers and programming in a format that is different from free-time voluntarily -based approached of code schools and clubs in Finland. I wanted to start building a way of being helpful as part of classroom training, instead of adding more hobbies on the side. There's still a big challenge ahead in getting all the teachers up to speed with what they could be teaching with regards to programming when it starts in all elementary schools for all grades in less than a year. 

I contacted my girl's teacher and asked if we could do something together. With a little discussion, we came to the conclusion that we could do a few classes with the focus of story telling with the kids. So we started making pictures and voice over to become a video of an alphabet story. 

It was wonderful to see what kinds of story ideas kids came up with from the letters. I was originally hoping to put these together into a CYOA-story (Choose Your Own Adventure), but the timeframe of 2*45 and a class size of 28 students minutes soon put me to a different idea. Just the collaboration would have required more time to make that happen.

The first session had two parts:

1. Hello Ruby book handout. Every kid in both 1st grade classes got a free copy to take home, as well as the teacher. That was 48 copies. Linda Liukas and her publisher were very nice and enabled a cheaper unit price for the batch, and my non-profit work was able to finance the books as gifts. The kids were very curious about the books and with guidance from me and the teachers, named their own copies so that they would not get lost or mixed up. 
2. Create a story with picture, and record it. I had each kid choose a letter of alphabet and draw a picture of a story they came up with. I asked them to come to the computer for recording, and showed them around on how to tell their story as a recording. I collected the hand-drawn pictures. 

In between the sessions, I did a bit of work on the outputs of the first session. I paired with my girl at home on some of the work, and taught her video editing basics to cut out all of me to leave just the kids. We scanned the hand-written pictures and included them side by side with the stories. My girl recorded a beginning and an end with a proper microphone and got excited about the idea of singing into the microphone. With the huge number of stories, the editing was quite much work, taking me 3-4 hours. I cut out the silences, and even transformed one word answers to something that resembles a story. 

With that preparation, our second session had three parts:

1. Creating credits image to end the video. The teacher set up a touching game to add a little randomness to the order of kids coming over to the computer to add their own name into signatures. When touched, it was their turn - and this created a positive sense of waiting for one's turn. With the projector, there was a lot of fun with this simple act: writing on the computer, getting your own name into the work and others reading what is on the screen as it emerged. Every kid wrote their own name with small (non-capital) letters, and watching that happen for a few kids, I realized there was a great intro into the programming we'd do later. I showed how with a computer I can select all 28 names and turn them into words that start with capital letters just with one function. 
2. Watching the video together. We then watched our end result. Everyone in their turn was looking like they were feeling a mixture of embarrassment and happiness, and the whole group was very focused on seeing their own stories. We talked briefly about the fact that the video is theirs and that I will not put it in the internet nor should they - that things like that should be agreed on in advance and everyone should agree. The class teacher pointed out clarify of speech and how much easier it was to understand what great messages you have to say if you articulate, and I was just thinking I missed a bit of editing when I did not raise the quiet voices to a higher volume. 
3. Talking and trying out programming. With the video created, we started talking about programming. We talked about someone having programmed the tool we could use to create videos, and programming as something that would allow us to create things we could imagine. We did two exercises from Hello Ruby book.
   We talked about small programs that consist of commands learning Ruby's dance moves. Then we talked about us being the computers running the dancing program and tried it with three repetitions. We ended the exercise talking about doing the moves 1000 times and how computers don't get tired of repeating things you program them to do. And we talked about the idea that we could tell also to stop when we say to stop.
   The second exercise was the debugging exercise that comes closer to my usual work. We looked at two three examples where the little instructions that computers could understand but that missed something. Like the idea that you would put your table cloth on top of your birthday cake, or that you would eat when you're full and say thank you when you are hungry. 

With a lot of laughter, sense of having created something that wasn't there without you, I ended the session. Next up would be hour of code, in just a few weeks. I like those exercises, but I find them one-sided: to become smart creatives, we need to want to create, not just solve given puzzles. I call for freedom and support in implementing one's own ideas, the thing that I remember loving but missing out in school system including the computer science studies in the university.

Creating with computers is the point for me. Programming is just a tool. Sometimes what we want to create require programming. Sometimes advanced use of existing programs. But the thing we should not have is fear. Believing computers do what you want, demystifying them is a core of what I do professionally, whether I feel like being a tester, a programmer or a product analyst - today. Most days I love being a tester too much to ever want to be anything else. 

Another experience in Mob Testing

Today, I had a visiting lecture at Aalto University of Applied Sciences. My theme was Testing in the industry, as it has been for any years in a row when they've invited me to share lessons from the trenches with the new aspiring computer science students. I love going and prioritize it high, after all it's my university. It's where I used to study, it's where I used to do research on testing and it's one of the major places on my career that has given me the space to grow with my interests in testing. 

I agreed on being there for a visiting lecture at XP2015 conference. Since then, the teacher changed, but the agreement of me doing a visiting lecture stayed. On previous years I've shared ideas about how I do my work and what I've learned about testing in these 20 years, but this year with the new teacher we came to another idea. What if I would just briefly tell what I do and how much I love what I do, and let the students experience my joy first hand in a mob testing session! 

We had 1,5 hours (2 university lectures) and about 15 students. When they came into the room and sneaked to take their places in the back row, I invited everyone to take their chair and come to the front of the class in a semi-circle. I had no slides to show, so we started with the application to test open on the screen, Eclipse with the Java-code and a browser window with mindmup in the background. 

We talked about the roles of a practice mob testing session, and the roles I assigned were those of the driver - no thinking at the keyboard, designated navigator and other navigators, that would navigate through the designated navigator. 

I introduced the application briefly. I told it was called Dark Function Editor, and I navigated the first driver as an example to get the application to a point where we could start testing. There's a limitation I would call a bug that prevents the application from working unless you realize not only start a project, but also add an animation within that project that I did not want them to get stuck with. This time I also did not force them to log it, and as I had shown it as if it was expected, no one seemed to consider it a bug. 

We rotated on three minutes, so the group kept moving on quite a high pace and organized to do that very fluently. The first few rotations were focused on trying out some features that ended up being just randomly picked by whoever was then navigating. The first navigators were clearly unsure of what to do, and referred very nicely to the help of their mob, and the group was contributing together quite early in the process. The first rotation was challenging, none of the navigators would volunteer ideas of where to start from with a blank sheet and I reminded that in testing, it's good to remember that if you freeze because you feel overwhelmed, it's good idea to do something, anything. There's no wrong or right when all is a mystery to us. 

The group found bugs, and one of the bugs was something I had not personally noticed before, even if I have now used the same application in quite many sessions. They noticed that if you added right kind of pictures in right order, it became obvious that the preview and actual layout with the order of pictures were reversed. They just got lucky to put pictures in order different that what my list of preprepared material had guided other groups to. I introduced the concept of writing bugs to the mind map. Then they run into things they were unsure of if they were bugs, and I introduced the concept of questions. With concept of questions, I also introduced the concept of color-coding different items in your mind map. With a few rotations, we had a few questions and a few bugs, all around the center item on the mindmap. 

However, as the facilitator I sensed that they were testing without a purpose. So I inflicted a constraint, asking them to focus on creating an animation that they could export, just a basic flow. They were lured by things they'd like to try occasionally, especially the mob volunteering things more actively than could be done and sometimes the driver and designated navigator would look a little overwhelmed. I kept advising that the final call of what gets done is on the designated navigator. With a few detours and me asking what was our purpose now, we got an animation done and some additional questions noted down. 

With four things around the center node in the mind map, I introduced them by taking navigation to the idea that we can have deeper hierarchies. We decided with the mob to have Bugs and Questions as nodes, and dragged things we had built by that time underneath. I knew I'd much rather have feature areas here, and bugs and questions color-coded under the appropriate feature area, but the group was not there yet. They had not yet understood there were feature areas you could name and use for categorization. 

At this point, I gave them another charter as their constraint. I pointed to a particular feature, and asked them to list all functionalities they could find. The feature I pointed to has visibly four buttons, and I've facilitated groups where testers think that they are done having listed those. I was curious to see how this group would do. They did great. 

Because of the projector resolution issues, they couldn't first see there were four buttons, and they paid attention to only two. This lead them by accident to create a good list of things, and they came up fluently with various features related to handling the list. They were trying out single clicks, learned that double click is a rename feature, and that right click wouldn't do anything even if that would be something you'd usually expect in a functionality like this. They accepted the right click and dismissed the information. They tried drag-and-drop and started playing little with the automatic and manual naming of the list items. I sensed they were starting to feel they might be close to done. 

I took over the navigation pausing my timer for the ongoing navigator, and navigated to add more items on the list. After that was done, I asked if people noticed another feature being revealed now - a scroll bar. With this revealed, there was more they learned. They noticed that the up/down arrows would also only work when there was more items, and with those keys they also started paying attention to other keyboard shortcuts. 

At some point someone in the mob made a surprising remark asking if there was an undo functionality. The product use, adding and removing things and having to recreate your test data after someone else had decided that deletion was a good thing to test now probably inspired it. And the group found the Edit menu, with an undo-redo -functionalities that turned out to extend to the feature we were looking at on the other side of the application. I pointed out the excellence of that realization and sharing the idea with the group - it opened the mob new doors of what features there could be, as they could start thinking of connections outside user interface proximity. 

The time was running low, and I stopped the navigator to take over with a piece of advice, pointing out the two buttons they had not noticed. With starting to add those buttons, someone in the mob suggested that perhaps the mind map should be restructured again, as it was hard to make sense of it with this many features. They used the last two rotations on getting started with restructuring the mind map, introducing concepts they had learned through exploration like naming the buttons function and creating a subcategory of actions for the buttons leaving tooltips of the buttons as another dimension to the actions. 

The map created turned out to be this: 

If they would have continued, they would have soon discovered that bugs and questions belong under their modeled areas. They would have visually seen more of areas of functionality they had considered from the listed features, and it would have inspired them to find more. There are still many features I've thought of in previous sessions that did not get listed. But they also had one thing they could name, but I had missed. 

We ended the session with a little retrospective. I asked people to share their observations in a round-robin style and we had little discussions about the observations, addressing e.g. Mob programming effectiveness and if anyone would do this in the industry. Many people pointed out how this mechanism enabled to build on others knowledge and get further than alone. And a few people pointed out how much fun it was. 

As an after-lecture discussion, I had a chat with the lecturer who said he will connect this experience to many of the concepts he will be teaching in the upcoming classes. He also said that mob was a very good teaching format, and I felt I have improved a lot even over the last few months on facilitating testing mobs and knowing when to step in to show a concept that will help learn more without stretching the group too far at once. We talked about an idea to take mob testing to published research articles - an idea I would want to support happening. They've done research on individual testing 10 hours vs. 5 people testing for 2 hours each, and having 5 people test 2 hours together would bring a whole new perspective into that setting. 

If you want to try teaching in this format, I would love to help you get started. And if you know of a team who would like to try this mob testing format with their own software, I'd like to collaborate to fine-tune my training approaches on this and do an experiment. 

Getting started with mob programming for User Interface Styles

While we've had some opportunities to try mob programming with my team, there's now an actual theme over a longer term a subgroup of my team will work on. It turns out that the subgroup consists of people who have not tried this way of working before AND that we need to do all of this in a remote setting. I'm looking forward to learning about how it works for us.

We've selected a theme: styles and the UI. There's a few reasons for this becoming a selection:

• There's a new user interface designer in our team that has not worked with major projects on styles before. So we know there's a significant skills ramp-up needed, and I don't know of a better way to do that than hands-on with a pair/group. 
• The current user interface programmer has not been strong on discipline. When sensing urgency in schedule, it's just easy to take a shortcut. And those shortcuts pile up. The style code looks messy and it's my personal testing nightmare: fix one thing, and many surprising places break. Cleanliness of style code would help us so much, whereas testing of styles in the extent they've been creeping to is just failure demand. So, I will test while bringing my idea of discipline to the group. 
• No programmer other than the user interface programmer currently touches the style code. It is too complicated to understand. We want to also transfer back some knowledge of how everyone in the team could work with it and clean it up as they go, and inviting them into the mob seems to be a good starting point for that encouragement.
• I want to try mobbing over long term and get more experiences in being the non-programmer (turning into a programmer) in a mob. And the others don't resist (at least yet), since the learning / sharing / improving the maintainability just makes sense.

Our styles, while they were CSS, used to be as clean as the rest of the code. Then we hired a user interface programmer who brought in LESS, changed all styles alone so that no others were involved and alianated everyone else by accident. It's not hard to see that touching something as wide and complicated, people do follow the instinct to flee (and leave the work for the specialized individual), even if that negatively affects our flow. I'm just glad testing isn't in this position in my team, it could be. Lesson: work together and share the work not to get there.

We had a "show-and-tell" session for the style code newbies of the team last week. We looked at our logic of structuring styles and I had two thoughts in my head: 1) now would be a great time to get that test-driven CSS lesson Llewellyn Falco could do with Agile Finland 2) reading code is separate from writing code - I know when I see a mess, while creating that mess myself would take a more focused learning effort. I made notes of four things to pay attention to:

• Use variables instead of the magic numbers. We really would want to keep things together better to follow what goes on.
• Simplify the overwriting chain. Minimal design over everything we could do. Less is better. And chaining makes up a mess of unpredictability.
• Scope with views. We have mechanisms to scope the styles, but we mostly keep things global. No wonder I see side effects of changes.
• No Important! -keyword unless absolutely have to. Use of these in chains have escalated in use of these almost everywhere, to make sure the last in chain will end up being applied. Discipline. 

Today we had our first scheduled mobbing session. It did not go quite as we'd hope. We started with 2 hours timebox and two goals in mind. We would use the new user interface designer's computer and first goal was that we would set things up that were needed for her to do the work. Second goal was styling of a tiny feature with minimalistic styles that would still fit both designer's ideas of beautiful UI (and the team's idea of maintainability, only valuable stuff added).

We only got work done on the first goal. We got the basics tried quickly, sharing voice over Skype and control of the computer over join.me, rotating on four minutes with the idea that our "expert" would skip his driving turn if we were in middle of something where we'd just be stuck without him navigating. With Visual Studio, we soon learned the computer in question had not been set up for the solution we needed to work on. With a few clicks, that was done. So we went to find the branch with the new feature to be styled and run into our first problem. The branch was not visible on the computer in question. We tried connections, no luck. We installed git bash and gui, thinking we could perhaps bring the branches in from the command line. First time installation gave us git bash that kept crashing every time we'd start it, and after rebooting (no effect) we reinstalled. But looking at things from command line did not take us much further. We pinged in more senior developers, still no clue. So we changed computer.

The starting setup on the second computer was faster, and we got to a point of Visual Studio and starting the application at about 40 minutes into our session. We started feeling the frustration of sharing the remote computer, as the feedback sluggishness of mouse movement made doing simple clicks harder than we're used to. We stopped the running application, synched the branch and went to run the application, only to see it crash at start. With theory of join.me and security settings, we turned off the sharing, no effect. So we decided to reboot the second computer too.

While waiting for reboot (it takes quite a while on Windows...) I asked my local colleague whose computer we tried using first if we could try one more thing - something I had suggested quite early on into the problem, something that should not be connected that was dismissed when I suggested it by the expert. We pulled latest of our integration branch and all of a sudden, the information of existing branches was available too. So by the time of reboot and reconnect, we had a working local environment.


Just before our remote expert could rejoin, we tried running the application, with the crash. We tried changing solution configurations between Development and Debug, and got a little further: login screen. But same error at login. The group was getting very amused with the luck we were having, and concluded that at least it was much nicer to be in this together than alone.

We called in yet another senior, to confirm our theory: the new feature relied on database changes that were available only in the developer's personal database. At 1,5 hours, we decided to call this session done and leave the second goal for next session, by which time we'd get the database changes in a common environment too.

A few observations of this session:

• The first goal turned out to be more complicated than we expected. No one, be it "code newbie", "style expert" or "senior developer / architect" had an immediate answer. Trying and googling were key. 
• If we would have tried the unlikely idea when it was presented instead of dismissing it, we would have saved 20 minutes. As a "code newbie", sometimes getting heard can be difficult. 
• Getting stuck and unstuck together seemed more efficient. Alone we would have just left it waiting for the "senior developer / architect" to hunt down an answer he did not have when we invited him - research needed. Together, we worked through the first problem and even managed to solve it. Surely, we could have done something else in the meanwhile. Like read the news.
• Minimizing the remoteness so that the one that must be remote is but the rest rotate locally seems a good idea with the connection sluggishness.
• We could improve from  here: the one test that fails the continuous delivery when the application doesn't start would be great for things like this. 
• Nothing I would pinpoint to my great tester abilities today. Perhaps next time. Next time is next week Monday. So experiment continues and everyone is still with me. 

 

Discomfort of defects

Last week, I was testing yet another new feature. And testing of it was very straighforward. First I tested to learn what the feature could do with simple data. Then I extended the data, and extended the variation I could bring in with the new feature. I made a few checklists to cover through relevant scenarios. It was about a day of work, nothing major.

I found 10 issues. That too is a very typical number. But what was atypical was that this time, probably because of business priority of the feature, all the bugs got fixed within one day.

As I was going through the fixes based on code commits and Jira notes, I noticed an interesting phenomenon. Only 1 out of the 10 issues I had raised had a comment. And the only comment was to say that I had found something that was _relevant_  - a mistake in how the code had been written.

There were 10 changes, so I would argue there were 10 items. But the developer, working towards no faults in his code, dismissed the other 9 and focused on the 1 as his personal feedback.

This again reminded me on how differently we think. For me, all 10 were relevant for the end user, and I really did not care whose fault it is that those exist. The developer worked against the idea that just-in-case he would ever need to defend himself, 9 of them were "new information he did not have when implementing the feature" and that he "had only one code defect".

We haven't classified things for anything but unfinished work for over a year, and still the culture of looking for one's own responsibility remains. There was no complaint on the other bugs, but the feeling can be seen in the comments - what gets acknowledged as "my bug" and what is just work on the list.