Tuesday, July 17, 2018

Pay To Speak - why care and other collected thoughts

I hold a strong belief that the world of conferences needs to change so that it is not #PayToSpeak. This means that while you are not necessarily making money for speaking (that should happen too), you are not paying money out of pocket for speaking.

Recently I spoke at a big conference that has no call for proposals but invites all speakers and says to pay their travel, including the following weekend's hotel nights to enjoy the beautiful city the conference is at. They have a policy of checking in with them on bookings or them doing bookings for you. So when they did my bookings for flights, they had me leave home 3:30 AM and return home 2:00 AM. That meant no public transport available to the airport. I could drive and park there, or I could take taxi. I chose the latter as cheaper option. When I arrived after 8 hours of being stuck on travel for something that would be a 2 hour direct flight, I had a taxi pick me up. However as I needed to get out immediately after my scheduled talk to still almost miss my flight, I had to find (and pay) the taxi myself. The flight tickets did not include any luggage, so I ended up paying 80 euros just to bring my stuff with me. Packing so that I can take it all inside means compromises on cosmetics I would rather not do while having to feel presentable. That's one of the women's problems, I guess.

The extra costs totaled to 180 dollars, which was more than the cheap flight they found me. Their view was that they wouldn't pay and that was yet another nail in the coffin killing my speaking willingness. Now it looks like they might pay, but I believe when the money is on my account.

So being against #PayToSpeak means that I believe that while it is reasonable to ask speakers to be considerate of costs (no business travel), it is not reasonable to optimize in ways where you just transfer the costs to them.

To be clear, many conferences are #PayToSpeak. Most in fact in the field of testing. A little less in the field of programming. A little less in the field of testing now that we are getting to a point of being respectable industry (including automation).

Why should the conferences care?

We've seen examples like Selenium Conference moving away from #PayToSpeak. The results they report are amazing.

  • 238 % increase in number of submissions - there are a lot of people with great content that cannot afford to teach in conferences that are pay to speak
  • new subjects, new speakers, new nationalities and new perspectives - all valuable for us learning in the field
  • percentage of women submitting growing from 10% to 40% - showing that the pay to speak might disproportionately impact underrepresented groups ability to teach in conferences

Surely you don't mean that we should pay newbie speakers?

I find that #PayToSpeak conferences present their audiences three groups of speakers:

  • Those so privileged financially that paying isn't an issue
  • Those with something to sell so that their companies pay them to deliver a sales pitch in disguise. Some better disguised than others. 
  • Those dreaming of public speaking experience finding no other options but paying their woes, sometimes delivering great talks. Believing paying is temporary. 
I believe that people in first category can opt out of payments in a conference that pays the expenses. In my experience they rarely do opt out on the out of pocket money, but many have opted out on profit sharing to leave money behind to support new speakers. People in the second category might become sponsors and pay more than just expenses to attend, and have their sessions branded as "industry practice / tool talks" which is often a way of saying it's selling a service or a tool. 

The third category is what makes me sad. 
This should not be the case. We should pay these speakers expenses. Our audiences deserve to learn from them.

As conference organizer, the thing I'm selling is good lessons from a (insert your favorite positive adjective) group of speakers. There are other ways of managing the risk of bad speakers than making sure your audience only gets to listen to the financially-well-off-and-privileged segment.

You could ensure the speakers with less of reputation get support and mentoring. With things like Speak Easy and folks like myself and Mark Dalgarno always popping up to volunteer in twitter, this is a real opportunity for conferences as well as individuals.

For Profit and Not For Profit

Finally, these words don't mean what you think they mean around conferences. You can have a not for profit conference with really expensive tickets and they can pay the speakers (this is the vision I build European Testing Conference towards - making real money to pay real money to speakers within a not for profit organization using profits to pay other conferences speaker's travel). You can have a for profit conference with ridiculously small cost and they still pay the speakers (Code Europe was one like this - they made their money out of selling participants to sponsors in various ways in my perspective).

Speaker's choices of where they show up with great material matters. But most of all, participants using money on the conferences matter most. Pay for the good players. Be a part of a better world of conferences.

Sunday, July 15, 2018

Testing does not improve quality - but a tester often does!

Being a self-proclaimed authority in exploratory testing, I find it fun when I feel the need of appealing to another authority. But the out of the blue comment the awesome Kelsey Hightower made today succinctly puts together something I feel I'm still struggling to say:  Testers do their work to save time for stakeholders next in chain. 
Actually, nothing in this tweet says that you need a *tester* to do this. It just refers to highly intellectual and time consuming activity, which to me implies that doing something like that might take a bit of time to focus.

With the European Testing Collaboration Calls, I've again been privileged to chat with people who trigger my focus to important bits. Yesterday it was someone stating their observation very much in sync with what Kelsey here is saying: for many of the organizations we look at that go for full hybrid roles, it turns out that the *minority perspective* of the exploratory testers tends to lose in the battle, and everyone just turns into programmers not even realizing why they have problems while in production on scale beyond "this is what we intended to build".

Today in prep for one of the Collaboration Calls, I got triggered with the sentence "testing does not improve quality". I sort of believe it doesn't, especially when it is overly focused on what we intended to build and verifying that. The bar is somewhere and it is not going up.

But as a tester, I've lived a career of raising the bar - through what I call testing. It might have started off like in one organization that 20 % of users were seeing big visible error messages, and that is where the bar was until I pointed out how to reproduce those issues so that the fixing could start. But I never stop where we are now, but look for the next stretch. When the basics are in place, we can start adding more, and optimizing. I have yet to find an organization where my tester work would have stalled, but that is a question of *attitude*. And that attitude goes well with being a tester that is valuable in their organization.

How do you raise the bar through your tester (or developer) role?

Feeling Pressured to An Opinion

It was one of those 40 collaboration calls I've been on to figure out what is people's contents that was special because of the way it ended up being set up. The discussion was as usual. There were a few of us having a discussion on an intriguing topic. The original topic was way too big for a single talk, and we collaborated on what would the pieces of focus look like that we would consider. 30 minute talks about everything between life and death don't do so well and don't end up selected, so we always seek something in a call that really has a fighting chance.

As I ended the call, a friend in the room expressed they had been listening in saying "You can't seriously consider you'd take *that* into the conference program".

I was taken back but was stupid enough to budge under pressure to confirm that I wasn't seriously thinking of it. Even though I actually am. I always am.

Because of my failure to - again - stick up to what I believe and let a verbal bully run over me, I felt like I wasn't true to my values. But I also learned that while in the moment I may be run over, I always come back to fix what I did wrong. So a few hours later, I expressed my annoyance of the style of communication, and recognizing the added bias of this negative interaction, I'm more carefully taking the advice of the two other excited co-organizers I had on that call.

Looking at the interaction a little more was funny in the light of a discussion a few hours later on how  awesome visual thinking strategies (a lesson by Lisa Crispin) are in identifying when you're doing an observation and when you're doing an inference.

Observations are the facts. The exact words the person would use in a call. What you can visually verify. What you can sense with your senses without adding judgment.

Inferences are not facts. They are your observations mixed with your experiences. They are your biases at play. And the reason we teach the difference in testing (and try to practice it), is to recognize the difference and seek fairness.

What the friend didn't see fit in the conference program was only unfit through their biases and experiences of when people are collaborative and when they are pushy. There's room for pushy people like them pressuring me to opinions I need to come back defending, so I'm sure there's room for all sorts of people.

Software is easy but people are hard. And combination of the two is fascinating.

Saturday, July 14, 2018

All of us have a test environment

This post is inspired by a text I saw fly by as I was reading stuff in the last hours: All of us have a test environment, but some of us are lucky enough to have a production environment too.

Test environments have been somewhat of a specialty of mine for the 25 years I've spent in testing, yet I rarely talk about them. So to celebrate that, I wanted to take a trip down memory lane.

Lesson One: Make it Clean

I started as a localization tester for Windows applications. After a few decades, I still remember the routines I was taught early on. As I was starting to test a new build (we got them twice a week back then), I would first need to clean my environment. It meant disk imaging software and resetting the whole of Windows operating system to a state close to factory settings. Back then it wasn't a problem that after doing something like this, you'd spend the next hours in receiving updates. We just routinely took ourselves to what we considered a clean state.

As you'd find a problem, the first thing people always would ask you was if it was tested on a clean machine. I can still remember how I felt with those questions, and the need of making sure I would never fail that check.

Lesson Two: You Don't Have to Make it Clean

Eventually, I changed jobs and obviously took with the a lot of the unspoken attitudes and ideas my first job had trained me in. I believed I knew what a test case looked like (numbered steps, y'all!) to an extent that I taught university students what I knew ruining some of them for a while.

I worked on another Windows application, in a company with less of a routine on cleanliness of the test environments, and learned a lot about the fact that when environments are realistic as opposed to clean, there's a whole category of problems of relevance that we are finding. It might have made sense to leave those out as long as we were focusing on localization testing, but it definitely did not make any sense now that I was doing functional testing.

I realized that we were not just testing our code, but our code in an environment. And that environment has a gazillion variables I could play with. Clean meant less variables in play and was useful for a purpose. But it definitely was not all there was.

Lesson Three: The Environment is Not My Machine but It's Also My Machine

Time moved forward, and application types I was testing became more varied. I ended up working with something I'd label as client - server and later on web. The Client - Server application environments no longer were as much under my personal control as the Windows applications I started off with. There was my machine with the client on it, but a huge dependency to a Server somewhere, often out of my reach. What version was where mattered. What I configured the Client to talk to mattered. And I learned of the concept of different test environments that would be under fairly regular new deliveries.

We had integration test environment, meaning the Server environment where we'd deliver new versions fairly often and that was usually a mess. We had system test environment, where we'd deliver selected versions as they were deemed good enough from whatever was done with the Integration test environment. And we had an environment that was copy of Production, as most realistic but also not a place where we could bring in versions.

For most people, these different environments were a list of addresses handed to them, but that was never my approach. I often ended up introducing new environments, rationalizing existing ones with rules, and knowing exactly what purpose each of them could give me with regards to how it impacted the flow of my testing.

Lesson Four: Sometimes They Cost a Million 

Getting a new environment wasn't always straightforward, it was usually a few months of making a business case for it and then shopping some rack servers we could hide in our server lab. I remember standing in front of one of these racks, listening to the humming both from it and the air conditioning needed to run that much hardware and being fascinated. Even if it took a few months arguing and a few more months delivering, it was still something that could be done.

But then I started working with Mainframes and a cost of a new environment went from some thousands to a million. It took me two years to get in a new environment while in this environment.

Being aware of the cost (not just hardware but the work to configure), I learned the environments we were working on in even more detail. I would know what data (which day's production copy scrambled) would reside where. I would know which projects would do what testing that would cause the data to change. In a long chain of backend environments, I knew which environments belonged together.

In particular, I knew how the environments were different from the production environment to the extent that I still think of a proud moment in my career when we were taking a multi-million project into production as big bang, and I had scheduled a test to happen in production as the first thing to do, one that we couldn't do elsewhere as the same kind of duplication and network topology wasn't available. And the test succeeded, meaning the application failed. It was one of those big problems and my proudness was centered around the fact that we managed to pinpoint and fix it within the 4 hour maintenance windows because we were prepared for it.

Lesson Five: It Shouldn't be Such a Specialty

Knowing the environments the way I did, I ended up being a go to person for people to check of which of the addresses to use. I felt frustrated that other people - in same kinds of positions that I was holding - did not seem to care enough to figure it out themselves. I was being more successful than others in my testing for knowing exactly what I was testing, and what pieces it consisted of. My tests were more relevant. I had less of "oops, wrong environment, wasn't supposed to work there".

So think about your environments and your attitude towards the environments? Are they in your control or are you in their control?

Tuesday, July 10, 2018

What is A/B Testing About?

Imagine you're building a feature. Your UX folks are doing what they often do, drawing sketches and having them tested on real users. You've locked down three variations: a red button, a rainbow button and a blue button. The UX tests show everyone says they want the red button. They tell you it attracts then, and red generally is most people's favorite color. You ask more people and the message is affirmative: red it is.

If you lived in a company that relies heavily on A/B tests, you would create three variations and make releases available with each variation. A percentage of your users would get the red button, and similarly for the two other colors. You'd have a *reason* why the button is there is the first place. Maybe it is supposed to engage the users to click and a click is ordering. Maybe it is supposed to engage users to click and a click is just showing you're still active within the system. Whatever the purpose, there is one. And with A/B tests, you'd see if your users are actually clicking, and if that clicking is actually driving forward the behaviors you were hoping for.

So with your UX tests, everyone says red, and with your A/B tests, you learn that while they say red, what they indeed do is blue. People say one thing, and do another. And when asked on why it is the way it is, they rationalize. A/B tests exist to an extent because people being asked is an unreliable source.

What fascinates me around A/B tests is the idea that as we are introducing variation, and combinations of variation, we are exploding the space that we have to test before delivering a product for a particular user to use. Sometimes I see people trusting that the features aren't intertwined and being ok with learning otherwise in production, thus messing the A/B tests when one of the variation combinations has significant functional bugs. But more often I see people not wanting to invest in variations unless variations are very simple like the example of color scheme of buttons.

A/B testing could give us so much more info on what of our "theories" of what matter to user really matter. But it needs to be preceded with A/B building of feature variations. I'm still on the fence with understanding how much effort and for what specific purposes organizations should be willing to invest to really hear what the users want.

Sunday, July 8, 2018

The New Tasks for an Engineering Manager

I've now been through three stages of transitioning to an Engineering Manager role.

First stage started off as my team interviewed me and decided they would be ok having me as their manager. People started acting differently (too many jokes!) even though nothing was really different.

Second stage started when my manager marked me as the manager in the personnel systems and I got new rights to see / hear stuff. I learned that while being a tester I never had to do mundane clicking, that was pretty much core my my new managerial responsibilities. Accepting people's hour reports (only if they insist doing them, we have an automated way for it too), people's vacations and expense reports.

Third stage started when I started finding the work undone while the engineering manager position was open. The recruitment process with all its steps. Supporting new people joining the organization. Saying goodbye to people when we agree the person and the work are not right for each other. Rewarding existing people and working towards their fair pay.

I found an Engineering Managers' Slack group, and have been fascinated with the types of things Engineering Managers talk about. A lot of this stuff is still things I was doing while identifying as "individual contributor".

I've found two weird powers I now have been trusted with: terminating someone's contract is just as easy in the systems than accepting hour reports (and there is something really alarming in that). And as a manager, I have access to proposing bonuses without having to do all the legwork I used to do to get people rewarded.

Officially one month into my new role and now one month on vacation. We'll see what the autumn brings. 

Sunday, June 24, 2018

Not looking for recipes of winning in a game but stopping the game

A long time ago, I learned a little game from James Bach: the beer game. It is a little exercise of trying to do something that people manage on a daily basis, buy a beer at a bar. Yet as the game progresses, it becomes clear that simple things can be hard. Your beer can come in a size you don't expect (you did not specify!), in temperatures you did not expect (you did not specify!) and with stuff you did not expect (you did not specify!). With the rules its played, you can only lose. And the only way to win is to stop the current rules.

Software development has a lot of aspects like this. If we end up in the defensive mode, one part of organization pitted against the other, we can always shift blame around. The estimates and predicting the schedules that are by nature filled with uncertainty can easily turn into a blame game. And from a perspective of an individual team, I can easily find others to blame: it would have taken us a week,  but we were blocked by another; it would have taken us a week, but we thought you meant X when you meant X+1; it would have taken us a week, but then quality is flexible and we did not think of that aspect of quality this time. Like the beer game, this is a game you cannot win. Trying is a sub optimization. Instead I find we need to look at the overall system, beyond an individual team.

Why do we want to have a date available? Because we don't have the stuff we need available today. Why we think the stuff we have today isn't sufficient? Because we believe that if we had more, it would be easier to sell? Why do we need stuff to be easier to sell? Because our sales people most likely are heavily paid on commissions. Is there other ways to set up the sales people's salaries? Most certainly yes.

I'm still in the process of asking why but I already know this: I'm not looking for ways to win in a game we shouldn't  be playing. I want to change the game into something we can all win together.

And for people thinking that "2-3 hours of estimating every now and then, no big deal". In scale, with uncertainty it is a big deal. And with people like me who feel every given estimate with their every night's sleep, it is a matter of health and sickness. I don't actively lie when I know there are better ways of doing things. Thus estimation is not a routine I take lightly.