Thursday, April 20, 2017

Time bombs in products

My desk is covered with post-it notes of things that I'm processing, and today, I seem to have taken a liking to doodling pictures of little bombs. My artistic talent did not allow me to post one here, but just speaking about it lets you know what I think of. I think of things that could be considered time bombs in our products, and ways to better speak of them.

There's one easy and obvious category of time bombs while working in a security company, and that is vulnerabilities. These typically have a few different parts in their life. There's the time when no one knows of them (that we know of). Then there's the time when we know of them but other don't (that we know of). Then there's the time when someone other than us knows of them and we know they know. When that time arrives, it really no longer matters much if we knew before or not, but fixing commences, stopping everything else. And there's times when we know, and let others know as there is an external mitigation / monitoring that people could do to keep themselves safe. We work hard to fix things we know of, before others know of them because working without an external schedule pressure is just so much nicer. And it is really the right  thing to do. The right thing isn't always easy and I love the intensity of analysis and discussions vulnerability related information causes here. It reminds me of the other places where the vulnerabilities were time bombs we just closed eyes on, and even publishing them wouldn't make assessing them a priority without a customer escalation.

Security issues, however, are not the only time bombs we have. Other relevant bugs are the same too. And with other relevant bugs, the question of timing sometimes becomes harder. For things that are just as easy to fix while in production and while developing an increment, timing can become irrelevant. This is what a lot of the continuous deployment approaches rely on - fast fixing. Some of these bugs though, when found have already caused a significant damage. Half of a database is corrupted. Communication between client and server has become irrecoverable. Computer fails to start unless you know how to go in through bios and hack registries so that starting up is again possible. So bugs with impacts other than inconvenience are ones that can bring a business down or slow it to a halt.

There's also the time bombs of bugs that are just hard to fix. At some point, someone gets annoyed enough with a slow website, and you've known for years it's a major architectural change to fix that one.

A thing that seems common with time bombs is that they are missing good conversations. The good conversations tends to lead to the right direction on deciding which ones we really need to invest on, right now. And for those not now, what is the time for them?

And all of this after we've done all we can to avoid having any in the first place. 

1 comment:

  1. Hmm... I was thinking of a different kind of time bombs - things we know will blow in our face at a certain date.
    For instance, if it so happens that some of the code we have now will still be alive in 2037, some stuff will be suddenly "expired".It's not an issue that will bother anyone in the next 19 years, so addressing it now does not make any sense - but there isn't any way of setting a reliable alert for 15 years from now, so hope is the strategy for fixing this.

    Another sort of time bomb that came to my mind was partial development - for instance, we developed a new mechanism to encrypt some of the data, but due to schedule pressure, did not develop a key-rotation mechanism. The encryption keys will expire (procedure prevents us from using keys after their crypto-priod is over), and the key rotation should be done before the first key expires. In this case, we remember the problem, but the schedule pressure isn't going away, and having to develop a feature with a given deadline is a pain in every way we plan our way forward.
    So, tick-tock.

    Bugs, security or otherwise, seem a bit more like a dud - it might go off and explode at any moment, but for now it's quiet and menacing.