Speculative Execution

I’m Rowan, a tech lead at Dunelm, writing stories about the intersection of Engineering, Automation and Psychology.

Part of my role as a Tech Lead is to help decide what can or should be the team’s next focus based on the dependencies we have on other teams for features as well as our own capacity.

If you’ve ever been in a work planning session you’ve inevitably heard “Well we need team X for that, but they can’t help us until next month” or “Jane Doe will let us know if we need solution A or B on monday” and people conclude they need to awkwardly sit on their hands for a bit. This feels pretty bad for everyone involved, the engineers want to get coding and the delivery lead looks on mournfully at their project roadmap as the deadlines get pushed back and lower priority work is picked up instead.

I was watching a video about the meltdown and spectre exploits and it reminded me of the hardware level optimisation used to avoid bottlenecks — a technique called speculative execution. Speculative execution works by reading ahead in the upcoming code and performing the outcome tasks based on an assumption about a decision.

In the example below if we usually pass the index check and can safely fetch the data from the array, it’s not unreasonable to fetch the output and have it ready while we wait on the potentially more time consuming decision to be officially made.

function getListItem(index: int) {
  if (index < list.length){ return list[index] }
  else { throw new Error("Out of bounds exception") }
}

Of course, if we’re wrong this time we end up doing work that’s not needed and have to throw it away for no benefit. On average though, predictions we’re confident about can speed up our CPU and execute work faster with less idle downtime.

Engineers and managers reading this, you probably use this technique without even realising it. Whether it gets called “Future Proofing”, “Working to specification”, “Feature flagged” or even “MVP” it’s not uncommon to do work that isn’t technically needed just yet, or avoid adding the extra bells and whistles if you think you know what the answer is going to be.

This technique lets teams spend more time on high priority tasks and features can ship as soon as official decisions (be they financial, legal or based on user feedback) get made which can be key for hitting key dates like big sales days. Naturally though, this approach should be taken with a little bit of caution — nobody likes doing work only to find it never ships. If work gets discarded on a regular basis lower priority work that could have been done gets missed and the team will get demotivated as they feel their work doesn’t matter.

It’s important to clarify here, not all work that doesn’t ship is a waste! Experiments that answer questions like “Is this a better/faster approach” or “How do users feel about us making this change” might tell us to keep the status quo and not ship new code, which is a good thing! Innovations and experiments with new technology can also yield advancements in personal or team capability that will impact later shipments.

So keep in mind the answer to “will we be asked to do this task?” actually has three answers; yes, no and we’re not certain. Being uncertain is fine! When certainty strikes though, consider doing (or not doing) work speculatively!