Innovating outside the box

It has been a while since I last posted. There are a number of reasons for this including recently making the decision to take on a new role as CTO at Twilio. While not directly related to this post the past week has given me some time to get back to the keyboard and write down some of my thoughts…

Today I wanted to spend a little bit of time thinking about innovation. Specifically, I wanted to articulate some of the ways I have approached problems that I have faced in the past and articulate why I feel that they have generally led to successful outcomes. First let me define innovation for the context of this article.

Innovation - the process of creating some output previously not known capable of generating repeatable value for some group of consumers

Consumers in the statement above don’t imply customers - value could be created for any group (employees within a company, for members of your community, etc). The key is that there is some value that can be taken and used in a repeatable fashion that has generated value for that group of consumers. Repeatable is important in my definition because I am not looking at innovation as the brute force undertaking of some task (I could cook a meal for someone - that itself is not innovative as the output, in this case the meal itself, is not generating repeatable value).

Ok - so now we understand what innovation looks like the next question you may have is why did “outside the box” in the title? The reason for this goes to the “value” component of the definition. Specifically, I have observed many innovations over time that are incremental in nature but today I want to focus in a bit deeper on how that value can be maximized and assert that often times one must think beyond incrementally improving something that already exists. There are generally two camps of people at this point:

• Group 1 will say - this is obvious - invention requires thinking about a problem in an entirely new way.
• Group 2 will say - this is non-sense as it assumes that there is always some better method to the problem.

While I think there is merit in both points of view (and group 2 would likely convince me that at the infinite an optimal solution to a given problem will have been discovered) - both responses focus on the fact that the problem to be solved is already known (is static; ie: we want to solve the problem as presented to us) and it is this premise that I feel is most often worth challenging. Stated differently - Innovation Outside the Box is about questioning the problem to be solved - not attempting to create a better tool to solve the problem as it is first presented to you.

Example: Picking items in a warehouse

I have started to see a number of solutions in the warehousing space pop-up that look at being able to manipulate a wide (infinite) variety of items much like humans can (a number of startups began popping up around 2014 which is 6 years ago now). Imagine for example a box of random items being placed in front of you and being asked to grab a specific item from the box. Now imagine attempting to build a robotic arm that can do the same. There has been some great innovation here both in the creation of new end effectors (the component of the arm that can grab the item), in the vision processing to enable the robot to “see” the various items and how to manipulate (grab) the item that was requested, and some innovations in the containers themselves and well as the process for filling those containers to make it easier for the robot to select a given item. Still, with all of this innovation the space is still relatively new and I might challenge our thinking to ask if we are really solving the correct problem (needing to select a defined item from a container that is holding multiple different items where the items are stored in roughly the way they were received in the warehouse).

There is no doubt tremendous value in being able to solve this generic problem (as the solution could easily be fit into existing processes) but the solutions themselves are formed from a premise that the method of human manipulation is optimal. It also presumes that the generic solution has significantly more value then an initial solution that could be delivered quickly. What if we actually started this process first by asking why this is the problem we are attempting to solve for? I suspect that much of the thinking surrounds the fact that today an item is not simply taken off a truck and placed on a shelf and later picked from a shelf and placed on a truck ready to ship. Instead an item is touched many times (taken off a truck, scanned and counted into a warehouse, periodically counted to make sure an item hasn’t been lost, picked into a container, moved from the container into a box to be shipped, etc). Under these conditions an item sitting in a warehouse might be touched say 15 times on average (note - I made that number up but it sounds reasonable). Having a generic solution that can manipulate items has a lot of value under these conditions. But what if an actual item only had to be manipulated twice - once when pulled off the truck and once when placed in a customer box? Would this robotic arm provide as much value? In other words - is perhaps the real problem the fact items are stored in their raw form? What if instead every item was placed into a container that itself made manipulation (grabbing any item) uniform (imagine placing all items in a plastic bag)? What if those bags were color coded and when placed in a container for the first time were placed such that only one color of bag existed in each container making the identification process easier? My point in asking these questions is to ensure that we understand “why” it was important to solve the generic problem and to challenge ourselves to determine if there is a better approximate solution that can deliver step function improvement faster.

Example: Periodic Reviews

How many of you have been part of some periodic review. At the moment I am thinking less about reviews setup to look at a design but rather the periodic style of reviews (like quarterly business reviews, Sprint Reviews, Monthly Scrum of Scrum Readouts, etc). Have you ever stopped to ask what the purpose (value) of the review is (ie: what problem was being solved)? I suspect that many will say that it is about communicating and informing everyone where a given project or team is relative to some development effort. While I do think communication is important; I question that this is the correct problem to be solved. Yes, the output is communication but the real problem more frequently is knowing WHAT needs to be communicated. In other words the forum of the review is perhaps LESS important then preparation for the forum. In that case perhaps the right question to then ask is how can we make sure everyone prepares but not presume that the everyone must publicly present? Amazon I feel came up with a really nice solution for this - it was a wheel that was spun (think Wheel of Fortune). The result was a random team being selected and being asked to present (this is the forcing function). The end result is that everyone wanted to be ready (no one would want to have to stand up and say they hadn’t prepared) and more importantly the process of preparation was achieved and all content was available. The meeting could be bound in terms of time even as an organization grew and when problems were discovered teams were more likely to proactively engage because all the information was available. The review session held value itself as well in creating a reasonable check and balance (mechanism) for detecting if preparation was being done properly (think of this as the Probability of leadership asking a question and being surprised - if that occurred with some frequency it could indicate a more systemic issue with what information was being collected). Again - innovation occurred by questioning the underlying problem to be solved.

Example: The SDLC

For the many software developers I suspect may have come across this page imagine the Software Development Lifecycle in which you work. Now ask yourself when is the last time it changed significantly? Better yet - ask yourself a simpler question about when a step was removed (disappeared) from the lifecycle? Chances are that if you are like me this hasn’t happened all that frequently. More then likely you have observed the opposite, that steps have been added (ok - I will admit that this one strikes close to home as I remember working to move through the CMMI ladder from Level 2 through Level 5 while I worked at Raytheon). The concept behind a defined SDLC process is fairly straightforward - to create some form of consistency in how software is built and shipped with a level of confidence and auditability while adding as little overhead as possible. What I have found in practice though is that is isn’t this overall goal that is debated, instead once an initial SDLC process is defined what I often see is that when a problem arises (say there was some bug shipped or if you live in a SaaS world something made it to production that caused in outage). At this point we often look at what step in the existing process may have failed and more often then not we attempt to plug the hole so that the same issue won’t occur again. While this may work well enough it dismisses the overall value that we were attempting to achieve and more specifically dismisses the “adding as little overhead as possible”. This is because it often leads to adding something without every thinking about removing something.

For this specific example I have more recently started to think about the distinction of choreography and orchestration. In other words, most of us have been trained to think of completing work in an assembly line fashion (Step A followed by Step B followed by Step C and so on). Instead what if there were not Steps but simply “Events” and those events were triggered when a specific set of criteria were met? In this world there is less of a Software Development Lifecycle but rather Software Development Events where a given Event occurred based on a form of matching function that considered the set of Inputs and a desired Output.

So, instead of assuming all code needed a code review instead we could say code is reviewed only when some measure of complexity is greater then a defined range AND the code in question doesn’t have previously reviewed test coverage AND we want an output of the code getting committed into the mainline branch. The event, a code review, would be the Event that matches the set of conditions and is thus triggered. Thinking in this way focuses on the value added. It also doesn’t presume that the event only occur in some defined sequence but instead creates focus on Events occurring whenever needed.

Wrapping things up

I personally believe that there is ample opportunity for innovation. I worry that often engineers conflate innovation with working on some bleeding edge technology when instead I would argue that engineers need to focus on identifying the correct problems to be solved. I wish more universities would help educate along these lines (don’t always focus on student writing more code or using the latest techniques but also given them the tools to be inquisitive). My first degree was in computer science and it perhaps stuck with me that I was a scientist first. I hope that some of these examples resonated and that perhaps now you too see the value in the pursuit of understanding the problem first and only then looking towards solutions.