In 2011, Marc Andreessen famously coined the phrase “software is eating the world.” Software is everywhere, and for decades it has been a ubiquitous part of our existence as digital versions of our physical dwellings. Unlike physical engineering, which has matured over millennia, software engineering is still a relatively immature industry. Great strides have been made since the days of punch cards but it is still a highly manual, labor intensive effort.
In this two-part blog post, I will describe a mindset shift happening in the industry toward an automated future where software writes itself, starting with a new approach to software engineering that is still surprisingly under the radar. In Part 1, I put a historical context around this mindset shift, while Part 2 focuses on a new approach that will serve as a bridge to world when automated agents build software based on data as opposed to human algorithms.
Imagine the World…
To start, let’s imagine what the world was like for an average person in 1920. They probably had a radio in their home, but they likely will not get an automobile or telephone for another 5–10 years even though both were invented several years prior. Fast forward to 2001; although cell phones had only been widely available since the mid-90’s, the average person already owned one. Smart phones will not be invented for a few years, yet they will be in almost everyone’s pocket a few years later.
These examples demonstrate a trend we are all familiar with. Over the last century, we have seen a dramatic increase in pace of new technology adoption. In the early 20th century, it took several decades for mass adoption of technologies like telephones, electricity and automobiles. By the early 21st century, technologies like smart phones and social media achieved mass adoption within roughly 5 years. This acceleration of technology adoption is indicative of two key trends that will shape our future.
Exponential Change
The first trend is the exponential pace of change and our inability as humans to adapt. Our primitive brain function, molded by evolution over millions of years, is no longer equipped to absorb and react effectively to the dizzying change happening all around us. Adam Grant provides some great examples of this in his book, “Think Again.” In 2011 we consumed five times the amount of information per day than twenty-five years prior. In 1950, it took fifty years for knowledge in medicine to double. By 1980 it was down to every 7 years, and by 2021 it was down to every 3.5 years. In order to cope with this change, we’ve had to come up with new models and tools to help us adapt, including crowdsourcing and the use of collective intelligence from peer networks.
This pace of change has real world impacts. Consider the top 10 companies in the S&P 500 in 2010 versus today — there has been a 60% turnover in just ten years. The average lifespan of an S&P company in 1958 was 61 years. Today it is less than 20 years. Clearly companies are having as much trouble adapting as individuals.
How to Adapt?
The key word is adaptability. Companies, especially entrenched enterprises, have renewed their focus on adaptability to achieve faster business outcomes and keep pace. Here again, Adam Grant gives a great anecdote about this in “Think Again” by comparing Blackberry and Apple.
In 1999, Mike Lazaridis, an accomplished entrepreneur, launched Blackberry as a wireless communication device for emails. It was like the TikTok for Gen X, and it took off like wildfire. By 2009 it represented half of the US smartphone market. But Blackberry has become a cautionary tale in the annals of corporate lore. When the iPhone launched in 2007, Lazaridis believed people would only want a device for checking emails, not carry a computer in their pocket for entertainment (whoops). When his researchers wanted to add a browser to the Blackberry, Mike refused and said it would drain the battery. Furthermore, he could not imagine people wanting to give up their coveted keyboard for an iPhone.
In 2008, Blackberry’s valuation was $70 billion. By 2014 their market share was down to less than 1%. Why? Apple.
In many respects, Apple’s adaptability allowed them to emerge so successfully from bankruptcy. And contrary to popular myth, it was not Steve Job’s clarity of vision that led to the iPhone. The truth is he was dead set against it. In 2004, Apple researchers first pitched Jobs on turning an iPod into a phone but Jobs rejected it outright. He said it would cannibalize iPod sales, and furthermore, he believed cell phone carriers would never allow a phone to be created with a superior user experience.
It was the persistence of his researchers who persuaded him to adapt his thinking. They showed how Apple could still be a computer company that simply added a phone, not become a phone company with computing capabilities. The result? Four years after the launch of the iPhone, it became half of Apple’s revenue.
Again, adaptability is the key. If we measure adaptability by how much faster an organization can achieve their business outcomes, then companies can facilitate this speed by creating bespoke solutions, or custom technology that fits their unique or specific needs without forcing additional people or process changes (assuming people and processes are optimized). These solutions create a user experience that fit like a glove, either because they perfectly match an organization’s existing processes or because they are so simple to use that it requires little to no switching costs from current processes.
But bespoke solutions are very hard to create. Why? Cost.
Solving the Money Problem
Companies have used two traditional levers to manage the cost of creating bespoke solutions, each with inconsistent results. First, they looked at customizing packaged solutions. The thinking was sound — why build a solution from scratch when you can modify something pre-built? The reality was rocky; anyone who has been through a project to customize an enterprise package like Salesforce or an ERP system knows how hard this can be. Some packaged systems are difficult to customize, and there is always the upgrade challenge to deal with. The result is usually more painful than expected with a resulting system that falls somewhere between bespoke and vanilla, feeling diluted and unsatisfying.
The second lever involves offshoring / nearshoring. Again, the thinking was sound — if bespoke solutions are labor intensive, then let’s find the lowest cost labor to keep costs down. Of course, the challenges with this option are also well documented. Bespoke technology solutions are often a moving target, making it hard for remote resources to adapt and respond due to time zone and cultural differences.
So what is the answer to the cost equation when the traditional levers don’t work? Automation provides an enticing new option.
The Automation Equation
Let’s return to our “imagine a world…” exercise and ask ourselves a question: what if we lived in a world where software wrote itself based on the unique specifications of a bespoke solution and lots of data? What if this is a third option beyond low-cost labor and packaged solutions customization to achieve our adaptability goals? Is automation in software engineering really possible? We will dissect the answer to these questions in Part 2.
