Much has been made about the transformative impacts of AI on business, yet recent studies are revealing that the success rates of companies implementing AI so far are shockingly low.
One MIT study showed that 95% of companies have failed to gain significant value from investments in AI. BCG found that out of 1,250 firms studied, only 5% were achieving value at scale.
Creating successful AI products is not just the task of the engineering department. There should be leadership from the C-suite and buy-in across the organization.
Despite these results, investments in AI continue to be made with the hopes of large payoffs in the near future. Even most skeptics will agree that AI/ML technologies are powerful innovations carrying the potential to transform products and even industries.
The question becomes how organizations can capture AI’s economic value.
The economic value of innovation is captured when the incorporation of new technology is positively reflected on a company’s balance sheet. If a company releases new products or services that add no value to customers or operations, they shouldn’t expect any improvements in earnings.
On the other hand, if they do create new value for customers, there’s no guarantee they’ll retain economic value. It could just as easily flow back to customers in the form of reduced prices or to suppliers (in this case, the model and chip producers) in the form of increased operating costs.
Why are so many companies failing to capture value from AI? While the issues are varied, they generally come down to strategic, organizational and technical factors.
Strategic: Align AI with business value
At the strategic level, organizations must treat AI/ML as they would any other technology: its role is to provide measurable value to customers and internal operations. The pressures of today’s environment can cause business leaders to believe that if they just incorporate some AI feature into their offerings, they won’t be left behind or investors and board members will be kept happy.
While these motivations are understandable, they’re disconnected from the customer experience and larger business goals and therefore unlikely to lead to any economic value. Organizations should remain product-driven in the face of innovative pressures. This means starting from customer needs and working backwards to technological solutions, rather than incorporating new tech and seeing how it can be marketed to customers.
Once the customer needs and business goals have been fleshed out, key performance indicators should be defined and the model’s performance must be measured by them.
In other words, it’s not enough to be satisfied with technical metrics like confusion matrices or accuracy numbers. The model’s impact should be tied to critical business concerns, whether that be reducing user churn, increasing click-through rate, or boosting sales from recommended products.
Organizational: Get buy-in across departments
Focus and disciplined scoping are major factors in the outcomes of an AI project. A study by McKinsey found that companies that incorporated AI successfully had a clear understanding of the business domain they wanted to transform, as well as the scope of the transformation. After this, they allocated resources to drive high-impact change across the domain.
It’s critical to note that creating successful AI products is not just the task of the engineering department. There should be leadership from the C-suite and buy-in across the organization.
This could look like redefining existing workflows, upskilling or hiring talent and making sure there’s consensus and collaboration across departments such as product, marketing, business development and engineering. The connection between product and engineering is especially important.
ML engineers need to sustain high rates of experimentation, and the product team must validate that the model’s behavior is aligned with product goals, as well as update the engineers as product requirements change, which in turn can affect experiments.
Technical: Start with strong infrastructure
On the technical side, companies should be aware that models are only a small part of a much larger dynamic system. The parts of this system include large amounts of data and metadata, training and serving pipelines, cloud infrastructure and compute provisions. Bringing such a system to production and maintaining it is a hard task with many points of failure.
With this in mind, companies should plan for failure while building for reliability, visibility and continuous improvement. Planning for failure may sound odd, but discovering risks only after releasing a product is a causal factor of failed projects. Managers should review product plans and identify areas that pose the highest risk, note what stakeholders would be impacted and to what degree, then put plans in place to mitigate or eliminate these negative impacts.
Building for reliability pertains to the mechanisms that are in place to contain adverse effects without disrupting service to end users. For example, systems can employ heuristics or non-AI algorithms as failsafes that can be defaulted to while an errant model is being triaged; they can also allow for rollbacks to revert to a previous model.
Visibility into the systems can increase the speed at which fires are put out. Keeping track of model artifacts, metadata, as well as the inputs and outputs of the system, will help engineers and managers troubleshoot problems that can be quite complex.
As the real world changes, models and AI systems must be continuously improved.
Based on this, companies should assume that at some point in time, they will have to retrain their models and design their systems accordingly. Automatic retraining allows companies to proactively improve models, mitigate their risks and reduce the introduction of manual errors.
All things being equal, the ability to continuously improve systems at high velocity confers an advantage on companies, enabling them to quickly adjust incorrect assumptions or oversights from the initial design stage.
The value of innovation
Warren Buffett has a saying that the advantage gained from new technology is like standing on your tiptoes in a large crowd. So long as you’re the only one on your toes, you’ll see above the crowd. But by the time the others wise up and join you, the advantage is neutralized.
In the long run, capturing the economic value of AI will depend on how long competitive advantages can be sustained. It’s true that the automation of labor-intensive tasks may reduce costs and increase earnings. However, once these cost reductions are reflected across an industry due to mass adoption, competitive pressures are likely to reduce prices, causing value to slip away from the firm.
It’s an iron law of business that low barriers to imitation yield meager profit for firms. To hedge against this, companies should assess what barriers to imitation and entry they have or can create, then consider how AI can be used strategically to strengthen those barriers.
As the economist Michael Porter notes, competitive barriers typically arise from different but interconnected activities that make it easy for a company to do things that competitors find almost impossible. This can range from providing low-cost, high-quality products to offering fast and free returns or producing with economies of scale. Therefore, concepts of standalone AI initiatives should be reframed as smaller components within a larger system of activities.
The percentage of companies that will actually capture the economic value of AI over time remains to be seen and depends on a whole host of factors untouched by this article. However, managers can improve their chances by avoiding known pitfalls, monitoring model performance and ensuring that performance metrics are tied to clear-cut business goals.
