According to a study paper by MIT, 95% of organisations implementing Generative AI have seen zero return on investment (ROI). Furthermore, they suggest that about 85% of projects will fall flat on their face for a variety of reasons. Since ChatGPT was unleashed on an unsuspecting public on 30th November 2022, everyone is building AI into their business model. Applications range from those seeking greater efficiency to revenue-generation and cybersecurity. So, after what seems like an eternity, happy birthday to ChatGPT (3 years old on 30th November) and OpenAI (10 years old on 20th November). So, what are leaders doing implementing AI when only 10% of projects are predicted to ever make a return? Read on.
Generative AI
GenAI is a technology that creates new content by ‘learning’ from large datasets. This machine learning uses algorithms to identify patterns and classify data which the model can use to predict answers via human-like outputs. The more high-quality data a model is trained on, the better the artificial intelligence (AI) predictions. It is particularly useful for generating derivative content, prototyping and summarising.
Immediately, some uses become clear. A huge dataset of images could be used to create ‘original’ content. Similarly, a ‘conversation’ can be had with text prompts where the best answer is predicted based on data and questions trained into the model. In organisations, repetitive tasks could be made more efficient where such a model can be ‘trained’.
AI return on investment
Investing in AI is no different to investing in any other software or technology. There are costs and benefits with an attempt to predict the present value of future cashflows. Unfortunately, especially for finance professionals, calculating the net present value (NPV) of a relatively unknown technology has few comparisons. Although rigour may be applied to the business case preparation and scenario modelling carried out, the benefits are less well-understood. The market remains relatively in its infancy with a smorgasbord of models and companies offering AI solutions. It would be a challenge to pinpoint a market-leader in every segment as of today, with various models competing for greater performance and accuracy.
What we don’t know is how many AI investments have been rejected by organisations. We also do not know how many decisions were taken based on optimism or because competitors are doing it. We would advise that finance professionals apply the usual rigour to business cases and investment committees. However, we would also recommend applying three simple tests to any potential AI investment.
3 simple tests before implementing AI
We recommend, as a minimum, the following 3 simple tests before implementing AI (in no particular order and solely based on our experience):
- Review your data and systems strategy
- Check that you capture all of the data points required and have a plan to capture data that you do not;
- Assess the quality of the data that you already have (rubbish in, rubbish out) before making any decision;
- Overlay your systems roadmap, end-of-life (EOL), consider if you require middleware and any unused AI features of your systems.
- Document the processes within the scope of AI
- Check that current processes are replicable in a system and identify non-standard, ad hoc activities or ‘fuzzy’ problems;
- Accurately measure the volumes of activities and time taken and consider the impact on human roles;
- Overlay proven AI capabilities with identified processes to improve, clearly recognising the limitations and expected accuracy level.
- Benchmark AI performance and lifetime costs
- Check any available benchmarking data for comparable solutions and identify the accuracy of the GenAI models based on a limited dataset;
- Analyse the costs of any hardware, cloud computing capacity, licencing of models, IT development resource and any pay-as-you-go costs per query;
- Overlay any quantifiable benefits, reductions in headcount, increases in revenue any other tangible benefit.
Intangible benefits from implementing AI
Part of the difficulty in calculating a return on investment are the intangible benefits that may follow. Improvements to employee experience (EX), customer experience (CX), product development, cybersecurity and competitive strategy are, at first, hard to measure. Some benefits, such as to EX and engagement, may be a pipedream to some. Many an article has flowed following the coining of the term AI ‘workslop’. That is, dealing with the training of, complexity of and varying degrees of inaccuracy of GenAI when your superiors expect large efficiencies. Similarly, product development is only valuable if you choose the correct strategic position and manage to monetise it later. Cybersecurity is an unknown, though the board may sleep better at night knowing it has improved.
With the world currently wondering if AI is all a big bubble waiting to burst (a la the dotcom bubble, but multiple times larger), investment may be somewhat a leap of faith. As with any competitive race, it may be a while before the true winners emerge. Disruptive business models may appear, which owe credit to GenAI. New products and discoveries may change lives as a result of AI. We may eventually migrate towards sophisticated software that can automate, analyse, classify and predict the next steps with little human input. At this time, unless used in a trial or in limited circumstances, we would advise to treat AI like any large change programme or system implementation.
Moving forward with AI and digital transformation
Here at Think Beyond, we work with businesses to solve challenges, find new growth and accelerate performance. That also means that we support you with good stewardship, careful investment decisions and thorough change programmes. A sizeable implementation of AI requires people, process and systems change. So, before jumping on the AI bandwagon, let’s make sure that your business is change and GenAI ready.
If you would like to speak to a consultant, there are several options to reach us.
Alternatively, why not check out our transformative service offerings.
Finally, why not read a related article about the challenges of system change.