Will Artificial Intelligence (AI) do customer research for me? Some. Will AI summarise the results for me? Yes. Are there any errors in the output? Probably. This is the dilemma facing many companies when toying with using AI. Yes, AI can create a survey for you. Yes, AI can analyse a pile of data that you give to it. Double yes to AI can summarise the findings in a report. Hmm, that’s all great and many are trialling, vetting and correcting the output of various AI tools. For this article, we are primarily focusing on customer research and the tools that users are likely to use in the workplace. So, let’s discuss customer research using AI for transcription and summarisation.
Customer research
Customer research is the study of the needs, preferences and attitudes of existing, lapsed or prospective customers using first-party, primary data collection techniques, observation and analysis to inform commercial decisions. Broadly speaking, we seek to understand why customers buy, why they don’t buy, where they buy, how they buy and their experiences. This may involve interviews, surveys, focus groups, observations or online studies.
Customer research using AI
Breaking this down further, AI can create a simple set of survey questions for you. AI can summarise the findings of the surveys into a report. Additionally, AI can transcribe audio feedback and summarise into a report. It can also collect data on interactions, depending on the research method used. So far, so good. As an employee at a research firm, a marketing agency or simply selling to consumers, this may turn a huge manual task into a faster one. Typically, much of this type of work is handed to junior employees to conduct the survey, telephone for interview or watch a focus group. Similarly, the report is a summary of the information collected, which AI is ideally suited for.
Using AI for transcription
In a typical scenario, an employee might be tempted to conduct an online interview using readily available corporate tools, particularly Microsoft Teams, Google Meet or Zoom. Most people know that these tools can record video and audio as well as create transcripts of the conversation. Here is part 1 of the problem. Although we would like to rely on it, these transcripts of the chat contain errors. Microsoft Teams is around 80-85% accurate. Google Meet is around 85-92% accurate. Zoom comes in at 75-80%.
All claim to be much higher in ‘ideal’ voice scenarios with a single speaker, good sound quality, low background noise, no accent, in plain English. The trouble is that you may have multiple speakers, variable sound quality, background noise (especially in an office), accents (not just by country but by region) and corporate jargon. None of this is published (why would you?) but the worst-case scenario dips below 50% accuracy. It should be noted that some third-party tools may be more accurate, requiring video or audio upload to their platform. This may of course involve a cost, especially if done regularly. This has other drawbacks which we will mention later.
Using AI for summarisation
An enterprising employee realises that they can plough all of this information into an AI tool to summarise. Gone are the couple of days to produce a glossy report for bosses. Once upon a time, someone would wrestle with a publishing tool to produce something for professional printing. That time was removed by modern software, printers and the ubiquitous availability of display screens. Now, the time to produce the report is drastically reduced. Many turn to well-known tools such as ChatGPT, Microsoft Copilot or Google Gemini to help them with this.
ChatGPT 4.5 has an estimated error rate when summarising information at 19%. In some cases, with simple information it is about 2%. It is however telling that such summarisation is deemed inadequate for healthcare purposes and both medical and legal tests have demonstrated higher error rates above 30%. Microsoft Copilot is similar with an error rate around 17% in normal use and ideal scenarios with simple information at about 1%. However, it can also go as high as 34% for complex information in specialist domains. Can we leave it to Google Gemini 3 to knock it out of the park? No, not just yet. Although the best case is well under 1% with simple information, the error rate is just under 14% on average. An older Gemini model pushed up towards a 40% error rate in more complex summarisation and in specialist domain fields.
Drawbacks for customer researchers using AI
At the beginning, AI may struggle to capture the contextual nuances of what you are attempting to discover with your research, leading to generic and low value surveys and scripts. There is also the perception of your existing, lapsed or prospective customers if they work out that AI created the research itself. Does this suggest they don’t care? Maybe they don’t know me? Perhaps they cut corners elsewhere?
Once the data or transcripts are gathered, what tool is this uploaded into? Is this a secure, locally trained and hosted instance or a free, public version in the cloud? Even it is paid for, what are the terms of use and how is that data stored and used? Is the ‘anonymous’ research really anonymous if the information can be used elsewhere to train another model? Also, what if your customer accidentally divulged sensitive information about their business or yours? If uploaded to a public model, that information could be regurgitated in someone else’s search query, research or report.
One of the major drawbacks is the error rate. For technical fields with domain expertise, such as legal, engineering, medicine or microbiology, there may be less training information for a model to base reliable answers upon. In the example above with Microsoft Teams and Copilot, with an 15% transcription error rate and a 17% summarisation error rate, we are 70% accurate in the final report. If the employee captured some of those errors, it could be 80% accurate and this is information that you want to use to refine products, marketing, experience, segmentation, pricing or strategy upon. That is a 1 in 5 chance that your strategic decision is incorrect – especially troubling if it requires investment. This also assumes that the research method is informed and the researcher knows what they are doing.
Informed customer research using AI or experts?
There are many ways to create, carry out, analyse and report on customer research. We would caution that the true power of such research is in the accuracy of feedback and the strategic choices that it enables. If that accuracy is compromised, it leads to the wrong decision. If that information is shared publicly, your advantage is nullified. Sure, it can save you time and money, but not all participants may be impressed by the approach you chose. Similarly, some legal cases have begun to appear, particularly involving advisory firms producing erroneous reports generated by AI. You have been warned.
If you would like to interact with humans to deal with other humans, then reach out to us and arrange an initial introduction. Alternatively, you can simply email the team.
Finally, why not read a related article about making a return on your AI initiative.