Why AI systems hallucinate facts and figures

It is built on sophisticated neural network architectures, trained on vast datasets and can perform various natural language processing tasks, such as language translation, text completion, and answering questions. LLMs can comprehend context, infer meanings, and generate coherent and contextually relevant text based on the input it receives.

In the absence of a concrete understanding of every learned parameter, the models may occasionally generate seemingly plausible information that is, in fact, a fabrication. This phenomenon, known as hallucination, occurs when the LLM extrapolates from its training data, creating information that appears accurate but lacks a factual basis.

LLMs are built on a type of neural network called a transformer model (Figure 2) and there is no way of working out exactly how these models arrive at specific conclusions. In other words, the output of an LLM is considered to be ‘Non-deterministic’.

This means that from the output, it is not possible to determine the input, meaning that detection of AI generated content can only be evaluated based on a margin of confidence, rather than a certain ‘true/false’ evaluation.

Hallucination in LLMs can be attributed, in part, to the challenges associated with overfitting. Overfitting occurs when a model becomes too closely tailored to its training data. As a result, the model may hallucinate information that aligns with the peculiarities of the training dataset.

For example, If the machine learning model was trained on a data set that contained mostly photos showing dogs outside in parks, it may learn to use grass as a feature for classification and may not recognise a dog inside a room. When faced with novel scenarios or inputs, LLMs may resort to generating responses based on superficial similarities to the learned data, leading to the production of inaccurate or hallucinated information.

The implications of LLMs hallucination extend beyond technical challenges, delving into ethical territory. As these systems become integral to decision-making processes in various fields, from healthcare to finance, the potential for disseminating misinformation raises concerns.

When an LLM hallucinates facts or figures, it may inadvertently contribute to the spread of false information (Figure 3), with consequences ranging from misinformation in news articles to inaccuracies in critical decision-making processes. 

Addressing the issue of hallucination requires a concerted effort to enhance the reference-ability of LLMs. Researchers are exploring methods to make neural networks more interpretable, allowing stakeholders to trace the decision-making processes of these systems. Additionally, accountability measures must be implemented to ensure responsible development and deployment of LLMs.

The road ahead involves refining algorithms, establishing robust evaluation frameworks, and fostering interdisciplinary collaboration to create LLMs that not only perform functionally, but also uphold ethical standards. As we navigate the evolving landscape of LLMs, a deeper understanding of hallucination paves the way for more transparent, accountable, and reliable artificial intelligence systems.

AI is a rapidly growing technology area in the UK and the UK government has recognised the shift in R&D more towards cloud-based data driven solutions by including qualifying expenditure on cloud computing and data licences in the categories of qualifying expenditure under the R&D schemes for accounting periods beginning on or after 1 April 2023.

UK Companies carrying out R&D can claim up to 15% credits (up to 16.2% for accounting periods beginning on or after 1 April 2024) in respect of qualifying R&D expenditure, across categories such as staffing costs, cloud computing, data licences, software, consumables, utilities, workers employed by third parties, and in some cases subcontracted R&D under the RDEC scheme.

HMRC have increased their compliance measures over the last few years, requiring more information to be submitted in support of claims and increasing the number of compliance checks it carries out. It is therefore now even more important than ever to have robust and supportable claims. KPMG, with experienced software developers and engineers, are well placed to assist with putting such R&D claims together.