Is It Real or Is It Fake? Detecting Misuse of AI-generated Content

As technology advances, the refinement of Artificial Intelligence (AI-)generated content also progresses. From text to visual art and audio, AI has become very proficient at creating content. The current state of this technology, known as Generative AI (GenAI), already makes it difficult to distinguish between content created by humans and content generated by computers. Do you, for example, know whether the title of this blog is crafted by us or generated by ChatGPT? Although using AI for content generation can be harmless, there is already evidence of criminals exploiting AI-generated content. Recently, a Hong Kong finance worker was convinced to transfer $25M to criminals during a video call with deepfake representations of his colleagues, including the CFO (CNN 04 Feb 2024).

Before delving further into GenAI, it is important to understand how generative models differ from other types of models: discriminative models. A discriminative model makes predictions on unseen data based on probabilities. A classic example of a discriminative model involves predicting whether a movie review is positive or negative, based on the words and phrases contained within it. This is done without generating new content, in contrast to generative models like ChatGPT and DALL·E, which are capable of autonomously generating content, or data, by learning patterns and structures from training data.

AI-generated content offers several promising opportunities, such as increased efficiency in content production, tailored content aligned with individual preferences, and the development of personal (digital) assistants that are indistinguishable from humans or even exceed human capabilities. However, every rose has its thorn: while human creators often have a deeper understanding of the context in which content is created because of their personal experiences and perspectives, AI models (currently) do not have the ability to fully comprehend the context in which they create. The AI-generated content may therefore not be entirely suitable for the intended context, which can result in inaccuracies and reduced reliability.

Additionally, (Gen)AI models are trained on large datasets, which may inadvertently contain biases. This can result in the reflection of societal biases in AI-generated content, which may lead to issues of fairness and impartiality. Moreover, there is a grave development in the exploitation of GenAI by malicious actors for purposes such as fraud, phishing attacks or the spread of misinformation. This can be done, for instance, by using deepfake (AI) technology, which poses risks by allowing the creation of realistic but fabricated audio and video content, raising concerns about identity theft. As an example, 404 Media Investigation uncovered an underground website, known as OnlyFake, which was capable of producing convincing images of counterfeit IDs for a mere $15 using GenAI (404 Media Investigations, 5 Feb 2024).

The remainder of this blog introduces various techniques aimed at assisting you in identifying AI-generated content. The first method focuses on detecting AI-generated content as a human. However, while it is generally still possible for observant individuals to distinguish AI-generated content, it is important to note that the described methods are not guaranteed to be foolproof. The last section is centered around identifying AI-generated content using AI technology.

Source of content: If you suspect that content is AI-generated, you should verify the credibility of the source before considering the content as authentic. If it is from a new or unfamiliar source, especially one with limited or no previous activity, the likelihood of the content being AI-generated is higher. Lack of a source at all should also raise your suspicion: AI systems excel at creating new information without basing it on a (proper) source.

Contextual Understanding: AI-models may struggle with grasping the full context of a topic. Human-generated content often demonstrates a deeper understanding of the subject matter, weaving in personal experiences, emotions and cultural references. If the content lacks a genuine connection to the context, it may be a hint that it is AI-generated. A simple example of AI-generated content that you may recognize are images where the people depicted have anatomical distortions, such as an unusual number of limbs.

Coherence and Logic: Humans naturally create textual content with a logical flow and coherence. AI-models, however, may struggle with maintaining a seamless narrative. Evaluate the content for abrupt transitions or illogical connections. If the information seems to lack a coherent structure, it might be the result of AI content generation. On the other hand, inconsistency in spelling and spelling mistakes could be an indicator that a text was human-written, as such errors are common among humans, whereas an AI model would generally have proper spelling because of its programming.

One advantage of using AI is its speed; AI systems can create many pieces of content within mere seconds. Despite the evident AI-generated nature of certain content pieces, within the same generation time, several pieces of content may also have been generated that appear realistic and could fool people. This creates the need for technology, as well as legislation, that can detect whether something is real or fake at the same rate as its creation. Luckily, as the ease of creating fake content has progressed, so has research into digital forensic techniques for detecting such content. Recently, McAfee revealed their AI-powered Deepfake Audio Detection technology, which was developed to defend consumers against the threat of (cyber)criminals utilizing AI-generated audio (McAfee, 8 Jan 2024).

Techniques that can be used to detect AI-generated content include investigations into the soundness of the content’s digital history obtained from its metadata. Deep learning algorithms can also be used to analyze the legitimacy of images by looking for inconsistencies in an image. Parts of the image might, for example, appear as if they have been pasted in, showing a contrast with the rest of the image, or the shadows and lighting within the image may show unrealistic characteristics. Additionally, statistical analyses can be used to assess the authenticity of textual data: is the vocabulary consistent and applicable considering the context, are verbs in the same tense, and is a sentence correctly phrased in singular of plural? The truth is, however, that even for these kinds of advanced analyses and tools, it remains difficult to determine with certainty what is real and what is fake, as humans may also make mistakes, and an AI model could deliberately introduce errors. Therefore, a human expert in the loop remains necessary.

In contrast or in addition to these post hoc analyses, there are certain techniques that can be used pre-emptively to make the detection of AI-generated content easier. Blockchain and provenance tracking can be used to prove the authenticity of data. Blockchain keeps track of data transactions and uses cryptography to prevent modification/altering of data, while provenance tracking records metadata of the content.

As AI continues to integrate into our daily lives, the ability to detect AI-generated content becomes increasingly vital. AI’s rapid evolution makes it difficult for legislation to keep up. Humans should therefore always look critically at content to distinguish between what is real and what is fake.

KPMG Forensic provides services to help organizations prevent potential exploitation by (criminals utilizing) GenAI, through assistance with the establishment of clear governance and the evaluation and integration of compliance measures, such as measures to prevent the exploitation of intellectual property data by AI models. Additionally, we provide data protection and validation services that can help identify the origin of data and authenticate received information. Want to know more? Visit Forensic Services or contact us directly.