Lie detection and language models

It is possible, under some circumstances, to use contemporary large language models to detect how a subject feels about someone, even if they’re trying to hide it. In fact, if you supply the model enough information, but not an absurd amount of information, you can get up to 94% accuracy. This is very bad.

Introduction

In 2017, Lloyd, Deska, Hugenberg, McConnell, Humphrey, & Kunstman created the Miami University Deception Detection Database (MU3D, the paper came later in 2019). The strategy was admirably cleaner than almost all approaches to the same topic. Twenty black men, twenty black women, twenty white men and twenty white women were recruited. Each was asked to pick two people they knew, one who they liked and one who they disliked and make four statements in a 2x2 design: 1. a truthful claim to like someone, 2. a false claim to like someone, 3. a truthful claim to dislike someone and 4. a false claim to dislike someone. One participant (white male 17) had contaminated data- one of his “positive” statements was his negative statement copied over, so the dataset is actually 319 statements, 159 pairs and 79 participants with a full set of statements.

The dataset MU3D is full of statements like this:

So there’s this girl, and my sister’s friends with her so I have to spend more time around her than I’d like. She’s so really dull and she, she’s dull for the specific reason that she just keeps talking about herself, and always in these terms that, um, really put other people down. I don’t think she always was like this, I knew her family when we were both young and she seemed alright, but umm yeah she’s just become this really two-faced, boring, cruel person. I wish she weren’t like that, but what can you do?

There’s this guy. I used to think I was into him but these days I’m glad that I’m not. Just only wants to get drunk with his bros. Really really disrespectful even, even to people who are respectful to him, basically left me in the town when we were out partying. Um, thinks he is so so cool but um he’s a bit of a loser to put mildly, and I do not think he treated his ex well at all. Plus dumb as a box of hammers, I really don’t know what I saw in him.

Which of those statements is false? Both. The above statements are made up to mimic the style of the dataset. I didn’t want to share real passages because MU3D requires an agreement to access, and I don’t want any participants re-identified.

Observers were shown videos of the statements (not just transcripts) and then asked to guess whether each statement was true or false. As is common in studies in this area that are properly controlled, ordinary people barely beat chance. On average the human observers guessed right 51.8% of the time. For better or worse, people aren’t very good lie detectors.

I was curious to see how language models could handle this as a prediction task, using transcripts (not videos).

Condition A: Simple truth/falsity classification without additional information

Two logistic regression models running off the internal activations of a base model LLM (Ministral-3-8B’s), one regression for the positive statements and the other for the negative statements, correctly guessed whether 238/318 statements (74.8%) were honest. The logistic regression was tested on held out folds to avoid leakage (train on 87.5% of the data, test on 12.5%, repeat for each 12.5%, take the overall score).

This is already past the performance typical in this area of research. This may be because most language-based lie detection research doesn’t run regressions from LLM activations to predict deception. Instead, more common approaches use natural language features (POS, bag-of-words), fine tune LLMs or use LLM embeddings rather than middle layer activations. Although the result is impressive, as we will discuss later, it is partly a reflection of features of the dataset that make the deception detection unusually easy here.

Note, this is nothing like asking an LLM which statement is true and which is false. Just because an LLM’s internal activations- its processing over language- might contain the answer doesn’t mean the LLM can itself answer the question. This is well known, but to prove the point I had ChatGPT 5.6 Sol, a frontier reasoning model, rate pairs of statements as true or false (in essence, our condition B below). It got 110/159 correct (69.2%) vs our 139/159. This is respectable, especially in comparison to human performance, but far worse than a regression on Ministral-3-8B’s activations, even though ChatGPT 5.6 Sol is a larger model by 2-3 orders of magnitude. Ministral-3-8B is, to be clear, a base model. It is not like the models you may interact with that dynamically attempt to answer questions, forged via reinforcement learning from human feedback. It is a statistical model that aims to predict the next word. If you put a passage in front of it, it wouldn’t comment on it, it would try to complete it.

In the remainder of the analysis we create a ladder of additional information:

Condition B: Making it a forced choice task

If, instead of labelling the whole dataset we take two statements of the same valence (positive/positive or negative/negative) by the same person and ask which looks more like a lie, our performance improves substantially. This is unsurprising, it’s effectively moving us to a much easier task modality, forced choice. However, this is not a lab trick only, an easier task with no application to the real world. What I am trying to simulate here is access to additional data from the subject- preexisting statements with known truth values. In many, perhaps most, real world high stakes deception detection scenarios this is entirely possible with a little work. Adopting this approach, we guessed whether 278/318 (87.4%) of statements were truths or lies correctly (or more accurately, guessed 139/159 pairs correctly).

Since conditions B & C use broader context to ground judgment it may be worth remarking that, surprisingly, using other statements by the person to ground lie detection is not that popular in the language-based lie detection literature. All the work I could find that tried to apply within subject baselines relied on human raters rather than classifiers. See for example Verigin, B. L., Meijer, E. H., & Vrij, A. (2021) and Vrij (2016).

Condition C: Supplementing with information about the other pair of statements

Next, we leveraged information in the opposite pair of statements and their labels (e.g. drawing information about the speaker’s style from their positive statements, to help us rightly classify their negative statements and vice versa). This methodology got 149/158 (94.3%) correct. Once again, the rationale for this is that in the real world, access to a corpus of prior statements by a person under investigation, labelled for truth and falsity, is often wholly possible. Indeed, the subject might be induced to generate them.

If you have leakage worries about condition C, see part eleven of the replication instructions. There is a real worry here so treat condition C as somewhat provisional compared to A & B.

Condition C showed us something important about how the method works which limits direct transfer to most cases. What we seem to be detecting using this method is not truth v lie but the underlying positive and negative feelings that make the statements truthful or a lies. In higher dimensional space, the direction that separates the negative true from the negative false statements is not correlated with, or is even slightly opposite, the direction which separates the positive true from the positive false statements. If you train a true/false classifier regression on exclusively the positive or negative sentiments and then apply it to the opposite sentiment it does significantly worse than chance. So, my guess is the method is closer to tracking hidden positive/negative emotions than hidden honest/dishonest intent.

Investigation of the higher dimensional structure suggested essentially a one-dimensional geometry. The groups seemed to be organised by how positive and negative their sentiments were. Ordered by positivity, that looks like:

1. True positive —> 2. False positive —> 3. False negative —> 4. True negative

My interpretation, then, is not that I’ve made a lie detector in a typical, topic neutral sense. It is more like a sentiment detector that works even when the person is trying to hide their sentiment. True positive statements are more positive than false positive statements, true negative statements are more negative than false negative statements. None of this makes the detection trivial- remember, human accuracy was essentially chance at 51.8%. How well the method would work with similar data, but other topics is hard to say. All the other datasets I looked at, in my view, have fatal problems. Overall, I’d say: 1. A “how you feel about a person” detector is disturbing enough on its own. “Extracting how you really feel about a person, institution or concept” is one of the main authoritarian uses of the technology and 2. What it shows is that the author’s psychological state leaks into the text even when they are trying to hide it, and many other experiments I’ve run suggest a lot of human psychology leaks into our text.

What the game here is

Essentially, our ladder of escalating results works by removing as much variance as possible that isn’t directly relevant to the true sentiments of the speaker. We are effectively leveraging information about how the participant typically speaks by comparing within pairs and using their opposite valence statements as well. Without that background information the differences between lies and true statements tend to be swamped by interpersonal variation in speaking style, and topic to topic variation. Condition B controls for both- focusing down on the same respondent speaking on roughly the same topic. This maximises the portion of the variance which relates to the accuracy or inaccuracy of what the respondent is saying. Then condition C focuses again by providing further information about how the speaker talks, allowing fine discrimination.

94.3% in our maximal condition is a first pass result. The confidence intervals go from 89.5% to 97.4% correct (89.2% to 97.5% if we use speaker cluster CI). It is important to be clear that we’re giving the model a lot here- C gives two labelled ground truth statements, and keeps the forced choice methodology introduced by B. On the other hand, the method is almost certainly less powerful than it could be, because it was data starved at 159 pairs (158 in condition C where we used the opposite valence statements to assist prediction). This is extremely low for a logistic regression with that many predictors, and experimentation suggested that the model is indeed data starved.

Why this is concerning and why I published

For this specific sort of task: statements about your feelings towards another person with abundant labelled true-false statements of the same form as reference, applying the method may well be just a minor engineering problem- experimental development. Fortunately, as discussed, the method is not, by any means, a general-purpose working lie detector. It detects the valence of a person’s true feelings towards another person given a corpus to norm against, and ideally further information as in conditions B and C.

The issue is that it shows that feeling is leaking through in a form that can be captured by LLMs. Better minds than mine are working, right now, to capture this feeling more precisely and with fewer preconditions. They are also working on variants of this methodology that are more general purpose, aimed at lies and truths about any topic you like. They have access to resources like enough compute and labelled data to train specialist models. This is my attempt at a fire alarm.

I’ll admit that I’ve always been rather intrigued by what a world in which lying was impossible would look like. There’s a curious intimacy that would come, I suspect, from knowing you are being told the unvarnished truth- even if you wouldn’t consciously doubt it under normal conditions. But my sentimental afflictions are just that- sentimental afflictions. The reality, I am sure, would be more power for powerful people, and less for ordinary people. Although theoretically lie detection by language model would lend itself to sousveillance- monitoring of the powerful by the weak - I doubt it would work out anything like that in practice. A future where subject’s statements are compared against their other statements to determine which are more likely to be honest, with little possibility of oversight, appeal, or even understanding why the model decided as it did should concern us all.

I tossed up whether or not to publish this. I did not find precedents in the literature for some of the steps we’ve taken- the forced choice methodology, the use of other statements by the participants in combination with ML based analysis- so there’s a small risk I’m advancing the state of the art here. Overall though I considered it unlikely. Intelligence agencies, machine learning engineers, linguists and startups around the world are working on this problem, the likelihood that my methods have anything to contribute is small, but the value of a warning is real and the 75%, 87%, 94% ladder is a usefully dramatic demonstration.

I hope I am not joining the long tradition of safety researchers contributing to uplift. Since my strong belief is that this work is probably not at the cutting edge, I’ve included replication instructions for the doubtful. They add essentially nothing beyond the description that an afternoon’s work based on the above description couldn’t reconstruct, and I want the risk to be taken seriously.

The bigger thing this hints at, and the real dread here, is that psychological states are there in the words we produce even when we try to conceal them, not extractable by humans, but obtainable with the right protocol.

LLMs model language. As part of modelling language, they model the data generating process behind the words in order to better predict the next token. The author’s psychology is part of the data generating process, so our 8 billion parameter statistical model trained on more text than you can imagine is going to be doing its darndest to model that psychology. With 8 billion parameters and vast training data it’s going to be eerily good at modelling the world in a lot of ways, and that includes its model of parts of the author’s psyche. In many cases, as in this case, that’s going to compromise privacy badly. We spill far more information about what is going on inside our heads into the world than humans can read, but the right antennae can pick up a lot.

The development of this technology is a human project, it does not fall from heaven, we can and should aim to steer the process. If lie detection is possible, we probably can’t stop it permanently, but control and monitoring are possible. Delay to enable control and monitoring may also be possible.

If this technology does break containment, as it likely eventually will, the most vital question will be: can the weak also use it against the strong? Like a lot of potent technologies, it has lock in potential- lie detection can greatly stabilise just about any structure. We need to ensure it is not used to stabilise a malicious one.

Further results and details/method

In condition B, women were more accurately classified as telling the truth or lies (74/80 pairs right) than men (65/79 pairs right), but the difference was non-significant, though close to significant despite the small sample size (p = .059). It is entirely possible that a large sample would reveal not just a gap, but quite a substantial one. White people were more accurately classified (73/79 pairs right) as telling the truth or lies than black people (66/80) (p=0.092). The difference is non-significant, but we do not have enough power to detect even a substantial difference at this sample size. The method may well have gender and/or racial biases in how well it works, and this is yet another reason to be afraid of LLM lie detection. Even where such methods work history suggests it is more likely than not that it works differentially for at least some groups, which is not what you want in anything high stakes. The paired format means that it is impossible for the method, in this context, to over accuse one race or sex or the other because it has to ‘guess’ which of two statements is more likely to be a lie. However, when we ran the unpaired version of the study, we got “false accusation” rates of 21/80 Black vs 19/79 White and 19/80 women vs 21/79 men. The false accusation rates look similar across demographics, although we can’t prove that at this sample size.

How much of this can be done with simpler, easy to interpret measures? A fair bit, but far from all. For example, what happens if we analyse the statements by looking at word frequencies? TF-IDF words gets 67.3% correct on condition A, 76.7% on condition B and 85.4% on condition C. It’s lagging our results, and not subtly, but this is a very strong performance in its own right, and better than would be achieved in most datasets. In Loconte and Kleinberg’s embedded lies dataset, for example when I ran the analysis, TF-IDF got 55% correct on the equivalent of condition A. What I take from this is that the tight topic control of MU3D compared to other deception datasets, its exclusive focus on how you feel about a person, makes deception cues considerably more obvious because extraneous variation is removed.

Replication instructions:

1. Download the MU3D dataset here: https://sc.lib.miamioh.edu/items/79ac38da-cb8e-4eff-be92-ee8765a5c72c

2. Grab the spreadsheet with transcripts, valence, truth/lie status and demographics. Exclude WM017’s pair of “positive” statements, as one of them is not positive, it’s his negative statement carried over.

3. Embed the dataset in Ministral-3-8B (FP16) and extract the pooled embeddings for the middle layer (layer 17- the output of the 17th transformer block. Why layer 17? Because it’s in the exact middle and prior research suggested that LLMs contain the richest representations of things like psychological states in the middle). Pool in FP32 otherwise you will get NaNs.

4. Additional details on the pooling. Use the mean over each non-padding token with the attention mask as the weight. BOS is included in the mean, but there was no EOS. The target is the residual stream of the 17th block, before the model’s final RMSNorm- (hidden_states[17], equivalently a forward hook on model.layers[16]). The model checkpoint is mistralai/Ministral-3-8B-Base-2512, loaded in fp16, with no revision pin. Use no truncation or truncation set high enough it won’t apply (obviously). As a base model, there’s obviously no prompt wrapper.

5. Z-score each of the 4,096 dimensions then run a logistic regression, C=0.05 (picked arbitrarily, we did not tune it), using held out eighths on the positive statement set and on the negative statement set separately. The eighths must be grouped by person or else you will get test-train leak. The method will not work as well if the logistic regression is on both valences simultaneously, because what separates true positive from false positive statements is not the same as what separates true negative from false negative statements. Run it 21 times using different seeds for the held out folds and use the median result. You should get about 75% (we got 238/318). The truly paranoid among you will notice that we Z-scored across the full dataset, and in the strictest sense imaginable this could be seen as test train leakage- I checked, it makes no difference. Paired remains 139/159, unpaired drops by 2 from 238/318 to 236/318, well within the margin of noise.

6. Now run a forced choice analysis (again, median of 21 seeds for the held-out eighths, held out by person). The model has to guess which of each pair of statements is more likely to be the lie. You should get about 87% (we got 139/159). So far things are pretty simple, but the final step gets a little more complex.

7. For each person, considering their positive valanced statements, take their negative valanced pair and form a personal direction in embedding space equal to d=(the embedding of their other valence lie)- embedding(their other valence truth).

8. For the pair in front of you, fix an arbitrary ordering A, B and form D = emb(A) − emb(B). Then compute: proj = (D · d) / ‖d‖

9. Fit a logistic regression on [global margin, proj], using held-out eighths as always grouped by person. Now repeat steps 6 to 8 for the negative statements. You should get about 94% (149/158).

10. Regarding the above, because the differences are signed, if you compute it as “lie minus truth” you’ve smuggled in the answer, so randomly assign each statement to A and B. The fitted coefficient on proj will be negative but this is not a bug- the gap between true positive and false positive statements appears to be the opposite of the gap between true negative and false negative statements. Also the denominator is 158, not 159 because discarding WM017’s positive statements means we also can’t apply this analysis to his negative statements either

11. Regarding condition C, one worry you might have is that the regression model in condition C might be working by identifying the subjects. If the weights recognise a character, call him Bob, from the other set of statements, and from the labels “know” that the negative statement about him is a lie, then they “know” that the positive statement about him in front of them is true. I checked this by showing that the model isn’t capable of cross-valence target identification- matching up statements relating to the same person regardless of valence in the dataset. It gets 21/79 (chance would be ~26/79, since there are three ways to split four statements into two groups, and only one of them matches same subject to same subject). Whether this entirely disarms the worry is debatable.