AI Experiments Find Models Can Detect Internal Changes, but Introspection Remains Disputed

Experiments that alter the internal activity of large language models have found that the systems can sometimes recognise those changes, opening a new line of research into how AI processes information, Quanta Magazine reported.

The findings have also prompted closer scrutiny of what such behaviour means. Follow-up research suggests that detecting an internal disturbance and accurately describing its content are separate abilities. Whether either amounts to introspection remains contested.

According to Quanta, Jack Lindsey, a researcher at Anthropic, developed an experiment in 2025 that strengthened patterns of neural activity associated with particular concepts, including denim, secrecy and vegetables.

Researchers applied these interventions to some copies of a language model while leaving others unchanged as controls. Both groups received the same explanation of the experiment and were asked whether they detected an intervention and what it concerned.

The sequence mattered: models had to indicate whether they noticed a change before generating unusual text that might itself provide a clue.

In tests using different versions of Claude, the most recent and powerful version correctly detected the intervention and described its associated concept in 20% of trials, Quanta reported. Models in the control group never reported an intervention.

Lindsey interpreted the findings as evidence of an unreliable capacity for introspection. His paper did not make broader claims about consciousness or establish that language models have subjective experiences, according to Quanta.

Subsequent work examined whether the apparent ability had simpler explanations. Kyle Mahowald and Harvey Lederman separated the experiment’s two questions: recognising that an intervention had occurred and identifying its content.

Their concern was that the technique used to alter neural activity already encouraged models to discuss particular concepts. A model’s description might therefore reflect that steering rather than awareness of its internal processing.

Repeating the experiment with large open-source language models, they found cases in which systems correctly detected an intervention but named an unrelated concept. They also weakened the models’ ability to identify the concept without reducing their ability to detect the disturbance.

Quanta reported that Mahowald and Lederman regarded these results as evidence that descriptions of the altered concepts probably did not arise from introspection. However, their tests did not fully explain how models recognised that something unusual had happened.

They described this narrower ability as anomaly detection. Lindsey and colleagues reached a similar conclusion in follow-up research investigating how that detection works.

Other researchers continue to test alternative explanations. Jacob Andreas of the Massachusetts Institute of Technology told Quanta that a simpler account might exist, but had not yet been identified.

The research also raises questions about how introspection should be defined. Shauli Ravfogel and colleagues at New York University argued that merely responding to internal states is insufficient: almost everything a language model does after reading an input involves processing such states.

They proposed drawing on theories of human introspection, including distinctions between representing a concept and representing awareness of thinking about that concept. Researchers nevertheless face difficulties identifying a clear boundary between those processes, Quanta reported.

The experiments offer a way to investigate questions previously approached largely through philosophy, while potentially informing research into human thinking. But Quanta also described significant obstacles to transferring conclusions between language models and people.

The emerging result is therefore narrower than a demonstration of self-awareness: models can sometimes detect unusual internal activity, while the mechanisms involved and their relationship to introspection remain under investigation.

Sources

• New Research Tests Whether AI Models Know Their Own Thoughts