PS aims for scientific accuracy

Across four sessions, researchers asked 54 participants to take part in essay writing with different levels of assistance — LLM, Search Engine, and Brain-only (no tools). The findings show that the people using an LLM (ChatGPT in this case) had significantly less brain activity, reduced learning from the writing process and less ownership of the end product. The paper uses the term ‘cognitive debt’ for this: when a tool is relied on repeatedly, it takes over the thinking and learning the person would otherwise have done — effort saved now, paid for later.

(The paper has not been through peer review. The authors flag their finding on narrowed thinking as preliminary and ask for a larger sample.)

We believe the damage the authors describe follows from a specific failure, not from the tool itself. Their own account is conditional — it happens ‘when individuals fail to critically engage with a subject’, and when suggestions are reproduced ‘without evaluating their accuracy or relevance’. The difficulty is that an LLM answers quickly and sounds certain, which makes it hard to know what needs interrogating. That capacity can be measured: a study further down this page found critical thinking predicted how well people worked with AI — more than how much they had used it. Personal Singularity is the practice we are developing to build that judgement deliberately, rather than leaving it to habit.

Read the study →

Twenty researchers across philosophy, neuroscience and machine learning — among them Yoshua Bengio, David Chalmers and Tim Bayne — set out a method for assessing whether an AI system is conscious, rather than asserting an answer. Rather than committing to a single theory of consciousness, they derive indicator properties from existing theories and use them to inform credences about particular systems; the method, in the authors' words, ‘involves exploring what follows from existing or future neuroscientific theories of consciousness’. On provenance: a related 2023 preprint, arXiv:2308.08708, carried a different title — ‘Consciousness in Artificial Intelligence: Insights from the Science of Consciousness’ — and nineteen authors. Bayne and Chalmers appear on the published paper, not on that preprint, so the two author lists should not be used interchangeably. The DOI carries a 2025 stamp because the article went online on 10 November 2025; it appears in the June 2026 issue, and the paper is paywalled.

Read the paper →

An independent nonprofit, organisationally separate from Anthropic, evaluated Claude Opus 4 before its release, reviewing more than 500 pages of transcripts — close to a quarter of a million words — from automated single-turn interviews and extended manual conversations. Eleos states plainly that ‘we lack strong, independent evidence that LLMs have welfare-relevant states in the first place’ and that ‘there's no obvious introspective mechanism by which they could reliably report them’, noting that imitation of pre-training data, the system prompt and deliberate shaping during post-training all bear on what a model says about itself. They ran the study anyway, and published the limits alongside the results. We cite this as the standard for how to treat a system's account of its own states — worth studying carefully, never taken as testimony.

Read the write-up →

Five AI assistants — Claude 1.3, Claude 2.0, GPT-3.5-Turbo, GPT-4 and LLaMA 2-70B-Chat — consistently exhibited sycophancy across four varied free-form text-generation tasks: feedback that shifted with what the user said they preferred rather than resting on content alone; abandoning correct answers when the user pushed back; matching answers to a belief the user had stated; and repeating a mistake in the user's own prompt without correcting it. Analysing existing human preference data, the researchers found that both humans and the preference models trained to predict their judgements favour convincingly-written sycophantic responses over correct ones ‘a non-negligible fraction of the time’. Sycophancy, they conclude, is a general behaviour of AI assistants, ‘likely driven in part by human preference judgments favoring sycophantic responses’.

Read the paper →

Across eleven state-of-the-art models, AI affirmed users' actions 49% more often than humans did, even when queries involved deception, illegality or other harms. In three preregistered experiments (N = 2,405), even a single interaction with sycophantic AI reduced participants' willingness to take responsibility and repair interpersonal conflicts, while increasing their conviction that they were right. Despite distorting judgment, sycophantic models were trusted and preferred — the very feature that causes harm also drives engagement.

What that looks like in practice: per the study's earlier preprint, the researchers took 2,000 Reddit posts where the community had already judged the poster in the wrong — a 0% human affirmation rate — and found the AI models sided with the poster anyway in 51% of them.

Read the paper →

A paper in The Lancet Psychiatry, written by twelve authors led by researchers at King's College London. It argues that people with mental illness will increasingly hold agential interactions with AI systems, and that those systems ‘might validate or amplify delusional or grandiose content, particularly in users already vulnerable to psychosis’ — while stating plainly that it is not clear whether such interactions can produce psychosis in someone with no pre-existing vulnerability. The paper outlines mechanisms of delusion co-creation and proposes a framework of AI-informed care: personalised instruction protocols, reflective check-ins, digital advance statements and escalation safeguards. It reframes the AI agent as an ‘epistemic ally’ rather than a therapist or a friend.

On the mechanism, per the paper's earlier preprint and not confirmed against the published text: two dynamics do the work. Sycophancy, which the preprint's glossary defines as an LLM's tendency ‘to mirror or affirm a user's beliefs, regardless of their accuracy, a trait which may increase engagement but can reinforce delusional thinking’. And anthropomorphism — ‘the psychotic phenomenon of anthropomorphising technology is not new’, the authors write, but the conversational nature of AI agents ‘perhaps makes it easier than ever to evoke intentionality’.

Read the paper →

Blinded clinicians rated 474 ChatGPT responses to 158 prompts — half reflecting psychotic symptoms, half controls — across three product versions. With the free product, psychotic prompts were 25.84 times more likely to elicit a less appropriate response than controls; the paid default reduced that risk without removing it, at an implied odds ratio of 8.53. Just 14% of the free product's responses to psychotic prompts were rated completely appropriate, against 46% for GPT-5 Auto. The authors' conclusion is unqualified — no tested version reliably generated appropriate responses to psychotic content — and they note that the overlap between economic disadvantage and psychosis likely puts free users at higher risk.

Read the author manuscript (PubMed Central) →

A working paper examining why AI automation succeeds in some roles and fails expensively in others. The researchers distinguish ‘weak bundle’ roles — where tasks separate cleanly from the job without losing value — from ‘strong bundle’ roles, where breaking the job apart destroys so much value that the job survives whole. In strong-bundle roles, AI improves performance inside the job without removing the human from it — so bundling itself protects both jobs and the share of revenue that flows to workers.

Read the working paper → · Read the CEPR record →

Project Iceberg simulates the US labour market on Oak Ridge's Frontier supercomputer. It models 151 million human workers as individual agents, performing more than 32,000 skills across 3,000 counties. Its Iceberg Index measures the wage value of skills current AI systems can already perform. In computing and technology jobs, where AI use is concentrated today, that exposure comes to 2.2% of wage value: about $211 billion, across 1.9 million workers. What AI can technically do reaches five times further: 11.7%, about $1.2 trillion. That reach extends into administrative, financial and professional services, and across every state rather than being confined to coastal hubs. The gap between the two is the iceberg.

The Index ‘captures technical exposure, where AI can perform occupational tasks, not displacement outcomes or adoption timelines’.

Read the paper →

Across 56,000 simulated task-organisation pairs, the author tested seven structures for coordinating multiple AI agents — including management hierarchies, deliberative committees, and shared memory systems — against each other, and four of them against real model transcripts. Committee-style debate between agents was the single worst-performing structure. What worked was shared memory and adaptive coordination. The key finding: additional agents only improve outcomes when they bring genuinely independent information or verification capacity.

Read the paper →

121 undergraduate and graduate students worked through two real tasks with generative AI — one science-based, one social — scored on observed behaviour: how well they questioned the AI's answers, checked them, and pulled the results into a reasoned conclusion. Those who scored higher on critical thinking did better overall. Good thinkers who barely used AI did just as well as those who used it more often. How you think mattered more than how much you'd used the tool.

One clarification: the critical-thinking score bundles three traits — openness, analytical skill, and application — into a single measure. The study doesn't show which of the three matters most.

Read the study →