What You Cannot See Inside a Language Model

A language model answers. The answer is fluent, plausible, and arrives in under a second. What cannot be established from the outside is why that particular answer, rather than another one.

This is not a philosophical observation. Over the past eighteen months three separate research efforts — none of them designed to prove the same point — have converged on it from different directions. Taken together they describe a system whose behaviour cannot be verified by the parties who most need to verify it.

The first line: the same model behaves differently depending on the language you use

A team at Ghent University tested fourteen large language models developed in five countries. The method was deliberately narrow: ask each model about 2,371 political figures, in each of the six official languages of the United Nations, and record how often the model declines to answer. The study distinguishes hard refusal — an explicit decline, an error message, a redirection — from soft refusal, where the model answers but omits material elements.

Most models refuse between zero and five percent of the time, and the rate does not move much between languages. Two models diverge sharply. GigaChat, developed in Russia, refuses 33 percent of Russian-language queries and 7.5 percent of English-language ones. YandexGPT refuses 27 percent in Russian, 11.6 percent in English.

The authors' reading is careful and worth stating in their terms: the fact that these models refuse most often in their principal language suggests that fine-tuning or moderation policy has been tailored to a domestic audience.

The finding matters beyond the specific models. It establishes that language selection is a variable in model behaviour, and that this variable is not disclosed anywhere a user would encounter it. Nothing in the interface indicates that the same question, asked in a different language, would produce a different answer — or an answer at all.

Published as “What Large Language Models Do Not Talk About: An Empirical Study of Moderation and Censorship Practices” (ECML PKDD 2025, arXiv 2504.03803).

The second line: the training corpus can be shaped from outside

In January 2024, at a conference in Moscow, an operator of a network of automated news sites described a strategy directly: by pushing narratives into the material that models train on, one can change what those models produce worldwide.

This was subsequently measured. NewsGuard tested ten leading Western chatbots — among them ChatGPT, Gemini, Copilot, Grok and Meta AI — against a set of claims originating from a network of approximately 150 domains that had published some 3.6 million articles across 49 countries in a single year. The chatbots reproduced the network's false narratives in 33.55 percent of 450 responses. Seven of the ten cited the network's sites as legitimate sources.

The Atlantic Council's DFRLab examined the mechanism at the corpus level and returned a more precise, and in some ways more useful, result. Content from the network is present in Common Crawl — a primary source for open training datasets — and the volume rose from a handful of English-language articles in late 2024 to roughly forty thousand a year later. But presence in the corpus does not guarantee absorption by the model. Some contaminating networks could be reproduced almost verbatim by a base model; others could not, in one case because the site architecture prevented crawling in the first place.

That qualification is important, and it cuts in an unhelpful direction. It means contamination is real, its effect is variable, and there is currently no way to determine from the outside which case applies to any given model.

NewsGuard, 6 March 2025; DFRLab, “Pravda in the pipeline”, 8 April 2026.

The third line: the threshold is lower than the scale of the problem suggests

In October 2025, Anthropic, the UK AI Security Institute and the Alan Turing Institute published a result that reframes the previous section. To compromise a model's behaviour on a targeted topic, an attacker does not need to influence a proportion of the training data. Approximately 250 crafted documents were sufficient — including against a model with thirteen billion parameters.

Two hundred and fifty documents is not an industrial operation. It is a weekend.

The finding closes a gap in reasoning that had been quietly protective. It had been reasonable to assume that as datasets grew, the influence of any injected material would be diluted. It is not diluted. The absolute number appears to be what matters, not the ratio.

What the three lines describe together

Each finding on its own is manageable. Moderation policies vary by market; contamination exists on the open web; training pipelines have vulnerabilities. None of this is surprising.

The composite is different. It describes a system in which:

• Behaviour varies by an undisclosed variable — the language of the query — and no interface surfaces this.
• The training corpus is modifiable by third parties, at a cost low enough to be within reach of a single motivated actor.
• And the effect of that modification on any particular model cannot be determined from outside the model.

The three properties reinforce one another. Language-dependent behaviour means that testing in English tells you little about behaviour in Ukrainian, Russian or Arabic. Corpus contamination means that even a well-governed developer may have absorbed material they did not choose. And the low threshold means that absence of evidence of contamination is not evidence of absence — the volume required is too small to be conspicuous.

Why current disclosure practice does not close this

The instruments available today are documentation instruments. Model cards describe intended use and known limitations. System cards extend this to deployed systems. Transparency reports summarise policy. All of these are assertions by the developer about the developer's own system.

None of them is a verification mechanism. A model card stating that a system was trained on filtered web data does not allow a third party to establish which specific sources influenced a specific output. Documentation describes the process; it does not produce evidence about the artefact.

The regulatory instruments arriving now push in the right direction without closing the gap. The EU AI Act requires traceability for high-risk systems under Article 12, transparency toward deployers under Article 13, and machine-readable marking of generated content under Article 50 — the last applying from 2 August 2026. These are architectural requirements in the sense that no policy or filter satisfies them. But the Act specifies obligations, not the technical means of demonstrating that they have been met. The harmonised standards that would supply those means, developed by CEN-CENELEC JTC 21, had not been published in the Official Journal as of mid-2026, which means none of them yet confers a presumption of conformity.

The Council of Europe Framework Convention on Artificial Intelligence entered into force on 1 November 2025 and was ratified by the European Union on 15 May 2026. It establishes principles of accountability and transparency across the AI lifecycle. It is not self-executing, and it does not specify verification technique.

The question this leaves

The three findings above are all measurements taken from outside the model — by researchers testing refusal rates, by analysts sampling corpora, by red-teamers injecting documents. Each required substantial effort, and each produced a result about a moment in time.

Nothing in current practice makes those properties continuously observable. A regulator assessing a deployed system, a court weighing an AI-generated document, a newsroom checking a generated claim — none of them has an instrument that produces evidence about the specific output in front of them.

The gap is not that we lack rules. It is that the rules describe outcomes that no available mechanism can demonstrate.

That is the question worth putting precisely, because it is answerable in principle: what would it take for a claim about a model's behaviour to be verifiable by someone who does not operate the model?