TECHNOLOGY × LANGUAGE

Large Language Models

2022 CE

History · Contemporary · Holocene · Meghalayan

Large language models, which ChatGPT brought into common use in November 2022, are neural networks trained on vast quantities of text to predict the next word.

Large language models are neural networks of the architecture known as the transformer, trained to predict the next word of a sequence of words. The model estimates, from very large quantities of text, what follows a given sequence, produces a word, adds it to the sequence and begins again. The capacity for conversation is not an added function: it results from this repeated prediction, which explains both the fluency of the output and its aptitude for asserting errors with assurance, a phenomenon designated in the discipline as hallucination. The opening to the public of the conversational agent ChatGPT in November 2022 made these systems of common use.

Why it matters

The measured effects followed within months. A randomised experiment conducted on 453 professionals and published by Science recorded a 40 per cent reduction in time spent on writing tasks and an 18 per cent improvement in their quality, the gap between the highest- and lowest-performing participants narrowing. A study in PNAS found that the choices of one such model in the classical behavioural games, bearing on trust, fairness and cooperation, were not statistically distinguishable from those of a randomly drawn human being, with an altruistic bias. Alan Turing proposed this criterion of indistinguishability in 1950, but for conversation and not for games.

Dating & uncertainty

The opening to the public dates from November 2022. Economic assessments of the share of occupations affected are under way and their results diverge.

Our species’ history as one day

23:59:581.1 s before midnight

if the 315,000 years of Homo sapiens were compressed into a single day