The Analytical
Bard Engine

Your words are punched upon a card at the clavier — a stroke to the column — and fed to the reader. The engine does not hear you — it receives marks.

In truth — a language model receives nothing but text: no voice, no view of the world, no notion of who is asking beyond what the words themselves carry. Everything that follows is determined by these characters and by the settings of the machine’s parts.

A note on the manner of address. To a chat engine you write whole questions; this cabinet asks instead for a few opening words to continue. There is no contradiction — a chat engine is a continuation engine too: the clerks lay the conversation out as a transcript (Operator: … Engine: —) and the machine simply continues the document, which lands, every time, precisely upon the engine’s reply. The instructions that govern its manners — the system prompt — are more cards still, read before yours. The clavier above the slot is no fancy, either: real cards were cut at just such a hand-punch, a stroke to the column — somebody must first turn words into holes.

The card is read and matched to slugs — standard pieces from the engine’s type-case. Each slug bears a catalogue number, and from here onward the engine handles numbers only.

In truth — text is broken into tokens from a fixed catalogue of roughly 50,000–200,000 pieces. Common words are one token; rarer words are cut into fragments (un·fathom·able). The machine never sees letters or words as you do — only these numbers.

This cabinet’s catalogue holds — slugs; a slug marked ✶ was cut fresh and is a stranger to it. The rack holds only 24 slugs — its context window. Push in more and the oldest fall off the far end, forgotten. (A modern engine’s rack holds 200,000 or more.)

Each catalogue number is looked up in the Great Ledger, and the slug is exchanged for a column of dial-settings — its position upon the map of meaning.

In truth — every token becomes an embedding: a list of thousands of numbers (this cabinet shows eight dials; a large engine uses more than ten thousand). Nobody wrote the ledger by hand — the settings were learned, and they place words used in similar ways at similar settings. Meaning, to the machine, is position.

Here is the celebrated mechanism. Every slug is joined to every earlier slug by an adjustable linkage; the engine draws some taut and lets others hang slack. A word discovers what it means by choosing what to regard.

In truth — this is attention, the invention (2017) that makes these engines possible. For each position the model computes how much every earlier token should bear upon it — how it finds its noun, how bank leans toward the river or the counting-house. Dozens of heads work in parallel, each minding a different sort of relation: shown here, a nickel head that favours the recent, a copper head that favours old acquaintances.

Depicted schematically — in the full engine these weights are computed afresh, at every layer, from the dial-settings themselves.

The columns now descend through the Mill — floor after identical floor of linkage-and-gearwork, each refining the settings handed down by the last.

In truth — attention plus a small calculating network make one layer, and layers are stacked by the dozen (a large model, near a hundred). A token’s numbers are revised at every floor — coarse matters of spelling and grammar low down, subtler matters of sense and intent above, or so the anatomists report. The adjustable parts — the parameters — number in the hundreds of billions, and none was set by hand: in training, the engine read a vast library, guessed each next token, and every screw was turned a hair’s-breadth whenever it guessed wrong. Repeated a very great many times, that is the whole of its education. A finishing school follows — training on human preference — to make it helpful, honest, and polite.

This cabinet, for comparison, owns — counting-wheels, learned from a library of — verse lines.

At the mill’s outfall the engine renders its judgment — not a word, but a pressure upon every gauge in the house: a score for every slug in the catalogue at once.

In truth — the model’s output at each step is a probability for every token in its vocabulary — all 50,000-odd, most of them minuscule. It does not decide the next word; it rates all possible ones. These gauges are live: they show this engine’s true arithmetic for the present context — and they obey the boiler lever at Station VII, should you care to move it.

One courtesy, disclosed: for the first two words after a fresh card the stop-mark and the full stops are held at nought and their pressure returned to the rest, so that a card of a single word cannot be answered with silence. From the third word the engine is naked, and may rest when it likes.

One slug must be drawn. The lots are cast in proportion to the gauges — and the boiler decides how strictly. Run cold, the favourite is all but certain; run feverish, and outsiders take their chance.

In truth — the next token is chosen by weighted lottery — sampling — and temperature reshapes the odds exactly as shown: low temperature sharpens the distribution toward the favourite (predictable, sometimes repetitious); high temperature flattens it (inventive, and at the extreme, incoherent). This is why the same question, asked twice, may be answered differently. Try the lever — the gauges obey at once.

The chosen slug is stamped upon the tape — and carried straight back to the rack, where it joins your question. The whole engine then turns over again, from card to lottery, to choose the word after. One revolution, one word.

In truth — a language model writes autoregressively: one token per pass, each pass re-reading all that came before. There is no finished answer waiting inside; each word is minted the moment before you read it. And the engine stops as it speaks — by predicting a special stop token (printed here as ∎), whereupon it rests.

The full engine keeps notes — a cache — so as not to redo old arithmetic on every revolution; but in principle, every turn reads the whole rack afresh.

Designed and built in a Victorian burst of enthusiasm for Shakespeare, the press sets the engine running line upon line — fourteen to the sheet, quatrains and couplet — in the hope, sincerely held and entirely vain, that new poetic masterpieces might emerge.

In truth — nothing can emerge but recombination. Every line is a walk through the sonnets the machine has read, spliced at the words Mr. Shakespeare himself repeated. The engine has read the poems as the quarto set them — line by line — so the shape of a single line it proposes for itself, down to the mark ⏎ by which it ends one; what makes the sheet look like a sonnet is the harness — the rule-book of cams above it, which owns the fourteen-line frame and the rhyme scheme. Each cam is a rule laid upon the lottery: it strikes forbidden slugs from the drum before the draw (the trade calls this constrained decoding), or steers the draw toward permitted ones. The cams forbid and steer; they cannot invent. Only the counting-wheels propose a word — and if the engine has never read a rhyme for compass, no rule can conjure one. Watch the Rhyme Cam fail, and mark it ✗ in the margin: that failure is the exhibit.

The trade term for the whole panel is the harness, and it is no small part of any modern engine: the good manners of a chat machine — its refusals, its formats, its stopping when it should — are largely rule-book rather than model. Two lessons follow. The harness cannot add knowledge, only shape it; and what it can do is bounded by what the engine has read — throw the Rhyme Cam at five sonnets, then at the full folio, and count the ✗ marks: the failures thin as the library grows, though not by a straight road. Rhyme is judged as the Bard would: by sound, with his own eye-rhymes and old pronunciations allowed (love/prove, eyes/lies) and marked in the margin as such — never a word against itself, nor against its own possessive; and, like him, the press does not stop for an imperfect rhyme. The press runs at its own boiler setting, 0.85, cooler than the crank's — of which more in the marginalia below. When the sheet is full the concordance beneath it underscores every run of three or more words that stands, in order, in the sonnets — a slip names the source line — and tallies how much of the sheet is quotation and how much is seam. It is the one question every reader of an engine's output should ask, and here it can be answered exactly, because the whole of what the machine has read is on the page.