Cognitive Psychology

Human Attention versus AI Attention

The word "attention" appears in two literatures that have almost nothing in common. In one it names a set of experimental findings about selection, capacity and awareness. In the other it names an arithmetic operation: a normalised similarity between one position in a sequence and every other position. This page has you do both, on the same sentence, so the difference is something you have produced rather than been told.

The numbers here are chosen by hand to be legible — no model was run, and nothing was learned

Learning objective

By the end you should be able to say precisely what a transformer attention weight is, and why a weight map is not an explanation of a model's output.

About 15 minutes. Nothing is timed. Nothing you do here is saved or sent anywhere.

Before you start — the language problem

A weight is a number, not a noticing. Throughout this page, the computational side is described in arithmetic terms only. A high weight on a position does not mean the system "focused on" or "cared about" or "was drawn to" that word. It means a dot product came out large and the softmax normalised it upwards. The habit of saying otherwise is the single commonest error in writing about these models, and it is easier to avoid here than anywhere else because you can see the arithmetic.

Nothing on the machine side is aware of anything. There is no experience, no report, no effort and no cost of the kind the human literature measures. The absence is the finding.

Answer the prediction below — it unlocks the first stage.
Stage 1: read two sentences that differ by three words and say what "it" refers to in each.
Stage 2: compute attention weights over the same sentence, by choosing a query position, a head, a temperature and a mask.
Stage 3: compare the two things you have just done, in the sorting challenge.
Read the debrief, which is where the one real rhyme and the divergences either side of it are set out.

This part needs JavaScript

The attention weights are computed in your browser, so this cannot work with JavaScript switched off. The rest of the page is readable without it.

If a transformer's attention weights for the word "it" land on the same noun you chose, what would that show? Very little on its own — two processes can produce the same output for unrelated reasons That the model resolves pronouns the way people do That the weights explain why the model produced the answer it did That the model was paying attention to the noun, in the ordinary sense

Key terms

Attention weight: A number saying how much one position contributes to the sum being computed for another position. The weights in a row add to one.
Softmax: The step that turns a row of raw scores into weights that are all positive and sum to one.
Temperature: A divisor applied to the scores before the softmax. A low temperature concentrates the weights on a few positions; a high one spreads them out.
Head: One of several weight patterns computed over the same sentence at the same time, each with its own scores.
Mask: A rule stopping a position from attending to certain others, such as everything later in the sentence.

The query

Compute the weights for which position?

The normalisation

Divides every score before normalising. Low concentrates, high flattens.

Mask later positions

An architectural rule, not a discovery about the sentence.

Presets

Look at another head

Which head?

Three multipliers applied to the query vector. They have no assigned meaning.

Attention weights over the ten token positions

Every position, with its key vector and score

For each token position: its three key numbers, the raw score against the query, and the resulting weight
Position	Token	Key vector	Score	Weight

What this readout is

Where these numbers came from

Complete stage 1 and stage 2 first — several of these are much easier once you have done both.

Eight statements, each to be sorted into human attention, transformer attention, both, or neither
Statement	True of…

A transformer attention weight is a softmax over query–key similarities: a set of positive numbers summing to one, deciding how much each position contributes to the next representation. It rhymes with selective attention in that both allocate limited influence, and diverges in almost every other way — there is no capacity limit, no serial bottleneck, no cost of switching, and nothing it feels like. The shared word is a useful metaphor and a poor argument.

Human Attention versus AI Attention

Learning objective

Before you start — the language problem

First, a prediction

Stage 1 — what does "it" refer to?

Before you move on

What you just did

Stage 2 — compute the weights yourself

One softmax over ten positions

What this readout is

Stage 3 — sort the statements

What this demonstrates