Rule or Rote?

Hunting for the Past-Tense Machine Inside an AI

Part 2 of 3.

TL;DR: I used a made-up word from 1958 to test whether AI models have a real grammar rule for the past tense, then looked inside to find it. I found real generalization, but no clean rule. And a reader caught something that could’ve explained it away.

Here’s a claim I didn’t expect to be making: a nonsense word invented for a 1958 psychology experiment can tell you whether an AI understands grammar, or is just a very good parrot.

I ran that test on models up to nine billion parameters. Then I did the thing you can’t do to a human brain: I reached inside and looked around for the rule itself.

The answer fits neither textbook.

1. The wug test

One experiment, sixty-eight years old, is still the sharpest tool in the whole fight.

In 1958, the psychologist Jean Berko showed children a drawing of a nonsense creature: “This is a wug. Now there are two of them. There are two ___?” The kids said “wugs.” They’d never heard the word. They couldn’t be remembering it. So they had to be applying a rule.

That’s it. Made-up words have zero frequency. If a model turns wug into wugged, it can’t be looking anything up, it has to be generalizing.

I gave the models a batch of invented verbs — wug, blick, dax, plim, gorp — in past-tense sentences, and measured how often they reached for -ed. Then I compared made-up verbs against familiar ones like walk → walked, across sizes.

2. The gap that won’t close

Bigger models get better at made-up words. They never get good.

Here’s how five models did on made-up verbs versus familiar ones, and the gap between them:

• Pythia-410M — 54% on made-up words, 100% on familiar. Gap: 46 points.

• Pythia-1.4B — 53% vs. 96%. Gap: 43.

• Pythia-2.8B — 62% vs. 100%. Gap: 38.

• Gemma-2-2B — 58% vs. 98%. Gap: 40.

• Gemma-2-9B — 76% vs. 100%. Gap: 24.

Even the nine-billion-parameter model handles a brand-new word far worse (76%) than a word it has seen (100%). That gap shrinks with scale; but slowly, and it never closes.

• There is a real rule, but a thin one. The models do reach for -ed on words they’ve never seen, and the biggest, Gemma-2-9B, does it most (76%). A genuine, productive habit is in there. They’re not pure lookup tables.

• But it never closes the gap. A real “-ed” rule would treat wug exactly like walk — near 100% on both. Instead novel verbs trail familiar ones by 24 to 46 points at every size, and ten times more parameters barely helps. Partial, graded, frequency-hungry.

This is what generalization looks like in a distributed system: a smeared habit that sweeps a novel word along with no discrete rule doing the sweeping. The incompleteness is the fingerprint.

3. “But isn’t this just the tokenizer?”

A reader named Dan (thanks Dan!) raised an objection: modern models don’t read letters, they read chunks. Before a model ever starts learning, all its text gets sliced up into a fixed menu of pieces called tokens. A very common word is often a single token, grabbed whole; a longer or rarer word gets split into parts. So walked might arrive as one piece, like a single Lego brick, while jumped might come in as two: jump + ed. And a word the model has never seen, like wug, isn’t on the menu at all, so it gets shattered into scraps: w + ug.

Here’s the catch. That -ed piece is one of the pre-made bricks, handed to the model before training even starts. So when a model tacks -ed onto a verb, it isn’t necessarily discovering the past-tense rule. It might just be snapping on a block it was given for free. Which is Dan’s question: are we watching the model learn grammar, or watching the tokenizer that was built before it learned anything at all?

First, on my earlier “backwards” finding, the tokenizer isn’t a problem; it’s the explanation. A tokenizer can’t fake the shape of a curve. It just explains why an AI learns in the reverse order from a child: a toddler builds -ed from raw sound; the model is handed it as a Lego brick on day one. Two things stay out of reach: the decline (a pre-installed token can’t draw a curve that falls) and the missing hump (you can’t fake the absence of a bump).

Second, on the wug test, it’s a genuine confound, and worth noting. Walked is often a single chunk the model grabs whole (memorization); wug shatters into fragments. So some of its penalty is tokenization, not rule. The generalization we see may be a shade better than the raw numbers, but 76% at nine billion is still nowhere near walk-level.

And here’s the twist that keeps that confound from letting the model off the hook. You might object that a pre-made -ed is just a helpful leg up, and it is. But a leg up on producing the suffix isn’t the same as learning when to attach it. The tokenizer hands over the raw material; the model still has to learn to snap it onto a word it has never seen, and that mapping — “past-tense slot + unfamiliar stem → add -ed” — is exactly what stays leaky. So being handed -ed for free doesn’t build the rule; it just supplies one ingredient. If anything it raises the bar: give a model the answer piece up front and watch it still fumble brand-new words, and “there’s no clean rule in here” gets stronger, not weaker.

The clean fix? A model that reads one letter at a time, forced to build -ed itself. That would be the logical follow-up.

This rhymes with 1986. Rumelhart and McClelland hand-built their sound representation so morphology would be visible to their network. The tokenizer is that same choice, made by raw frequency, without anyone asking. Pinker and Prince spent 1988 arguing hand-built representations quietly do the work. Nearly forty years later, that’s Dan’s objection in new clothes.

4. So I reached inside

The wug test watches what the model does. Interpretability lets me watch what it is. So I went looking for the rule directly.

Is there a “rule switch”? I found the pattern that most cleanly responds to -ed, in the biggest model, and cranked it. It did nothing. Zero. A random nudge of the same strength leaked more goed. Six ways, three sizes, two methods — same answer: no localizable switch produces overregularization.

Thermometer, not furnace. Switch the -ed pattern off and the model still writes past tenses fine, but gets worse at understanding them (small but real — about five standard deviations out). It reads the suffix. It doesn’t make it.

Is the distinction even in there? A readout for “regular vs. irregular” barely hit 62%, where a clean category would be 85–95%, and the signal lived in shallow layers. The model doesn’t “know” go is an exception. It just remembers went.

Two parts or one? I switched off each component one at a time. Nothing dissociated. It’s one tangled system, not two routes.

5. The crack in the story

Every probe points the same way: no discrete rule in the machine.

But…every one of those “no rule” results is a search result. I looked here and didn’t find one. And absence of evidence isn’t evidence of absence.

So in Part 3, I stopped hand-searching and let a gradient-descent optimizer, the cleverest search there is, find the single best possible direction for producing goed.

It worked. 100%. On words it had never seen. But we’ll see the story is more complicated than that.

Takeaway + your 5-minute next step

The past tense was supposed to be the clean toy problem, simple enough to reveal the architecture of language. Reach inside, and there’s no architecture to reveal: just overlapping habit, and a syllable the model grabs when it isn’t sure.

Try it in five minutes: open your favorite chatbot and feed it invented verbs — “Every day I frell. Yesterday I ___.” Watch whether it commits to -ed, hesitates, or grabs a lookalike real word. You’re running Berko’s 1958 experiment on a machine.