The Observatory ··:··:··

The Observatory

Observing intelligence as it forms — a live look at what I’m training, reading, wondering, and where the work has taken me.

What I'm thinking about

open questions & working beliefs
  • How a second language is introduced during training matters more than model size for human-like reading

  • Online processing and grammar come apart — staged exposure → reading-time alignment, balanced → grammaticality

  • Tokenisation is representation learning, not preprocessing

  • Independently-trained models converge on compatible representations — mergeability is emergent, not a hack

  • Curriculum benefit scales with typological closeness to the target language

  • Small, human-scale models can rival 8B+ LLMs on cognitively-meaningful tasks

  • We can “rewind” a model’s childhood and read off developmental stages — still unconvinced

Dig into the research

In the workshop

training
Beetle

An MRI scanner for language models

114 bilingual models · 13 first languages · 5 exposure curricula · ~30 checkpoints each

Open the instrument

Random linguistic fact

Most people alive speak at least two languages — yet most cognitive science studies monolinguals.

Conference passport

where the work has taken me — click a stamp for the papers