Nonlinearity · an AI research lab
Anything that learns from its own best stops being linear.
Trying more, on its own, climbs a straight line: steady and slow. Keep what works, learn from it and try again: the line bends upward, and keeps bending as long as every part of the loop keeps improving. People have always worked this way. Models now do too, and much faster.
search alonelearning from the best
I
Thought goes out, and comes back in.
People learn in two moves: put thought outside the head, then take it back in. Everything written down that can be read back is the record. Every new medium made the first move cheaper and faster: speech, writing, print, code, the web. The second stayed slow. Each person still had to read, practice and learn it themselves.
A language model is the first medium that makes the second move too. Trained on the record, it takes it in; asked, it writes back out. Out and in now happen in one system, at machine speed. That is why the medium became a learner, and why the next step is models improving models.
II
Some things only need to work once.
Code that runs, a proof, a working design, a research result. Any single attempt may fail, and nobody knows in advance which one will succeed. One success is enough, as long as we can tell it is real. That is discovery.
Whether a success is within reach depends on three things: how likely an attempt is to be right, how many attempts we can afford and how often a wrong answer passes for right. An exam gives one attempt, so only the chance of being right, p, matters. Research gives many, and then all three matter.
III
A loop, then a spiral.
Once found, a success becomes part of the record. Working code is copied, a tool is rebuilt, a pipeline just runs. Discovery stays uncertain; what it finds is kept.
A model is probabilistic and can forget. What it writes down does not: text, code and results stay put. So what is kept stays kept. But if nothing else improves, the loop climbs one curve, flattens and stops at its weakest part.
When each turn also improves training, search and verification, the limits rise, and the loop opens into a spiral: each turn larger than the last.
IV
The world, the models, and us.
People and models are two learners in one world. Each builds on it and learns from it, so each learns from what the other leaves behind.
Part of the world changes on its own; the rest is what learners build. Every result left behind makes the next one easier to reach. Both build on the same world, so their two loops share one record and turn as one spiral.
About
We build loops that keep their best.
Nonlinearity is an independent AI research lab working on systems that close the loop on their own: search, keep what works, learn from it and search again. We work across training, search and verification, and on where the three meet.
We think general intelligence is already here. What it still lacks is the loop that makes it compound, and that loop is starting to close: AI now does a growing share of AI research itself. We want it to turn as fast as it can.
Founded by Yufa Zhou, a CS PhD student at Duke with papers at ICLR, NeurIPS and ICML.
- contact
- hi@nonlinearity.net
- scholar
- Google Scholar
Research
Each part of the loop is a research area.
Make right answers more likely, afford more attempts, make checks harder to fool and close the loop.
a rule of thumb
log 1/p≲min(log N,log 1/ε)↺
log 1/pTraininghow likely an attempt is to be right
- Pre-training & dataWhat the learned distribution holds, and what it can compose.
- Post-trainingTurn what passed into weights, so the next task needs fewer attempts.
- Science of scalingWhy a recipe this simple keeps producing new capability.
log NSearchhow many attempts we can afford
- Test-time searchSample, branch, select: buy more attempts with compute.
- AgentsAct with tools over long horizons, and put work where it can be tested.
- World modelsSimulate attempts cheaply so more can be afforded, at the price of a check that is easier to fool.
- SystemsMake each turn of the loop cheap enough to leave running.
log 1/εVerificationhow rarely the check is fooled
- BenchmarksDecide what counts as passing, and resist being gamed.
- EnvironmentsBuild places where an attempt can actually be run and judged.
- InterpretabilityCheck what was learned, not only what was scored.
↺The loopwhat works becomes training data
- AutoresearchModels that run the research loop end to end.
- Self-improvementWhere the loop compounds, and where it stalls.
Projects will appear under these areas as they ship.
Writing
Essays and notes from Nonlinearity.
Essays lay out the view. Notes report what each project found.
- forthcomingThe LoopPeople and models run the same cycle: put work out where it can be judged, take back what survived. Why that cycle bends a straight line, why each curve still flattens, and what it takes for the loop to keep turning.essay