weekly reel February 11, 2024
Yowdy frendz. This week’s music is the Outer Wilds OST.
Outer Wilds: Echoes of the Eye (The Lost Reels) by Andrew Prahlow (bandcamp.com)
People seem to be falling for two rather thoughtless extremes:
- “LLMs are AGI, they work like the human brain, they can reason, etc.”
- “LLMs are dumb and useless.”
The reality is that LLMs are not AGI — they're a big curve fit to a very large dataset. They work via memorization and interpolation. But that interpolative curve can be tremendously useful, if you want to automate a known task that’s a match for its training data distribution.
Memorization works, as long as you don’t need to adapt to novelty. You don’t need intelligence to achieve usefulness across a set of known, fixed scenarios.
In fact, that’s the entire story of the field of AI so far: achieve increasing levels of usefulness and automation, while bypassing the problem of creating intelligence.