> Do they merely memorize training data and reread it out loud, or are they picking up the rules of English grammar and the syntax of C language?

This is a false dichotomy. Functionally the reality is in the middle. They "memorize" training data in the sense that the loss curve is fit to these points but at test time they are asked to interpolate (and extrapolate) to new points. How well they generalize depends on how well an interpolation between training points works. If it reliably works then you could say that interpolation is a good approximation of some grammar rule, say. It's all about the data.

Big thread at the time https://news.ycombinator.com/item?id=34474043

Idk from when even id this article? Got me LLMs currently are broke and the majority is already aware of this.

Copilot fails the cleanly refactor complex Java methods in a way that I’m better of writing that stuff by my own as I have to understand it anyways.

And the news that they don’t scale as predicted is too bad compared to how weak they currently perform…

It turns out our word for "surface statistics" is "world model".

Honestly, I think it’s somewhere in between. LLMs are great at spotting patterns in data and using that to make predictions, so you could say they build a sort of "world model" for the data they see. But it’s not the same as truly understanding or reasoning about the world, it’s more like theyre really good at connecting the dots we give them.

They dont do science or causality theyre just working with the shadows on the wall, not the actual objects casting them. So yeah, they’re impressive, but let’s not overhype what they’re doing. It’s pattern matching at scale, not magic. Correct me if I am wrong.