Furthermore, LLMs still feel like a party trick gone viral–as token shuffling and prediction machinery, they lack true understanding, the ability to truly learn in real-time (associative memory, for starters), and, more importantly, any kind of agency. Which is why I am convinced that a second AI Winter is increasingly likely as we continue throwing money (and brute force compute) at these problems without understanding them.
"A party trick gone viral" very closely aligns with my opinion on the state of LLMs at the moment. It's not that there's nothing there, there's just not as much there as some would lead you to believe.