It's such madness that all this resource usage is for LLMs that are barely useful (at least for me). So many billions in resources wasted for machine learning systems that mimic human speech, it's not intelligent in any sense.

Now we see models showing the user it's "thinking" as some sort of intelligent agent, when really, ploys like that is to prop up the AI sector's stock price.

Surely LLMs in this form can't be the future of AGI?

Discussions:

(87 points, 1 day ago, 97 comments) https://news.ycombinator.com/item?id=43555898

(47 points, 1 day ago, 45 comments) https://news.ycombinator.com/item?id=43562005

I really don't get why we got here. The big 5 providers have enough money to hire/poach some scraping/indexing experts. They could avoid lots of grief and issues by just indexing in a non stupid way. Even if they still ignored robots.txt, they could fly under the radar with a few well placed exceptions. But now they're risking losing easy access to a lot of the internet by just doing a really bad job of a thing they really rely on.

I get that it could be incompetence in one or two places, but as far as I know all the big providers fail here... so am I missing something obvious? Is it really just greed and vc money burning scrapefest?

Which companies are doing this? Why aren't they just downloading the Wikipedia databases as per https://en.wikipedia.org/wiki/Wikipedia:Database_download ?

From the Wikipedia OKRs linked from the Wikipedia article that is the source of the Ars article:

https://meta.wikimedia.org/wiki/Wikimedia_Foundation_Annual_...

> Key result FA2.1: Generate 5,000,000 views from short-form video content across all owned channels by the end of H1.

Wikipedia is actively saying it should pivot to short-form video to engage young people...