A blog post said most AI models are trained on less than 10% of the data they could use, and I can't stop thinking about it

Stella Bennett · 2026-10-10T07:05:03.101983+00:00

I was reading a machine learning blog by a researcher named Cody who works at a small lab in Austin, and he mentioned that a lot of large language models only use a tiny slice of the text out there. He said something like 90% of the data gets filtered out because it's too messy or duplicated or whatever. That number really hit me because I always assumed these things just hoover up the whole internet. I mean, we're already running out of good training text, but we're also throwing away most of what we have? Has anyone else seen a concrete source for how much data actually gets used versus discarded, because I can't find a clear answer.

Librear

Loading…

A blog post said most AI models are trained on less than 10% of the data they could use, and I can't stop thinking about it | AI Innovations - Librear