n Sat, Dec 6, 2025 at 7:18 PM Brent Meeker <[email protected]> wrote:
*>>> These data centers have been sucking up and processing data >>> accumulated over 70yrs or more and condensing it into neural nets. Isn't >>> there some point of diminishing returns in this process?* >> >> *>>> Companies are making a multi-trillion dollar bet that there is not a >> point of diminishing returns. And I think that's probably a pretty good >> bet, * > > > > *Why? Do you think there's a lot more to be sucked up? * > *No, but I think there's a lot more ways to think about the facts that we already know, and even more important I think there are a lot more ways to think about thinking and to figure out ways of learning faster. * *People have been saying for at least the last two years that synthetic data doesn't work and we're running out of real data so AI improvement is about to hit a ceiling; but that hasn't happened because high quality synthetic data can work if used correctly. For example, in the process called "AI distillation" a very large AI model supplies synthetic data to a much smaller AI model and asks it a few billion questions about that data and tells it when it made a correct answer and when it has not. After a month or two the small model becomes much more efficient and is nearly as capable as the far larger one, sometimes even more so; it has been able to do this not by thinking more but by thinking smarter. After that the small model is scaled up and is allowed access to much more computing hardware, and then the process is repeated and it starts teaching a much smaller model. * *That's why, far from hitting a ceiling, almost every week we learn about another significant improvement in AI technology, the magnitude of improvement that pre 2020 only occurred once or twice a decade. I**t's so common now that most people have gotten used to it and such announcements bore them, ho-hum just another revolutionary development and AI. I'm reminded of the boiling a frog parable where a frog is placed in cold water that is gradually warmed up until it boils and kills the frog and the frog doesn't jump out because it doesn't notice the gradual increase in temperature. * *John K Clark See what's on my new list at Extropolis <https://groups.google.com/g/extropolis>* rvv > > -- You received this message because you are subscribed to the Google Groups "Everything List" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion visit https://groups.google.com/d/msgid/everything-list/CAJPayv3foPWETM%2BHYhTh4gVojQJDfPwSyB9PyY%3DrZc%3DVrOPH3Q%40mail.gmail.com.
