Regarding James Bowery's proposal for DeepMind to fund the Hutter Prize to spur research in language modeling, I generally agree. But:
1. Google (DeepMind's biggest customer) is less interested in modeling 1GB of text on a single CPU vs 1 exabyte on billions of CPUs (plus a few zettabytes of pictures and video). 2. The Hutter prize now requires open source submissions (although there are none since this rule change). With a large prize, Google will likely demand IP rights. The big players in AI have figured out that the way forward is a globally distributed network of specialists. I mean, we have been doing this with human brains for centuries, but this approach only goes so far due to our cognitive limits. AGI is really a step in the wrong direction. We don't need a billion Jack of all Trades. We need billions of narrow experts. The Large Text Benchmark, which is the original Hutter Prize but without prize money or hardware constraints, has produced some valuable research. We can now say that neural network based language models top all others. But that means the top ranked algorithms take a week to compress 1 GB with 32 GB of RAM and thousands of GPU cores. The Hutter Prize submissions are just tweaks to run on puny hardware. Without prize money, the benchmark feeds on the competitive nature of programming (mostly among young males) and the opportunity to showcase your work and get noticed by tech companies (how I was able to retire at 59). On Sat, Jan 2, 2021, 1:27 PM James Bowery <[email protected]> wrote: > DeepMind’s big losses, and the questions around running an AI lab > <https://venturebeat.com/2020/12/27/deepminds-big-losses-and-the-questions-around-running-an-ai-lab/amp/> > > According to the article, the losses in the hundreds of millions per year > are primarily due to the high costs of: > > 1. intensive use of its parent company's deep learning hardware > (despite probaby being used at a discount), and > 2. top talent in machine learning/AI which sometimes command 7 figure > salaries. > > So I have a suggestion for Shane Legg: > > Divert a mere 1% per year of the company's budget to increasing the > underwriting for The Hutter Prize For Lossless Compression of Human > Knowledge. > > This addresses both of the cost centers by discovering: > > 1. Machine learning algorithms that reduce hardware requirements. > 2. Undervalued talented individuals from anywhere in the world. > > Not a penny of this gets spent without advancing, in an objectively > verifiable manner, both of these cost-reductions. > > Moreover, if anyone should _get_ this, it should be Shane Legg -- a > founder of DeepMind. > *Artificial General Intelligence List <https://agi.topicbox.com/latest>* > / AGI / see discussions <https://agi.topicbox.com/groups/agi> + > participants <https://agi.topicbox.com/groups/agi/members> + delivery > options <https://agi.topicbox.com/groups/agi/subscription> Permalink > <https://agi.topicbox.com/groups/agi/Tee75800a5ab37f5b-M7f8659eb51da74498e12cf62> > ------------------------------------------ Artificial General Intelligence List: AGI Permalink: https://agi.topicbox.com/groups/agi/Tee75800a5ab37f5b-Mb1a8c2970e940f312c8a0558 Delivery options: https://agi.topicbox.com/groups/agi/subscription
