The Large Text Compression Benchmark's exclusion of the compressor from the metric has its upside "to spur research in language modeling" since, were it underwritten by the big players, it would motivate "a globally distributed network of specialists".
However, my OP was addressing a narrow concern: DeepMind's cost centers. That said, I do have my often stated/not-so-hidden agenda in my suggested cost reduction measure: I want to raise awareness of the value of approximating Solomonoff Induction in rank-ordering unified models of society based on the enormous resources, both computational and informational. But then, I am rather biased toward averting a bloody fratricidal Thirty Years War over politics. Seems to me that would cost something like $20T easily, so reducing its likelihood by a percent or so would more than justify adequate resources toward applying such discipline to "The National Conversation". On Tue, Jan 5, 2021 at 9:36 AM Matt Mahoney <[email protected]> wrote: > Regarding James Bowery's proposal for DeepMind to fund the Hutter Prize to > spur research in language modeling, I generally agree. But: > > 1. Google (DeepMind's biggest customer) is less interested in modeling 1GB > of text on a single CPU vs 1 exabyte on billions of CPUs (plus a few > zettabytes of pictures and video). > > 2. The Hutter prize now requires open source submissions (although there > are none since this rule change). With a large prize, Google will likely > demand IP rights. > > The big players in AI have figured out that the way forward is a globally > distributed network of specialists. I mean, we have been doing this with > human brains for centuries, but this approach only goes so far due to our > cognitive limits. AGI is really a step in the wrong direction. We don't > need a billion Jack of all Trades. We need billions of narrow experts. > > The Large Text Benchmark, which is the original Hutter Prize but without > prize money or hardware constraints, has produced some valuable research. > We can now say that neural network based language models top all others. > But that means the top ranked algorithms take a week to compress 1 GB with > 32 GB of RAM and thousands of GPU cores. The Hutter Prize submissions are > just tweaks to run on puny hardware. > > Without prize money, the benchmark feeds on the competitive nature of > programming (mostly among young males) and the opportunity to showcase your > work and get noticed by tech companies (how I was able to retire at 59). > > On Sat, Jan 2, 2021, 1:27 PM James Bowery <[email protected]> wrote: > >> DeepMind’s big losses, and the questions around running an AI lab >> <https://venturebeat.com/2020/12/27/deepminds-big-losses-and-the-questions-around-running-an-ai-lab/amp/> >> >> According to the article, the losses in the hundreds of millions per year >> are primarily due to the high costs of: >> >> 1. intensive use of its parent company's deep learning hardware >> (despite probaby being used at a discount), and >> 2. top talent in machine learning/AI which sometimes command 7 figure >> salaries. >> >> So I have a suggestion for Shane Legg: >> >> Divert a mere 1% per year of the company's budget to increasing the >> underwriting for The Hutter Prize For Lossless Compression of Human >> Knowledge. >> >> This addresses both of the cost centers by discovering: >> >> 1. Machine learning algorithms that reduce hardware requirements. >> 2. Undervalued talented individuals from anywhere in the world. >> >> Not a penny of this gets spent without advancing, in an objectively >> verifiable manner, both of these cost-reductions. >> >> Moreover, if anyone should _get_ this, it should be Shane Legg -- a >> founder of DeepMind. >> > *Artificial General Intelligence List <https://agi.topicbox.com/latest>* > / AGI / see discussions <https://agi.topicbox.com/groups/agi> + > participants <https://agi.topicbox.com/groups/agi/members> + delivery > options <https://agi.topicbox.com/groups/agi/subscription> Permalink > <https://agi.topicbox.com/groups/agi/Tee75800a5ab37f5b-Mb1a8c2970e940f312c8a0558> > ------------------------------------------ Artificial General Intelligence List: AGI Permalink: https://agi.topicbox.com/groups/agi/Tee75800a5ab37f5b-Ma0fb05fd3823efefdd247c13 Delivery options: https://agi.topicbox.com/groups/agi/subscription
