On my large text benchmark I accept results submitted by others that I don't or can't test myself. The purpose of the rules is to allow others to verify results independently. Thus, the software has to be available on a public website with free download for at least a trial period. It doesn't need to be source code, but something that a lot of people can run.
If you claim a new record but you have to rent all of Amazon AWS for a year to run it (expected to cost $55 billion in 2021 at 10^14 operations per dollar) then I will still post your claim. But don't be surprised if some people have doubts. The Hutter prize is different, of course. Since there is prize money, it has to be practical to test submissions ourselves. On Sat, May 29, 2021, 2:29 PM <[email protected]> wrote: > "the test hardware, operating system, compiler, and programming language > implementing the decompressor must be general purpose, available to the > public, and not specifically designed to improve the ranking on this > benchmark. (A Win32 or Linux executable or C/C++ program meets this > requirement)." > > "There is no restriction on compression/decompression time, memory, or > disk space." > > Matt seems to say no restriction, yet it must be publicly available....I > think you should specify clearly here what surpasses "publicly available", > because only some can afford the best GPU or beowulf/ supercomputer. It > seems Matt is against the 'let's see who has the most cash' contest since > he disallows testing such machine even if only 1 such machine exist. > Beowulfs can cost a lot or be hard to use and give a lot of performance > increase I think. The best GPU may not give much more an increase than a > common GPU. In this sense, the goal of the contest would be using the > common buyable computer then, simply to make sure you can and do use what > you can practically. The GPU use should cost ex. under 1,000USD. This is > perhaps more useful than the 'who has more cash contest'. Do we still need > a such contest as well? Not really, you can just check how much money some > person has or what they have done/ own. > *Artificial General Intelligence List <https://agi.topicbox.com/latest>* > / AGI / see discussions <https://agi.topicbox.com/groups/agi> + > participants <https://agi.topicbox.com/groups/agi/members> + > delivery options <https://agi.topicbox.com/groups/agi/subscription> > Permalink > <https://agi.topicbox.com/groups/agi/Te9633f76cfbb22e5-M2ef1d712bcc58d2afb9ad854> > ------------------------------------------ Artificial General Intelligence List: AGI Permalink: https://agi.topicbox.com/groups/agi/Te9633f76cfbb22e5-Mb9c7e8a81d7df51dde61470a Delivery options: https://agi.topicbox.com/groups/agi/subscription
