Lynn W. Taylor wrote: > Paul D. Buck wrote: >> The best benchmark for a system is the actual running application. This >> truism has been long known as established fact. The only reason >> synthetic benchmarks are used is that they are a lot easier to port to >> other systems. > > The moment you pick one work unit as your "reference" work unit, you > have a synthetic benchmark -- and a circular argument.
A vital point is to decide _what_ we are referencing against. Is it: s...@home; A mythical cobblestone of cpu cached integer rate and cpu cached floating point rate; A golden reference PC of a certain architecture; A mythical "statistically averaged" 'computer' bistromathematically determined from the current s...@home participants; Or? Sooo... First question is what are we referencing against? At the moment we seem to be doing an add-hoc mix of all of the above with various 'fiddle-factors'. From: http://boinc.berkeley.edu/trac/wiki/AutoFlops the premise/basis of the whole idea of credits is based upon: "should be roughly proportional to *FLOPs* performed". Can we get the idea of *ALL resources* to be included in there right from the outset? I'll start with a list of: network usage; RAM max usage; disk storage; turn-around time; success rate; availability; data quality; single float calculations; double float calculations; single integer calculations; double integer calculations; single logic operations; double logic operations; others... and then have a weighted summation of all those to give a cobblestones+ value that in some proportion reflects the resources utilised. Hence, such as quake-catcher or other monitoring projects can reward for the availability and network usage of their more dedicated participants. Long compute projects such as CPDN can additionally reward for a participant keeping with the same one or two year simulation for a completed run, and also additionally reward for the disk space and RAM space utilised. For the various primes projects, the old cobblestone perhaps is fine. "How" to do that is for another thread ;-) (Sorry, if this has already been discussed, then add a reference to that trac page please.) Regards, Martin -- -------------------- Martin Lomas m_boincdev ml1 co uk.ddSPAM.dd -------------------- _______________________________________________ boinc_dev mailing list [email protected] http://lists.ssl.berkeley.edu/mailman/listinfo/boinc_dev To unsubscribe, visit the above URL and (near bottom of page) enter your email address.
