Slightly long to try to 'wrap-up'. For those in a hurry: If the clear measurement is not seen as important, then we will continue to have arbitrary drifting credits that drift and noone can say by how much.
I don't think any "autoflops" /can/ fix a moving target. Further comments: Lynn W. Taylor wrote: > > > Martin wrote: > >> [...] >>> Credit is defined in a measurable way: >>> >>> Claimed Credit=([whetstone]+[drystone])*wu_cpu_time/1728000 >>> >>> See: >>> <http://www.boinc-wiki.info/BOINC_FAQ:_Credit#How_is_credit_calculated_in_BOINC.3F> >>> >>> >>> >>> Where we get in trouble is when we start talking about something >>> other than the formula above -- at least as far as what is canon. >>> >>> The "mythical 100 cobblestone computer" is an arbitrarily chosen box >>> such that when you use the formula above, you get 100 cobblestones >>> per day, if you use the original method of claiming credit. >> >> The 'problem' is that we have a range of hosts with different >> architectures that can run the synthetic benchmark and "look like" a >> 100 "cobblestone computer" and yet one host can have the performance >> of a garden trowel and another host with the exact same benchmark >> result can have the performance of a tractor-and-backhoe for whatever >> project. > > ... which the definition does not take into account. > > So, either we follow the definition, do the best we can to approximate > it, or replace the definition with something less dependent on > architecture. > > You're arguing against the definition. I'm arguing a different and concrete way to measurably and consistently factor in 'imponderables' such as architecture so that we can have the cobblestones AND have other measures also, all included consistently into a continuing credits system. So: "Part 1" is to reference against an actual physical standard. This is where we reference all WUs NIST-style against what has come to be called an "Etalon computer". It would be impractical to run every WU through the Etalon hence use a hierarchy of calibrations so that the Etalon only has to run a sample of WUs to maintain a calibration of how many Cobblestones are used for a particular type of WU. The other non-cobblestone resources used that are implicit in what we have as "credits" are still included in the "k" in: credits = k * cobblestones The "k" factor is recalculated so that the "Etalon" calibrated 'imponderables' match the results of the present fiddle-factor from Eric's script for s...@h. Hence, the present credits continue and stay fixed at their present nominal 'value' for s...@h. We also get real measurement numbers to show how the Etalon compares to the Eric median mythical computer. And that's it, the credits are fixed to a solid real-world reference. (Along with previous comments about maintaining an Etalon and recalibrating when replacing that with new hardware for a new Etalon perhaps every two years or so or whatever.) "Part 2": We all know that the Cobblestones only measure a part of the resources used in a computer system and that a non-cobblestones-"fiddle-factor" is used to make the numbers 'look nice'. "Part 2" is that we add in real resource measurements to reduce the effect of the "fiddle-factor" required so that the credits are more honest. Hence: credits = A * Cobblestones + B * ResourceB + C * ResourceC etc for as far as is seen to be useful. Better could be to use: credits = W * whetstones + D * dhrystones + B * ResourceB + C * ResourceC ... The factors there represent the weighting for what is arbitrarily judged to be the relative importance for the credits. I expect none will be a simple "1" due to needing to match the present fiddle-factors. That's extended the idea a bit ;-) I think the competing 'hang-ups' are: Eric's 'fix' is a fix for s...@h and other FLOPS-counting projects only. Other projects voluntarily (manually?) adjust their credits to keep ad-hoc parity with s...@h in 'some way'. Cobblestones and hence the credits don't make sense for non-FLOPS projects. And for the cobblestone itself: Perhaps the Cobblestone itself is 'flawed' in that we have the dhrystone/whetstone benchmark and yet we don't count the different types of integer and floats operations differently that are actually used in the WUs, even though they can take very different lengths of time to execute. Should a trig operation be counted as say 8x a float addition?... The Cobblestone assumes we have hosts where integer arithmetic and floating point arithmetic are the same cost in CPU time. Whereas usually, integer arithmetic is faster than for floats... Also, an all-integer application gets awarded for the CPU time as if floats were also used. And vice-versa for all-floats applications. A carefully optimised application that does indeed make good use of floats and integer arithmetic executed in parallel actually gets less credit even though better use is made of the CPU! Cobblestones gives equal weighting to floats and integer arithmetic, even though float operations are more expensive and take longer than integer arithmetic. Indeed, an integer optimised CPU (high dhrystone score) would claim very high credit for less work for a floats intensive WU if CPU time is used for the credits! In summary: Is there any clear advantage for using real physical hardware as a golden reference for performance and credits? My view is yes. You get a clear measurement for the credits from which you can usefully deduce performance and that permits meaningful comparisons to be made. You can also calibrate other performance measures if useful, with exactly the same system. The disadvantage is that you have the complication of making and propagating the calibration. If the clear measurement is not seen as important, then we will continue to have arbitrary drifting credits that drift and noone can say by how much. Regards, Martin See: How is credit calculated in BOINC? http://www.boinc-wiki.info/BOINC_FAQ:_Credit#How_is_credit_calculated_in_BOINC.3F Whetstone http://en.wikipedia.org/wiki/Whetstone_(benchmark) Dhrystone http://en.wikipedia.org/wiki/Dhrystone Whetstone Benchmark Detailed Results On PCs http://www.roylongbottom.org.uk/whetstone%20results.htm Dhrystone Benchmark Results On PCs http://www.roylongbottom.org.uk/dhrystone%20results.htm -- -------------------- Martin Lomas m_boincdev ml1 co uk.ddSPAM.dd -------------------- _______________________________________________ boinc_dev mailing list [email protected] http://lists.ssl.berkeley.edu/mailman/listinfo/boinc_dev To unsubscribe, visit the above URL and (near bottom of page) enter your email address.
