Though it is going to seem like I don't like the proposal (I do like it actually, in many ways) we need to think on all the objections so that flaws can be fixed.
So, in no particular order: - The proposal does not seem to address multi-threaded applications (Aqua), applications that span resource types (Einstein with CPU +CUDA), NCI tasks (even if only to exclude them as a consideration), tasks that use only portions of the resource (Not sure we have any projects doing this yet) - The proposal does not address Integer heavy projects (Prime Grid?) or the distinction between single precision projects (the majority) and projects that are DP heavy (MIlkyway) - The proposal suggests counting the passes through the main loop but does not seem to make any specific use of that information that would lead to better results (as far as I can tell). To put it another way, why would the summation of the FLOPS number resulting from the execution time of each iteration through the loop times the benchmark score or GPU rating be more accurate than a simple multiplication of those ratings against the total execution time. If we eliminate the counting of the execution loops we eliminate the need to alter the current applications and also make this more amenable for legacy applications that use the wrapper (YoYo anyone?) - The proposal states that the project no longer needs to specify FLOPS estimates but now requires them to note the number of "App Units" which is essentially a stand-in for a bucket of FLOPS ... or to put it another way, a FLOPS estimate ... - If I always specify my app units as 1 (one) then in essence I have the old benchmark score times the execution time way of calculating credit ... which sucked swamp water ... - The proposal does not specify how the information is propagated across projects so that the "learning" on project "A" can improve project "B". - The proposal does not address the instability of the current benchmark. - Though the GPU speed is derived from the GPU model it is not clear to me if that is going to be completely accurate. For one thing, especially with over-clocking common with GPUs you can have two GPUs of the same model with significantly different speeds. In some cases exceeding 20%; in that I don't have that many GPUs I cannot say if the reported speeds are adjusted up, or down accordingly. This also does not address user applied clock modifications. - The proposal still requires the project to make initial scoring estimates no matter how wrong. - If the project's scoring estimates are badly off initially how is that going to "bias" the awards for initial tasks vs. later tasks? - We also do not specify the expectations and outcomes for projects that opt out of this system. - Are we going to attempt in the implementation of this system to bring back the original standard definition of what a Cobblestone means and attempt to adjust participating projects to that definition? Or are we going to implement the currently debased Cobblestone value and move on from here. If so, what is the new definition of a Cobblestone? As an example, Collatz (I think it is Collatz) does try to tie their award to the original definition by running the applications on a reference system and adjusting from there ... because if we are not going to re-establish the standard as originally defined we should, and must, so state ... otherwise purists (like me) are going to probably bring it up ... - The proposal does not explain how we might use the information gained from projects that do have broadest coverage across resource types and application classes (Collatz and MW) where the same task is used and the results are intended to be comparable no matter how they were processed. In other words, some of those nagging issues with architecture differences might be exposed if we collect and process the information correctly. This might be more of a side benefit question, but, ... still ... So, now the real question ... Is this a serious proposal? Should we really spend time working on it to make something that might work? Because as critical as this post and others might seem ... this proposal is still at least a recognition that what we have is not what we want ... and I think it can be improved upon ... On Aug 28, 2009, at 12:45 PM, David Anderson wrote: > I'm coming around to the viewpoint that projects shouldn't be expected > to supply estimates of job duration or application performance. > I think it's feasible to maintain these estimates dynamically, > based on actual job runtimes. > I've sketched a set of changes that would accomplish this: > http://boinc.berkeley.edu/trac/wiki/AutoFlops > Comments welcome. > > BTW, a bonus of the proposed design is that it provides > a project-independent credit-granting policy. _______________________________________________ boinc_dev mailing list [email protected] http://lists.ssl.berkeley.edu/mailman/listinfo/boinc_dev To unsubscribe, visit the above URL and (near bottom of page) enter your email address.
