I am not sure that some other points are well considered as well... RAC is unstable, always has been, partly because of delays in validation, but partly for other reasons (never clearly identified, but, aside for SaH RAC fanatics I don't know of anyone that relies or trusts the numbers) but if cross-project parity is a chimera, then cross-project RAC is even more so... and a one project experiment with the new credit award system does not a proof make of its validity and accuracy. Nor that it works in the face of many other projects using the variety of awards systems we have in the field. Perhaps this would only affect those of us that support a wider variety of projects than SaH, but, BOINC is not just SaH, though it is often treated as such.
Secondly, if you are going to use RAF (not sure if this is new-new, or just recent, but this is the first I have heard of this tracking parameter), what is the point of also using RAC? ... they are effectively the same measurement under different guises... though this raises Richard's point about the accuracy of RAC and credit granting with equal concerns about the effectiveness and accuracy of our measurement of FLOPS (though the use of both may provide some damping and some correction for errors introduced by the alternative measure)... The next point is that for some projects, especially NCIs like WUProp RS of 100 can never be reached because of workload limiting... and other projects have similar, though usually harder to reach daily limits as well... Next I would raise the point that our method of rating the GPUs is self limited in the sense that there are three different classes of work being run on GPUs ... SP, DP, and integer ... and the cards are rated only for one of the three classes. So, on my Nvidia cards I run GPU Grid which uses SP and have a "valid" measure of performance, but I also run MW for which I don't, and Collatz also for which I also don't have adequate measurements of performance. Because one card may perform far better than another in DP than SP because of internal architecture even when the SP stats are roughly identical ... and I have never been that comfortable with either the CPU benchmarks or the current rating system for GPUs with regard for actual project performance. Looking at the proposal's example 1 I can see the logic and the correctness thereof, however, the RAC for the CPU is going to be on the order of (in several of my systems) a factor of 50 to 60 different (or more). which begs the question, will this new system not also cause "pegging" to limits because of these efficiency differences as well... I am probably not being clear about what this concern is, but, it seems to me that we have equal potential for unbalance because of the differential in processing capability. In my case I would most likely see this in the Nvidia and ATI system because for the most part I don't run CPU/GPU projects without turning off one side or the other (usually CPU side, if you can run on the GPU why on earth would you waste CPU time? But that is just me.). Last consideration is work availability's ... the simulator I know does not reflect well the variety of work availabilities from projects like LHC where work is almost never available and when it is is only there in small batches, to MW which almost always has work but has a queue length limiter that cuts off issue at x tasks on hand regardless of use and return speeds (though the pernicious effects of this may be reduced by the recent change to remove FIFO for GPU tasks, assuming that has actually made it into a current build), to projects like Virtual Prairie which has work available in batches with each batch usually being available for a month or so before a several month long drought, to SIMAP that has monthly batches, yet I never really see the dedication to running SIMAP I would expect based on my RS and the fact that you only usually get work one week of four... and so on... The current two week RAC window does not support these diversities either ... But fundamentally, the biggest concern is how do we measure the effectiveness of the fix when we have such inadequate tools to measure what BOINC is actually doing. Productivity wise... I used to use BOINC View to log and then analyze the logs for things like how much work was really done .. but that tool is no longer supported well and there is nothing to replace its like... BOINC does emit the "Job Log" though what the data is, is not specified (no log headers) and I am not sure the current content is sufficient to do much with these logs to allow us to figure out if the new system is working as expected. On Oct 26, 2010, at 3:27 PM, Richard Haselgrove wrote: > My first reaction, on reading this, was that resource share should be based > on work done, measured directly, rather than by using credit as a surrogate. > But I've read the design document, and you've covered that point - and also > taken into account the immediate consequence, which is that projects which > over-inflate their credits get automatically penalised with a reduced > effective resource share, and hence lower work rate. I'm happy with that. > > But I'm worried by any proposal which uses credit as an input to drive any > functional aspect of BOINC. As a trivial example, just an hour ago someone > on a message board celebrated that, according to > http://aqua.dwavesys.com/server_status.php, AQUA has now "surpassed 200,000 > GFlops of compute power". BOINCstats is currently displaying 169,077.8 > GigaFLOPS. I suspect the difference may be due to AQUA's current > transitioner troubles, which results in a 'lumpy' allocation of credit, and > hence RAC. I suspect that the poster's conclusion, that "This would place > a...@h at Rank #22 in the Top500 list" > http://www.top500.org/list/2010/06/100, > would be treated with a certain amount of scepticism by the developers of > LINPACK. > > If credit is to be used as an input, it will, and should, be subject to > greater scrutiny with regards to derivation, methodology, and, dare I say, > accuracy. > > There has only been one large-scale trial (so far as I know) of the > third-generation "Credit New" scoring algorithm, and that is the s...@home > project - where it has been running, with some interim fine-tuning, since > early June 2010. We are fortunate that SETI, prior to this roll-out, had one > of the more mature flops-based credit algorithms - though one fatally flawed > by the flop-count discrepancy between CPU and CUDA applications processing > identical work for the project. > > But, with care, one can still obtain the 'claimed' credit (flop-count based) > for pre-validation SETI tasks, and compare them with the granted credit for > the same tasks post-validation. > > I performed this analysis over the weekend of 27/28 August 2010, and plotted > the results for 980 tasks processed - exclusively by CUDA applications - on > four of my hosts (I can supply raw data, and details of the methodology, for > anyone interested). I had planned to repeat the analysis before publishing > the results, but SETI's server problems have scuppered that plan. Here is > the single-weekend outcome: > > http://img231.imageshack.us/img231/1146/seticlaimedvsgrantedcre.png > > The legend shows SETI hostIDs: > > 4292666 - GTX 470 (Fermi, Windows XP) > 2901600 - 9800GTX+ (Windows 7) > 3751792) > 3755243) - 9800GT, Windows XP > > I am worried by the non-deterministic value of the individual credit awards, > even though the average figures seem reasonable. I think we should > double-check that all is going to plan with Credit New, before we start > building greater edifices on foundations, possibly, of sand. > > ----- Original Message ----- > From: "David Anderson" <[email protected]> > To: "BOINC Developers Mailing List" <[email protected]> > Sent: Tuesday, October 26, 2010 10:13 PM > Subject: [boinc_dev] proposed scheduling policy changes > > >> Experiments with the client simulator using Richard's scenario >> made it clear that the current scheduling framework >> (based on STD and LTD for separate processor types) is fatally flawed: >> it may divide resources among projects in a way that makes no sense >> and doesn't respect resource shares. >> >> In particular, resource shares, as some have already pointed out, >> should apply to total work (as measured by credit) >> rather than to individual processor types. >> If two projects have equal resource shares, >> they should ideally have equal RAC, >> even if that means that one of them gets 100% of a particular processor >> type. >> >> I think it's possible to do this, >> although there are difficulties due to delayed credit granting. >> I wrote up a design for this: >> http://boinc.berkeley.edu/trac/wiki/ClientSchedOctTen >> Comments are welcome. >> >> BTW, the new mechanisms would be significantly simpler than the old ones. >> This is always a good sign. >> >> -- David >> _______________________________________________ >> boinc_dev mailing list >> [email protected] >> http://lists.ssl.berkeley.edu/mailman/listinfo/boinc_dev >> To unsubscribe, visit the above URL and >> (near bottom of page) enter your email address. >> > > > _______________________________________________ > boinc_dev mailing list > [email protected] > http://lists.ssl.berkeley.edu/mailman/listinfo/boinc_dev > To unsubscribe, visit the above URL and > (near bottom of page) enter your email address. _______________________________________________ boinc_dev mailing list [email protected] http://lists.ssl.berkeley.edu/mailman/listinfo/boinc_dev To unsubscribe, visit the above URL and (near bottom of page) enter your email address.
