On Mon, Nov 03, 2003 at 12:28:51AM +0000, Ian Clarke wrote: > Martin Stone Davis wrote: > >Ian Clarke wrote: > >>As I have said on a few occasions, the best measurement we could have > >>of NGR's performance would be a mean difference between estimated and > >>actual response times. This measures exactly how well NGR is doing the > >>job it is supposed to be doing, and can thus be used to guage the > >>effectiveness of modifications to the NGR algorithm in terms of their > >>effect on routing. > >> > >Could you point me to something that justifies this? I don't understand > >how this can me true. > > Well, it should really be self-evident if you look at what NGR is trying > to do. NGR works by estimating which node will have the lowest response > time, and routing to that node. Obviously, the difference between the > estimated and actual response times will be lower if the estimate is > better, and thus this averaged over time will tell us how well NGR is > estimating routing times, and therefore how optimal its decisions are > likely to be.
How exactly do you reconcile this with the fact that the NGRouting estimate subtracts pLegitDNF from pDNF when calculating it's estimate? Surely we would have to calculate it twice, once without the legit DNFs correction, for your stat to be meaningful? > > > From what I understand, for every key requested, you would measure how > >long we estimated tSuccess, and then subtract the actual time for > >success. You then take the average of all these. > > Well, the average of the difference (ie. never negative), yes. Well, if it's only for successes, then it really doesn't tell us much. It would have to be for ALL requests... which makes it quite fiddly. > > >But so what? There's no reason to think that this average would be > >larger in build 6264 (where we picked the worst possible node) than in > >the later corrected versions. My eHealth, however, would measure such a > >difference. > > I don't claim that this will allow us to judge the performance of all > aspects of the node's operation, but it would allow us to measure how > well NGR is making its routing decisions. > > Ian. -- Matthew J Toseland - [EMAIL PROTECTED] Freenet Project Official Codemonkey - http://freenetproject.org/ ICTHUS - Nothing is impossible. Our Boss says so.
signature.asc
Description: Digital signature
_______________________________________________ Devl mailing list [EMAIL PROTECTED] http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/devl
