On Tue, Nov 25, 2003 at 09:09:35PM -0500, Ed Tomlinson wrote: > On November 25, 2003 05:05 pm, Toad wrote: > > Hypothetical: > > Routing works, so we have a 20% success ratio. > > The average filesize is 200kB (this is about right on the current > > network, check your datastore - but maybe we need to gather more > > accurate stats on it). > > We have a 256kbps uplink i.e. 32kB/sec, of which we can use all (this is > > optimistic). > > We get a mere 10kqph incoming, and accept all of it. > > > > I will now demonstrate that this is impossible: > > 10kqph * 0.2 = 2kqph. > > 2000 * 200kB = 409,600,000 bytes > > 409,600,000 bytes / 3600 seconds = 113,777 bytes per second, for > > trailers alone, assuming no connection and search overhead. > > > > So bandwidth is indeed the limiting factor, and we need to reject > > queries based on bandwidth usage. But I fear that routing may not work > > at all in this case. > > We need to reduce queries _or_ QR. If we reduce the rate of queries we get > the same effect in a much more bandwidth way. This is the gist of the suggestion > I put forth on Friday.
We need to reduce total queries. This includes retries due to QR. > > When a node QRs its telling us, I have enough work for now. What I think is that > we should backoff depending on how much of that work we gave the node and > on how much time it has had to process our work. The load on the node is not proportional to the number of queries - at least if it is, it's a pretty weak correlation. All your proposal will do is when we finally do find a node that doesn't constantly QR (probably because it has loads of bandwidth and a massive thread limit), when it once rejects a query because of routingTime or a transient load spike caused by a bunch of incoming connections, we will then back it off until the end of the age! > > Ed In any case, it appears that the main thing we can do to reduce the number of queries is to reduce the number of QRs. We can do this by: * Reducing the number of queries rejected due to thread usage by implementing more NIO. * Reducing CPU usage and thus queries rejected due to routingTime, and threads used by negotiations, by implementing multiplexing. * Reconsidering messageSendTimeRequest and routingTime based QR thresholds. Routing may recover as is if we cut the number of queries - that is the essence of the idea that we need to handle routing and load balancing separately. I'm not sure I agree with it... My searchFailedCount stats range between 0.92 and 2.57 over the last 24 hours. This suggests we are still rejecting FAR too many queries. -- Matthew J Toseland - [EMAIL PROTECTED] Freenet Project Official Codemonkey - http://freenetproject.org/ ICTHUS - Nothing is impossible. Our Boss says so.
signature.asc
Description: Digital signature
_______________________________________________ Devl mailing list [EMAIL PROTECTED] http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/devl
