On October 04, 2003 10:21 am, Toad wrote: > On Fri, Oct 03, 2003 at 11:40:25PM +0100, Toad wrote: > > On Fri, Oct 03, 2003 at 12:40:03PM -0700, Ian Clarke wrote: > > > Toad wrote: > > > >On Sun, Sep 28, 2003 at 12:21:05AM -0500, Brandon Low wrote: > > > >>Also, it is possible that inserts are __slow__ due (indirectly) to > > > >> the asyncronizing of trailer sends. I've tried to explain my view > > > >> of this to toad, but lets see if mentioning it here gets me > > > >> anywhere: With trailers async, there is no local limit on how many > > > >> trailers we can start, that is we will keep queueing up trailers > > > >> till we are blue in the face, causing tremendous slowness in the > > > >> actual transfer of said trailers. IT IS MY __HUMBLE__ opinion that > > > >> this behavior is less desirable than having threads block on trailer > > > >> sends forcing the node to limit itself on how many trailers it can > > > >> start. > > > > > > I agree with Brandon. > > > > > > Would you rather go to a restaurant where they just kept accepting more > > > customers, slowing the service afforded to their existing customers, or > > > would you prefer that a restaurant did its best to serve its existing > > > customers, and didn't accept new customers if existing customers would > > > suffer? > > > > > > I would prefer the latter, and would point out that the latter > > > restaurant may well end up serving as many if not more customers > > > overall than the former. > > > > The solution as I hammered out with edt recently, is to make the send > > queue length accurate (by only including data we actually have in cache > > ready to send, this defeats some DoS attacks and makes the whole thing > > much more accurate when dealing with stream-through requests), and to > > reject all queries when the send queue exceeds some factor multiplied by > > the bandwidth limit. He said he'd have a go at implementing it; I am > > still working on PeerHandler. Do we have your approval? My main concern > > was that a simple limit on the number of trailers moving was going to be > > a major problem, but the above scheme should work. > > There is a major problem with this. > > We have no control over how fast data is sent out from the send queue if > we limit the send queue - so we could end up using very low bandwidth > and still rejecting all incoming requests, for no good reason.
No. This does not have to be the case. > The right solution is exactly what we have now - if the output bandwidth > usage is over some fraction of the limit, reject all queries. What we have now works as designed. Its lets us start far more work than we have the resources to do. > What we have now is apparently not working, this is most likely due to > some problem with the exact ratios, bugs in the bandwidth limiter or > similar problems. You could try increasing lowLevelBWLimitMultiplier > (default currently 1.2, used to be 1.5, maybe it is too low), disabling > the hard limiter completely (doLowLevelOutputLimiting=false), or > decreasing the outLimitCutoff (currently 0.9, try 0.75). This should > produce similar results to what you have with your other limiting > schemes - in theory - modulo bugs. What I have do is setup a feedback loop that finds the smallest sendQueueSize such that we are using our bandwidth. I we do not use the bandwidth it increases the window and the bandwidth usage goes up. The converse is also true. Ed _______________________________________________ Devl mailing list [EMAIL PROTECTED] http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/devl
