On Friday 30 November 2007 03:17, Michael Rogers wrote: > Matthew Toseland wrote: > > But the > > practical effect was reports from various places, not least on my own node, > > of the node using a lot more bandwidth and doing a lot more work than it > > would otherwise have done. > > If the problem is just that nobody's node is reaching the bandwidth > limit, would it be possible to tweak the various load thresholds > (MAX_PING_TIME etc) until nodes seem to be using the configured amount > of bandwidth?
I'm not sure what you are suggesting or why it would improve throughput. Increasing MAX_PING_TIME would have no effect, for example, because most nodes mostly reject on bandwidth liability. But the point I am making is *we don't even limit effectively on bandwidth liability* : busy-looping until a request gets through shouldRejectRequest() improves performance significantly, therefore backoff and AIMD is not supplying enough requests to the front end of the current load limiting system. > > > When a node has spare capacity in terms of the number of requests it can > > carry, it sends a message to this effect, inviting requests, to each of its > > peers. A node receiving such a message will normally send the queued request > > closest location-wise to the sender, although timing concerns may factor in > > also. Within a short period several requests will have been received, and the > > node will accept one and reject the others. > > This sounds pretty similar to the current system - when a peer rejects a > request we raise a flag and don't send it any more requests until the > flag is lowered. But the flag is currently lowered by a timer, and > you're saying it should be lowered by an explicit signal from the peer? Yes. Well really it's a form of token passing, but I'm trying to make it simple and obviously correct. > That makes sense in principle, but whether it works in practice depends > on whether nodes can estimate their capacity well enough to send the > signal at the right time. > > > How to define spare capacity? Initially we simply have an arbitrary number of > > requests. We can tune this, or introduce an algorithm to tune it, later. > > This is the real problem: a node doesn't know when it's busy. I'm not > sure it makes sense to change the signalling between nodes until this > problem is solved. We have lots of ways of estimating capacity at the moment: the thread limit, bandwidth liability, ... We can use all of these to work out whether we can carry more requests, and we can add an additional one, the number of running requests (including those queued). > > When you say "an arbitrary number of requests" do you mean that when the > queue reaches a certain length the node considers itself busy? Yes. > What > happens when a local client dumps a load of requests on the queue - do > we tell all the peers to back off until we've processed all the local > requests? It can't. We are not talking about the same queue. Local requests from the various client-layer queues have to go through exactly the same process. > > > One obvious problem is that requests which are not close to any of our nodes > > may not get sent for a very long time, and worse, they may clog up the node > > so that requests which are closer to the target don't get through. > > I don't see how the head of the queue could be blocked but some other > item in the queue could be sendable. Either a peer has handled twice the > median number of requests, in which case it's embargoed for all > requests, or it hasn't, in which case we can send it the request at the > head of the queue. I mean that requests queued may not be successfully forwarded because they are too far away from any of our peers' locations, yet since they don't go away, our peers cannot send us any more requests which are closer to the target. I believe what I said about this is sufficient. > > (Incidentally we need to recalculate the median when a peer disconnects, > otherwise we could deadlock with all the fast peers embargoed and all > the slow peers disconnected.) Of course. > > Cheers, > Michael -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available URL: <https://emu.freenetproject.org/pipermail/devl/attachments/20071130/c892f9ce/attachment.pgp>
