The variety of ideas being thrown around about how to load balance is quite reminiscent of the alchemy we used for routing pre-NGR. Perhaps we should rethink load-balancing.
So, lets start with what we are trying to achieve:
The goal of load balancing is to have a node receive as many requests as it can handle but no more. Our current mechanism for achieving this relies on an effectively binary negative feedback mechanism that just isn't working. The solution is probably to make the feedback contain more information.
So: A node should be able to establish how many queries it can handle without problems in a given stretch of time (I assume this should be reasonably easy to do experimentally). What if a node told other nodes how many requests each of them are permitted to send to it in a given time-period to ensure that the total such number of incoming requests aren't exceeded?
Now, obviously many nodes might not meet their quota, in which case part of their quota could be redistributed to other nodes in some manner, although they would have the right to reclaim their quota should they need to send more requests than they have been doing.
Still a somewhat unformed idea, but perhaps a basis for a non-alchemaic (sp?) approach to load-balancing?
Ian.
Just some more thoughts on this:
The total available request quota could be adjusted constantly, such that when a node is experiencing overload it can lower the quota, if it isn't it can increase it.
A node would therefore know, for all of its references, how many requests it is permitted to send (within a time-period) and NGR could even take account of that information in some way, although that is probably over-ambitious for an initial implementation.
We would need to think about the algorithm for allowing nodes that send more requests to soak up unused quota for nodes which are under-using their allocation while allowing nodes that are underusing their quota to reclaim their unused proportion when they need it.
Now, it would be better if we didn't need to resort to this approach, so we should continue to try to get the QR mechanism to work, but this is probably a good last-resort.
Ian.
I *think* I understand (on the basis of IRC chat) what you're proposing here:
By default a new node gets an equal allocation of the quota; but if it doesn't use it up, it will get a smaller allocation next time, and if it uses it all up, it will get the allocation given up by other nodes which don't use it.
What if a node tries to get greedy and query more than his allocation? I believe you've said that then *all* requesting nodes would be punished at once and no one would be able to query. The so-called "greedy" node couldn't benefit---he'd really have to be mean or stupid. Have I got this right?
If so, doesn't this allow for a really easy DoS attack? Wouldn't a malicious node only have to use a small portion of bandwidth to exceed his allocation and halt queries for everyone else?
Or would you instead use a negative trust system? If so, continue that part of the discussion in 'Can "negative trust" work?'.
Yet another possibility is to not use a per-node quota system, but instead use a public quota system. A node tells everyone "I'll be able to handle up to x queries FROM EVERYONE IN TOTAL for the next y seconds" and then strictly enforces that statement. (It may be best if the node chose x to be a bit higher than he really expects to be able to handle. That way, he doesn't risk his node being under-utilized if it turns out he has more capacity than expected.)
The public quota system avoids making it so much easier for a malicious node to force us to QR. Also, we retain benefit of having made our QR:s more informative, since querying nodes know that if they receive one QR during the time period, it will be pointless to send any more queries until the y seconds are over.
-Martin
_______________________________________________ Devl mailing list [EMAIL PROTECTED] http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/devl
