OK, Here is a possible solution that addresses most of the current issues we are currently seeing.
First there is a queue that all incoming requests go into. Obviously if the queue is empty and we have threads waiting, it should be processed immediately. Otherwise, because it is to our advantage to try to improve both the average request time, and the reliability of the network, we will process requests out of both ends of the queue. One end is a LIFO. This would be the default end. When a new thread becomes available, it grabs a request from this end and removes it from the queue, and sends a query accept. On the other end (the FIFO end) we should check if we have previously seen that key before. If we have, send an accept and process the request. If not, mark it as such, and leave it in the queue. Then to prevent the queue from getting to large there should be some assumed timeout, where the requesting node just assumes it was dropped if it has been more than a given amount of time without an accept. If that much time has gone by and the request is still in the queue, then drop it, but don't bother to send any message back. Because Freenet inherently uses more outgoing bandwidth, (some of the stuff has to come from our store) as soon as that becomes saturated, or we are out of threads, we need to backoff our processing of requests. In that case, it would benefit the network if we let other nodes know the situation, so they can backoff, rather than just having lots of requests fall off the queue. So when requests start falling off the queue, we should start QRing incoming requests in proportion to our % overload. So, ideally speaking we accept exactly the number that we can process in any given period of time. Now to prevent any single node, overloading you at the expense of others, we should reject incoming requests with the probability of (FractionOfCurrentlyPendingRequestsFromThisNode * Current%OfOverload). This will insure that a node that is using more of the resources will be more likely to be rejected. Also so that other nodes know how likely they are to be rejected, it would be usefull if a query reject contained these two numbers. However the problem remains that regardless of the amount of resources that any node uses they would get the same performance. To solve this, instead of simply grabbing each request from the queue, on the LIFO end, each query should have a probability of being skipped over with a probability of FractionOfCurrentlyPendingRequestsFromThisNode. This way other nodes get more of a chance. Of course if it happens to skip through all the entries in the queue, it should just start from the beginning again. That way eventually one will be selected to be processed. This solves the problem such that, even under load the average successful time is fast. If we are overloaded, we don't necessarily waste bandwidth to tell others we are overloaded and we don't drop requests that are close to actually locating the data. Finally it prevents any node from nudging out others in terms of both queries accepted or performance time. And our node is capable of giving this information to the requesting node, so that they are able to non-alchemistically guess the relative performance that they will get even as the load changes. _______________________________________________ Devl mailing list [EMAIL PROTECTED] http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/devl
