Fair queueing automatically propagates load back to source was Re: [freenet-dev] Alternative congestion control algorithm

2006-04-24 Thread Matthew Toseland
On Fri, Apr 21, 2006 at 10:09:00PM +0100, Michael Rogers wrote: > > This is what a token bucket achieves. Each neighbour's bucket is filled > with tokens at the same rate. (The bucket is actually just implemented > as a counter, where the 'size' of the bucket is the upper limit of the > counter.)

Re: Fair queueing automatically propagates load back to source was Re: [freenet-dev] Alternative congestion control algorithm

2006-04-24 Thread Matthew Toseland
On Fri, Apr 21, 2006 at 10:09:00PM +0100, Michael Rogers wrote: > > This is what a token bucket achieves. Each neighbour's bucket is filled > with tokens at the same rate. (The bucket is actually just implemented > as a counter, where the 'size' of the bucket is the upper limit of the > counter.)

Fair queueing automatically propagates load back to source was Re: [freenet-dev] Alternative congestion control algorithm

2006-04-21 Thread Michael Rogers
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Matthew Toseland wrote: > - If we simply move on to the next node (since backed off nodes are > ignored for routing at present and that seems to work), then to some > degree the load will go to where it is least overloaded... ...but at the cost of

Fair queueing automatically propagates load back to source was Re: [freenet-dev] Alternative congestion control algorithm

2006-04-21 Thread Matthew Toseland
On Fri, Apr 21, 2006 at 10:09:00PM +0100, Michael Rogers wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > Matthew Toseland wrote: > > Is there a non-alchemical way to define such a queueing algorithm? > > Yup, see my message from the following morning about token buckets. I > guess fr

Fair queueing automatically propagates load back to source was Re: [freenet-dev] Alternative congestion control algorithm

2006-04-21 Thread Michael Rogers
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Matthew Toseland wrote: > Is there a non-alchemical way to define such a queueing algorithm? Yup, see my message from the following morning about token buckets. I guess from the flood of messages that you've just dealt with a week's worth of mail. :-)

[freenet-dev] Alternative congestion control algorithm

2006-04-21 Thread Matthew Toseland
On Fri, Apr 14, 2006 at 10:24:53AM +0100, Michael Rogers wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > Sorry to contradict what I said yesterday, but last night I tried to go > through the argument from first principles and came to a different > conclusion. > > What are we trying t

[freenet-dev] Alternative congestion control algorithm

2006-04-21 Thread Matthew Toseland
On Thu, Apr 13, 2006 at 05:42:23PM +0100, Michael Rogers wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > OK, I think this is the fix: keep a separate rate for each pair of > neighbours, and adjust it in response to both local and non-local > RejectedOverloads. > > E F > |

Fair queueing automatically propagates load back to source was Re: [freenet-dev] Alternative congestion control algorithm

2006-04-21 Thread Matthew Toseland
On Thu, Apr 13, 2006 at 05:39:58PM +0100, Michael Rogers wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > Matthew Toseland wrote: > > I mean we won't get enough information to have a useful and accurate > > rate for each pair. > > I see your point, but hopefully it won't be necessary

Re: Fair queueing automatically propagates load back to source was Re: [freenet-dev] Alternative congestion control algorithm

2006-04-21 Thread Michael Rogers
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Matthew Toseland wrote: > - If we simply move on to the next node (since backed off nodes are > ignored for routing at present and that seems to work), then to some > degree the load will go to where it is least overloaded... ...but at the cost of

Re: Fair queueing automatically propagates load back to source was Re: [freenet-dev] Alternative congestion control algorithm

2006-04-21 Thread Matthew Toseland
On Fri, Apr 21, 2006 at 10:09:00PM +0100, Michael Rogers wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > Matthew Toseland wrote: > > Is there a non-alchemical way to define such a queueing algorithm? > > Yup, see my message from the following morning about token buckets. I > guess fr

Re: Fair queueing automatically propagates load back to source was Re: [freenet-dev] Alternative congestion control algorithm

2006-04-21 Thread Michael Rogers
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Matthew Toseland wrote: > Is there a non-alchemical way to define such a queueing algorithm? Yup, see my message from the following morning about token buckets. I guess from the flood of messages that you've just dealt with a week's worth of mail. :-)

Re: [freenet-dev] Alternative congestion control algorithm

2006-04-21 Thread Matthew Toseland
On Fri, Apr 14, 2006 at 10:24:53AM +0100, Michael Rogers wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > Sorry to contradict what I said yesterday, but last night I tried to go > through the argument from first principles and came to a different > conclusion. > > What are we trying t

Re: [freenet-dev] Alternative congestion control algorithm

2006-04-21 Thread Matthew Toseland
On Thu, Apr 13, 2006 at 05:42:23PM +0100, Michael Rogers wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > OK, I think this is the fix: keep a separate rate for each pair of > neighbours, and adjust it in response to both local and non-local > RejectedOverloads. > > E F > |

Fair queueing automatically propagates load back to source was Re: [freenet-dev] Alternative congestion control algorithm

2006-04-21 Thread Matthew Toseland
On Thu, Apr 13, 2006 at 05:39:58PM +0100, Michael Rogers wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > Matthew Toseland wrote: > > I mean we won't get enough information to have a useful and accurate > > rate for each pair. > > I see your point, but hopefully it won't be necessary

[freenet-dev] Alternative congestion control algorithm

2006-04-14 Thread Michael Rogers
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Sorry to contradict what I said yesterday, but last night I tried to go through the argument from first principles and came to a different conclusion. What are we trying to achieve with load balancing / congestion control? 1. Reduce load on overloade

Re: [freenet-dev] Alternative congestion control algorithm

2006-04-14 Thread Michael Rogers
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Sorry to contradict what I said yesterday, but last night I tried to go through the argument from first principles and came to a different conclusion. What are we trying to achieve with load balancing / congestion control? 1. Reduce load on overloade

[freenet-dev] Alternative congestion control algorithm

2006-04-13 Thread Michael Rogers
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 OK, I think this is the fix: keep a separate rate for each pair of neighbours, and adjust it in response to both local and non-local RejectedOverloads. E F | | A---B---C---D D sends a RejectedOverload in response to traffic from A. C redu

[freenet-dev] Alternative congestion control algorithm

2006-04-13 Thread Michael Rogers
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Matthew Toseland wrote: > I mean we won't get enough information to have a useful and accurate > rate for each pair. I see your point, but hopefully it won't be necessary to have a very accurate rate for pairs that don't communicate much. The rate onl

[freenet-dev] Alternative congestion control algorithm

2006-04-13 Thread Matthew Toseland
On Thu, Apr 13, 2006 at 04:45:45PM +0100, Michael Rogers wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > Your first diagram: E F | | A---B---C---D > > That seems a bit much. Can it be avoided? I worry that we won't get > > enough data if we have to keep one for each pair.

[freenet-dev] Alternative congestion control algorithm

2006-04-13 Thread Michael Rogers
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 > That seems a bit much. Can it be avoided? I worry that we won't get > enough data if we have to keep one for each pair. The rate's only adjusted when we get data so I don't think it will be a problem - if there's very little traffic between a given

[freenet-dev] Alternative congestion control algorithm

2006-04-13 Thread Matthew Toseland
On Thu, Apr 13, 2006 at 04:06:02PM +0100, Michael Rogers wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > Hmm, my suggestion needs to be modified to deal with the following scenario: > > E F > | | > A---B---C---D > > D sends a RejectedOverload to C, which reduces its rate

[freenet-dev] Alternative congestion control algorithm

2006-04-13 Thread Michael Rogers
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hmm, my suggestion needs to be modified to deal with the following scenario: E F | | A---B---C---D D sends a RejectedOverload to C, which reduces its rate to D. If A carries on sending at a high rate then C sends a RejectedOverload to B,

[freenet-dev] Alternative congestion control algorithm

2006-04-13 Thread Matthew Toseland
Yes, it might. I like your suggestion, but need some feedback from Ian on it. On Thu, Apr 13, 2006 at 01:59:02PM +0100, Michael Rogers wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > Matthew Toseland wrote: > > Yep. One request send rate for all neighbours combined. > ... > > If it's

[freenet-dev] Alternative congestion control algorithm

2006-04-13 Thread Michael Rogers
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Matthew Toseland wrote: > Yep. One request send rate for all neighbours combined. ... > If it's local, it backs off. Either way it reduces its overall send > rate via AIMD. This seems like it might lead to a chain reaction: when a request is rejected,

[freenet-dev] Alternative congestion control algorithm

2006-04-13 Thread Matthew Toseland
On Thu, Apr 13, 2006 at 12:46:41PM +0100, Michael Rogers wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > Sounds good, but could you clarify a couple of points? > > The "request send rate" applies to forwarded requests as well as > originated requests, right? Yep. One request send ra

[freenet-dev] Alternative congestion control algorithm

2006-04-13 Thread Michael Rogers
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Sounds good, but could you clarify a couple of points? The "request send rate" applies to forwarded requests as well as originated requests, right? And there's a separate rate for requests sent/forwarded to each neighbour (as opposed to one rate for a

[freenet-dev] Alternative congestion control algorithm

2006-04-13 Thread Matthew Toseland
I propose the following congestion control algorithm: Each node tracks all requests passing through it, and uses them to establish a current request send rate by AIMD (for each type of request). It sends a request every time it can, using a fair queueing algorithm - one queue from each node, and o

Re: [freenet-dev] Alternative congestion control algorithm

2006-04-13 Thread Michael Rogers
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 OK, I think this is the fix: keep a separate rate for each pair of neighbours, and adjust it in response to both local and non-local RejectedOverloads. E F | | A---B---C---D D sends a RejectedOverload in response to traffic from A. C redu

Re: [freenet-dev] Alternative congestion control algorithm

2006-04-13 Thread Michael Rogers
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Matthew Toseland wrote: > I mean we won't get enough information to have a useful and accurate > rate for each pair. I see your point, but hopefully it won't be necessary to have a very accurate rate for pairs that don't communicate much. The rate onl

Re: [freenet-dev] Alternative congestion control algorithm

2006-04-13 Thread Ian Clarke
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 This looks good, but I think it would be imprudent to deploy it without simulating it. Is there an easy way we could simulate this? Ian. On 13 Apr 2006, at 07:12, Matthew Toseland wrote: Yes, it might. I like your suggestion, but need some fee

[freenet-dev] Alternative congestion control algorithm

2006-04-13 Thread Ian Clarke
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 This looks good, but I think it would be imprudent to deploy it without simulating it. Is there an easy way we could simulate this? Ian. On 13 Apr 2006, at 07:12, Matthew Toseland wrote: > Yes, it might. I like your suggestion, but need some feed

Re: [freenet-dev] Alternative congestion control algorithm

2006-04-13 Thread Matthew Toseland
On Thu, Apr 13, 2006 at 04:45:45PM +0100, Michael Rogers wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > Your first diagram: E F | | A---B---C---D > > That seems a bit much. Can it be avoided? I worry that we won't get > > enough data if we have to keep one for each pair.

Re: [freenet-dev] Alternative congestion control algorithm

2006-04-13 Thread Michael Rogers
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 > That seems a bit much. Can it be avoided? I worry that we won't get > enough data if we have to keep one for each pair. The rate's only adjusted when we get data so I don't think it will be a problem - if there's very little traffic between a given

Re: [freenet-dev] Alternative congestion control algorithm

2006-04-13 Thread Matthew Toseland
On Thu, Apr 13, 2006 at 04:06:02PM +0100, Michael Rogers wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > Hmm, my suggestion needs to be modified to deal with the following scenario: > > E F > | | > A---B---C---D > > D sends a RejectedOverload to C, which reduces its rate

Re: [freenet-dev] Alternative congestion control algorithm

2006-04-13 Thread Michael Rogers
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hmm, my suggestion needs to be modified to deal with the following scenario: E F | | A---B---C---D D sends a RejectedOverload to C, which reduces its rate to D. If A carries on sending at a high rate then C sends a RejectedOverload to B,

Re: [freenet-dev] Alternative congestion control algorithm

2006-04-13 Thread Matthew Toseland
Yes, it might. I like your suggestion, but need some feedback from Ian on it. On Thu, Apr 13, 2006 at 01:59:02PM +0100, Michael Rogers wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > Matthew Toseland wrote: > > Yep. One request send rate for all neighbours combined. > ... > > If it's

Re: [freenet-dev] Alternative congestion control algorithm

2006-04-13 Thread Michael Rogers
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Matthew Toseland wrote: > Yep. One request send rate for all neighbours combined. ... > If it's local, it backs off. Either way it reduces its overall send > rate via AIMD. This seems like it might lead to a chain reaction: when a request is rejected,

Re: [freenet-dev] Alternative congestion control algorithm

2006-04-13 Thread Matthew Toseland
On Thu, Apr 13, 2006 at 12:46:41PM +0100, Michael Rogers wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > Sounds good, but could you clarify a couple of points? > > The "request send rate" applies to forwarded requests as well as > originated requests, right? Yep. One request send ra

Re: [freenet-dev] Alternative congestion control algorithm

2006-04-13 Thread Michael Rogers
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Sounds good, but could you clarify a couple of points? The "request send rate" applies to forwarded requests as well as originated requests, right? And there's a separate rate for requests sent/forwarded to each neighbour (as opposed to one rate for a

[freenet-dev] Alternative congestion control algorithm

2006-04-13 Thread Matthew Toseland
I propose the following congestion control algorithm: Each node tracks all requests passing through it, and uses them to establish a current request send rate by AIMD (for each type of request). It sends a request every time it can, using a fair queueing algorithm - one queue from each node, and o