On Monday 29 Aug 2011 19:28:50 Ian Clarke wrote: > Ok, thinking about it, here is a proposal, or rather, the beginning of a > proposal. I'm assuming that we get rid of NLM, fair sharing, and anything > else intended to control load, and replace it with this. We will absolutely > need to simulate this before we write a single line of code to deploy it.
I have justified fair sharing in my other reply. However alternate mechanisms might replace it. I agree we should simulate, although this might be significant work. > > The core idea is that a node will include a floating point number in > response to any kind of request showing how close that node is to being > overloaded. 0.0 would mean its doing nothing, 1.0 would mean that it is > completely saturated and must reject requests. Clearly the goal is to avoid > getting anywhere near to 1.0. > > A node tracks several things: All of these apply to both locally originated requests and remotely originated requests? And are they for only our direct peers? > > - What is the overall average load reported by responses this node has > received > - What is the average load reported by responses this node has received, > per remote node > - What is the average load reported by responses this node forwarded, per > remote node Ahhh, this one could be interesting - you could use it to penalise nodes which spam excessively. > > I think, given these metrics, we should be able to do the following: > > - Limit our overall rate of initiating local requests based on the global > average reported load We can already do this on the basis of AIMD's, based purely on behaviour of local requests. However, taking more data into account might result in more accurate results - although it might also dilute the feedback. In any case, this part is "safe": it's not going to cause feedback problems, on its own. Conversely, it doesn't deal with problems such as network bottlenecks. However, while AIMD's are based on the *global* average load (or rather the load of all the nodes along the chain), it sounds like you are proposing to only report the *local* load of our direct peers? How are you going to compute the global average load? > - Limit our rate of local requests based on the average load of the > connection to the peer it would need to be forwarded to Not sure I follow here. Are you proposing that we initiate more requests in keyspace areas where there is lower load? > - Detect when remote peers are abusing the system by disregarding load - > as evidenced by a significantly higher average load in replies forwarded to > them Okay, that is useful - although I doubt it would be an unambiguous detection, and what would we do about it? We would have to reject requests, and if we're not sure, or if it could be a downstream troublemaker, couldn't that result in misrouting and make things worse? On the other hand, any malicious node would be most easily detected by the nodes directly connected to it, so this could work much the same way as fair sharing - provided that it can reject or terminate the requests. Arguably fair sharing achieves the same objective. It could be extended in various ways e.g. if we are relaying a lot of slow-down notices to a peer we could start to think about squeezing the peer's allocation. > > Of course, lots of questions: > > - How do we compute the averages? A decaying running average of some > kind? What time period? Yay more tunables. :( > - How do we translate load into a desired rate of requests? And even more tunables. :( > - What criteria indicate that a peer is abusing the system? What is the > remedy? In the current system, don't accept their requests. In ANY system, there will always be a borderline below which they can get away with it. We can either try to make this as small as possible or try not to hit legitimate traffic too hard. > > This is basically control theory stuff, and we'll need robust answers to > these questions before we proceed (ideally with a theoretical rather than > experimental foundation). IMHO small tweaks to AIMDs, combined with fair sharing, will achieve much the same without having to wait years for our theoreticians to wake up. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: This is a digitally signed message part. URL: <https://emu.freenetproject.org/pipermail/devl/attachments/20110829/c06395ac/attachment.pgp>
