CC'ing Martin in case he has any ideas.

On Thursday 27 January 2011 19:01:28 Dennis Nezic wrote:
> On Thu, 27 Jan 2011 18:49:57 +0000, Matthew Toseland wrote:
> > On Thursday 27 January 2011 17:17:15 Dennis Nezic wrote:
> > > On Thu, 27 Jan 2011 16:55:58 +0000, Matthew Toseland wrote:
> > > > On Wednesday 26 January 2011 20:14:52 Dennis Nezic wrote:
> > > > > On Wed, 26 Jan 2011 19:59:43 +0000, Matthew Toseland wrote:
> > > > > > On Wednesday 26 January 2011 19:42:38 Dennis Nezic wrote:
> > > > > > > On Wed, 26 Jan 2011 19:38:55 +0000, Matthew Toseland wrote:
> > > > > > > > On Monday 24 January 2011 22:28:36 you wrote:
> > > > > > > > > On Mon, 24 Jan 2011 18:29:14 +0000, Matthew Toseland
> > > > > > > > > wrote:
> > > > > > > > > > Freenet 0.7.5 build 1336 is now available. Please
> > > > > > > > > > upgrade, it will be mandatory on Friday. Much of this
> > > > > > > > > > build is intended to try to improve network
> > > > > > > > > > performance and particularly to prevent transfer
> > > > > > > > > > failures especially on realtime requests.
> > > > > > > > > > 
> > > > > > > > > > Details:
> > > > > > > > > > - Fix some block transfer bugs related to transfer
> > > > > > > > > > coalescing, disconnects and reassigning to self after
> > > > > > > > > > a request has taken too long.
> > > > > > > > > > - Keep on transferring the data if we need it for a
> > > > > > > > > > local request, and explain why this is safe in
> > > > > > > > > > comments. However it will go away when we get rid of
> > > > > > > > > > receiver-side transfer cancels anyway.
> > > > > > > > > > - Fix even more bugs related to request forking on
> > > > > > > > > > timeout.
> > > > > > > > > > - Always drop the queued messages when we disconnect
> > > > > > > > > > due to a timeout.
> > > > > > > > > > - Eliminate turtle transfers, they are no longer
> > > > > > > > > > necessary and involve unnecessary complexity and
> > > > > > > > > > transfer cancels. We will soon eliminate receiver
> > > > > > > > > > cancels too which will further simplify matters.
> > > > > > > > > > - Remove timed out filters more often.
> > > > > > > > > > - Show totals for backoff times.
> > > > > > > > > > - Fixes to auto-testing code.
> > > > > > > > > 
> > > > > > > > > This build still can't manage to handle input bandwidth
> > > > > > > > > sanely, on a congested connection -- it entirely
> > > > > > > > > consumes my already busy connection. (Aka. my freenet
> > > > > > > > > is still unusable here.)
> > > > > > > > 
> > > > > > > > Limiting input bandwidth usage accurately is
> > > > > > > > extraordinarily difficult and most people have fat pipes
> > > > > > > > downstream.
> > > > > > > > 
> > > > > > > > Probably your peers are resending packets constantly.
> > > > > > > 
> > > > > > > So, umm, make them stop, after say a minute of flooding?
> > > > > > > 
> > > > > > And disconnect completely? That kind of defeats the point
> > > > > > doesn't it?
> > > > > 
> > > > > No, that doesn't defeat the point. The point is to have a
> > > > > running Freenet, which is simply not possible at the moment,
> > > > > since it will flood my internet connection to an unusable state.
> > > > 
> > > > A running Freenet that disconnects from all your peers and
> > > > constantly announces because your connection is broken? How is
> > > > that useful?
> > > 
> > > My connection isn't broken. I actually took much pain to somewhat
> > > guarantee that my Freenet gets the bandwidth I allocate it.
> > > 
> > > 
> > > > > Moreover, what's the problem with "completely disconnecting"
> > > > > from a peer, after it continues to flood us for many minutes?
> > > > 
> > > > You are flooding it, not the other way around. It's entirely your
> > > > fault for not having the bandwidth to handle the data. Freenet is
> > > > tested and developed on typical connections which have lots of
> > > > downstream bandwidth and not much upstream bandwidth. And Freenet
> > > > is only doing exactly what TCP would do! Admittedly TCP has a
> > > > different algorithm for estimating round trip times, and a more
> > > > precise congestion control mechanism. But I have yet to be
> > > > convinced that this is a serious issue for any other person than
> > > > you and therefore worth spending significant amounts of developer
> > > > time on. I'm also not sure exactly what would fix it...
> > > 
> > > I really think you haven't understood the problem yet. This isn't
> > > just a bumpy averaging of bursting input -- this is a *sustained 5+
> > > minute flood that completely uses up 100KB/s downstream connection,
> > > and 5x the bandwidth I allocate it*. You're not sure what would fix
> > > it? Like I mentioned a couple times here already, *how about after
> > > a minute or two, telling my connected peers to SLOW DOWN*?
> > 
> > The reason they are sending data is that they are NOT receiving your
> > acknowledgements. So sending yet more control data won't help.
> 
> Well that sure sounds like a recipe for disaster. How long are peers
> supposed to keep pushing data to un-acknowledging peers, before they
> do something?

We disconnect from a node if we haven't received anything from it in 60 seconds.
> 
> (My last flood occurred for over 10 minutes, and then managed to stop.
> I believe all 5 of my connected strangers were listed as BackedOff
> during the flood. I will try to provide more details and more testing.)

What was your upstream bandwidth doing during this period? The reason for 
resends is that the original messages were not acked.

Several possibilities occur to me:
1. Maybe we are not acking all of the pending packets, due to limitations of 
the ack compression code. However IMHO this is unlikely due to the fact that 1) 
we will send packets with only acks even when we are out of bandwidth (and then 
not send data packets to compensate), and 2) we always use the same packet 
number sequence, even when we rekey, so long gaps are unlikely. It would be 
possible to rewrite the format to support acking a wider range if necessary. A 
complicating factor is that packets that only contain acks also get a packet 
number, although they don't need to be acked; but this does increase the chance 
of not being able to ack everything.
2. What if the problem is that a lot of packets get acked in one packet, and 
then that packet is lost?
3. Congestion control currently operates at the wrong level. We control the 
number of block transfer messages which are queued and awaiting sending and 
acking, not the number of messages actually in flight. If we have data to 
resend we send it before we start sending new data; but if the sender is not 
limited by bandwidth (which is clearly the case here), it won't throttle back 
until it has quite a lot of data in flight (the maximum window size, around 
250KB, which for a single connection could be 60 seconds plus). We really 
should implement congestion control at the low level.
4. Our RTT calculation is not very accurate, we should use something closer to 
TCP's algorithm.

I am still not convinced this is a critical bug, however I do see highish 
retransmits sometimes. Thoughts, Martin?

Attachment: signature.asc
Description: This is a digitally signed message part.

_______________________________________________
Support mailing list
Support@freenetproject.org
http://news.gmane.org/gmane.network.freenet.support
Unsubscribe at http://emu.freenetproject.org/cgi-bin/mailman/listinfo/support
Or mailto:support-requ...@freenetproject.org?subject=unsubscribe

Reply via email to