On Saturday 08 Jun 2013 15:28:27 Vladislav Sterzhanov wrote: > 2013/6/8 Matthew Toseland <[email protected]> > > > The previously proposed next steps were (there are bugs for these): > > 1) allow including packets more than 255 apart (via a bit flag probably) > > 2) allow sending ranges (again use a bit flag) > > 3) allow including packets in a range that we've already sent an ack for, > > if it makes it shorter. > > > > It feels like reorganizing it to range-only principle could produce the > significant improvements in the majority of cases. Of course, needs > testing.
I don't know whether we need to efficiently encode single acks, maybe we don't.
We do IMHO need delta compression as now to save space (which is not
incompatible with ranges), but we need some way to override it so we can
acknowledge packets or ranges that are far apart, in case something wierd
happens.
> Concerning #3 - the extra packets could be included not only for
> the means of reducing the ack cost, but also for providing robustness in
> case of lossy connections in each direction, especially if it turns out to
> be a zero-cost ack. This is one of the options that could be triggered
> on-the-fly depending on the state of the connection (this one is quite
> simple since it requires changes in the behaviour of nodes independently of
> each other).
Right. If it reduces cost we should do it. If it increases cost, but also
increases resiliency, we could do it anyway depending on the circumstances.
Obviously we'd need updated unit tests for all this.
>
> > Interesting idea. We could still use relative offsets most of the time for
> > the start of the range. Or if we're only interested in rearranging, we
> > could just send a flag, or a counter, for the number of times when we've
> > received packet n and then received packet n-1?
>
> It's just an idea how to utilize the range-acking further, if it turns out
> to be a more efficient way. Curious, how this would affect unstable
> connections, perhaps, we will need to introduce the flags, but, as for me,
> it seems not the most natural way to do it.
Delta compression means sending the ranges in any order other than that of the
packet sequence numbers will be less efficient. So it probably makes sense to
have a flag or a counter?
>
> > Hmmm, interesting. There are some delays here; most things get queued for
> > 100ms (coalescing delay). The packet encoding logic is in NPFPacket.java.
> > Where is the retransmit logic now? NewPacketFormat I think?
> >
> > NewPacketFormat.createPacket() is *always* used for sending a packet. We
> > resend at the byte-range level; we never resend exactly the same packet.
> > But we *do* mark packets as failed, and thus resend their contents:
> > NewPacketFormat.SentPacket.lost() is called by
> > NewPacketFormatKeyContext.checkForLostPackets()
> >
> > NewPacketFormat is per-peer, and tracks the messages in flight;
> > NewPacketFormatKeyContext is per-key, and tracks the packets.
> >
> > NewPacketFormatKeyContext.checkForLostPackets is worth a look. I *think*
> > what is going on here is we only implement "slow" retransmission. I'm not
> > sure that the timeout is sensible though, it looks like 1 RTT plus 220ms
> > for coalescing at both ends.
>
> Thanks for this one, need to think this over more thoroughly.
Right.
>
> > Let me know if you need anything. Hope some of the hints above will help.
>
> Of course they will. Thank you once again.
>
> > Please let us know where the repository is when you start coding.
>
> Sure, I just want to make the things clearer and plan the whole thing up
> before setting up the sails.
Yes, that's fine.
>
> > Also, it sounds like you are hoping to give some before and after
> > measurements? Is this "in vivo", or could you implement them as unit tests
> > (test/) / integration tests (src/freenet/node/simulator/)? (A few
> > benchmarks are set up this way, and there is support for randomly dropping
> > packets).
>
> Yeah, I hope to introduce at least basic testing - it's vital for valuing
> the changes I make. I think this should precede the actual changing of the
> use-proven code - so need to work this out in the near future, thanks for
> directions. Some additional hints about it would help - did not yet have a
> chance to implement testing of the functionality of this kind.
>
Ok. Well, what I said above: We have top-level tests, which use actual Node
instances, several of them within a single JVM, and which can take a long time
and large amounts of resources. And we have unit tests, most of which are much
lower level, which are much cheaper, and run during a normal compile, but there
are options to run longer lived / heavier unit tests ("extensive" and
"benchmark"), which trigger properties in TestProperty.java.
signature.asc
Description: This is a digitally signed message part.
_______________________________________________ Devl mailing list [email protected] https://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl
