Re: [6lo] Mirja Kühlewind's Discuss on draft-ietf-6lo-fragment-recovery-13: (with DISCUSS and COMMENT)
Hello Mirja: I published 20 with this addition. https://www.ietf.org/rfcdiff?url2=draft-ietf-6lo-fragment-recovery-20 I guess we're close to be done by now. In that case, I must thanks you again and deeply. Be safe, Pascal > -Original Message- > From: Mirja Kuehlewind > Sent: vendredi 20 mars 2020 18:12 > To: Pascal Thubert (pthubert) > Cc: 6lo-cha...@ietf.org; The IESG ; 6lo@ietf.org; > draft-ietf-6lo- > fragment-recov...@ietf.org; Carles Gomez ; Martin > Duke ; Benjamin Kaduk > Subject: Re: Mirja Kühlewind's Discuss on draft-ietf-6lo-fragment-recovery-13: > (with DISCUSS and COMMENT) > > Hi Pascal, > > The proposed text below and in your previous email look good! Thanks! > > I would appreciate and recommend to add also a shorter version of your good > explanation below to make the reader better understand which factor is the > limiting one when using a certain parameter setting and respectively which > factor could be tuned if any dynamic adaptation is implemented. > > Thanks! > Mirja > > P.S.: Off for today but will check for an update tomorrow in order to clear my > discuss! > > > > > On 20. Mar 2020, at 17:15, Pascal Thubert (pthubert) > wrote: > > > > Hello Mirja > > > > I missed this while working on yesterday's message. > > > > Let's see below: > > > > > >> Okay! :-) > >> > >> About the use of ECN, I agree as you say there should only be a few > >> fragments and not increasing might be okay. However, you would need > >> to clarify that the window is reset for each new datagram, I guess, right? > > > > Agreed. Let's change appendix C, more below; > > > >> Also I don’t think you > >> necessarily need to reduce to 1 on CE marking but maybe halve the > >> window or something. Or you leave this open like “If an E flag is > >> received the window SHOULD be reduced, at least by 1 and at max to 1. > >> Halving the window for each E flag received, could be a good > >> compromise but needs further experimentation.”… > > > > Thanks for this text! Appendix C now becomes: > > " > > Congestion on the forward path can also be indicated by an Explicit > > Congestion Notification (ECN) mechanism. Though whether and how ECN > > [RFC3168] is carried out over the LoWPAN is out of scope, this draft > > provides a way for the destination endpoint to echo an ECN indication > > back to the fragmenting endpoint in an acknowledgment message as > > represented in Figure 4 in Section 5.2. While the support of echoing > > the ECN at the reassembling endpoint is mandatory, this specification > > only provides a minimalistic behaviour on the fragmenting endpoint. > > If an E flag is received the window SHOULD be reduced, at least by 1 > > and at max to 1. Halving the window for each E flag received, could > > be a good compromise but needs further experimentation. A very > > simple implementation may just reset the window to 1 so the fragments > > are sent and acknowledged one by one. Note that the action on ECN > > only applies till the end of the datagram and the next datagram uses > > the configured Window_Size again. > > > > " > > > > I'd like to fully converge before I publish again > > > >> I wonder if it would be good to say a bit more about the recommended > >> values for the window size, as I think 32 will usually in most > >> network not limit transmission (and the limiting value will be IFG) > >> while with a size of 3, that's very conservative to not overload the > >> network (and will be slow than the limits induced by IFG). Is my > understanding correct? > > > > I'd believe so: > > > > Note that the exact use of the ack and the window_size is left to > implementation. The optimistic one could send all the fragments up to > window_size and ask for an ack only on the last one, wait for the bitmap, > which > means a gap of RTT/2, and resend the losses. > > Then again we want to leave room for learning. The pessimistic > implementation could set the bit on the first fragment to check that the path > works and open the window only upon receiving the ack. It could then set an > ack bit again on the second fragment and then use the window as a credit to > send up to window_size before it is blocked. In that case, if the ack comes > back > before the window starves, the gating factor is the IFG. If the ack does not > arrive in time, the window_size is the gating factor. > > > > If the sender uses the window_size as a credit, the window of 3 will be the > gating factor that starves the sender and causes gaps longer than IFG as soon > as you have more than 3 hops in a TSCH network and 5-6 hops in a single > frequency mesh. I guess that's what you mean by slower. The more hops the > more it will hurt. > > > > I initially used nb_of_hops as a rule of a thumb. That was an approximation > > of > RTT/(XMIT+IFG) in the case one fragment needs to progress only to the second > hop before the next can be sent (case of TSCH). > > > > The assumptions behind are that
[6lo] I-D Action: draft-ietf-6lo-fragment-recovery-20.txt
A New Internet-Draft is available from the on-line Internet-Drafts directories. This draft is a work item of the IPv6 over Networks of Resource-constrained Nodes WG of the IETF. Title : 6LoWPAN Selective Fragment Recovery Author : Pascal Thubert Filename: draft-ietf-6lo-fragment-recovery-20.txt Pages : 32 Date: 2020-03-20 Abstract: This draft updates RFC 4944 with a protocol to forward individual fragments across a route-over mesh and recover them end-to-end, with congestion control capabilities to protect the network. The IETF datatracker status page for this draft is: https://datatracker.ietf.org/doc/draft-ietf-6lo-fragment-recovery/ There are also htmlized versions available at: https://tools.ietf.org/html/draft-ietf-6lo-fragment-recovery-20 https://datatracker.ietf.org/doc/html/draft-ietf-6lo-fragment-recovery-20 A diff from the previous version is available at: https://www.ietf.org/rfcdiff?url2=draft-ietf-6lo-fragment-recovery-20 Please note that it may take a couple of minutes from the time of submission until the htmlized version and diff are available at tools.ietf.org. Internet-Drafts are also available by anonymous FTP at: ftp://ftp.ietf.org/internet-drafts/ ___ 6lo mailing list 6lo@ietf.org https://www.ietf.org/mailman/listinfo/6lo
Re: [6lo] Mirja Kühlewind's Discuss on draft-ietf-6lo-fragment-recovery-13: (with DISCUSS and COMMENT)
Hi Pascal, The proposed text below and in your previous email look good! Thanks! I would appreciate and recommend to add also a shorter version of your good explanation below to make the reader better understand which factor is the limiting one when using a certain parameter setting and respectively which factor could be tuned if any dynamic adaptation is implemented. Thanks! Mirja P.S.: Off for today but will check for an update tomorrow in order to clear my discuss! > On 20. Mar 2020, at 17:15, Pascal Thubert (pthubert) > wrote: > > Hello Mirja > > I missed this while working on yesterday's message. > > Let's see below: > > >> Okay! :-) >> >> About the use of ECN, I agree as you say there should only be a few fragments >> and not increasing might be okay. However, you would need to clarify that the >> window is reset for each new datagram, I guess, right? > > Agreed. Let's change appendix C, more below; > >> Also I don’t think you >> necessarily need to reduce to 1 on CE marking but maybe halve the window or >> something. Or you leave this open like “If an E flag is received the window >> SHOULD be reduced, at least by 1 and at max to 1. Halving the window for >> each E flag received, could be a good compromise but needs further >> experimentation.”… > > Thanks for this text! Appendix C now becomes: > " > Congestion on the forward path can also be indicated by an Explicit > Congestion Notification (ECN) mechanism. Though whether and how ECN > [RFC3168] is carried out over the LoWPAN is out of scope, this draft > provides a way for the destination endpoint to echo an ECN indication > back to the fragmenting endpoint in an acknowledgment message as > represented in Figure 4 in Section 5.2. While the support of echoing > the ECN at the reassembling endpoint is mandatory, this specification > only provides a minimalistic behaviour on the fragmenting endpoint. > If an E flag is received the window SHOULD be reduced, at least by 1 > and at max to 1. Halving the window for each E flag received, could > be a good compromise but needs further experimentation. A very > simple implementation may just reset the window to 1 so the fragments > are sent and acknowledged one by one. Note that the action on ECN > only applies till the end of the datagram and the next datagram uses > the configured Window_Size again. > > " > > I'd like to fully converge before I publish again > >> I wonder if it would be good to say a bit more about the recommended values >> for the window size, as I think 32 will usually in most network not limit >> transmission (and the limiting value will be IFG) while with a size of 3, >> that's >> very conservative to not overload the network (and will be slow than the >> limits >> induced by IFG). Is my understanding correct? > > I'd believe so: > > Note that the exact use of the ack and the window_size is left to > implementation. The optimistic one could send all the fragments up to > window_size and ask for an ack only on the last one, wait for the bitmap, > which means a gap of RTT/2, and resend the losses. > Then again we want to leave room for learning. The pessimistic implementation > could set the bit on the first fragment to check that the path works and open > the window only upon receiving the ack. It could then set an ack bit again on > the second fragment and then use the window as a credit to send up to > window_size before it is blocked. In that case, if the ack comes back before > the window starves, the gating factor is the IFG. If the ack does not arrive > in time, the window_size is the gating factor. > > If the sender uses the window_size as a credit, the window of 3 will be the > gating factor that starves the sender and causes gaps longer than IFG as soon > as you have more than 3 hops in a TSCH network and 5-6 hops in a single > frequency mesh. I guess that's what you mean by slower. The more hops the > more it will hurt. > > I initially used nb_of_hops as a rule of a thumb. That was an approximation > of RTT/(XMIT+IFG) in the case one fragment needs to progress only to the > second hop before the next can be sent (case of TSCH). > > The assumptions behind are that an rfrag_ack takes the same path and the same > time as the rfrag, which is mostly correct. So if we spread the fragments > with IFG, keeping busy one hop out of 2 on both directions means aligning the > window_size of the nb of hops. As you pointed out this is not a congestion > control measure, it is just the window that is not "slower" than IFG, per > your expression. The text that uses RTT/(XMIT+IFG) being equivalent I'm happy > either way. Because knowing the RTT is the same problem as knowing the number > of hops so we do not need to indicate both. > > 3 is when you h ave no idea of either RTT or nb of hops. it is > conservative and will starve the TSCH sender if we have more than 3 hops. It > also protects
Re: [6lo] Mirja Kühlewind's Discuss on draft-ietf-6lo-fragment-recovery-13: (with DISCUSS and COMMENT)
Hello Mirja I missed this while working on yesterday's message. Let's see below: > Okay! :-) > > About the use of ECN, I agree as you say there should only be a few fragments > and not increasing might be okay. However, you would need to clarify that the > window is reset for each new datagram, I guess, right? Agreed. Let's change appendix C, more below; > Also I don’t think you > necessarily need to reduce to 1 on CE marking but maybe halve the window or > something. Or you leave this open like “If an E flag is received the window > SHOULD be reduced, at least by 1 and at max to 1. Halving the window for > each E flag received, could be a good compromise but needs further > experimentation.”… Thanks for this text! Appendix C now becomes: " Congestion on the forward path can also be indicated by an Explicit Congestion Notification (ECN) mechanism. Though whether and how ECN [RFC3168] is carried out over the LoWPAN is out of scope, this draft provides a way for the destination endpoint to echo an ECN indication back to the fragmenting endpoint in an acknowledgment message as represented in Figure 4 in Section 5.2. While the support of echoing the ECN at the reassembling endpoint is mandatory, this specification only provides a minimalistic behaviour on the fragmenting endpoint. If an E flag is received the window SHOULD be reduced, at least by 1 and at max to 1. Halving the window for each E flag received, could be a good compromise but needs further experimentation. A very simple implementation may just reset the window to 1 so the fragments are sent and acknowledged one by one. Note that the action on ECN only applies till the end of the datagram and the next datagram uses the configured Window_Size again. " I'd like to fully converge before I publish again > I wonder if it would be good to say a bit more about the recommended values > for the window size, as I think 32 will usually in most network not limit > transmission (and the limiting value will be IFG) while with a size of 3, > that's > very conservative to not overload the network (and will be slow than the > limits > induced by IFG). Is my understanding correct? I'd believe so: Note that the exact use of the ack and the window_size is left to implementation. The optimistic one could send all the fragments up to window_size and ask for an ack only on the last one, wait for the bitmap, which means a gap of RTT/2, and resend the losses. Then again we want to leave room for learning. The pessimistic implementation could set the bit on the first fragment to check that the path works and open the window only upon receiving the ack. It could then set an ack bit again on the second fragment and then use the window as a credit to send up to window_size before it is blocked. In that case, if the ack comes back before the window starves, the gating factor is the IFG. If the ack does not arrive in time, the window_size is the gating factor. If the sender uses the window_size as a credit, the window of 3 will be the gating factor that starves the sender and causes gaps longer than IFG as soon as you have more than 3 hops in a TSCH network and 5-6 hops in a single frequency mesh. I guess that's what you mean by slower. The more hops the more it will hurt. I initially used nb_of_hops as a rule of a thumb. That was an approximation of RTT/(XMIT+IFG) in the case one fragment needs to progress only to the second hop before the next can be sent (case of TSCH). The assumptions behind are that an rfrag_ack takes the same path and the same time as the rfrag, which is mostly correct. So if we spread the fragments with IFG, keeping busy one hop out of 2 on both directions means aligning the window_size of the nb of hops. As you pointed out this is not a congestion control measure, it is just the window that is not "slower" than IFG, per your expression. The text that uses RTT/(XMIT+IFG) being equivalent I'm happy either way. Because knowing the RTT is the same problem as knowing the number of hops so we do not need to indicate both. 3 is when you h ave no idea of either RTT or nb of hops. it is conservative and will starve the TSCH sender if we have more than 3 hops. It also protects the intermediate node that can be very constrained, to at most 3 fragments per datagram that it relays. Note that I'm not aware of any full duplex radio available in LLNs (though there are prototypes for 5G). So this is really a corner case. As written 32 indicates the last fragment of the datagram, which is usually way less than 32. When used, as you understood well, the only protection is IFG, and I'm must say I'm quite happy with the progress we made on that text together. Please let me know, should I change something more? Pascal ___ 6lo mailing list 6lo@ietf.org https://www.ietf.org/mailman/listinfo/6lo
Re: [6lo] Mirja Kühlewind's Discuss on draft-ietf-6lo-fragment-recovery-13: (with DISCUSS and COMMENT)
Hello Mirja: Continuing from yesterday night. I published 18 and then 19 for this discussion. So the total diff is https://tools.ietf.org/tools/rfcdiff/rfcdiff.pyht?url1=https://tools.ietf.org/id/draft-ietf-6lo-fragment-recovery-17.txt=https://tools.ietf.org/id/draft-ietf-6lo-fragment-recovery-19.txt Based on the discussion below, 7.1 would become: " 7.1. Protocol Parameters The management system SHOULD be capable of providing the parameters listed in this section and an implementation MUST abide by those parameters and in particular never exceed the minimum and maximum configured boundaries. An implementation should consider the generic recommendations from the IETF in the matter of congestion control and rate management for IP datagrams in [RFC8085]. An implementation may perform a congestion control by using a dynamic value of the window size (Window_Size), adapting the fragment size (Fragment_Size), and may reduce the load by inserting an inter-frame gap that is longer than necessary. In a large network where nodes contend for the bandwidth, a larger Fragment_Size consumes less bandwidth but also reduces fluidity and incurs higher chances of loss in transmission. This is controlled by the following parameters: inter-frame gap: The inter-frame gap indicates the minimum amount of time between transmissions. The inter-frame gap controls the rate at which fragments are sent, the ratio of air time, and the amount of memory in intermediate nodes that a particular datagram will use. It can be used as a flow control, a congestion control, and/ or a collision control measure. It MUST be set at a minimum to a value that protects the propagation of one transmission against collision with next [FRAG-FWD]. In a wireless network that uses the same frequency along a path, this may represent the time for a frame to progress over multiple hops (more in Section 4.2). It SHOULD be augmented beyond this as necessary to protect the network against congestion. " Where section 4.2 is where we indicate that the IFG is "inserted between transmissions to the same next hop" > -Original Message- > From: Pascal Thubert (pthubert) > Sent: jeudi 19 mars 2020 20:12 > To: Mirja Kuehlewind > Cc: Pascal Thubert (pthubert) ; 6lo- > cha...@ietf.org; The IESG ; 6lo@ietf.org; draft-ietf-6lo- > fragment-recov...@ietf.org; Carles Gomez ; Martin > Duke ; Benjamin Kaduk > Subject: Re: Mirja Kühlewind's Discuss on draft-ietf-6lo-fragment-recovery-13: > (with DISCUSS and COMMENT) > > Hello Mirja > > > > > > >> > >> > >> Please see below the discussion: > >> > > -- > > --- > > - > > DISCUSS: > > - > > > >> > >> Note that all classical wireless interfaces perform ARQ. On one hop, you > >> get > the same effect of multiplying the traffic in the air vs. what the transport > see. > The factor can be high, e.g., 64. On a mesh, we get the additional effect of > multiple flows converging on a same node. > > > > Yes but with only “one hop”/the network you are connected to directly, and > there is usually also some kind of back-off mechanism that reacts to > congestion/collision/contention on that layer. > > > > > >> > >>> > >>> To be clear the request of this discuss is to give a normative > >>> recommendation for the default value of the window size and/or inter- > frame gap. > >> > >> Yes, and since there is no great expectation that ECN will be implemented, > that must be reasonable. > >> Also we want to agree on the proposed mechanism to drop the window to 1 > in case of congestion notification, or is that behind us already? > > > > Dropping to 1 on CE mark is fine. However, when do you increase the window > again? If you want to say something here, you have to specify that as well. > > > If we keep things really simple it would not. Note that this applies to a > single > data gram and that’s usually just a few fragments. > > We could double at each round trip but by the time it takes effect the > datagram will be done... > > > > >> > >> > >>> > >>> Further note, as you allow to adapt both the window and the > >>> inter-frame gap dynamically, you actually implement two control > >>> mechanisms here. I actually recommend to only use the inter-frame > >>> gap and don’t have window here. You say a couple of times in your > >>> reply below, that the window determines the ask- rate, however, it > >>> is not clear to me because the ack rate should be a parameter at the > >>> receiver and not at the sender (maybe I don’t remember things > >>> correctly because it’s a while back since I read the draft and I > >>> couple find anything about this in the draft quickly). If the window size > however does define the ack rate, then maybe you should rename that > parameter
[6lo] I-D Action: draft-ietf-6lo-fragment-recovery-19.txt
A New Internet-Draft is available from the on-line Internet-Drafts directories. This draft is a work item of the IPv6 over Networks of Resource-constrained Nodes WG of the IETF. Title : 6LoWPAN Selective Fragment Recovery Author : Pascal Thubert Filename: draft-ietf-6lo-fragment-recovery-19.txt Pages : 31 Date: 2020-03-20 Abstract: This draft updates RFC 4944 with a protocol to forward individual fragments across a route-over mesh and recover them end-to-end, with congestion control capabilities to protect the network. The IETF datatracker status page for this draft is: https://datatracker.ietf.org/doc/draft-ietf-6lo-fragment-recovery/ There are also htmlized versions available at: https://tools.ietf.org/html/draft-ietf-6lo-fragment-recovery-19 https://datatracker.ietf.org/doc/html/draft-ietf-6lo-fragment-recovery-19 A diff from the previous version is available at: https://www.ietf.org/rfcdiff?url2=draft-ietf-6lo-fragment-recovery-19 Please note that it may take a couple of minutes from the time of submission until the htmlized version and diff are available at tools.ietf.org. Internet-Drafts are also available by anonymous FTP at: ftp://ftp.ietf.org/internet-drafts/ ___ 6lo mailing list 6lo@ietf.org https://www.ietf.org/mailman/listinfo/6lo
Re: [6lo] Mirja Kühlewind's Discuss on draft-ietf-6lo-fragment-recovery-13: (with DISCUSS and COMMENT)
Hi Pascal, Okay! :-) About the use of ECN, I agree as you say there should only be a few fragments and not increasing might be okay. However, you would need to clarify that the window is reset for each new datagram, I guess, right? Also I don’t think you necessarily need to reduce to 1 on CE marking but maybe halve the window or something. Or you leave this open like “If an E flag is received the window SHOULD be reduced, at least by 1 and at max to 1. Halving the window for each E flag received, could be a good compromise but needs further experimentation.”… I wonder if it would be good to say a bit more about the recommended values for the window size, as I think 32 will usually in most network not limit transmission (and the limiting value will be IFG) while with a size of 3, that's very conservative to not overload the network (and will be slow than the limits induced by IFG). Is my understanding correct? Mirja > On 19. Mar 2020, at 20:12, Pascal Thubert (pthubert) > wrote: > > Hello Mirja > > > >> >>> >>> >>> Please see below the discussion: >>> >> - >> - >> DISCUSS: >> - >> >>> >>> Note that all classical wireless interfaces perform ARQ. On one hop, you >>> get the same effect of multiplying the traffic in the air vs. what the >>> transport see. The factor can be high, e.g., 64. On a mesh, we get the >>> additional effect of multiple flows converging on a same node. >> >> Yes but with only “one hop”/the network you are connected to directly, and >> there is usually also some kind of back-off mechanism that reacts to >> congestion/collision/contention on that layer. >> >> >>> To be clear the request of this discuss is to give a normative recommendation for the default value of the window size and/or inter-frame gap. >>> >>> Yes, and since there is no great expectation that ECN will be implemented, >>> that must be reasonable. >>> Also we want to agree on the proposed mechanism to drop the window to 1 in >>> case of congestion notification, or is that behind us already? >> >> Dropping to 1 on CE mark is fine. However, when do you increase the window >> again? If you want to say something here, you have to specify that as well. > > > If we keep things really simple it would not. Note that this applies to a > single data gram and that’s usually just a few fragments. > > We could double at each round trip but by the time it takes effect the > datagram will be done... > >> >>> >>> Further note, as you allow to adapt both the window and the inter-frame gap dynamically, you actually implement two control mechanisms here. I actually recommend to only use the inter-frame gap and don’t have window here. You say a couple of times in your reply below, that the window determines the ask- rate, however, it is not clear to me because the ack rate should be a parameter at the receiver and not at the sender (maybe I don’t remember things correctly because it’s a while back since I read the draft and I couple find anything about this in the draft quickly). If the window size however does define the ack rate, then maybe you should rename that parameter respectively. >>> >>> The ack is not for flow control (as we do not have it) but in support of >>> ARQ. The possibility to use it for congestion control is a desirable side >>> effect. The fragmenting endpoint FSM may want to wait and see how things >>> went for the fragments that it already sent. E.g., there's the case where >>> the fragmenting endpoint would use an ack on the first fragment for a >>> number of reasons such as check that a path is available, that the MTU is >>> OK or assess an initial RTT. It may maximize the number of fragments in >>> flight for congestion control. But whether to do any of that is left to >>> implementation (so far). >>> >>> However, if there is really a need for a window, I still recommend to talk less about adapting this value dynamically and make clear that having a fixed value is the recommended default. Therefore I recommend to remove the parameter MinWindowSize and MaxWindowSize because these should actually not be parameters than can be configured but are actual bounds. If someone decides to implement dynamic window adaption, they can decide to re-introduce these parameter again and make them configurable but it doesn’t need to be part of this spec. >>> >>> I can see that, yes. I still like the idea to drop to 1 in case of ECN. >>> Do you suggest to remove that as well? >>> If so, should we augment the inter-frame gap in case of ECN? >>> That may be better though not simpler to specify or implement. >> >> That’s an option as well. Again when you reduce something you might as
