On 27 May 2016 at 07:29, Adam Vitkovsky <[email protected]> wrote: >> James Bensley >> Sent: Thursday, May 26, 2016 4:37 PM >> >> Running BFD in no echo mode (which is what I usually do) we see the >> average interval is a lovely 47ms (but I've never had 100% clarification of >> why): >> Rx Count: 3314443, Rx Interval (ms) min/max/avg: 1/72/47 last: 36 ms ago Tx >> Count: 3310865, Tx Interval (ms) min/max/avg: 1/72/47 last: 40 ms ago >> >> > Isn’t it because in no echo mode it does just half of the work it has to in > echo mode? > > In no echo each participant has to just fire hellos at a given rate and reset > hold-down timer upon receiving neighbour's hello. > > In echo mode each participant has to fire hellos at a given rate and reset > timers upon receiving its own hello. > And in addition to that it has to receive loop back and transmit neighbour's > hellos at the rate the neighbour is sending them. > > And according to your little test it looks like the looping back is lot more > process intensive then just resetting holdup timer.
In no echo mode the local node is sending control packets at the specified interval of 50ms to the remote node, the remote node is checking that incomming packet and marking the link as "still up". In echo mode the local node sends echo packets at the specified interval of 50ms, the remote node loops them back in hardware (because they are UDP packets they shouldn't be punted to the CPU) and the local node receives and checks it's own echo packets. So in no echo mode the local node originates packets and checks packet received from the remote node. In echo mode the local node originates and checks its own packets only. Either way its basically the same amoutn of work, and in both cases setting the timers to 50ms I would expect to mean there must be some level of hardware offload here? On 27 May 2016 at 08:41, Saku Ytti <[email protected]> wrote: > On 27 May 2016 at 09:29, Adam Vitkovsky <[email protected]> wrote: > The design goal was opposite. Echo mode was supposed to be the > performant and easy to implement in HW. Results look fishy, I've seen > others report similar in NXOS. This is what is confusing me (as above). > I wouldn't bother with echo mode anyhow, much poorer interop than with > control mode. And many platforms manage to control mode in HW just > fine. You say you wouldn't bother with echo mode, however if I disable echo mode on the ASR9001 and ASR920, two devices that "support BFD in hardware" as you can see in the below output from the ASR9001, the ASR920 isn't managing to keep to the 50ms timmers and the ASR9001 is just scraping by; Intervals between echo packets: Tx: Number of intervals=100, min=52 ms, max=54 ms, avg=52 ms Last packet transmitted 86 s ago Rx: Number of intervals=99, min=50 ms, max=5310 ms, avg=105 ms Last packet received 86 s ago Latency of echo packets (time between tx and rx): Number of packets: 100, min=1 ms, max=5 ms, avg=2940 us It seems for these platforms that do "support BFD in hardware" disabling echo mode switches back to CPU based forwarding of control packets at the configured interval (50ms) based on the outputs directly above (how sloppy the timers have become, before they were pretty much exactly 50ms). However for platforms that don't support BFD in hardware, disabling echo modes "seems" to give the performance one can only get with hardware accelerated forarding (as per the outputs in my original email). Further example... 7600 <> ME3600X with echo mode on (default in IOS) both configured with 500ms interval and 3x multiplier: 7600 (WS-X6704-10GE): Rx Count: 219, Rx Interval (ms) min/max/avg: 1/1000/873 last: 548 ms ago Tx Count: 218, Tx Interval (ms) min/max/avg: 756/1000/877 last: 504 ms ago ME3600X: Rx Count: 1305954, Rx Interval (ms) min/max/avg: 1/1008/883 last: 680 ms ago Tx Count: 1302978, Tx Interval (ms) min/max/avg: 1/1016/881 last: 932 ms ago 7600 <> ME3600X with no echo mode configured (still with 500ms and 3x multiplier): 7600 (WS-X6704-10GE): Rx Count: 50, Rx Interval (ms) min/max/avg: 1/512/408 last: 224 ms ago Tx Count: 49, Tx Interval (ms) min/max/avg: 1/500/416 last: 144 ms ago ME3600X: Rx Count: 67, Rx Interval (ms) min/max/avg: 1/500/431 last: 176 ms ago Tx Count: 68, Tx Interval (ms) min/max/avg: 1/512/423 last: 336 ms ago For some reason these platforms behave in the exact opposite manner to platforms that support hardware offload for BFD (echo mode). We've been rolloing out ME3600s/ME3800s/7600s with echo mode disabled and ASR920s and ASR9000s with echo mode enabled, so overall we get the 50ms failover effect as desired, but it would be nice to know whats going on (how are we getting these surprisingly good results?) without having to go to spend an age with TAC. Cheers, James. _______________________________________________ cisco-nsp mailing list [email protected] https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
