Hello, On Wed, Nov 13, 2019 at 8:35 PM Saku Ytti <[email protected]> wrote: > > On Wed, 13 Nov 2019 at 18:27, Matt Corallo <[email protected]> wrote: > > > This sounds like a bug on Cloudflare’s end (cause trying to do anycast TCP > > is... out of spec to say the least), not a bug in ECN/ECMP. > > Not true. Hash result should indicate discreet flow, more importantly > discreet flow should not result into two unique hash numbers. Using > whole TOS byte breaks this promise and thus breaks ECMP. > > Platforms allow you to configure which bytes are part of hash > calculation, whole TOS byte should not be used as discreet flow SHOULD > have unique ECN bits during congestion. Toke has diagnosed the problem > correctly, solution is to remove TOS from ECMP hash calculation.
In fact I believe everything beyond the 5-tuple is just a bad idea to base your hash on. Here are some examples (not quite as straight forward than the TOS/ECN case here): TTL: https://mailman.nanog.org/pipermail/nanog/2018-September/096871.html IPv6 flow label: https://blog.apnic.net/2018/01/11/ipv6-flow-label-misuse-hashing/ https://pc.nanog.org/static/published/meetings/NANOG71/1531/20171003_Jaeggli_Lightning_Talk_Ipv6_v1.pdf https://www.youtube.com/watch?v=b0CRjOpnT7w Lukas
