Hi Brian,
On 11-02-2022 15:49, Brian King wrote:
Hello Wikitech,
I’m a new SRE on the Search team. As Ryan and I are in the middle of relocating
many large shards on our Elastic cluster, I wanted to ask your thoughts about
using jumbo frames and/or LACP for our physical Elastic nodes. We’re also
moving to new hardware 10Gbps networking, so it seems like a good time to start
optimizing our network settings.
Let us know what you think (and please feel free to suggest any other
optimizations).
Unexpected question on this list. Link aggregation (lag) is an easy way
to add more bandwidth without changing the logical setup. In core
networks I'm used to only use lags (even if it has only one member) for
future growth. LACP is the protocol of choice because it's open and
widely supported. Do use fast mode (interval of 1 second instead of 30
seconds). For critical links you can also use microBFD, but that seems a
bit overkill in your case. If you have switches that support MC-LAG, you
can connect to different switches and only have reduced bandwidth when
one of the switches is unavailable.
Every network should be build for jumbo frames with an physical mtu
around 9216 (different vendors, different ways of calculating it) and an
IP mtu of 9000. Gives better performance for bulk traffic like backup,
probably less in other cases. Usually no problem with standard switches
and routers, but firewalls and load balancers might be problematic.
Do buy network interfaces that support the proper offloading because
otherwise it will hit your CPU. I recall some cases in the past where
the slightly cheaper cards didn't support vxlan offloading and the
likes. Not sure if that's still a thing these days.
Oh and if you're not using 10GBASE-T: Do buy right optics. 10GBASE-SR
(multimode, 850 nm, usually black) and 10GBASE-LR (singlemode, 1310 nm,
usually blue) don't mix :-)
Maarten
_______________________________________________
Wikitech-l mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
