G'day Tom and Haoyu, I'm trying to join the discussion about "draft: Scale-Up Network Header (SUNH)", but I just joined the mail list, so I don't know if posting to the subject line will do it. ( Apologies if this breaks threading )
Drafts: https://datatracker.ietf.org/doc/draft-herbert-sunh/ https://datatracker.ietf.org/doc/html/draft-song-ship-edge-05 It seems like the discussion centers on the address length. The SUNH "1.1. Problem statement" is very clear " 8% overhead in a 256 byte packet, and the forty bytes of IPv6 header would be about 16% overhead " Absolutely minimizing overhead makes sense currently, but for how long do we expect this to be true? Tom, since you've been talking to people who run the largest AI clusters in the world, you expect this to hold true for the foreseeable future. Tom - I wonder if draft-herbert-sunh would benefit from a small summary, maybe with a table, that compares the proposed addressing to other protocols that are common within data centers? For example, comparing protocols by their header, address lengths, and "overhead" - PCIe ( IEEE have paywalls, so it's hard to find a good source. Maybe this: https://www.pearsonhighered.com/assets/samplechapter/0/3/2/1/0321156307.pdf ) - Infiniband ( addressing scheme found here on page 625 https://hjemmesider.diku.dk/~vinter/CC/Infinibandchap42.pdf ) - Ethernet - Ethernet with 802.1q ( and qnq ) - IPv4 - IPv6 - SUNH ... Now that the context is established, explain why 16 bits were chosen for the source/destination address. I guess, but it's not in the document; You were considering the number of hosts in the domain. Nit pick (sorry). "care must be taken to ensure the minimum packet size is maintained". Might help to explain why. Re section "TCP and UDP in SUNH". I remember recently Stuart from Apple saying something pretty interesting about UDP: "If IP had port numbers, you wouldn't really need a UDP header at all." Multicast? It might be worth mentioning multicast and explaining why it isn't discussed. e.g. No requirement for this, or it might be considered in the future if a need arises. Haoyu - I really like your draft-song-ship-edge-05 Hierarchical addressing stuff: a) This reminds me of good old fiber channel addressing, and I suppose the more modern Infiniband/RDMA. b) The words "variable length" are scary because variability clearly isn't ideal for hardware. I guess when you say "variable length" you don't actually mean the addresses would vary dynamically, but that there could be a range of set fixed length addressing that could be selected for different deployment scenarios? c) One core concept of draft-song-ship-edge-05, is that traffic destined for IoT devices needs a long, unique address, while the traffic _sourced_ from these devices towards the data center can have a much smaller destination address. I recall Geoff Huston discussing IPv6 at a recent NANGO, where he commented that because of the pervasive use of anycast by a relatively small number of CDNs, that the Internet might only need a /24 worth of addresses for 99% of all traffic. Other network protocols with asymmetric addresses include: - PCIe (Requester vs Completer addressing) - In InfiniBand / RDMA, requests carry full destination addressing (QPN + LID/GID + path), while responses omit it and are routed implicitly using the established queue-pair and path state, making the addressing directionally asymmetric. - QUIC has explicit directional asymmetry in connection IDs -- Regards, Dave Seddon _______________________________________________ Int-area mailing list -- [email protected] To unsubscribe send an email to [email protected]
