On 6/13/2022 6:33 AM, Michael Eriksson wrote:
Hi Christian,
Thanks for progressing this!
Single vs multiple packet number spaces
I used to think that a single packet number (PN) space was a cleaner and
simpler solution, partly because of less difference with singlepath QUIC. A
later insight is that this is mainly an issue if you add multipath to an
existing singlepath stack. With a stack designed for multipath from the
beginning, multiple PN spaces actually seem more straightforward and efficient.
With a careful design of multiple PN spaces, there is very little difference
between a singlepath connection and a multipath-enabled connection that only
uses one path.
Additionally: At Ericsson Research, we have Rebecka Alfredsson finishing her
M.Sc. thesis and prototype of multipath QUIC hardware offload. During that
work, we have realized that multiple PN spaces are more or less needed for
multipath QUIC offload. The reason is the compressed packet numbers, holes in
the packet number sequence makes it difficult to expand the packet number on
the receiving side.
Both single and multiple packet number spaces
What you suggest is to use both single and multiple PN spaces, which I think is
a mistake. The main reason is the increased complexity and double
implementation.
The proposal does not suggest to use both single and multiple number
spaces "in general". It actually suggests that multiple number spaces is
the default. The proposal makes an exception for the zero-length
identifier case, because we actually got feedback about the importance
of that scenario from several implementations, and also because these
implementations expressed their preference for using a single number
space in that case.
The proposal carefully outlines options for "pure multipath"
implementations that don't want to handle the extra complexity:
* on the receiving data part, the implementations can avoid the
complexity by simply using non-zero-length CID. This is entirely under
the implementation control.
* on the sending data part, the implementations can avoid the complexity
by either not negotiating multipath if the peer uses zero-length CID, or
falling back to a degraded mode in which the implementation uses just
one path at a time.
I do believe this is the right compromise between the competing
suggestions of multiple implementations.
Zero-length connection identifiers
The stated need for a single PN space is to support zero-length connection identifiers
(CIDs). I think that multiple PN spaces can be used also for zero-length CIDs if the IP
address and port number ("socket" in practice) is used for demultiplexing on
the receiver side.
Zero-length CIDs are very seldom, if ever, used on the server side. The client
will normally use separate local network interfaces (e.g., cellular and Wi-Fi)
for the different paths, and thus have separate sockets for the different paths
anyway. The server can handle a NAT rebinding, since the different paths are
separated by the connection ID and the event is handled as in RFC 9000 (i.e.,
path validation).
The "multiple sockets" assumption is not true. Many implementations use
a single UDP socket, and use parameters of `sendmsg` to specify the
outgoing interface and the outgoing source address.
The main scenario of zero-length CID is indeed "zero length on the
client", "defined length on the server". This is the scenario that some
implementers explicitly want to support. These same implementers insist
on using a single number space for packets sent by the server to the client.
Some implementations do use zero-length CID on the server side. This is
quite rare, and I believe limited in practice to peer-to-peer scenarios.
Identifying path by IP and port numbers is rather fragile, especially in
the presence of NAT and load balancers.
Path Identification
Each path needs to have an identity for mainly two reasons:
* to refer to the path in signaling frames, e.g., PATH_ABANDON.
* to avoid reusing encryption nonces when using multiple PN spaces
The current draft uses the destination CID sequence number to create the nonce.
Since zero-length CIDs don't have sequence numbers, they can't be used with
multiple PN spaces.
I suggest that the sequence number of the server's CID is used to identify the
path in both directions, including for nonce generation. This has two direct
advantages:
* It allows multiple PN spaces for zero-length CIDs on the client side
* The path has the same identity in both directions, which reduces the
complexity and risk for errors
"PN Space ID" should be removed from the specification and Path ID used
everywhere.
The current specification is the result of quite some discussions. It is
definitely a compromise, but I think it is the correct compromise.
Summary
I suggest that only multiple PN spaces are used for multipath QUIC. I have
shown how zero-length CIDs can be used on the client side and argue that they
will not be used on servers anyway.
Well, we disagree.
-- Christian Huitema
