On 6/16/2022 2:08 AM, Michael Eriksson wrote:
You claim that a single PN space makes hardware offload easier, which is against my understanding after having supervised Rebecka Alfredsson's master thesis project on hardware offload for multipath QUIC. As I wrote in a previous message, receive-side hardware offload needs to expand the received compressed packet number to a full packet number to be able to decrypt the packet. This is hard to do if there are holes in the packet number sequence, which will happen if you have a single PN space. To be explicit, different paths can arrive on different offloading NICs or be handled by different threads due to load-balancing mechanisms similar to receive-side scaling (RSS). The same problem could affect more abstract designs, such as those based on P4.
I think that the PN expansion issue is not decisive. Yes, there will be holes. An interface specific encryption offload engine will only see a fraction of the traffic. But if that fraction is large enough, no problem -- even if the increase is not monotonous, the stream of packets is sufficient to maintain a good idea of the current PN number range. If the fraction is too low for that, that also means that the interface is very lightly used. In that case, the overhead of an occasional synchronization between hardware and software can be absorbed with little impact on performance.
With the single space solution, the same encryption logic is used regardless of whether the connection supports multipath or not. I think that makes implementation of crypto offload simpler. The same encryption key and IV are used for all paths, and that too makes encryption offload simpler, especially when handling key rotations.
But all this is only relevant if we go back to the drawing board and insist on picking exactly one solution. That's not what the unifying PR proposes. With that PR, the single number space is only used in conjunction with zero-length identifiers. The node deploying crypto offload will only need to deal with PN number holes if it chooses to use zero-length identifiers. In practice, the kind of big servers that deploy encryption offload don't do that. They tend to sit behind load balancers, and the defined CID are very much required to support multipath through a load balancer. So, I don't think that we have a problem in practice.
-- Christian Huitema
