Re: [nvo3] Alignment and Ethernet encapsulation

[Joe Touch]

On 9/20/2016 11:38 AM, Tom Herbert wrote:
> On Tue, Sep 20, 2016 at 10:55 AM, Joe Touch  wrote:
>>
>> On 9/20/2016 10:47 AM, Tom Herbert wrote:
>>> On Tue, Sep 20, 2016 at 10:35 AM, Joe Touch  wrote:
 ...
 If you've been handed Ethernet to encapsulate, then that is the header
 whose alignment you should be optimizing.

 As you point out, you can't align both IP and Ethernet to 4-byte
 boundaries at the same time.
>>> You can if you insert a two byte pad before the encapsulated Ethernet
>>> header like ETHERIP (RFC3378 does).
>> You CAN do a lot of things, but you shouldn't.
>>
>> If you're handed ethernet to encapsulate, then THAT is the header whose
>> access should be aligned.
>>
>> You have no business ASSUMING that you need to optimize for IP header
>> access. All you're doing is messing up the ethernet access.
>>
> Sorry, but I disagree with that statement. Keeping IP headers aligned
> for stack processing is a HUGE concern in real implementation.
It is, and something that handles Ethernet messages should already know
what to do here.

So if it already expects to shift the packet over as it copies it in,
then you've just defeated that. The same is true for non-IP inside
Ethernet, which might have its own padding to re-align its contents.

I.e., if it's ethernet, treat it as ethernet. That's what an L2 tunnel
should do.

Joe


___
nvo3 mailing list
nvo3@ietf.org
https://www.ietf.org/mailman/listinfo/nvo3

Re: [nvo3] Alignment and Ethernet encapsulation

[Tom Herbert]

On Tue, Sep 20, 2016 at 10:55 AM, Joe Touch  wrote:
>
>
> On 9/20/2016 10:47 AM, Tom Herbert wrote:
>> On Tue, Sep 20, 2016 at 10:35 AM, Joe Touch  wrote:
>>> ...
>>> If you've been handed Ethernet to encapsulate, then that is the header
>>> whose alignment you should be optimizing.
>>>
>>> As you point out, you can't align both IP and Ethernet to 4-byte
>>> boundaries at the same time.
>> You can if you insert a two byte pad before the encapsulated Ethernet
>> header like ETHERIP (RFC3378 does).
>
> You CAN do a lot of things, but you shouldn't.
>
> If you're handed ethernet to encapsulate, then THAT is the header whose
> access should be aligned.
>
> You have no business ASSUMING that you need to optimize for IP header
> access. All you're doing is messing up the ethernet access.
>
Sorry, but I disagree with that statement. Keeping IP headers aligned
for stack processing is a HUGE concern in real implementation.
Probably every device driver on the planet will put in a two byte pad
in receiver buffers to ensure that the Ethernet payload (and hence IP
header, EHs, TCP headers, etc.). We need two byte alignment of
Ethernet headers, and four byte alignment on IP protocols. If we don't
have this a lot of things will break (as VXLAN is currently broken on
SPARC)! Maybe in theory alignment is no longer considered an issue and
is just a historical footnote, but on real devices in deployment today
unaligned protocol headers still can be problematic.

Tom


> Frankly, anything that already parses Ethernet then IP ought to be able
> to efficiently handle the difference in the two alignments - and you
> might be messing THAT up.
>
> RFC3378 defines a header used for ETHERIP; that header may or may not be
> relevant to other ethernet-in-X encapsulations. The field needed to be
> there for versioning and was expanded from 8 to 16 bits, which certainly
> helps align the *ethernet* header that comes after it compared to the
> earlier use of an 8-bit field. There's no mention of aligning for the
> purposes of accessing the IP header, which again, I disagree with.
>
> Joe
>
>

___
nvo3 mailing list
nvo3@ietf.org
https://www.ietf.org/mailman/listinfo/nvo3

Re: [nvo3] Alignment and Ethernet encapsulation

[Joe Touch]

On 9/20/2016 10:47 AM, Tom Herbert wrote:
> On Tue, Sep 20, 2016 at 10:35 AM, Joe Touch  wrote:
>> ...
>> If you've been handed Ethernet to encapsulate, then that is the header
>> whose alignment you should be optimizing.
>>
>> As you point out, you can't align both IP and Ethernet to 4-byte
>> boundaries at the same time.
> You can if you insert a two byte pad before the encapsulated Ethernet
> header like ETHERIP (RFC3378 does).

You CAN do a lot of things, but you shouldn't.

If you're handed ethernet to encapsulate, then THAT is the header whose
access should be aligned.

You have no business ASSUMING that you need to optimize for IP header
access. All you're doing is messing up the ethernet access.

Frankly, anything that already parses Ethernet then IP ought to be able
to efficiently handle the difference in the two alignments - and you
might be messing THAT up.

RFC3378 defines a header used for ETHERIP; that header may or may not be
relevant to other ethernet-in-X encapsulations. The field needed to be
there for versioning and was expanded from 8 to 16 bits, which certainly
helps align the *ethernet* header that comes after it compared to the
earlier use of an 8-bit field. There's no mention of aligning for the
purposes of accessing the IP header, which again, I disagree with.

Joe


___
nvo3 mailing list
nvo3@ietf.org
https://www.ietf.org/mailman/listinfo/nvo3

Re: [nvo3] Alignment and Ethernet encapsulation

[Joe Touch]

Hi, Tom,


On 9/20/2016 9:13 AM, Tom Herbert wrote:
> ...
> For new encapsulation protocols please consider the effects of IP
> header alignment in the presence of Ethernet encapsulation. Defining
> Ethernet encapsulation with the two byte padding like in ETHERIP may
> help a lot to make implementation of Ethernet encapsulation feasible
> on CPU HW.

IMO, alignment needs to be handled within each encapsulation layer
independently. I don't think it's useful to expect new encapsulation
layers to have to make sure every layer of an encapsulated packet is
aligned - just the first one ought to be sufficient. The rest is the
responsibility of whomever added the other layers already in place.

So yes, it's useful to make sure the encapsulated packet starts on a
boundary that is 4-byte aligned, but the rest *needs to be* someone
else's problem.

Joe

___
nvo3 mailing list
nvo3@ietf.org
https://www.ietf.org/mailman/listinfo/nvo3

[nvo3] Alignment and Ethernet encapsulation

[Tom Herbert]

Hello,

This issue came up on the Linux netdev list. The Oracle engineers are
trying to deal with the ramifications of unaligned IP headers (not
aligned to four bytes) when doing Ethernet encapsulation (VXLAN, GRE
with TEB for instance). The problem is that certain CPU architectures,
such as SPARC (and maybe PowerPC?), do not handle unaligned accesses
very well. Every unaligned access causes a trap to software which
kills any semblance of performance. x86 has shift logic so we don't
see any issues there (except for unaligned accesses across a page
boundary).

The Linux stack is written to assume that IP protocols are properly
four byte aligned and in non-encapsulation cases Ethernet headers are
padded to be 16 bytes. The problem seems to be specific when
performing Ethernet header encapsulation and there is no padding of
the inner Ethernet header like would be done in ETHERIP. Either the
outer or the inner IP headers will be misaligned. There does not seem
to be an easy fix for this without rewriting the stack or doing copies
on every packet.

For new encapsulation protocols please consider the effects of IP
header alignment in the presence of Ethernet encapsulation. Defining
Ethernet encapsulation with the two byte padding like in ETHERIP may
help a lot to make implementation of Ethernet encapsulation feasible
on CPU HW.

Thanks,
Tom

___
nvo3 mailing list
nvo3@ietf.org
https://www.ietf.org/mailman/listinfo/nvo3