** Description changed: After a new Kernel e.g. 4.4.0-131 is installed on our Ubuntu 16.04 LTS clients, unneeded (waste) 14 bytes paddings are unexpectedly present after PPPoE payload in client's Ethernet frames which are sent towards a - NAS access router. The content of the waste padding is not all zero + NAS access router. The content of such a waste padding is not all zero filled bytes, but rather parts of memory, sometimes with readable text. - Sometimes the waste padding is more than 14 bytes. For example, a PPPoED - PADI Frame with "amf/application/12" (18 bytes) shown as padding in - Wireshark: + Thus it is also a security issue. Sometimes the waste padding is more + than 14 bytes. For example, a PPPoED PADI frame (single tagged VLAN) + with text "amf/application/12" (18 bytes) at the end of the frame shown + as padding in Wireshark: 0000 ff ff ff ff ff ff 00 0a cd 2a ea 9f 81 00 00 07 ÿÿÿÿÿÿ..Í*ê..... 0010 88 63 11 09 00 00 00 0c 01 01 00 00 01 03 00 04 .c.............. 0020 d9 1f 00 00 61 6d 66 2f 61 70 70 6c 69 63 61 74 Ù...amf/applicat 0030 69 6f 6e 2f 31 32 ion/12 - It is a critical bug of Kernel 4.4.0-131 and some previous releases - (since July 2018). + It is a critical bug of Kernel 4.4.0-131. - The affected client Ethernet frames are large enough (>64 bytes), and - thus they do not need to be padded. Padding should be included and is - required in Ethernet frames only to achieve the minimum 64 byte size of - an Ethernet frame sent on wire. For a packet larger than e.g. 100 bytes - no padding is needed in Ethernet frame. However we see such 14 Byte - paddings also in large client Ethernet Frames (in PPPoE/PPP/IP session - packets), with payload size of several hundred bytes (200, 300, 400 - bytes and more). + Some affected client Ethernet frames are large enough (>64 bytes), and + thus they do not need to be padded for Ethernet. Padding should be + included and is required in Ethernet frames only to achieve the minimum + 64 byte size of an Ethernet frame sent on wire. For a packet larger than + e.g. 100 bytes no padding is needed in Ethernet frame. However we can + see such 14 Byte paddings also in large client Ethernet frames (in + PPPoE/PPP/IP session packets), with payload size of several hundred + bytes (200, 300, 400 bytes and more). Our Ubuntu 16.04 LTS Clients are always updated via apt update / apt upgrade early enough, after a new Ubuntu update is released in Internet. - Our statistics after analyzing the collected archived pcap traces - (Clients with Ubuntu 16.04 LTS and the latest Kernel updates installed) - can help to identify the date when the kernel bug first appeared: + Our statistics after analyzing the collected archived pcap traces of our + clients with Ubuntu 16.04 LTS and the latest Kernel installed can help + to identify the date and the version when the kernel bug first appeared: - From 15.Jul.2018 till 29.Jul.2018 - OK, still no waste padding - 10.07.2018 - 14 byte padding (PADI etc) - a kernel BUG ! - 12.07.2018 - 14 byte padding (PADI etc) - a kernel BUG ! - 13.07.2018 - 14 byte padding (PADI etc) - a kernel BUG ! - 19.07.2018 - 14 byte padding (PADI etc) - a kernel BUG ! + From 15.06.2018 to 29.06.2018 - still no waste padding on all clients (OK) + 02.07.2018 - OK, still no waste padding on all clients (OK) + 03.07.2018 - OK, still no waste padding on all clients (OK) + 04.07.2018 - OK, still no waste padding on all clients (OK) + 05.07.2018 - OK, still no waste padding on all clients (OK) + 06.07.2018 - OK, still no waste padding on all clients (OK) - A kernel version affected with this BUG: + !!! Kernel Update to 4.4.0-131 on 09-Jul-2018 !!! + + 09.07.2018 - 14 byte padding (PADI, PADR, etc) on all updated clients - a kernel BUG ! + 10.07.2018 - 14 byte padding (PADI, PADR, etc) on all updated clients - a kernel BUG ! + 12.07.2018 - 14 byte padding (PADI, PADR, etc) on all updated clients - a kernel BUG ! + 13.07.2018 - 14 byte padding (PADI, PADR, etc) on all updated clients - a kernel BUG ! + 19.07.2018 - 14 byte padding (PADI, PADR, etc) on all updated clients - a kernel BUG ! + ... + + So the padding issue occurred on our clients for the first time on + 09-Jul-2018, after the Linux Kernel was updated to 4.4.0-131 on our + Ubuntu 16.04 LTS clients. Still then the issue is present on all updated + clients. The kernel bug disappeared and the padding behavior is OK again + (without bug), after clients were downgraded to and booted with an older + kernel version like e.g. 4.4.0-116 (older than 4.4.0-131). + + The kernel version affected with this BUG: @client:~$ uname -a Linux client 4.4.0-131-generic #157-Ubuntu SMP Thu Jul 12 15:51:36 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux + + @client:~$ cat /proc/version_signature + Ubuntu 4.4.0-131.157-generic 4.4.134 @client:~$ lsb_release -rd Description: Ubuntu 16.04.5 LTS Release: 16.04 - The kernel bug does not occur on Ubuntu 16.04 LTS Client and waste - paddings (14 byte or 18 byte) are not present in Client frames, if an - old Kernel e.g. 4.4.0-116 (of 12-Feb-2018) is installed on the client - and activated at boot time. + @client:~$ lsb_release -ci + Distributor ID: Ubuntu + Codename: xenial + + The kernel bug does not occur on Ubuntu 16.04 LTS clients and waste + paddings (14 byte on single-stacked VLAN or 18 byte on double-stacked + VLAN) are not present in client frames, if an old Kernel e.g. 4.4.0-116 + (of 12-Feb-2018) is installed on the client and activated at boot time. This old kernel 4.4.0-116 is still without the padding bug: @client:~$ uname -a Linux client 4.4.0-116-generic #140-Ubuntu SMP Mon Feb 12 21:23:04 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux A PADI packet with unneeded padding is included as attachment. It was captured via tcpdump directly on the Ubuntu Client. It is a padding - inserted by Ubuntu, not by ethernet HW or driver. If it were padding - included by Ethernet adapter or driver, we could not capture it on the - same host and we could not see such real padding for this client packet - in Wireshark. + inserted by Ubuntu, not by the HW or driver. If it were padding included + by Ethernet adapter or driver to reach the minimum 64 byte frame size, + we could not capture such real padding on the WAN interface of the same + host and could not see such real padding of this client frame in + Wireshark. The issue affects both PPPoED client packets (PADI, PADR) and session PPPoE/PPP/IP client packets. We expect no padding is included after - PPPoE payload in Ethernet Frames if the packets are not small (>64 byte) + PPPoE payload in Ethernet frames if the packets are not small (>64 byte) and thus are not required to be padded for Ethernet. Such correct behavior (without waste paddings) can be observed on Ubuntu 16.04 LTS - with old Kernel versions released before June 2018, like e.g. 4.4.0-116. + with old Kernel versions released before July 2018, like e.g. 4.4.0-116. Please implement the fix in the next Kernel update.
-- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1784684 Title: Waste padding after PPPoE payload Status in linux package in Ubuntu: Confirmed Bug description: After a new Kernel e.g. 4.4.0-131 is installed on our Ubuntu 16.04 LTS clients, unneeded (waste) 14 bytes paddings are unexpectedly present after PPPoE payload in client's Ethernet frames which are sent towards a NAS access router. The content of such a waste padding is not all zero filled bytes, but rather parts of memory, sometimes with readable text. Thus it is also a security issue. Sometimes the waste padding is more than 14 bytes. For example, a PPPoED PADI frame (single tagged VLAN) with text "amf/application/12" (18 bytes) at the end of the frame shown as padding in Wireshark: 0000 ff ff ff ff ff ff 00 0a cd 2a ea 9f 81 00 00 07 ÿÿÿÿÿÿ..Í*ê..... 0010 88 63 11 09 00 00 00 0c 01 01 00 00 01 03 00 04 .c.............. 0020 d9 1f 00 00 61 6d 66 2f 61 70 70 6c 69 63 61 74 Ù...amf/applicat 0030 69 6f 6e 2f 31 32 ion/12 It is a critical bug of Kernel 4.4.0-131. Some affected client Ethernet frames are large enough (>64 bytes), and thus they do not need to be padded for Ethernet. Padding should be included and is required in Ethernet frames only to achieve the minimum 64 byte size of an Ethernet frame sent on wire. For a packet larger than e.g. 100 bytes no padding is needed in Ethernet frame. However we can see such 14 Byte paddings also in large client Ethernet frames (in PPPoE/PPP/IP session packets), with payload size of several hundred bytes (200, 300, 400 bytes and more). Our Ubuntu 16.04 LTS Clients are always updated via apt update / apt upgrade early enough, after a new Ubuntu update is released in Internet. Our statistics after analyzing the collected archived pcap traces of our clients with Ubuntu 16.04 LTS and the latest Kernel installed can help to identify the date and the version when the kernel bug first appeared: From 15.06.2018 to 29.06.2018 - still no waste padding on all clients (OK) 02.07.2018 - OK, still no waste padding on all clients (OK) 03.07.2018 - OK, still no waste padding on all clients (OK) 04.07.2018 - OK, still no waste padding on all clients (OK) 05.07.2018 - OK, still no waste padding on all clients (OK) 06.07.2018 - OK, still no waste padding on all clients (OK) !!! Kernel Update to 4.4.0-131 on 09-Jul-2018 !!! 09.07.2018 - 14 byte padding (PADI, PADR, etc) on all updated clients - a kernel BUG ! 10.07.2018 - 14 byte padding (PADI, PADR, etc) on all updated clients - a kernel BUG ! 12.07.2018 - 14 byte padding (PADI, PADR, etc) on all updated clients - a kernel BUG ! 13.07.2018 - 14 byte padding (PADI, PADR, etc) on all updated clients - a kernel BUG ! 19.07.2018 - 14 byte padding (PADI, PADR, etc) on all updated clients - a kernel BUG ! ... So the padding issue occurred on our clients for the first time on 09-Jul-2018, after the Linux Kernel was updated to 4.4.0-131 on our Ubuntu 16.04 LTS clients. Still then the issue is present on all updated clients. The kernel bug disappeared and the padding behavior is OK again (without bug), after clients were downgraded to and booted with an older kernel version like e.g. 4.4.0-116 (older than 4.4.0-131). The kernel version affected with this BUG: @client:~$ uname -a Linux client 4.4.0-131-generic #157-Ubuntu SMP Thu Jul 12 15:51:36 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux @client:~$ cat /proc/version_signature Ubuntu 4.4.0-131.157-generic 4.4.134 @client:~$ lsb_release -rd Description: Ubuntu 16.04.5 LTS Release: 16.04 @client:~$ lsb_release -ci Distributor ID: Ubuntu Codename: xenial The kernel bug does not occur on Ubuntu 16.04 LTS clients and waste paddings (14 byte on single-stacked VLAN or 18 byte on double-stacked VLAN) are not present in client frames, if an old Kernel e.g. 4.4.0-116 (of 12-Feb-2018) is installed on the client and activated at boot time. This old kernel 4.4.0-116 is still without the padding bug: @client:~$ uname -a Linux client 4.4.0-116-generic #140-Ubuntu SMP Mon Feb 12 21:23:04 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux A PADI packet with unneeded padding is included as attachment. It was captured via tcpdump directly on the Ubuntu Client. It is a padding inserted by Ubuntu, not by the HW or driver. If it were padding included by Ethernet adapter or driver to reach the minimum 64 byte frame size, we could not capture such real padding on the WAN interface of the same host and could not see such real padding of this client frame in Wireshark. The issue affects both PPPoED client packets (PADI, PADR) and session PPPoE/PPP/IP client packets. We expect no padding is included after PPPoE payload in Ethernet frames if the packets are not small (>64 byte) and thus are not required to be padded for Ethernet. Such correct behavior (without waste paddings) can be observed on Ubuntu 16.04 LTS with old Kernel versions released before July 2018, like e.g. 4.4.0-116. Please implement the fix in the next Kernel update. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1784684/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : [email protected] Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
