Bug#889827: linux-image-4.15.0-rc8-amd64: IP offloading does not work

[Ben Hutchings]

Control: tag -1 moreinfo

On Thu, 2018-02-22 at 17:50 +0100, Vincent Danjean wrote:
> Le 07/02/2018 à 15:46, Vincent Danjean a écrit :
> [...]
> >   So, it seems there is an issue with checksum offloading with this driver
> > and hardware.
> >   Note that I got this bug with previous kernels (such as 4.14*) but I did 
> > not
> > note the exact versions and I did not think to disable offloading at this 
> > time.
> 
>   I just found this on the internet. I think it is exactly my problem.
> https://patchwork.kernel.org/patch/10072391/
> 
>   I do not know if a patch has been committed eventually, or if the
> real problem has been fixed.
> 
>   For now, I manually run "sudo ethtool -K en-wired rx off" when the
> problem occurs.

Interesting.  So it sounds like the USB device is working correctly but
the hub (or wiring) is faulty.

There is an upstream commit that looks like it should fix this:
https://git.kernel.org/linus/0b1655143df00ac5349f27b765b2ed13a3ac40ca
This went into 4.15-rc9, so this should now be fixed in unstable with
version 4.15.4-1.

Does the new version work for you?

Ben.

-- 
Ben Hutchings
[W]e found...that it wasn't as easy to get programs right as we had
thought. ... I realized that a large part of my life from then on was
going to be spent in finding mistakes in my own programs. - Maurice
Wilkes, 1949


signature.asc
Description: This is a digitally signed message part

Bug#889827: linux-image-4.15.0-rc8-amd64: IP offloading does not work

[Vincent Danjean]

Le 22/02/2018 à 17:50, Vincent Danjean a écrit :
> Le 07/02/2018 à 15:46, Vincent Danjean a écrit :
> [...]
>>   So, it seems there is an issue with checksum offloading with this driver
>> and hardware.
>>   Note that I got this bug with previous kernels (such as 4.14*) but I did 
>> not
>> note the exact versions and I did not think to disable offloading at this 
>> time.
> 
>   I just found this on the internet. I think it is exactly my problem.
> https://patchwork.kernel.org/patch/10072391/
> 
>   I do not know if a patch has been committed eventually, or if the
> real problem has been fixed.

It seems to be committed just after the rc8:
https://github.com/torvalds/linux/commit/0b1655143df00ac5349f27b765b2ed13a3ac40ca

So, this bug should be workaround from the current 4.15 Debian kernel package.
I will report when I will reboot on this kernel.

  Regards,
Vincent

> 
>   For now, I manually run "sudo ethtool -K en-wired rx off" when the
> problem occurs.
> 
>   Regards,
> Vincent
> 
>>   Regards,
>> Vincent 
> 
> 
> 


-- 
Vincent Danjean   GPG key ID 0xD17897FA vdanj...@debian.org
GPG key fingerprint: 621E 3509 654D D77C 43F5  CA4A F6AE F2AF D178 97FA
Unofficial pkgs: http://moais.imag.fr/membres/vincent.danjean/deb.html
APT repo:  deb http://people.debian.org/~vdanjean/debian unstable main

Bug#889827: linux-image-4.15.0-rc8-amd64: IP offloading does not work

[Vincent Danjean]

Le 07/02/2018 à 15:46, Vincent Danjean a écrit :
[...]
>   So, it seems there is an issue with checksum offloading with this driver
> and hardware.
>   Note that I got this bug with previous kernels (such as 4.14*) but I did not
> note the exact versions and I did not think to disable offloading at this 
> time.

  I just found this on the internet. I think it is exactly my problem.
https://patchwork.kernel.org/patch/10072391/

  I do not know if a patch has been committed eventually, or if the
real problem has been fixed.

  For now, I manually run "sudo ethtool -K en-wired rx off" when the
problem occurs.

  Regards,
Vincent

>   Regards,
> Vincent 



-- 
Vincent Danjean   GPG key ID 0xD17897FA vdanj...@debian.org
GPG key fingerprint: 621E 3509 654D D77C 43F5  CA4A F6AE F2AF D178 97FA
Unofficial pkgs: http://moais.imag.fr/membres/vincent.danjean/deb.html
APT repo:  deb http://people.debian.org/~vdanjean/debian unstable main

Bug#889827: linux-image-4.15.0-rc8-amd64: IP offloading does not work

[Vincent Danjean]

Le 07/02/2018 à 22:22, Ben Hutchings a écrit :
> Control: tag -1 upstream patch moreinfo
> 
> Does the attached patch fix it?

Nope.

Before the patch, I trigger the error with this kind of command:
ssh aya dd if=/dev/zero bs=8192 | dd of=/dev/null bs=8192

I got this kind of messages:
$ ssh aya dd if=/dev/zero bs=8192 | dd of=/dev/null bs=8192
ssh_dispatch_run_fatal: Connection to 10.77.0.3 port 22: message authentication 
code incorrect
9151+0 enregistrements lus
9151+0 enregistrements écrits
74964992 bytes (75 MB, 71 MiB) copied, 0,859281 s, 87,2 MB/s

I got at most a few GB before the error trigger (and usually only a few MB)


I do not observe error with a reverse ssh:
dd if=/dev/zero bs=8192 | ssh aya dd of=/dev/null bs=8192

I do not observe error with rx offload disabled:
sudo ethtool -K en-wired rx off tx on


I recompiled the kernel according to the kernel handbook with
bash debian/bin/test-patches -f amd64 
../0001-r8152-Fix-conditions-for-setting-ip_summed-CHECKSUM_.patch
I do not install the resulting package but I removed the module and load the 
new one:
# rmmod r8152
# insmod 
debian/linux-image-4.15.0-rc8-amd64/lib/modules/4.15.0-rc8-amd64/kernel/drivers/net/usb/r8152.ko
Note: I observed with "tail -f /var/log/syslog" in another terminal the
removal of my internet interface with the first command and its come back
with the second one.

With the new (patched) r8152 module, I get exactly the same results.
Error with "ssh aya dd if=/dev/zero bs=8192 | dd of=/dev/null bs=8192"
No more error if I disable rx offload.

  Regards,
Vincent

>  Instructions for building a patched
> kernel package can be found at:
> https://kernel-handbook.alioth.debian.org/ch-common-tasks.html#s-common-official
> 
> Ben.
> 


-- 
Vincent Danjean   GPG key ID 0xD17897FA vdanj...@debian.org
GPG key fingerprint: 621E 3509 654D D77C 43F5  CA4A F6AE F2AF D178 97FA
Unofficial pkgs: http://moais.imag.fr/membres/vincent.danjean/deb.html
APT repo:  deb http://people.debian.org/~vdanjean/debian unstable main




signature.asc
Description: OpenPGP digital signature

Bug#889827: linux-image-4.15.0-rc8-amd64: IP offloading does not work

[Ben Hutchings]

Control: tag -1 upstream patch moreinfo

Does the attached patch fix it?  Instructions for building a patched
kernel package can be found at:
https://kernel-handbook.alioth.debian.org/ch-common-tasks.html#s-common-official

Ben.

-- 
Ben Hutchings
Reality is just a crutch for people who can't handle science fiction.
From abc83f38abd2d62ce5a443e0b2a40a76d0b75cb8 Mon Sep 17 00:00:00 2001
From: Ben Hutchings 
Date: Wed, 7 Feb 2018 21:05:37 +
Subject: [PATCH] r8152: Fix conditions for setting ip_summed =
 CHECKSUM_UNNECESSARY

Currently r8152_rx_csum() returns CHECKSUM_UNNECESSARY for any packet
which the hardware reports as being IPv4 and not having a layer-4
checksum error, even if the layer-4 protocol was not recognised.
This will incorrectly include, for example, SCTP packets.  It also
appears that some TCP packets with valid headers and bad checksums
also trigger this bug.

Use the same logic for IPv4 as for IPv6.  We don't need to check the
hardware IP header checksum flag because that's always checked in
software.

Reported-by: Vincent Danjean 
Signed-off-by: Ben Hutchings 
---
 drivers/net/usb/r8152.c | 11 +--
 1 file changed, 1 insertion(+), 10 deletions(-)

diff --git a/drivers/net/usb/r8152.c b/drivers/net/usb/r8152.c
index 0657203ffb91..c3c8e2d74f8b 100644
--- a/drivers/net/usb/r8152.c
+++ b/drivers/net/usb/r8152.c
@@ -1845,16 +1845,7 @@ static u8 r8152_rx_csum(struct r8152 *tp, struct rx_desc *rx_desc)
 	opts2 = le32_to_cpu(rx_desc->opts2);
 	opts3 = le32_to_cpu(rx_desc->opts3);
 
-	if (opts2 & RD_IPV4_CS) {
-		if (opts3 & IPF)
-			checksum = CHECKSUM_NONE;
-		else if ((opts2 & RD_UDP_CS) && (opts3 & UDPF))
-			checksum = CHECKSUM_NONE;
-		else if ((opts2 & RD_TCP_CS) && (opts3 & TCPF))
-			checksum = CHECKSUM_NONE;
-		else
-			checksum = CHECKSUM_UNNECESSARY;
-	} else if (opts2 & RD_IPV6_CS) {
+	if (opts2 & (RD_IPV4_CS | RD_IPV6_CS)) {
 		if ((opts2 & RD_UDP_CS) && !(opts3 & UDPF))
 			checksum = CHECKSUM_UNNECESSARY;
 		else if ((opts2 & RD_TCP_CS) && !(opts3 & TCPF))


signature.asc
Description: This is a digitally signed message part

Bug#889827: linux-image-4.15.0-rc8-amd64: IP offloading does not work

[Vincent Danjean]

Package: src:linux
Version: 4.15~rc8-1~exp1
Severity: normal

  Hi,

  Since I got my laptop, I sometimes see some issues when transfering big
chunks of data ("dd if=big-partition ... | ssh ...", or just plain "apt 
upgrade").
With the ssh, I got error messages about wrong protocol (sorry, I did not
keep the error message). With apt, I got such kind of messages while downloading
the packages:
[...]
Réception de:82 http://ftp.fr.debian.org/debian unstable/main amd64 libgdal20 
amd64 2.2.3+dfsg-2 [5 327 kB]
Err:82 http://ftp.fr.debian.org/debian unstable/main amd64 libgdal20 amd64 
2.2.3+dfsg-2
  Somme de contrôle de hachage incohérente
  Hashes of expected file:
   - SHA256:832ddfc6014fd3be7f7cf61191e6eff2136e09d1636832bc09c63b4afb63e29e
   - MD5Sum:e9f3cd0628a30760a8fec012ba48a1fe [weak]
   - Filesize:5327124 [weak]
  Hashes of received file:
   - SHA256:e52c862d27a7fcb3cdadfd2d6afe4645f68940bae0cc238f0d0cc184081875ba
   - MD5Sum:21a29fff8d706b6a0233735ca2e78bcd [weak]
   - Filesize:5327124 [weak]
  Last modification reported: Tue, 06 Feb 2018 09:59:18 +
[...]

  In both cases, the error is transient (if I retry, it generally works or
it fails at another place) and occurs on high-speed network (ssh on a local
GB network, apt on a local GB network and then a high speed ISP network (the
French Renater network)). I did not observe issues on small transferts nor
on transfert on slower networks (no issues with apt on a ADSL link).
  Recently, I though about TCP offloading and experiment a bit. And, indeed,
I observe no more issues if I type:
$ sudo ethtool -K en-wired rx off tx off
before my "sudo apt upgrade -yd"
And I got issues again if I set offload on again:
$ sudo ethtool -K en-wired rx on tx on
$ sudo apt-get clean
$ sudo apt upgrade -yd

  So, it seems there is an issue with checksum offloading with this driver
and hardware.
  Note that I got this bug with previous kernels (such as 4.14*) but I did not
note the exact versions and I did not think to disable offloading at this time.

  Regards,
Vincent 


-- Package-specific info:
** Version:
Linux version 4.15.0-rc8-amd64 (debian-ker...@lists.debian.org) (gcc version 
7.2.0 (Debian 7.2.0-19)) #1 SMP Debian 4.15~rc8-1~exp1 (2018-01-15)

** Command line:
BOOT_IMAGE=/boot/vmlinuz-4.15.0-rc8-amd64 root=/dev/mapper/eyak-root ro 
apparmor=0 security= swapaccount=1 cgroup_enable=memory atkbd.softraw=0 
vsyscall=emulate quiet

** Tainted: W (512)
 * Taint on warning.

** Kernel log:
[71838.653455] usb 3-1: Product: USB2807 Hub
[71838.653456] usb 3-1: Manufacturer: Microchip
[71838.672701] hub 3-1:1.0: USB hub found
[71838.672826] hub 3-1:1.0: 7 ports detected
[71838.746169] thunderbolt :07:00.0: 301: uid: 0x8086537e20a87610
[71838.746272] thunderbolt :07:00.0:  Port 0: 8086:1578 (Revision: 4, TB 
Version: 1, Type: Port (0x1))
[71838.746273] thunderbolt :07:00.0:   Max hop id (in/out): 7/7
[71838.746273] thunderbolt :07:00.0:   Max counters: 8
[71838.746274] thunderbolt :07:00.0:   NFC Credits: 0x80
[71838.746792] thunderbolt :07:00.0:  Port 1: 8086:1578 (Revision: 4, TB 
Version: 1, Type: Port (0x1))
[71838.746793] thunderbolt :07:00.0:   Max hop id (in/out): 15/15
[71838.746793] thunderbolt :07:00.0:   Max counters: 16
[71838.746794] thunderbolt :07:00.0:   NFC Credits: 0x780
[71838.747305] thunderbolt :07:00.0:  Port 2: 8086:1578 (Revision: 4, TB 
Version: 1, Type: Port (0x1))
[71838.747305] thunderbolt :07:00.0:   Max hop id (in/out): 15/15
[71838.747306] thunderbolt :07:00.0:   Max counters: 16
[71838.747306] thunderbolt :07:00.0:   NFC Credits: 0x0
[71838.747838] thunderbolt :07:00.0:  Port 3: 8086:1578 (Revision: 4, TB 
Version: 1, Type: Port (0x1))
[71838.747838] thunderbolt :07:00.0:   Max hop id (in/out): 15/15
[71838.747839] thunderbolt :07:00.0:   Max counters: 16
[71838.747839] thunderbolt :07:00.0:   NFC Credits: 0x3c0
[71838.748344] thunderbolt :07:00.0:  Port 4: 8086:1578 (Revision: 4, TB 
Version: 1, Type: Port (0x1))
[71838.748345] thunderbolt :07:00.0:   Max hop id (in/out): 15/15
[71838.748345] thunderbolt :07:00.0:   Max counters: 16
[71838.748346] thunderbolt :07:00.0:   NFC Credits: 0x3c0
[71838.748346] thunderbolt :07:00.0: 301:5: disabled by eeprom
[71838.748448] thunderbolt :07:00.0:  Port 6: 8086:1578 (Revision: 4, TB 
Version: 1, Type: PCIe (0x100102))
[71838.748448] thunderbolt :07:00.0:   Max hop id (in/out): 8/8
[71838.748449] thunderbolt :07:00.0:   Max counters: 2
[71838.748449] thunderbolt :07:00.0:   NFC Credits: 0x80
[71838.748587] thunderbolt :07:00.0:  Port 7: 8086:1578 (Revision: 4, TB 
Version: 1, Type: PCIe (0x100101))
[71838.748588] thunderbolt :07:00.0:   Max hop id (in/out): 8/8
[71838.748588] thunderbolt :07:00.0:   Max counters: 2
[71838.748589] thunderbolt :07:00.0:   NFC Credits: 0x80
[71838.748749] thunderbolt :07:00.0:  Port 8: 8086:1578 (Revision: 4, TB