Bug#927825: arm: mvneta driver used on Armada XP GP boards does not receive packets (regression from 4.9)
Hello, On Tue, Apr 30, 2019 at 10:12:27AM +0200, Uwe Kleine-König wrote: > On Thu, Apr 25, 2019 at 09:17:32PM +0200, Aurelien Jarno wrote: > > On 2019-04-25 14:50, Aurelien Jarno wrote: > > > On 2019-04-23 22:16, Aurelien Jarno wrote: > > > > Source: linux > > > > Version: 4.19.28-2 > > > > Severity: important > > > > > > > > After upgrading hartmann.debian.org (an armhf buildd using an Armada XP > > > > GP board) from buster to stretch, the ethernet device is not working > > "upgrading from buster to stretch" doesn't make sense. I think you meant > from stretch to buster. > > > > > > > More precisely the board is a "Marvell Armada XP Development Board > > > DB-MV784MP-GP" > > > > > > > anymore. Using tcpdump on both the buildd and a remote host, it appears > > > > that the packets correctly leave the board and that the reception side > > > > fails. > > If you can send to a remote host at least ARP (or ND) must be working, > so some reception still works, right? > > > > > The module used for the ethernet device is mvneta. The corresponding DT > > > > compatible entry is "marvell,armada-xp-neta". > > > > > > > > > > I have started a "bisection" with the kernels from snapshot. This is > > > what I have found so far: > > > > > > This one works: > > > - linux-image-4.19.0-rc6-armmp-lpae_4.19~rc6-1~exp1_armhf.deb > > > > > > The following ones don't: > > > - linux-image-4.19.0-rc7-armmp-lpae_4.19~rc7-1~exp1_armhf.deb > > > - linux-image-5.0.0-trunk-armmp_5.0.2-1~exp1_armhf.deb > > > > > > My guess (I don't have time to try more now) is that the issue is caused > > > by the following change: > > > > > > | [ Uwe Kleine-König ] > > > | * [armhf] enable MVNETA_BM_ENABLE and CAN_FLEXCAN as a module > > > > > > > I confirm this is the issue. Disabling MVNETA_BM_ENABLE on kernel > > 4.19.28-2 fixes the issue. Note that it breaks the ABI. > > A colleague happens to work with an XP based machine with a (nearly) > vanilla kernel based on 5.1.0-rc6 and there enabling MVNETA_BM_ENABLE > doesn't render networking nonfunctional. > > Looking through the changes to drivers/net/ethernet/marvell/mvneta* > between 5.0 and 5.1-rc6 there isn't something that would explain a fix > though. There doesn't seem to be a good explanation in the debian > specific patches either. > > So this problem is either machine specific or it works with the mvneta > driver builtin. (I didn't double check, but guess that my colleague uses > =y and the Debian kernel =m). Well, or I missed something. > > Is it possible to test a few things on hartmann? I'd suggest: > > - try (vanilla) 5.1-rc6 with MVNETA=y > - try an older kernel (maybe 4.6 as the buffer manager stuff was >introduced in dc35a10f68d3 ("net: mvneta: bm: add support for >hardware buffer management") which made it into 4.6-rc1) with >MVNETA_BM_ENABLE=[ym]. Thanks to Steve McIntyre I got access to a DB-MV784MP-GP and did the second test. I used mvebu_v7_defconfig and enabled CGROUPS and AUTOFS4 (to please systemd) and MVNETA_BM_ENABLE=y on 4.6. The latter breaks networking similar to newer kernel versions. So I guess the buffer management never worked on that board. I don't have the time to debug this issue further, but will disable buffer management for the Debian kernel again. Best regards Uwe signature.asc Description: PGP signature
Bug#927825: arm: mvneta driver used on Armada XP GP boards does not receive packets (regression from 4.9)
Hi, On 2019-05-01 00:04, Aurelien Jarno wrote: > On 2019-04-30 10:12, Uwe Kleine-König wrote: > > > > More precisely the board is a "Marvell Armada XP Development Board > > > > DB-MV784MP-GP" > > > > > > > > > anymore. Using tcpdump on both the buildd and a remote host, it > > > > > appears > > > > > that the packets correctly leave the board and that the reception side > > > > > fails. > > > > If you can send to a remote host at least ARP (or ND) must be working, > > so some reception still works, right? > > I have to try again, but what i have seen is the ARP requests from > hartmann arriving to the other hosts on the subnet. Steve McIntyre > (added in Cc:) confirmed me on IRC being able to reproduce the issue on > another board. I confirm that. Basically on the other hosts of the same subnet, I can see the ARP requests: 18:23:45.979860 ARP, Request who-has 172.28.17.1 tell 172.28.17.18, length 46 18:23:47.002990 ARP, Request who-has 172.28.17.1 tell 172.28.17.18, length 46 18:23:48.027262 ARP, Request who-has 172.28.17.1 tell 172.28.17.18, length 46 18:23:52.004248 ARP, Request who-has 172.28.17.1 tell 172.28.17.18, length 46 18:23:53.019252 ARP, Request who-has 172.28.17.1 tell 172.28.17.18, length 46 18:23:54.043276 ARP, Request who-has 172.28.17.1 tell 172.28.17.18, length 46 18:23:58.027937 ARP, Request who-has 172.28.17.1 tell 172.28.17.18, length 46 172.28.17.1 is the gateway, 172.28.17.18 is hartmann.d.o. > > Is it possible to test a few things on hartmann? I'd suggest: > > > > - try (vanilla) 5.1-rc6 with MVNETA=y I have tried 5.1-rc7 with: CONFIG_MVNETA_BM_ENABLE=y CONFIG_MVNETA=y CONFIG_MVNETA_BM=y and also with CONFIG_MVNETA_BM_ENABLE=m CONFIG_MVNETA=m CONFIG_MVNETA_BM=m And the mvneta network driver is not able to receive data in both cases. Best regards, Aurelien -- Aurelien Jarno GPG: 4096R/1DDD8C9B aurel...@aurel32.net http://www.aurel32.net signature.asc Description: PGP signature
Bug#927825: arm: mvneta driver used on Armada XP GP boards does not receive packets (regression from 4.9)
On 2019-04-30 10:12, Uwe Kleine-König wrote: > [Adding the mvebu guys and netdev to Cc] > > Hello, > > On Thu, Apr 25, 2019 at 09:17:32PM +0200, Aurelien Jarno wrote: > > On 2019-04-25 14:50, Aurelien Jarno wrote: > > > On 2019-04-23 22:16, Aurelien Jarno wrote: > > > > Source: linux > > > > Version: 4.19.28-2 > > > > Severity: important > > > > > > > > After upgrading hartmann.debian.org (an armhf buildd using an Armada XP > > > > GP board) from buster to stretch, the ethernet device is not working > > "upgrading from buster to stretch" doesn't make sense. I think you meant > from stretch to buster. You're correct. To be more precise the kernel is there the issue, so that should even be upgrading from the stretch kernel to the buster kernel. > > > More precisely the board is a "Marvell Armada XP Development Board > > > DB-MV784MP-GP" > > > > > > > anymore. Using tcpdump on both the buildd and a remote host, it appears > > > > that the packets correctly leave the board and that the reception side > > > > fails. > > If you can send to a remote host at least ARP (or ND) must be working, > so some reception still works, right? I have to try again, but what i have seen is the ARP requests from hartmann arriving to the other hosts on the subnet. Steve McIntyre (added in Cc:) confirmed me on IRC being able to reproduce the issue on another board. > > > > The module used for the ethernet device is mvneta. The corresponding DT > > > > compatible entry is "marvell,armada-xp-neta". > > > > > > > > > > I have started a "bisection" with the kernels from snapshot. This is > > > what I have found so far: > > > > > > This one works: > > > - linux-image-4.19.0-rc6-armmp-lpae_4.19~rc6-1~exp1_armhf.deb > > > > > > The following ones don't: > > > - linux-image-4.19.0-rc7-armmp-lpae_4.19~rc7-1~exp1_armhf.deb > > > - linux-image-5.0.0-trunk-armmp_5.0.2-1~exp1_armhf.deb > > > > > > My guess (I don't have time to try more now) is that the issue is caused > > > by the following change: > > > > > > | [ Uwe Kleine-König ] > > > | * [armhf] enable MVNETA_BM_ENABLE and CAN_FLEXCAN as a module > > > > > > > I confirm this is the issue. Disabling MVNETA_BM_ENABLE on kernel > > 4.19.28-2 fixes the issue. Note that it breaks the ABI. > > A colleague happens to work with an XP based machine with a (nearly) > vanilla kernel based on 5.1.0-rc6 and there enabling MVNETA_BM_ENABLE > doesn't render networking nonfunctional. > > Looking through the changes to drivers/net/ethernet/marvell/mvneta* > between 5.0 and 5.1-rc6 there isn't something that would explain a fix > though. There doesn't seem to be a good explanation in the debian > specific patches either. > > So this problem is either machine specific or it works with the mvneta > driver builtin. (I didn't double check, but guess that my colleague uses > =y and the Debian kernel =m). Well, or I missed something. Yes the debian kernel uses: CONFIG_MVNETA_BM_ENABLE=m CONFIG_MVNETA=m CONFIG_MVNETA_BM=m > Is it possible to test a few things on hartmann? I'd suggest: > > - try (vanilla) 5.1-rc6 with MVNETA=y > - try an older kernel (maybe 4.6 as the buffer manager stuff was >introduced in dc35a10f68d3 ("net: mvneta: bm: add support for >hardware buffer management") which made it into 4.6-rc1) with >MVNETA_BM_ENABLE=[ym]. Ok, I'll try that in the next days. I just do not have remote power, only serial console, I hope the kernels will boot enough to be able to roll back to another kernel. Best regards, Aurelien -- Aurelien Jarno GPG: 4096R/1DDD8C9B aurel...@aurel32.net http://www.aurel32.net signature.asc Description: PGP signature
Bug#927825: arm: mvneta driver used on Armada XP GP boards does not receive packets (regression from 4.9)
[Adding the mvebu guys and netdev to Cc] Hello, On Thu, Apr 25, 2019 at 09:17:32PM +0200, Aurelien Jarno wrote: > On 2019-04-25 14:50, Aurelien Jarno wrote: > > On 2019-04-23 22:16, Aurelien Jarno wrote: > > > Source: linux > > > Version: 4.19.28-2 > > > Severity: important > > > > > > After upgrading hartmann.debian.org (an armhf buildd using an Armada XP > > > GP board) from buster to stretch, the ethernet device is not working "upgrading from buster to stretch" doesn't make sense. I think you meant from stretch to buster. > > > > More precisely the board is a "Marvell Armada XP Development Board > > DB-MV784MP-GP" > > > > > anymore. Using tcpdump on both the buildd and a remote host, it appears > > > that the packets correctly leave the board and that the reception side > > > fails. If you can send to a remote host at least ARP (or ND) must be working, so some reception still works, right? > > > The module used for the ethernet device is mvneta. The corresponding DT > > > compatible entry is "marvell,armada-xp-neta". > > > > > > > I have started a "bisection" with the kernels from snapshot. This is > > what I have found so far: > > > > This one works: > > - linux-image-4.19.0-rc6-armmp-lpae_4.19~rc6-1~exp1_armhf.deb > > > > The following ones don't: > > - linux-image-4.19.0-rc7-armmp-lpae_4.19~rc7-1~exp1_armhf.deb > > - linux-image-5.0.0-trunk-armmp_5.0.2-1~exp1_armhf.deb > > > > My guess (I don't have time to try more now) is that the issue is caused > > by the following change: > > > > | [ Uwe Kleine-König ] > > | * [armhf] enable MVNETA_BM_ENABLE and CAN_FLEXCAN as a module > > > > I confirm this is the issue. Disabling MVNETA_BM_ENABLE on kernel > 4.19.28-2 fixes the issue. Note that it breaks the ABI. A colleague happens to work with an XP based machine with a (nearly) vanilla kernel based on 5.1.0-rc6 and there enabling MVNETA_BM_ENABLE doesn't render networking nonfunctional. Looking through the changes to drivers/net/ethernet/marvell/mvneta* between 5.0 and 5.1-rc6 there isn't something that would explain a fix though. There doesn't seem to be a good explanation in the debian specific patches either. So this problem is either machine specific or it works with the mvneta driver builtin. (I didn't double check, but guess that my colleague uses =y and the Debian kernel =m). Well, or I missed something. Is it possible to test a few things on hartmann? I'd suggest: - try (vanilla) 5.1-rc6 with MVNETA=y - try an older kernel (maybe 4.6 as the buffer manager stuff was introduced in dc35a10f68d3 ("net: mvneta: bm: add support for hardware buffer management") which made it into 4.6-rc1) with MVNETA_BM_ENABLE=[ym]. Best regards Uwe signature.asc Description: PGP signature
Bug#927825: arm: mvneta driver used on Armada XP GP boards does not receive packets (regression from 4.9)
On 2019-04-25 14:50, Aurelien Jarno wrote: > On 2019-04-23 22:16, Aurelien Jarno wrote: > > Source: linux > > Version: 4.19.28-2 > > Severity: important > > > > After upgrading hartmann.debian.org (an armhf buildd using an Armada XP > > GP board) from buster to stretch, the ethernet device is not working > > More precisely the board is a "Marvell Armada XP Development Board > DB-MV784MP-GP" > > > anymore. Using tcpdump on both the buildd and a remote host, it appears > > that the packets correctly leave the board and that the reception side > > fails. > > > > The module used for the ethernet device is mvneta. The corresponding DT > > compatible entry is "marvell,armada-xp-neta". > > > > I have started a "bisection" with the kernels from snapshot. This is > what I have found so far: > > This one works: > - linux-image-4.19.0-rc6-armmp-lpae_4.19~rc6-1~exp1_armhf.deb > > The following ones don't: > - linux-image-4.19.0-rc7-armmp-lpae_4.19~rc7-1~exp1_armhf.deb > - linux-image-5.0.0-trunk-armmp_5.0.2-1~exp1_armhf.deb > > My guess (I don't have time to try more now) is that the issue is caused > by the following change: > > | [ Uwe Kleine-König ] > | * [armhf] enable MVNETA_BM_ENABLE and CAN_FLEXCAN as a module > I confirm this is the issue. Disabling MVNETA_BM_ENABLE on kernel 4.19.28-2 fixes the issue. Note that it breaks the ABI. Aurelien -- Aurelien Jarno GPG: 4096R/1DDD8C9B aurel...@aurel32.net http://www.aurel32.net
Bug#927825: arm: mvneta driver used on Armada XP GP boards does not receive packets (regression from 4.9)
On 2019-04-23 22:16, Aurelien Jarno wrote: > Source: linux > Version: 4.19.28-2 > Severity: important > > After upgrading hartmann.debian.org (an armhf buildd using an Armada XP > GP board) from buster to stretch, the ethernet device is not working More precisely the board is a "Marvell Armada XP Development Board DB-MV784MP-GP" > anymore. Using tcpdump on both the buildd and a remote host, it appears > that the packets correctly leave the board and that the reception side > fails. > > The module used for the ethernet device is mvneta. The corresponding DT > compatible entry is "marvell,armada-xp-neta". > I have started a "bisection" with the kernels from snapshot. This is what I have found so far: This one works: - linux-image-4.19.0-rc6-armmp-lpae_4.19~rc6-1~exp1_armhf.deb The following ones don't: - linux-image-4.19.0-rc7-armmp-lpae_4.19~rc7-1~exp1_armhf.deb - linux-image-5.0.0-trunk-armmp_5.0.2-1~exp1_armhf.deb My guess (I don't have time to try more now) is that the issue is caused by the following change: | [ Uwe Kleine-König ] | * [armhf] enable MVNETA_BM_ENABLE and CAN_FLEXCAN as a module Add Uwe as Cc: so that he can comment on the change. Aurelien -- Aurelien Jarno GPG: 4096R/1DDD8C9B aurel...@aurel32.net http://www.aurel32.net
Bug#927825: arm: mvneta driver used on Armada XP GP boards does not receive packets (regression from 4.9)
Source: linux Version: 4.19.28-2 Severity: important After upgrading hartmann.debian.org (an armhf buildd using an Armada XP GP board) from buster to stretch, the ethernet device is not working anymore. Using tcpdump on both the buildd and a remote host, it appears that the packets correctly leave the board and that the reception side fails. The module used for the ethernet device is mvneta. The corresponding DT compatible entry is "marvell,armada-xp-neta".