Bug#820622: linux-image-4.5.0-trunk-armmp-lpae: raspberry pi 2: smsc95xx 1-1.1:1.0 eth0: kevent 2 may have been dropped

2020-09-09 Thread Gunnar Wolf 
fixed 820622 4.19.132-1
thanks

Hello,

I have not seen this message appear in my RPi2, using stock Debian
kernels starting at 4.19 (even a bit before that, I think). I do not
use the LPAE kernel, as I understand it does not make much sense in
the RPi2 (sold only with 1GB RAM, and not expandable), so probably
this specific kernel should be tested, but I believe the bug can be
now safely closed.


signature.asc
Description: PGP signature

Bug#820622: linux-image-4.5.0-trunk-armmp-lpae: raspberry pi 2: smsc95xx 1-1.1:1.0 eth0: kevent 2 may have been dropped

2016-05-07 Thread Diederik de Haas 
On Saturday 07 May 2016 12:44:41 Ben Hutchings wrote:
> If this patch reduces the risk of
> failed buffer allocations then I think it's still a win.

I agree (fwiw).

signature.asc
Description: This is a digitally signed message part.

Bug#820622: linux-image-4.5.0-trunk-armmp-lpae: raspberry pi 2: smsc95xx 1-1.1:1.0 eth0: kevent 2 may have been dropped

2016-05-07 Thread Ben Hutchings 
On Tue, 2016-05-03 at 09:13 -0700, Vagrant Cascadian wrote:

> On 2016-05-02, Vagrant Cascadian wrote:
> > 
> > On 2016-05-02, Ben Hutchings wrote:
> > > 
> > > On Sun, 2016-05-01 at 18:20 -0700, Vagrant Cascadian wrote:
> > > > 
> > > > On 2016-04-28, Ben Hutchings wrote:
> > > > > 
> > > > > Could you test with turbo_mode re-enabled and with this patch applied?
> > > > > 
> > > > > Also could you test network receive throughput (e.g. with netperf -t
> > > > > TCP_STREAM, sending *to* the RPi) in these three different
> > > > > configurations:
> > > > Ok, if I understood you correctly...
> > > > 
> > > > Installed netperf on another machine, and ran:
> > > > 
> > > >   netperf -t TCP_STREAM 10.0.0.50
> > > That is not correct syntax; you need to put a -H before the IP address.
> > Ok. Never used netperf before... will try again!
> Ok, this time with:
> 
>   netperf -l 60 -t TCP_STREAM -H 10.0.0.50
> 
> 4.5.2-1, with turbo disabled:
>   
>   MIGRATED TCP STREAM TEST from 0.0.0.0 () port 0 AF_INET to 10.0.0.50 ()
>   port 0 AF_INET : demo
>   Recv   SendSend
>   Socket Socket  Message  Elapsed
>   Size   SizeSize Time Throughput
>   bytes  bytes   bytessecs.10^6bits/sec
>   
>    87380  16384  1638460.04  93.95

Interesting - it's very close to saturating the link even without
turbo.  (The maximum possible TCP throughput over and Ethernet with
standard MTU is about 94% of the underling bit rate.)

> 4.5.2-1, with turbo enabled:
>   
>   MIGRATED TCP STREAM TEST from 0.0.0.0 () port 0 AF_INET to 10.0.0.50 ()
>   port 0 AF_INET : demo
>   Recv   SendSend
>   Socket Socket  Message  Elapsed
>   Size   SizeSize Time Throughput
>   bytes  bytes   bytessecs.10^6bits/sec
>   
>    87380  16384  1638460.03  94.15
>   
>   
> 4.5.2, patched with turbo enabled:
>   
>   MIGRATED TCP STREAM TEST from 0.0.0.0 () port 0 AF_INET to 10.0.0.50 ()
>   port 0 AF_INET : demo
>   Recv   SendSend
>   Socket Socket  Message  Elapsed
>   Size   SizeSize Time Throughput
>   bytes  bytes   bytessecs.10^6bits/sec
>   
>    87380  16384  1638460.04  94.15

So the patch doesn't seem to hurt performance at all.

From your previous mail:

> For what it's worth, the patched driver did still appear to eventually
> generate the error logs reported:
> 
>   smsc95xx 1-1.1:1.0 eth0: kevent 2 may have been dropped
> 
> So at best, it reduces the problem, rather than solving it.

I think the basic problem is you're giving this machine more tasks than
will comfortably fit in its memory, added to which the swap device is
slow (or maybe you disabled swap?).  If this patch reduces the risk of
failed buffer allocations then I think it's still a win.

Ben.

-- 
Ben Hutchings
Editing code like this is akin to sticking plasters on the bleeding stump
of a severed limb. - me, 29 June 1999

signature.asc
Description: This is a digitally signed message part

Bug#820622: linux-image-4.5.0-trunk-armmp-lpae: raspberry pi 2: smsc95xx 1-1.1:1.0 eth0: kevent 2 may have been dropped

2016-05-03 Thread Vagrant Cascadian 
On 2016-05-02, Vagrant Cascadian wrote:
> On 2016-05-02, Ben Hutchings wrote:
>> On Sun, 2016-05-01 at 18:20 -0700, Vagrant Cascadian wrote:
>>> On 2016-04-28, Ben Hutchings wrote:
>>> > Could you test with turbo_mode re-enabled and with this patch applied?
>>> > 
>>> > Also could you test network receive throughput (e.g. with netperf -t
>>> > TCP_STREAM, sending *to* the RPi) in these three different
>>> > configurations:
>>> Ok, if I understood you correctly...
>>> 
>>> Installed netperf on another machine, and ran:
>>> 
>>>   netperf -t TCP_STREAM 10.0.0.50
>>
>> That is not correct syntax; you need to put a -H before the IP address.
>
> Ok. Never used netperf before... will try again!

Ok, this time with:

  netperf -l 60 -t TCP_STREAM -H 10.0.0.50

4.5.2-1, with turbo disabled:
  
  MIGRATED TCP STREAM TEST from 0.0.0.0 () port 0 AF_INET to 10.0.0.50 ()
  port 0 AF_INET : demo
  Recv   SendSend
  Socket Socket  Message  Elapsed
  Size   SizeSize Time Throughput
  bytes  bytes   bytessecs.10^6bits/sec
  
   87380  16384  1638460.04  93.95
  
4.5.2-1, with turbo enabled:
  
  MIGRATED TCP STREAM TEST from 0.0.0.0 () port 0 AF_INET to 10.0.0.50 ()
  port 0 AF_INET : demo
  Recv   SendSend
  Socket Socket  Message  Elapsed
  Size   SizeSize Time Throughput
  bytes  bytes   bytessecs.10^6bits/sec
  
   87380  16384  1638460.03  94.15
  
  
4.5.2, patched with turbo enabled:
  
  MIGRATED TCP STREAM TEST from 0.0.0.0 () port 0 AF_INET to 10.0.0.50 ()
  port 0 AF_INET : demo
  Recv   SendSend
  Socket Socket  Message  Elapsed
  Size   SizeSize Time Throughput
  bytes  bytes   bytessecs.10^6bits/sec
  
   87380  16384  1638460.04  94.15


live well,
  vagrant


signature.asc
Description: PGP signature

Bug#820622: linux-image-4.5.0-trunk-armmp-lpae: raspberry pi 2: smsc95xx 1-1.1:1.0 eth0: kevent 2 may have been dropped

2016-05-02 Thread Vagrant Cascadian 
On 2016-05-02, Ben Hutchings wrote:
> On Sun, 2016-05-01 at 18:20 -0700, Vagrant Cascadian wrote:
>> On 2016-04-28, Ben Hutchings wrote:
>> > Could you test with turbo_mode re-enabled and with this patch applied?
>> > 
>> > Also could you test network receive throughput (e.g. with netperf -t
>> > TCP_STREAM, sending *to* the RPi) in these three different
>> > configurations:
>> Ok, if I understood you correctly...
>> 
>> Installed netperf on another machine, and ran:
>> 
>>   netperf -t TCP_STREAM 10.0.0.50
>
> That is not correct syntax; you need to put a -H before the IP address.

Ok. Never used netperf before... will try again!

For what it's worth, the patched driver did still appear to eventually
generate the error logs reported:

  smsc95xx 1-1.1:1.0 eth0: kevent 2 may have been dropped

So at best, it reduces the problem, rather than solving it.


live well,
  vagrant


signature.asc
Description: PGP signature

Bug#820622: linux-image-4.5.0-trunk-armmp-lpae: raspberry pi 2: smsc95xx 1-1.1:1.0 eth0: kevent 2 may have been dropped

2016-05-02 Thread Ben Hutchings 
On Sun, 2016-05-01 at 18:20 -0700, Vagrant Cascadian wrote:
> Control: tag 820622 -moreinfo
> 
> On 2016-04-28, Ben Hutchings wrote:
> 
> > 
> > Could you test with turbo_mode re-enabled and with this patch applied?
> > 
> > Also could you test network receive throughput (e.g. with netperf -t
> > TCP_STREAM, sending *to* the RPi) in these three different
> > configurations:
> Ok, if I understood you correctly...
> 
> Installed netperf on another machine, and ran:
> 
>   netperf -t TCP_STREAM 10.0.0.50

That is not correct syntax; you need to put a -H before the IP address.

[...]
> To my untrained eye, doesn't look like a significant difference between
> any of the modes.
[...]

This is not surprising as you did a loopback test on the other machine!

Ben.

-- 
Ben Hutchings
Life is what happens to you while you're busy making other plans.
   - John Lennon

signature.asc
Description: This is a digitally signed message part

Processed: Re: Bug#820622: linux-image-4.5.0-trunk-armmp-lpae: raspberry pi 2: smsc95xx 1-1.1:1.0 eth0: kevent 2 may have been dropped

2016-05-01 Thread Debian Bug Tracking System 
Processing control commands:

> tag 820622 -moreinfo
Bug #820622 [src:linux] linux-image-4.5.0-trunk-armmp-lpae: raspberry pi 2: 
smsc95xx 1-1.1:1.0 eth0: kevent 2 may have been dropped
Removed tag(s) moreinfo.

-- 
820622: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=820622
Debian Bug Tracking System
Contact ow...@bugs.debian.org with problems

Bug#820622: linux-image-4.5.0-trunk-armmp-lpae: raspberry pi 2: smsc95xx 1-1.1:1.0 eth0: kevent 2 may have been dropped

2016-05-01 Thread Vagrant Cascadian 
Control: tag 820622 -moreinfo

On 2016-04-28, Ben Hutchings wrote:

> Could you test with turbo_mode re-enabled and with this patch applied?
>
> Also could you test network receive throughput (e.g. with netperf -t
> TCP_STREAM, sending *to* the RPi) in these three different
> configurations:

Ok, if I understood you correctly...

Installed netperf on another machine, and ran:

  netperf -t TCP_STREAM 10.0.0.50

Where 10.0.0.50 is the raspberry pi 2.

Ran netperf twice for each combination, rebooting the raspberry pi 2
between changes in turbo_mode or kernel. Double-checked
/sys/module/smsc95xx/parameters/turbo_mode contained N when booted with
turbo_mode=0, and Y when turbo_mode=1.

To my untrained eye, doesn't look like a significant difference between
any of the modes. None of them triggered the kernel messages that
prompted the bug report.


> 1. turbo_mode=0

  MIGRATED TCP STREAM TEST from 0.0.0.0 () port 0 AF_INET to localhost () port 
0 AF_INET : demo
  Recv   SendSend
  Socket Socket  Message  Elapsed
  Size   SizeSize Time Throughput
  bytes  bytes   bytessecs.10^6bits/sec
  
   87380  16384  1638410.021781.22

  MIGRATED TCP STREAM TEST from 0.0.0.0 () port 0 AF_INET to localhost () port 
0 AF_INET : demo
  Recv   SendSend
  Socket Socket  Message  Elapsed
  Size   SizeSize Time Throughput
  bytes  bytes   bytessecs.10^6bits/sec
  
   87380  16384  1638410.021847.21


> 2. turbo_mode=1, current driver

  MIGRATED TCP STREAM TEST from 0.0.0.0 () port 0 AF_INET to localhost () port 
0 AF_INET : demo
  Recv   SendSend  
  Socket Socket  Message  Elapsed  
  Size   SizeSize Time Throughput  
  bytes  bytes   bytessecs.10^6bits/sec  
  
   87380  16384  1638410.021812.38   
  MIGRATED TCP STREAM TEST from 0.0.0.0 () port 0 AF_INET to localhost () port 
0 AF_INET : demo
  Recv   SendSend  
  Socket Socket  Message  Elapsed  
  Size   SizeSize Time Throughput  
  bytes  bytes   bytessecs.10^6bits/sec  
  
   87380  16384  1638410.021843.34   


> 3. turbo_mode=1, patched driver

  MIGRATED TCP STREAM TEST from 0.0.0.0 () port 0 AF_INET to localhost () port 
0 AF_INET : demo
  Recv   SendSend  
  Socket Socket  Message  Elapsed  
  Size   SizeSize Time Throughput  
  bytes  bytes   bytessecs.10^6bits/sec  
  
   87380  16384  1638410.031822.99   
  MIGRATED TCP STREAM TEST from 0.0.0.0 () port 0 AF_INET to localhost () port 
0 AF_INET : demo
  Recv   SendSend  
  Socket Socket  Message  Elapsed  
  Size   SizeSize Time Throughput  
  bytes  bytes   bytessecs.10^6bits/sec  
  
   87380  16384  1638410.031824.52   


live well,
  vagrant


signature.asc
Description: PGP signature

Bug#820622: linux-image-4.5.0-trunk-armmp-lpae: raspberry pi 2: smsc95xx 1-1.1:1.0 eth0: kevent 2 may have been dropped

2016-04-28 Thread Ben Hutchings 
On Thu, 2016-04-28 at 17:30 +0200, Diederik de Haas wrote:
> On Thursday 28 April 2016 17:12:27 Ben Hutchings wrote:
> > 
> > Could you test with turbo_mode re-enabled and with this patch applied?
> Which patch? (or do I need more coffee?)
> I did find a commit in git which may be related, but I didn't see a direct 
> reference to this issue.
> https://anonscm.debian.org/cgit/kernel/linux.git/commit/?h=sid=76d0845dce198a77d063aac386886c0bc8c3330b

D'oh.  Here it is.

Ben.

-- 
Ben Hutchings
All extremists should be taken out and shot.From 01983960918ad1e491dce39fbfd75d4f603ca3d6 Mon Sep 17 00:00:00 2001
From: Ben Hutchings 
Date: Thu, 28 Apr 2016 16:44:43 +0200
Subject: [PATCH] smsc95xx: Limit RX burst size to order-1 to avoid allocation
 failures

The smsc95xx driver enables RX burst mode or 'turbo mode' by default,
in which the hardware packs multiple packets into a single URB of size
~17 K (order-3, assuming 4 K pages).  The Raspberry Pi boards use this
driver and have as little as 512 MiB RAM, so when they have been
running for a while heavy network traffic tends to result in
allocation failures.

Currently the Raspbian distribution works around this by disabling
turbo mode by default, at the cost of network performance.  But we
should be able to get some benefit from burst mode without using
such large buffers.  Limit the burst size to 2 pages, i.e. order-1.

References: https://bugs.debian.org/820622
Reported-by: Vagrant Cascadian 
Signed-off-by: Ben Hutchings 
---
 drivers/net/usb/smsc95xx.c | 8 ++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/drivers/net/usb/smsc95xx.c b/drivers/net/usb/smsc95xx.c
index 66b3ab9f614e..56d16b9568eb 100644
--- a/drivers/net/usb/smsc95xx.c
+++ b/drivers/net/usb/smsc95xx.c
@@ -35,8 +35,12 @@
 #define SMSC_DRIVER_VERSION		"1.0.4"
 #define HS_USB_PKT_SIZE			(512)
 #define FS_USB_PKT_SIZE			(64)
-#define DEFAULT_HS_BURST_CAP_SIZE	(16 * 1024 + 5 * HS_USB_PKT_SIZE)
-#define DEFAULT_FS_BURST_CAP_SIZE	(6 * 1024 + 33 * FS_USB_PKT_SIZE)
+/* Maximum URB size to limit risk of allocation failures */
+#define MAX_URB_SIZE			SKB_MAX_ORDER(NET_IP_ALIGN, 1)
+#define DEFAULT_HS_BURST_CAP_SIZE	\
+	min_t(size_t, 16 * 1024 + 5 * HS_USB_PKT_SIZE, MAX_URB_SIZE)
+#define DEFAULT_FS_BURST_CAP_SIZE	\
+	min_t(size_t, 6 * 1024 + 33 * FS_USB_PKT_SIZE, MAX_URB_SIZE)
 #define DEFAULT_BULK_IN_DELAY		(0x2000)
 #define MAX_SINGLE_PACKET_SIZE		(2048)
 #define LAN95XX_EEPROM_MAGIC		(0x9500)


signature.asc
Description: This is a digitally signed message part

Bug#820622: linux-image-4.5.0-trunk-armmp-lpae: raspberry pi 2: smsc95xx 1-1.1:1.0 eth0: kevent 2 may have been dropped

2016-04-28 Thread Diederik de Haas 
On Thursday 28 April 2016 17:12:27 Ben Hutchings wrote:
> Could you test with turbo_mode re-enabled and with this patch applied?

Which patch? (or do I need more coffee?)
I did find a commit in git which may be related, but I didn't see a direct 
reference to this issue.
https://anonscm.debian.org/cgit/kernel/linux.git/commit/?h=sid=76d0845dce198a77d063aac386886c0bc8c3330b

signature.asc
Description: This is a digitally signed message part.

Processed: Re: Bug#820622: linux-image-4.5.0-trunk-armmp-lpae: raspberry pi 2: smsc95xx 1-1.1:1.0 eth0: kevent 2 may have been dropped

2016-04-28 Thread Debian Bug Tracking System 
Processing control commands:

> tag -1 patch moreinfo
Bug #820622 [src:linux] linux-image-4.5.0-trunk-armmp-lpae: raspberry pi 2: 
smsc95xx 1-1.1:1.0 eth0: kevent 2 may have been dropped
Added tag(s) moreinfo and patch.

-- 
820622: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=820622
Debian Bug Tracking System
Contact ow...@bugs.debian.org with problems

Bug#820622: linux-image-4.5.0-trunk-armmp-lpae: raspberry pi 2: smsc95xx 1-1.1:1.0 eth0: kevent 2 may have been dropped

2016-04-28 Thread Ben Hutchings 
Control: tag -1 patch moreinfo

On Wed, 2016-04-13 at 11:43 -0700, Vagrant Cascadian wrote:
> On 2016-04-10, Vagrant Cascadian wrote:
> > 
> > On 2016-04-10, Ben Hutchings wrote:
> > > 
> > > On Sun, 2016-04-10 at 22:49 +0200, Diederik de Haas wrote:
> > > > 
> > > > At https://plugwash.raspbian.org/linux_3.18/ you can find the
> > > > things for the 
> > > > latest raspbian.org kernel.
> > > Thanks.  That has a patch to turn off the turbo_mode module
> > > parameter
> > > by default.
> > > 
> > > Vagrant, could you try putting "options smsc95xx turbo_mode=N" in
> > > /etc/modprobe.d/smsc95xx.conf ?
> > Added to one of the two machines, updated the initramfs (just in
> > case
> > the module gets loaded in initramfs), and rebooted ... will see if
> > that
> > works.
> > 
> > The other option I was pointed to was to update to the latest boot
> > firmware, though I haven't tried that yet:
> New boot firmware didn't help. Configuring the module to not use
> turbo_mode did.

Could you test with turbo_mode re-enabled and with this patch applied?

Also could you test network receive throughput (e.g. with netperf -t
TCP_STREAM, sending *to* the RPi) in these three different
configurations:

1. turbo_mode=0
2. turbo_mode=1, current driver
3. turbo_mode=1, patched driver

(Unfortunately netperf is non-free due to a slightly weird licence.
Also take care to remove it afterwards as the server allows a trivial
network DoS if you leave it running...)

Ben.

-- 
Ben Hutchings
All extremists should be taken out and shot.


signature.asc
Description: This is a digitally signed message part

Bug#820622: linux-image-4.5.0-trunk-armmp-lpae: raspberry pi 2: smsc95xx 1-1.1:1.0 eth0: kevent 2 may have been dropped

2016-04-13 Thread Vagrant Cascadian 
On 2016-04-10, Vagrant Cascadian wrote:
> On 2016-04-10, Ben Hutchings wrote:
>> On Sun, 2016-04-10 at 22:49 +0200, Diederik de Haas wrote:
>>> At https://plugwash.raspbian.org/linux_3.18/ you can find the things for 
>>> the 
>>> latest raspbian.org kernel.
>>
>> Thanks.  That has a patch to turn off the turbo_mode module parameter
>> by default.
>>
>> Vagrant, could you try putting "options smsc95xx turbo_mode=N" in
>> /etc/modprobe.d/smsc95xx.conf ?
>
> Added to one of the two machines, updated the initramfs (just in case
> the module gets loaded in initramfs), and rebooted ... will see if that
> works.
>
> The other option I was pointed to was to update to the latest boot
> firmware, though I haven't tried that yet:

New boot firmware didn't help. Configuring the module to not use
turbo_mode did.

live well,
  vagrant


signature.asc
Description: PGP signature

Bug#820622: linux-image-4.5.0-trunk-armmp-lpae: raspberry pi 2: smsc95xx 1-1.1:1.0 eth0: kevent 2 may have been dropped

2016-04-11 Thread Vagrant Cascadian 
On 2016-04-10, Vagrant Cascadian wrote:
> On 2016-04-10, Ben Hutchings wrote:

>> Vagrant, could you try putting "options smsc95xx turbo_mode=N" in
>> /etc/modprobe.d/smsc95xx.conf ?
>
> Added to one of the two machines, updated the initramfs (just in case
> the module gets loaded in initramfs), and rebooted ... will see if that
> works.

Haven't seen any messages since adding those module options, so that's
at least a viable workaround.


live well,
  vagrant


signature.asc
Description: PGP signature

Bug#820622: linux-image-4.5.0-trunk-armmp-lpae: raspberry pi 2: smsc95xx 1-1.1:1.0 eth0: kevent 2 may have been dropped

2016-04-11 Thread Bjørn Mork 
Ben Hutchings  writes:
> On Mon, 2016-04-11 at 09:55 +0200, Bjørn Mork wrote:
>> Ben Hutchings  writes:
>> > 
>> > On Sun, 2016-04-10 at 11:15 -0700, Vagrant Cascadian wrote:
>> > > 
>> > > It works for the most part, but floods syslog with messages:
>> > > 
>> > > [501966.870273] net_ratelimit: 35702 callbacks suppressed
>> > > [501966.875438] smsc95xx 1-1.1:1.0 eth0: kevent 2 may have been dropped
>> > > 
>> > > It seems to function ok, though I'm not sure if there's degraded
>> > > network performance...
>> > [...]
>> > 
>> > I understand network performance on all RPi models is poor due to lack
>> > of a built-in Ethernet MAC and the poor design of the USB interface.
>> > Also that message is a generic error message from the usbnet core and
>> > is not specific to the smsc95xx driver.
>> Yes, kevent 2 is "EVENT_RX_MEMORY", and this error affects any usbnet
>> based device under memory pressure.  The usbnet framework just doesn't
>> handle the case where GFP_KERNEL allocations fail. How could it? 
>
> What I don't understand, given this, is why Vagrant didn't report any
> OOM warnings.  (I wondered whether the __GFP_NOWARN flag was wrongly
> being added somewhere, but I don't see that.)

Maybe I'm wrong?  Looking at this again, I notice that rx_submit()
always will call usb_submit_urb(urb, GFP_ATOMIC) regardless of the input
mem_flags, because it's holding a spinlock when submitting the urb.  So
it doesn't necessarily help much that the urb and skb are allocated with
GFP_KERNEL.

Exactly what allocations usb_submit_urb() may end up with seems to be
depending on the host controller. So the error spam could very well be
caused by a problematic host controller.  At least partly.

>> The only viable "fix" I can see is by preallocating and recycling all
>> buffers instead of the repeated allocations done by usbnet now.  But
>> that's a major refactoring of usbnet.
>> 
>> The log message spam could of course be fixed, but the rate-limiting is
>> "good enough".
>
> How is anyone supposed to know what "kevent 2" is though?

By looking it up in include/linux/usb/usbnet.h :)

Yes, that could certainly be improved by translating the number back to
a readable description.  

> The drivers that support batching into large RX buffers should
> automatically fall back to smaller buffers if this happens, or whenever
> they're running on a small (for some definition of small) machine.  I
> understand that the dynamic fallback may be tricky though.

I don't think the smsc95xx use any large buffers.  Looks like it's an
ethernet packet + 12 bytes.  And usbnet limits the total rx queue to
60 * 1518 bytes.


Bjørn

Bug#820622: linux-image-4.5.0-trunk-armmp-lpae: raspberry pi 2: smsc95xx 1-1.1:1.0 eth0: kevent 2 may have been dropped

2016-04-11 Thread Ben Hutchings 
On Mon, 2016-04-11 at 09:55 +0200, Bjørn Mork wrote:
> Ben Hutchings  writes:
> > 
> > On Sun, 2016-04-10 at 11:15 -0700, Vagrant Cascadian wrote:
> > > 
> > > It works for the most part, but floods syslog with messages:
> > > 
> > > [501966.870273] net_ratelimit: 35702 callbacks suppressed
> > > [501966.875438] smsc95xx 1-1.1:1.0 eth0: kevent 2 may have been dropped
> > > 
> > > It seems to function ok, though I'm not sure if there's degraded
> > > network performance...
> > [...]
> > 
> > I understand network performance on all RPi models is poor due to lack
> > of a built-in Ethernet MAC and the poor design of the USB interface.
> > Also that message is a generic error message from the usbnet core and
> > is not specific to the smsc95xx driver.
> Yes, kevent 2 is "EVENT_RX_MEMORY", and this error affects any usbnet
> based device under memory pressure.  The usbnet framework just doesn't
> handle the case where GFP_KERNEL allocations fail. How could it? 

What I don't understand, given this, is why Vagrant didn't report any
OOM warnings.  (I wondered whether the __GFP_NOWARN flag was wrongly
being added somewhere, but I don't see that.)

> The only viable "fix" I can see is by preallocating and recycling all
> buffers instead of the repeated allocations done by usbnet now.  But
> that's a major refactoring of usbnet.
> 
> The log message spam could of course be fixed, but the rate-limiting is
> "good enough".

How is anyone supposed to know what "kevent 2" is though?

The drivers that support batching into large RX buffers should
automatically fall back to smaller buffers if this happens, or whenever
they're running on a small (for some definition of small) machine.  I
understand that the dynamic fallback may be tricky though.

Ben.

-- 
Ben Hutchings
Humans are not rational beings; they are rationalising beings.

signature.asc
Description: This is a digitally signed message part

Bug#820622: linux-image-4.5.0-trunk-armmp-lpae: raspberry pi 2: smsc95xx 1-1.1:1.0 eth0: kevent 2 may have been dropped

2016-04-11 Thread Bjørn Mork 
Ben Hutchings  writes:
> On Sun, 2016-04-10 at 11:15 -0700, Vagrant Cascadian wrote:
>> It works for the most part, but floods syslog with messages:
>> 
>> [501966.870273] net_ratelimit: 35702 callbacks suppressed
>> [501966.875438] smsc95xx 1-1.1:1.0 eth0: kevent 2 may have been dropped
>> 
>> It seems to function ok, though I'm not sure if there's degraded
>> network performance...
> [...]
>
> I understand network performance on all RPi models is poor due to lack
> of a built-in Ethernet MAC and the poor design of the USB interface.
> Also that message is a generic error message from the usbnet core and
> is not specific to the smsc95xx driver.

Yes, kevent 2 is "EVENT_RX_MEMORY", and this error affects any usbnet
based device under memory pressure.  The usbnet framework just doesn't
handle the case where GFP_KERNEL allocations fail. How could it? 

The only viable "fix" I can see is by preallocating and recycling all
buffers instead of the repeated allocations done by usbnet now.  But
that's a major refactoring of usbnet.

The log message spam could of course be fixed, but the rate-limiting is
"good enough".


Bjørn

Bug#820622: linux-image-4.5.0-trunk-armmp-lpae: raspberry pi 2: smsc95xx 1-1.1:1.0 eth0: kevent 2 may have been dropped

2016-04-10 Thread Vagrant Cascadian 
On 2016-04-10, Ben Hutchings wrote:
> On Sun, 2016-04-10 at 22:49 +0200, Diederik de Haas wrote:
>> At https://plugwash.raspbian.org/linux_3.18/ you can find the things for the 
>> latest raspbian.org kernel.
>
> Thanks.  That has a patch to turn off the turbo_mode module parameter
> by default.
>
> Vagrant, could you try putting "options smsc95xx turbo_mode=N" in
> /etc/modprobe.d/smsc95xx.conf ?

Added to one of the two machines, updated the initramfs (just in case
the module gets loaded in initramfs), and rebooted ... will see if that
works.

The other option I was pointed to was to update to the latest boot
firmware, though I haven't tried that yet:

  https://wiki.gentoo.org/wiki/Raspberry_Pi#Troubleshooting

  Troubleshooting

Problem: dmesg is full of smsc95xx 1-1.1:1.0: eth0: kevent 2 may
have been dropped and/or page allocation failure messages:
Solution: Try to update all firmware files in /boot,
especially fixup.dat.
Solution: If the former fails, add smsc95xx.turbo_mode=N to
kernel parameters, or vm.min_free_kbytes=4096 to /etc/sysctl.conf


live well,
  vagrant


signature.asc
Description: PGP signature

Bug#820622: linux-image-4.5.0-trunk-armmp-lpae: raspberry pi 2: smsc95xx 1-1.1:1.0 eth0: kevent 2 may have been dropped

2016-04-10 Thread Ben Hutchings 
On Sun, 2016-04-10 at 22:49 +0200, Diederik de Haas wrote:
> On Sun, 10 Apr 2016 20:14:19 +0100 Ben Hutchings  > wrote:
> > 
> > I checked the latest Raspbian kernel I could find
> > (,3.16.7-ckt11-1+deb8u4+rpi1) and they don't have any patches to that
> > driver (other than adding a module parameter for no good reason).
> Where did you look?

Under http://archive.raspbian.org/raspbian/pool/main/l/linux/

> At https://plugwash.raspbian.org/linux_3.18/ you can find the things for the 
> latest raspbian.org kernel.

Thanks.  That has a patch to turn off the turbo_mode module parameter
by default.

Vagrant, could you try putting "options smsc95xx turbo_mode=N" in
/etc/modprobe.d/smsc95xx.conf ?

Ben.

-- 
Ben Hutchings
Humans are not rational beings; they are rationalising beings.


signature.asc
Description: This is a digitally signed message part

Bug#820622: linux-image-4.5.0-trunk-armmp-lpae: raspberry pi 2: smsc95xx 1-1.1:1.0 eth0: kevent 2 may have been dropped

2016-04-10 Thread Diederik de Haas 
On Sun, 10 Apr 2016 20:14:19 +0100 Ben Hutchings  wrote:
> I checked the latest Raspbian kernel I could find
> (,3.16.7-ckt11-1+deb8u4+rpi1) and they don't have any patches to that
> driver (other than adding a module parameter for no good reason).

Where did you look?
At https://plugwash.raspbian.org/linux_3.18/ you can find the things for the 
latest raspbian.org kernel.

signature.asc
Description: This is a digitally signed message part.

Bug#820622: linux-image-4.5.0-trunk-armmp-lpae: raspberry pi 2: smsc95xx 1-1.1:1.0 eth0: kevent 2 may have been dropped

2016-04-10 Thread Vagrant Cascadian 
On 2016-04-10, Ben Hutchings wrote:
> On Sun, 2016-04-10 at 11:15 -0700, Vagrant Cascadian wrote:
>> Thanks for enabling support for raspberry pi 2!
>
> Thanks for confirming it works, because I was working without any
> access to hardware.

For what it's worth, I've not been able to get the 4.4.x kernels from
Debian to even show console output with an otherwise identical setup. So
there must be something missing from the backported rpi patchset.


> I see you've updated flash-kernel to support it as
> well.

Yeah, I tested it with the u-boot-rpi package.


>> It works for the most part, but floods syslog with messages:
>> 
>> [501966.870273] net_ratelimit: 35702 callbacks suppressed
>> [501966.875438] smsc95xx 1-1.1:1.0 eth0: kevent 2 may have been dropped
>> 
>> It seems to function ok, though I'm not sure if there's degraded
>> network performance...
> [...]
>
> I understand network performance on all RPi models is poor due to lack
> of a built-in Ethernet MAC and the poor design of the USB interface.
> Also that message is a generic error message from the usbnet core and
> is not specific to the smsc95xx driver.
>
> Were you using the network heavily, or do these messages appear
> regardless of the level of network traffic?

They have pretty steady network traffic. I haven't monitored network
traffic with enough granularity to really correlate if the messages
occur at the same time as peak network usage, though.

vnstat --hours output:

 eth0 19:58 
  ^ r   
  | r   
  | r   
  | r   
  | rrr 
  |  r  r  r rr 
  |  r  r t r  r r   r  r r 
  |  r  r t r  r r   r  r  r  r 
  |  r  r  r rt r   r  r r r r  r  r  r 
  |  r  r  r rt r   r  r   tr  r r  r  r  rt r  r  r  r 
 -+---> 
  |  20 21 22 23 00 01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 16 17 18 19

 h  rx (KiB)   tx (KiB)  h  rx (KiB)   tx (KiB)  h  rx (KiB)   tx (KiB) 
20337,957 35,64904 53,478 17,62812252,402  3,730
21248,322  4,69005107,901  3,20213 67,155 18,629
22119,047 32,82706105,470  5,66914170,309  2,387
23 39,099  4,73607 50,548 68,11015 66,700 81,917
00172,232258,62608 50,122  3,18816397,451 11,037
01574,549 10,33709166,691  2,46017246,196 15,701
02 14,133  1,08810337,027  6,79818215,436 15,847
03312  2,16711  9,29362519366,292  8,472

And syslogs of the messages:

Apr 10 08:33:35 rpi2b kernel: [467762.422149] net_ratelimit: 3837 callbacks 
suppressed
Apr 10 08:33:35 rpi2b kernel: [467762.427247] smsc95xx 1-1.1:1.0 eth0: kevent 2 
may have been dropped
--
Apr 10 08:33:42 rpi2b kernel: [467769.491572] net_ratelimit: 5501 callbacks 
suppressed
Apr 10 08:33:42 rpi2b kernel: [467769.496651] smsc95xx 1-1.1:1.0 eth0: kevent 2 
may have been dropped
--
Apr 10 09:01:45 rpi2b kernel: [469453.280337] net_ratelimit: 996 callbacks 
suppressed
Apr 10 09:01:45 rpi2b kernel: [469453.285330] smsc95xx 1-1.1:1.0 eth0: kevent 2 
may have been dropped
--
Apr 10 09:09:53 rpi2b kernel: [469940.843408] net_ratelimit: 11182 callbacks 
suppressed
Apr 10 09:09:53 rpi2b kernel: [469940.848708] smsc95xx 1-1.1:1.0 eth0: kevent 2 
may have been dropped
--
Apr 10 09:11:13 rpi2b kernel: [470020.740828] net_ratelimit: 4669 callbacks 
suppressed
Apr 10 09:11:13 rpi2b kernel: [470020.745911] smsc95xx 1-1.1:1.0 eth0: kevent 2 
may have been dropped
--
Apr 10 09:28:45 rpi2b kernel: [471072.895953] net_ratelimit: 2945 callbacks 
suppressed
Apr 10 09:28:45 rpi2b kernel: [471072.901033] smsc95xx 1-1.1:1.0 eth0: kevent 2 
may have been dropped
--
Apr 10 09:50:03 rpi2b kernel: [472350.602289] net_ratelimit: 157500 callbacks 
suppressed
Apr 10 09:50:03 rpi2b kernel: [472350.607546] smsc95xx 1-1.1:1.0 eth0: kevent 2 
may have been dropped
--
Apr 10 09:50:08 rpi2b kernel: [472356.167786] net_ratelimit: 6353 callbacks 
suppressed
Apr 10 09:50:08 rpi2b kernel: [472356.172989] smsc95xx 1-1.1:1.0 eth0: kevent 2 
may have been dropped
--
Apr 10 09:51:59 rpi2b kernel: [472466.940525] net_ratelimit: 1425 callbacks

Bug#820622: linux-image-4.5.0-trunk-armmp-lpae: raspberry pi 2: smsc95xx 1-1.1:1.0 eth0: kevent 2 may have been dropped

2016-04-10 Thread Ben Hutchings 
Control: tag -1 moreinfo

On Sun, 2016-04-10 at 11:15 -0700, Vagrant Cascadian wrote:
> Package: src:linux
> Version: 4.5-1~exp1
> Severity: normal
> 
> Thanks for enabling support for raspberry pi 2!

Thanks for confirming it works, because I was working without any
access to hardware.  I see you've updated flash-kernel to support it as
well.

> It works for the most part, but floods syslog with messages:
> 
> [501966.870273] net_ratelimit: 35702 callbacks suppressed
> [501966.875438] smsc95xx 1-1.1:1.0 eth0: kevent 2 may have been dropped
> 
> It seems to function ok, though I'm not sure if there's degraded
> network performance...
[...]

I understand network performance on all RPi models is poor due to lack
of a built-in Ethernet MAC and the poor design of the USB interface.
Also that message is a generic error message from the usbnet core and
is not specific to the smsc95xx driver.

Were you using the network heavily, or do these messages appear
regardless of the level of network traffic?

I checked the latest Raspbian kernel I could find
(,3.16.7-ckt11-1+deb8u4+rpi1) and they don't have any patches to that
driver (other than adding a module parameter for no good reason).

Ben.

-- 
Ben Hutchings
I'm always amazed by the number of people who take up solipsism because
they heard someone else explain it. - E*Borg on alt.fan.pratchett

signature.asc
Description: This is a digitally signed message part

Processed: Re: Bug#820622: linux-image-4.5.0-trunk-armmp-lpae: raspberry pi 2: smsc95xx 1-1.1:1.0 eth0: kevent 2 may have been dropped

2016-04-10 Thread Debian Bug Tracking System 
Processing control commands:

> tag -1 moreinfo
Bug #820622 [src:linux] linux-image-4.5.0-trunk-armmp-lpae: raspberry pi 2: 
smsc95xx 1-1.1:1.0 eth0: kevent 2 may have been dropped
Added tag(s) moreinfo.

-- 
820622: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=820622
Debian Bug Tracking System
Contact ow...@bugs.debian.org with problems

Bug#820622: linux-image-4.5.0-trunk-armmp-lpae: raspberry pi 2: smsc95xx 1-1.1:1.0 eth0: kevent 2 may have been dropped

2016-04-10 Thread Vagrant Cascadian 
Package: src:linux
Version: 4.5-1~exp1
Severity: normal

Thanks for enabling support for raspberry pi 2!

It works for the most part, but floods syslog with messages:

[501966.870273] net_ratelimit: 35702 callbacks suppressed
[501966.875438] smsc95xx 1-1.1:1.0 eth0: kevent 2 may have been dropped

It seems to function ok, though I'm not sure if there's degraded
network performance...

Not sure if the version of the boot firmware and/or u-boot would likely
impact this, or running stretch/sid vs. jessie.

I also experience this on a second rpi2, similarly configured.


live well,
  vagrant


-- Package-specific info:
** Version:
Linux version 4.5.0-trunk-armmp-lpae (debian-kernel@lists.debian.org) (gcc 
version 5.3.1 20160316 (Debian 5.3.1-12) ) #1 SMP Debian 4.5-1~exp1 (2016-03-20)

** Command line:
root=/dev/mapper/rpi2b-root console=ttyAMA0,115200

** Not tainted

** Kernel log:
[496926.383178] smsc95xx 1-1.1:1.0 eth0: kevent 2 may have been dropped
[496926.390069] smsc95xx 1-1.1:1.0 eth0: kevent 2 may have been dropped
[496926.397026] smsc95xx 1-1.1:1.0 eth0: kevent 2 may have been dropped
[496926.405843] smsc95xx 1-1.1:1.0 eth0: kevent 2 may have been dropped
[496926.412441] smsc95xx 1-1.1:1.0 eth0: kevent 2 may have been dropped
[496926.419085] smsc95xx 1-1.1:1.0 eth0: kevent 2 may have been dropped
[496926.425923] smsc95xx 1-1.1:1.0 eth0: kevent 2 may have been dropped
[496926.433675] smsc95xx 1-1.1:1.0 eth0: kevent 2 may have been dropped
[496926.440371] smsc95xx 1-1.1:1.0 eth0: kevent 2 may have been dropped
[496926.447153] smsc95xx 1-1.1:1.0 eth0: kevent 2 may have been dropped
[496931.412674] net_ratelimit: 6206 callbacks suppressed
[496931.417762] smsc95xx 1-1.1:1.0 eth0: kevent 2 may have been dropped
[496932.093933] smsc95xx 1-1.1:1.0 eth0: kevent 2 may have been dropped
[496932.108465] smsc95xx 1-1.1:1.0 eth0: kevent 2 may have been dropped
[496932.164059] smsc95xx 1-1.1:1.0 eth0: kevent 2 may have been dropped
[496932.208265] smsc95xx 1-1.1:1.0 eth0: kevent 2 may have been dropped
[496932.226975] smsc95xx 1-1.1:1.0 eth0: kevent 2 may have been dropped
[496932.246691] smsc95xx 1-1.1:1.0 eth0: kevent 2 may have been dropped
[496932.265630] smsc95xx 1-1.1:1.0 eth0: kevent 2 may have been dropped
[496932.287754] smsc95xx 1-1.1:1.0 eth0: kevent 2 may have been dropped
[496932.294860] smsc95xx 1-1.1:1.0 eth0: kevent 2 may have been dropped
[497554.490001] net_ratelimit: 285 callbacks suppressed
[497554.495003] smsc95xx 1-1.1:1.0 eth0: kevent 2 may have been dropped
[497554.501775] smsc95xx 1-1.1:1.0 eth0: kevent 2 may have been dropped
[497554.510097] smsc95xx 1-1.1:1.0 eth0: kevent 2 may have been dropped
[497554.516646] smsc95xx 1-1.1:1.0 eth0: kevent 2 may have been dropped
[497554.523284] smsc95xx 1-1.1:1.0 eth0: kevent 2 may have been dropped
[497554.531653] smsc95xx 1-1.1:1.0 eth0: kevent 2 may have been dropped
[497554.538317] smsc95xx 1-1.1:1.0 eth0: kevent 2 may have been dropped
[497554.545981] smsc95xx 1-1.1:1.0 eth0: kevent 2 may have been dropped
[497554.554849] smsc95xx 1-1.1:1.0 eth0: kevent 2 may have been dropped
[497554.561473] smsc95xx 1-1.1:1.0 eth0: kevent 2 may have been dropped
[498179.500059] net_ratelimit: 5196 callbacks suppressed
[498179.505139] smsc95xx 1-1.1:1.0 eth0: kevent 2 may have been dropped
[498190.972627] smsc95xx 1-1.1:1.0 eth0: kevent 2 may have been dropped
[498192.138691] smsc95xx 1-1.1:1.0 eth0: kevent 2 may have been dropped
[498193.264114] smsc95xx 1-1.1:1.0 eth0: kevent 2 may have been dropped
[498193.270634] smsc95xx 1-1.1:1.0 eth0: kevent 2 may have been dropped
[498193.278020] smsc95xx 1-1.1:1.0 eth0: kevent 2 may have been dropped
[498193.284598] smsc95xx 1-1.1:1.0 eth0: kevent 2 may have been dropped
[498193.291305] smsc95xx 1-1.1:1.0 eth0: kevent 2 may have been dropped
[498193.298322] smsc95xx 1-1.1:1.0 eth0: kevent 2 may have been dropped
[498193.305610] smsc95xx 1-1.1:1.0 eth0: kevent 2 may have been dropped
[498193.312196] smsc95xx 1-1.1:1.0 eth0: kevent 2 may have been dropped
[498667.628700] net_ratelimit: 6384 callbacks suppressed
[498667.633789] smsc95xx 1-1.1:1.0 eth0: kevent 2 may have been dropped
[498741.372264] smsc95xx 1-1.1:1.0 eth0: kevent 2 may have been dropped
[498741.379072] smsc95xx 1-1.1:1.0 eth0: kevent 2 may have been dropped
[498741.385927] smsc95xx 1-1.1:1.0 eth0: kevent 2 may have been dropped
[498741.392619] smsc95xx 1-1.1:1.0 eth0: kevent 2 may have been dropped
[498741.399375] smsc95xx 1-1.1:1.0 eth0: kevent 2 may have been dropped
[498741.406110] smsc95xx 1-1.1:1.0 eth0: kevent 2 may have been dropped
[498741.412718] smsc95xx 1-1.1:1.0 eth0: kevent 2 may have been dropped
[498741.419497] smsc95xx 1-1.1:1.0 eth0: kevent 2 may have been dropped
[498741.426310] smsc95xx 1-1.1:1.0 eth0: kevent 2 may have been dropped
[498741.433143] smsc95xx 1-1.1:1.0 eth0: kevent 2 may have been dropped
[501393.624239] net_ratelimit: 2507 callbacks suppressed
[501393.629322] smsc95xx 1-1.1:1.0 eth0: kevent 2 may have been dropped