Re: serial/ftdi_sio byte loss / performance regression

2013-06-07 Thread Tomaž Šolc
On 06/06/2013 11:58 AM, Johan Hovold wrote:
 On Thu, Jun 06, 2013 at 10:50:36AM +0200, Tomaž Šolc wrote:
 I have noticed that the ftdi_sio serial driver in recent kernel versions
 has very bad performance when used through the Python's serial library.

 As a test case I have a custom device that will send a continuous block
 of 5k characters once every few seconds over a RS-232 line (115200 baud)
 to an Olimex programmer (based on FT2232C, also tried one with FT2232H).

 Programmer is connected to a Linux system where a simple Python script
 reads the device:

 import serial
 comm = serial.Serial(/dev/ttyUSB0, 115200)
 while True:
  line = comm.readline()

 With kernels before 3.7.0 the script reads uncorrupted data while using
 newer kernels (including 3.9.4) the Python script sees heavy byte loss.
 top shows an 95% idle CPU. Only very slow transmissions (on the
 order of tens of bytes per second) will come through uncorrupted.

 Using git-bisect, I have found the commit that introduced this problem:

 6f602912c9d0c84c2edbd446dd9f72660b701605
 usb: serial: ftdi_sio: Add missing chars_in_buffer function

 This might also be related with the unusual way Python serial library
 reads the device. It uses select() with no timeout and single byte
 read()s in a loop. strace output:

 select(4, [3], [], [], NULL)= 1 (in [3])
 read(3, D, 1) = 1
 select(4, [3], [], [], NULL)= 1 (in [3])
 read(3, E, 1) = 1
 ...

 With sufficiently large read()s the byte loss can be eliminated.

 With the commit above, each select() now causes an additional round trip
 over USB to read the state of the hardware buffer. It's possible that
 constant status querying triggers some bug in the hardware or the query
 is simply too slow and causes overflows in the hardware buffer.
 
 You're absolutely right. This is a known issue (the select overhead)
 that was just recently fixed by commit a37025b5c7 (USB: ftdi_sio: fix
 chars_in_buffer overhead) in v3.10-rc3. Care to give v3.10-rc4 a try?

Johan, thanks for the prompt reply. I have tried v3.10-rc4 and indeed
the issue is fixed.

Best regards
Tomaž
--
To unsubscribe from this list: send the line unsubscribe linux-usb in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


serial/ftdi_sio byte loss / performance regression

2013-06-06 Thread Tomaž Šolc
Hi

I have noticed that the ftdi_sio serial driver in recent kernel versions
has very bad performance when used through the Python's serial library.

As a test case I have a custom device that will send a continuous block
of 5k characters once every few seconds over a RS-232 line (115200 baud)
to an Olimex programmer (based on FT2232C, also tried one with FT2232H).

Programmer is connected to a Linux system where a simple Python script
reads the device:

import serial
comm = serial.Serial(/dev/ttyUSB0, 115200)
while True:
line = comm.readline()

With kernels before 3.7.0 the script reads uncorrupted data while using
newer kernels (including 3.9.4) the Python script sees heavy byte loss.
top shows an 95% idle CPU. Only very slow transmissions (on the
order of tens of bytes per second) will come through uncorrupted.

Using git-bisect, I have found the commit that introduced this problem:

6f602912c9d0c84c2edbd446dd9f72660b701605
usb: serial: ftdi_sio: Add missing chars_in_buffer function

This might also be related with the unusual way Python serial library
reads the device. It uses select() with no timeout and single byte
read()s in a loop. strace output:

select(4, [3], [], [], NULL)= 1 (in [3])
read(3, D, 1) = 1
select(4, [3], [], [], NULL)= 1 (in [3])
read(3, E, 1) = 1
...

With sufficiently large read()s the byte loss can be eliminated.

With the commit above, each select() now causes an additional round trip
over USB to read the state of the hardware buffer. It's possible that
constant status querying triggers some bug in the hardware or the query
is simply too slow and causes overflows in the hardware buffer.

Thanks for your help
Tomaž
--
To unsubscribe from this list: send the line unsubscribe linux-usb in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html