Re: Overly conservative xHCI bandwidth estimation

[Lu, Baolu]

On 11/09/2015 05:27 PM, Steinar H. Gunderson wrote:

On Mon, Nov 09, 2015 at 01:02:50PM +0800, Lu, Baolu wrote:

We've got below facts:

1. One card works well w/ or w/o LPM enabled.

No, this isn't right. One card works well without LPM; it does work with LPM
but not well. There are ping timeouts that cause disconnects, especially
during the first packets.


I wasn't aware of this kind of issue.




2. Two cards work well w/o LPM, but not work w/ LPM. The error
 is "not enough bus bandwidth".

Correct.


We can come to a conclusion that "LPM consumes extra usb bus
bandwidth, this extra bandwidth could cause device not to work
due to limited bandwidth resources."

I have a patch attached in this email. It can disable LPM for any
devices during run time. Do you mind having a try? You can follow
below steps:

You saw Alan's blacklisting patch, right? It works perfectly for me.
But I'll be happy to try yours as well.


Yes, I know that you already have a patch and it works for you.
I am in favor of Alan's patch. You can go ahead with that patch.

I just want to find a way that users can use an LPM-unfriendly
device, but don't need to customize a kernel or wait until the
the quirk patch gets merged.



/* Steinar */


Thanks,
Baolu
--
To unsubscribe from this list: send the line "unsubscribe linux-usb" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Overly conservative xHCI bandwidth estimation

[Lu, Baolu]

On 11/07/2015 07:05 PM, Steinar H. Gunderson wrote:

Can you cat /sys/bus/usb/devices/2-2/power/usb3_hardware_lpm_u1
>and /sys/bus/usb/devices/2-2/power/usb3_hardware_lpm_u2 for several
>times? Don't use that device when cat these two files.

I waited a bit longer, and indeed, now it consistently shows enabled on both
cards.

   klump:~> cat /sys/bus/usb/devices/2-1/power/usb3_hardware_lpm_u1
   enabled
   klump:~> cat /sys/bus/usb/devices/2-1/power/usb3_hardware_lpm_u2
   enabled
   klump:~> cat /sys/bus/usb/devices/2-2/power/usb3_hardware_lpm_u1
   enabled
   klump:~> cat /sys/bus/usb/devices/2-2/power/usb3_hardware_lpm_u2
   enabled


That's good. Thank you for your time.

We've got below facts:

1. One card works well w/ or w/o LPM enabled.
2. Two cards work well w/o LPM, but not work w/ LPM. The error
is "not enough bus bandwidth".

We can come to a conclusion that "LPM consumes extra usb bus
bandwidth, this extra bandwidth could cause device not to work
due to limited bandwidth resources."

I have a patch attached in this email. It can disable LPM for any
devices during run time. Do you mind having a try? You can follow
below steps:

1) apply the attached two patches on top of the latest 4.3 kernel.
0001-usb-core-lpm-fix-usb3_hardware_lpm-sysfs-node.patch
0002-usb-core-lpm-add-sysfs-node-for-usb3-lpm-permit.patch
2) build and install the kernel.
3) boot your machine with the new kernel.
4) insert two  Blackmagic Design device into USB3 root port.
5) disable LPM for devices in step 4).

# echo "0" > /sys/bus/usb/devices/<1st_device_path>/port/usb3_lpm_permit
# echo "0" > /sys/bus/usb/devices/<2nd_device_path>/port/usb3_lpm_permit

You can check your setting by reading the sysfs file. They should read 0.

6) start the two devices at the same time, can they work at the same time?

Thanks,
Baolu


For fun, I tried also with a USB 2.0 cable. U1 and U2 comes up as disabled,
but of course, the device doesn't work.

/* Steinar */


>From 836c0ab187bcc5a08ac3a6d55136203f922ad95b Mon Sep 17 00:00:00 2001
From: Lu Baolu 
Date: Fri, 6 Nov 2015 09:53:59 +0800
Subject: [PATCH 2/3] usb: core: lpm: add sysfs node for usb3 lpm permit

USB3 LPM is default on in Linux kernel if both xHCI host controller
and the USB devices declare to be LPM-capable. Unfortunately, some
devices are known to work well with LPM disabled, but to be broken
if LPM is enabled, although it declares the LPM capability.  Users
won't be able to use this kind of devices, until someone puts them
in the kernel blacklist and gets the kernel upgraded.

This patch adds a sysfs node to permit or forbit USB3 LPM U1 or U2
entry for a port. The settings apply to both before and after device
enumeration. Supported values are "0" - neither u1 nor u2 permitted,
"u1" - only u1 is permitted, "u2" - only u2 is permitted, "u1_u2" -
both u1 and u2 are permitted.

Signed-off-by: Lu Baolu 
Signed-off-by: Zhuang Jin Can 
---
 Documentation/ABI/testing/sysfs-bus-usb | 11 
 drivers/usb/core/hub.c  | 15 +-
 drivers/usb/core/hub.h  |  5 +-
 drivers/usb/core/port.c | 89 -
 4 files changed, 116 insertions(+), 4 deletions(-)

diff --git a/Documentation/ABI/testing/sysfs-bus-usb b/Documentation/ABI/testing/sysfs-bus-usb
index 136ba17..4165c37 100644
--- a/Documentation/ABI/testing/sysfs-bus-usb
+++ b/Documentation/ABI/testing/sysfs-bus-usb
@@ -189,6 +189,17 @@ Description:
 		The file will read "hotplug", "wired" and "not used" if the
 		information is available, and "unknown" otherwise.
 
+What:		/sys/bus/usb/devices/.../(hub interface)/portX/usb3_lpm_permit
+Date:		November 2015
+Contact:	Lu Baolu 
+Description:
+		Some USB3.0 devices may not support usb3 lpm well. usb3_lpm_permit
+		attribute allows enabling/disabling usb3 lpm of a port. It takes
+		effect both before and after a usb device is enumerated. Supported
+		values are "0" if both u1 and u2 are NOT permitted, "u1" if only u1
+		is permitted, "u2" if only u2 is permitted, "u1_u2" if both u1 and
+		u2 are permitted.
+
 What:		/sys/bus/usb/devices/.../power/usb2_lpm_l1_timeout
 Date:		May 2013
 Contact:	Mathias Nyman 
diff --git a/drivers/usb/core/hub.c b/drivers/usb/core/hub.c
index 94a10e0..023e39f 100644
--- a/drivers/usb/core/hub.c
+++ b/drivers/usb/core/hub.c
@@ -4012,6 +4012,8 @@ EXPORT_SYMBOL_GPL(usb_unlocked_disable_lpm);
 void usb_enable_lpm(struct usb_device *udev)
 {
 	struct usb_hcd *hcd;
+	struct usb_hub *hub;
+	struct usb_port *port_dev;
 
 	if (!udev || !udev->parent ||
 			udev->speed != USB_SPEED_SUPER ||
@@ -4031,8 +4033,17 @@ void usb_enable_lpm(struct usb_device *udev)
 	if (udev->lpm_disable_count > 0)
 		return;
 
-	usb_enable_link_state(hcd, udev, USB3_LPM_U1);
-	usb_enable_link_state(hcd, udev, USB3_LPM_U2);
+	hub = usb_hub_to_struct_hub(udev->parent);
+	if 

Re: Overly conservative xHCI bandwidth estimation

[Steinar H. Gunderson]

On Fri, Nov 06, 2015 at 08:52:52AM +0800, Lu, Baolu wrote:
>> I reproduced the U1/U2 disconnect issues several times. I didn't try the
>> issue of not enough bandwidth for two devices.
> Can you please try the not enough bandwidth issue?

It doesn't work. I can use one card (it takes 3-4 tries to get it started due
to LPM bugs, but it works), but not two.

Note that I can use one card even if the other is plugged in. I just can't
use both at the same time.

> Can you cat /sys/bus/usb/devices/2-2/power/usb3_hardware_lpm_u1
> and /sys/bus/usb/devices/2-2/power/usb3_hardware_lpm_u2 for several
> times? Don't use that device when cat these two files.

I waited a bit longer, and indeed, now it consistently shows enabled on both
cards.

  klump:~> cat /sys/bus/usb/devices/2-1/power/usb3_hardware_lpm_u1
  enabled
  klump:~> cat /sys/bus/usb/devices/2-1/power/usb3_hardware_lpm_u2
  enabled
  klump:~> cat /sys/bus/usb/devices/2-2/power/usb3_hardware_lpm_u1
  enabled
  klump:~> cat /sys/bus/usb/devices/2-2/power/usb3_hardware_lpm_u2
  enabled

For fun, I tried also with a USB 2.0 cable. U1 and U2 comes up as disabled,
but of course, the device doesn't work.

/* Steinar */
-- 
Homepage: http://www.sesse.net/
--
To unsubscribe from this list: send the line "unsubscribe linux-usb" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Overly conservative xHCI bandwidth estimation

[Steinar H. Gunderson]

On Thu, Nov 05, 2015 at 04:12:24PM +0800, Lu, Baolu wrote:
> 1) apply the attached patch on top the latest kernel.
> 2) build and install the kernel.
> 3) boot your machine with the new kernel.

Do you want this on top of Alan's LPM-disabling patch or on a clean
4.3.0 tree?

> 4) insert one  Blackmagic Design device into USB3 root port.
> 7) wait for a few seconds
> 6) check below sysfs nodes:

I guess there's no step 5. :-)

/* Steinar */
-- 
Homepage: http://www.sesse.net/
--
To unsubscribe from this list: send the line "unsubscribe linux-usb" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Overly conservative xHCI bandwidth estimation

[Lu, Baolu]

Hi Steinar,

Do you mind trying attached patch? This is not a fix but to verify
whether LPM really works when only one device is used. You can
follow below steps:

1) apply the attached patch on top the latest kernel.
2) build and install the kernel.
3) boot your machine with the new kernel.
4) insert one  Blackmagic Design device into USB3 root port.
7) wait for a few seconds
6) check below sysfs nodes:

$ cat /sys/bus/usb/devices//power/usb3_hardware_lpm_u1
$ cat /sys/bus/usb/devices//power/usb3_hardware_lpm_u2

Can you please post values of above two sysfs nodes?
A copy of dmesg will also be helpful.

Thanks,
Baolu

On 11/05/2015 02:23 AM, Steinar H. Gunderson wrote:

On Mon, Nov 02, 2015 at 04:00:42PM -0500, Alan Stern wrote:

That commit was included in (approximately) the 4.1.5 or later stable
kernel, and it is included in 4.2.  You should be able to put one of
those on a bootable USB stick.

I tried going down this route. After some back and forth, I realize that...
the machine has only two USB ports, so I can't boot off USB stick and still
test how it behaves with two cards. (I could buy a hub, but that would surely
introduce new potential errors in the mix.)

So I'm afraid I can't help you at this point. My recommendation to include
the patch still stands, though.

/* Steinar */


>From c8cd71a0703cf26cfca9d89a3ad451f79c7232af Mon Sep 17 00:00:00 2001
From: Lu Baolu 
Date: Thu, 5 Nov 2015 14:08:48 +0800
Subject: [PATCH 1/2] usb: core: LPM: fix usb3_hardware_lpm sysfs node

Commit 655fe4effe0f ("usbcore: add sysfs support to xHCI usb3
hardware LPM") introduced usb3_hardware_lpm sysfs node. This
doesn't show the correct status of USB3 U1 and U2 LPM status.

This patch fixes this by replacing usb3_hardware_lpm with two
nodes, usb3_hardware_lpm_u1 (for U1) and usb3_hardware_lpm_u2
(for U2), and recording the U1/U2 LPM status in right places.

Signed-off-by: Lu Baolu 
---
 Documentation/ABI/testing/sysfs-bus-usb | 16 +---
 Documentation/usb/power-management.txt  | 11 ++-
 drivers/usb/core/hub.c  | 29 +
 drivers/usb/core/sysfs.c| 28 
 include/linux/usb.h |  2 ++
 5 files changed, 62 insertions(+), 24 deletions(-)

diff --git a/Documentation/ABI/testing/sysfs-bus-usb b/Documentation/ABI/testing/sysfs-bus-usb
index 3a4abfc..136ba17 100644
--- a/Documentation/ABI/testing/sysfs-bus-usb
+++ b/Documentation/ABI/testing/sysfs-bus-usb
@@ -134,19 +134,21 @@ Description:
 		enabled for the device. Developer can write y/Y/1 or n/N/0 to
 		the file to enable/disable the feature.
 
-What:		/sys/bus/usb/devices/.../power/usb3_hardware_lpm
-Date:		June 2015
+What:		/sys/bus/usb/devices/.../power/usb3_hardware_lpm_u1
+		/sys/bus/usb/devices/.../power/usb3_hardware_lpm_u2
+Date:		November 2015
 Contact:	Kevin Strasser 
+		Lu Baolu 
 Description:
 		If CONFIG_PM is set and a USB 3.0 lpm-capable device is plugged
 		in to a xHCI host which supports link PM, it will check if U1
 		and U2 exit latencies have been set in the BOS descriptor; if
-		the check is is passed and the host supports USB3 hardware LPM,
+		the check is passed and the host supports USB3 hardware LPM,
 		USB3 hardware LPM will be enabled for the device and the USB
-		device directory will contain a file named
-		power/usb3_hardware_lpm. The file holds a string value (enable
-		or disable) indicating whether or not USB3 hardware LPM is
-		enabled for the device.
+		device directory will contain two files named
+		power/usb3_hardware_lpm_u1 and power/usb3_hardware_lpm_u2. These
+		files hold a string value (enable or disable) indicating whether
+		or not USB3 hardware LPM U1 or U2 is enabled for the device.
 
 What:		/sys/bus/usb/devices/.../removable
 Date:		February 2012
diff --git a/Documentation/usb/power-management.txt b/Documentation/usb/power-management.txt
index 4a15c90..0a94ffe 100644
--- a/Documentation/usb/power-management.txt
+++ b/Documentation/usb/power-management.txt
@@ -537,17 +537,18 @@ relevant attribute files are usb2_hardware_lpm and usb3_hardware_lpm.
 		can write y/Y/1 or n/N/0 to the file to	enable/disable
 		USB2 hardware LPM manually. This is for	test purpose mainly.
 
-	power/usb3_hardware_lpm
+	power/usb3_hardware_lpm_u1
+	power/usb3_hardware_lpm_u2
 
 		When a USB 3.0 lpm-capable device is plugged in to a
 		xHCI host which supports link PM, it will check if U1
 		and U2 exit latencies have been set in the BOS
 		descriptor; if the check is is passed and the host
 		supports USB3 hardware LPM, USB3 hardware LPM will be
-		enabled for the device and this file will be created.
-		The file holds a string value (enable or disable)
-		indicating whether or not USB3 hardware LPM is
-		enabled for the device.
+		enabled for the device and these files will be created.
+		The files hold a string value 

Re: Overly conservative xHCI bandwidth estimation

[Steinar H. Gunderson]

On Fri, Nov 06, 2015 at 08:24:15AM +0800, Lu, Baolu wrote:
> Yeah, sorry about it.
> 
> 1) apply the attached patch on top the latest clean kernel.
> 2) build and install the kernel.
> 3) boot your machine with the new kernel.
> 4) insert one  Blackmagic Design device into USB3 root port.
> 5) wait for a few seconds
> 6) check below sysfs nodes:
> 
> $ cat /sys/bus/usb/devices//power/usb3_hardware_lpm_u1
> $ cat /sys/bus/usb/devices//power/usb3_hardware_lpm_u2

OK, what I sent you this earlier today should be exactly this; I think I
might not have sent the results to the list, though.

/* Steinar */
-- 
Homepage: http://www.sesse.net/
--
To unsubscribe from this list: send the line "unsubscribe linux-usb" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Overly conservative xHCI bandwidth estimation

[Lu, Baolu]

On 11/06/2015 02:45 AM, Steinar H. Gunderson wrote:

On Thu, Nov 05, 2015 at 04:12:24PM +0800, Lu, Baolu wrote:

1) apply the attached patch on top the latest kernel.

I applied on top of a clean 4.3.0.


$ cat /sys/bus/usb/devices//power/usb3_hardware_lpm_u1
$ cat /sys/bus/usb/devices//power/usb3_hardware_lpm_u2

klump:~> cat /sys/bus/usb/devices/2-2/product
Intensity Shuttle
klump:~> cat /sys/bus/usb/devices/2-2/power/usb3_hardware_lpm_u1
disabled
klump:~> cat /sys/bus/usb/devices/2-2/power/usb3_hardware_lpm_u2
disabled


Thank you.

Have you set CONFIG_PM? Can you reproduce the problem with this kernel?




Can you please post values of above two sysfs nodes?
A copy of dmesg will also be helpful.

Attached.

/* Steinar */


--
To unsubscribe from this list: send the line "unsubscribe linux-usb" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Overly conservative xHCI bandwidth estimation

[Steinar H. Gunderson]

On Fri, Nov 06, 2015 at 08:39:28AM +0800, Lu, Baolu wrote:
> Have you set CONFIG_PM?

Yes. CONFIG_PM=y.

> Can you reproduce the problem with this kernel?

I reproduced the U1/U2 disconnect issues several times. I didn't try the
issue of not enough bandwidth for two devices.

/* Steinar */
-- 
Homepage: http://www.sesse.net/
--
To unsubscribe from this list: send the line "unsubscribe linux-usb" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Overly conservative xHCI bandwidth estimation

[Lu, Baolu]

On 11/06/2015 08:45 AM, Steinar H. Gunderson wrote:

On Fri, Nov 06, 2015 at 08:39:28AM +0800, Lu, Baolu wrote:

Have you set CONFIG_PM?

Yes. CONFIG_PM=y.


Can you reproduce the problem with this kernel?

I reproduced the U1/U2 disconnect issues several times. I didn't try the
issue of not enough bandwidth for two devices.


Can you please try the not enough bandwidth issue?

Can you cat /sys/bus/usb/devices/2-2/power/usb3_hardware_lpm_u1
and /sys/bus/usb/devices/2-2/power/usb3_hardware_lpm_u2 for several
times? Don't use that device when cat these two files.

I am hoping that LPM works when only one device is used (hence
there is enough bus bandwidth).



/* Steinar */


Thanks,
Baolu
--
To unsubscribe from this list: send the line "unsubscribe linux-usb" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Overly conservative xHCI bandwidth estimation

[Steinar H. Gunderson]

On Mon, Nov 02, 2015 at 04:00:42PM -0500, Alan Stern wrote:
> That commit was included in (approximately) the 4.1.5 or later stable 
> kernel, and it is included in 4.2.  You should be able to put one of 
> those on a bootable USB stick.

I tried going down this route. After some back and forth, I realize that...
the machine has only two USB ports, so I can't boot off USB stick and still
test how it behaves with two cards. (I could buy a hub, but that would surely
introduce new potential errors in the mix.)

So I'm afraid I can't help you at this point. My recommendation to include
the patch still stands, though.

/* Steinar */
-- 
Homepage: http://www.sesse.net/
--
To unsubscribe from this list: send the line "unsubscribe linux-usb" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Overly conservative xHCI bandwidth estimation

[Lu, Baolu]

Thanks for the information. I will let you know if I have anything.

Thanks,
Baolu

On 11/01/2015 04:20 AM, Steinar H. Gunderson wrote:

On Wed, Oct 21, 2015 at 09:49:16AM +0800, Lu, Baolu wrote:

I could spend some time on this issue a week later.
I'd like to check whether there is any bugs in the driver itself.
Otherwise, blacklist this specific device for LPM.

I don't know if anything happened here, but if you need information for a
blacklist, the two devices I've tested (which both have this issue) are:

  - 1edb:bd3b (Blackmagic Design Intensity Shuttle)
  - 1edb:bd4f (Blackmagic Design UltraStudio SDI)

/* Steinar */


--
To unsubscribe from this list: send the line "unsubscribe linux-usb" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Overly conservative xHCI bandwidth estimation

[Steinar H. Gunderson]

On Mon, Nov 02, 2015 at 11:04:42AM -0500, Alan Stern wrote:
> To make your life easier, here's a patch which blacklists these two 
> devices for Link Power Management.  Please let me know if it works 
> okay.

OK, this worked brilliantly. I can now use both cards at the same time,
and I also see no LPM timeouts anymore. (Tested on top of 4.3.0, with
CONFIG_PM=y.)

/* Steinar */
-- 
Homepage: http://www.sesse.net/
--
To unsubscribe from this list: send the line "unsubscribe linux-usb" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Overly conservative xHCI bandwidth estimation

[Steinar H. Gunderson]

On Mon, Nov 02, 2015 at 01:28:09PM -0500, Alan Stern wrote:
>> Thanks for testing.  I will submit the patch in two weeks, when the
>> current merge window closes.
> Or maybe not.  This may not be the best way to solve the problem, since
> you found that the two video cards worked okay with one xHCI host
> controller but not with another.  Maybe what we really need to do is 
> blacklist some controllers for LPM, rather than the devices.

Note that the two controllers were running different kernels. The one that
worked had a kernel before LPM was turned on, I believe.

/* Steinar */
-- 
Homepage: http://www.sesse.net/
--
To unsubscribe from this list: send the line "unsubscribe linux-usb" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Overly conservative xHCI bandwidth estimation

[Alan Stern]

On Mon, 2 Nov 2015, Steinar H. Gunderson wrote:

> On Mon, Nov 02, 2015 at 11:04:42AM -0500, Alan Stern wrote:
> > To make your life easier, here's a patch which blacklists these two 
> > devices for Link Power Management.  Please let me know if it works 
> > okay.
> 
> OK, this worked brilliantly. I can now use both cards at the same time,
> and I also see no LPM timeouts anymore. (Tested on top of 4.3.0, with
> CONFIG_PM=y.)

Thanks for testing.  I will submit the patch in two weeks, when the
current merge window closes.

Alan Stern

--
To unsubscribe from this list: send the line "unsubscribe linux-usb" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Overly conservative xHCI bandwidth estimation

[Alan Stern]

On Mon, 2 Nov 2015, Alan Stern wrote:

> On Mon, 2 Nov 2015, Steinar H. Gunderson wrote:
> 
> > On Mon, Nov 02, 2015 at 11:04:42AM -0500, Alan Stern wrote:
> > > To make your life easier, here's a patch which blacklists these two 
> > > devices for Link Power Management.  Please let me know if it works 
> > > okay.
> > 
> > OK, this worked brilliantly. I can now use both cards at the same time,
> > and I also see no LPM timeouts anymore. (Tested on top of 4.3.0, with
> > CONFIG_PM=y.)
> 
> Thanks for testing.  I will submit the patch in two weeks, when the
> current merge window closes.

Or maybe not.  This may not be the best way to solve the problem, since
you found that the two video cards worked okay with one xHCI host
controller but not with another.  Maybe what we really need to do is 
blacklist some controllers for LPM, rather than the devices.

Alan Stern

--
To unsubscribe from this list: send the line "unsubscribe linux-usb" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Overly conservative xHCI bandwidth estimation

[Steinar H. Gunderson]

On Mon, Nov 02, 2015 at 11:04:42AM -0500, Alan Stern wrote:
> To make your life easier, here's a patch which blacklists these two 
> devices for Link Power Management.  Please let me know if it works 
> okay.

Thanks! I'll give it a shot.

/* Steinar */
-- 
Homepage: http://www.sesse.net/
--
To unsubscribe from this list: send the line "unsubscribe linux-usb" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Overly conservative xHCI bandwidth estimation

[Alan Stern]

On Mon, 2 Nov 2015, Steinar H. Gunderson wrote:

> On Mon, Nov 02, 2015 at 02:38:41PM -0500, Alan Stern wrote:
> >> Note that the two controllers were running different kernels. The one that
> >> worked had a kernel before LPM was turned on, I believe.
> > Oh.  Can you try running a more recent kernel on that computer?  Does 
> > it fail with the same "Not enough bandwidth for altsetting 4" error?
> 
> Unfortunately I cannot easily; it's under someone else's administration.
> Perhaps I could find a live USB stick, but I'm not immediately sure if
> there's anything out there with e.g. 4.3 yet.

You don't need 4.3.  Anything released within the last few years 
will contain the LPM code.

> But I did run an older kernel on the computer with the problem (as part of
> the bisection that showed LPM to be breaking it), and it worked then.

Well yes, we know that that controller works okay when LPM is turned 
off.  The question is whether the other controller needs the same 
workaround.

Alan Stern

--
To unsubscribe from this list: send the line "unsubscribe linux-usb" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Overly conservative xHCI bandwidth estimation

[Steinar H. Gunderson]

On Mon, Nov 02, 2015 at 03:32:46PM -0500, Alan Stern wrote:
> You don't need 4.3.  Anything released within the last few years 
> will contain the LPM code.

According to the bisect, the patch that broke it was from June this year
(2d2a316765d956bc5cb6bb367b2ec52ca59ab8e9), so it would really seem I need
something quite new.

The kernel on the other machine is Ubuntu's 3.19.0-28-generic, as far as I
can see.

/* Steinar */
-- 
Homepage: http://www.sesse.net/
--
To unsubscribe from this list: send the line "unsubscribe linux-usb" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Overly conservative xHCI bandwidth estimation

[Alan Stern]

On Mon, 2 Nov 2015, Steinar H. Gunderson wrote:

> On Mon, Nov 02, 2015 at 01:28:09PM -0500, Alan Stern wrote:
> >> Thanks for testing.  I will submit the patch in two weeks, when the
> >> current merge window closes.
> > Or maybe not.  This may not be the best way to solve the problem, since
> > you found that the two video cards worked okay with one xHCI host
> > controller but not with another.  Maybe what we really need to do is 
> > blacklist some controllers for LPM, rather than the devices.
> 
> Note that the two controllers were running different kernels. The one that
> worked had a kernel before LPM was turned on, I believe.

Oh.  Can you try running a more recent kernel on that computer?  Does 
it fail with the same "Not enough bandwidth for altsetting 4" error?

Alan Stern


--
To unsubscribe from this list: send the line "unsubscribe linux-usb" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Overly conservative xHCI bandwidth estimation

[Steinar H. Gunderson]

On Mon, Nov 02, 2015 at 02:38:41PM -0500, Alan Stern wrote:
>> Note that the two controllers were running different kernels. The one that
>> worked had a kernel before LPM was turned on, I believe.
> Oh.  Can you try running a more recent kernel on that computer?  Does 
> it fail with the same "Not enough bandwidth for altsetting 4" error?

Unfortunately I cannot easily; it's under someone else's administration.
Perhaps I could find a live USB stick, but I'm not immediately sure if
there's anything out there with e.g. 4.3 yet.

But I did run an older kernel on the computer with the problem (as part of
the bisection that showed LPM to be breaking it), and it worked then.

/* Steinar */
-- 
Homepage: http://www.sesse.net/
--
To unsubscribe from this list: send the line "unsubscribe linux-usb" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Overly conservative xHCI bandwidth estimation

[Alan Stern]

On Sat, 31 Oct 2015, Steinar H. Gunderson wrote:

> On Wed, Oct 21, 2015 at 09:49:16AM +0800, Lu, Baolu wrote:
> > I could spend some time on this issue a week later.
> > I'd like to check whether there is any bugs in the driver itself.
> > Otherwise, blacklist this specific device for LPM.
> 
> I don't know if anything happened here, but if you need information for a
> blacklist, the two devices I've tested (which both have this issue) are:
> 
>  - 1edb:bd3b (Blackmagic Design Intensity Shuttle)
>  - 1edb:bd4f (Blackmagic Design UltraStudio SDI)

To make your life easier, here's a patch which blacklists these two 
devices for Link Power Management.  Please let me know if it works 
okay.

Alan Stern



Index: usb-4.3/include/linux/usb/quirks.h
===
--- usb-4.3.orig/include/linux/usb/quirks.h
+++ usb-4.3/include/linux/usb/quirks.h
@@ -47,4 +47,7 @@
 /* device generates spurious wakeup, ignore remote wakeup capability */
 #define USB_QUIRK_IGNORE_REMOTE_WAKEUP BIT(9)
 
+/* device can't handle Link Power Management */
+#define USB_QUIRK_NO_LPM   BIT(10)
+
 #endif /* __LINUX_USB_QUIRKS_H */
Index: usb-4.3/drivers/usb/core/hub.c
===
--- usb-4.3.orig/drivers/usb/core/hub.c
+++ usb-4.3/drivers/usb/core/hub.c
@@ -124,6 +124,10 @@ struct usb_hub *usb_hub_to_struct_hub(st
 
 int usb_device_supports_lpm(struct usb_device *udev)
 {
+   /* Some devices have trouble with LPM */
+   if (udev->quirks & USB_QUIRK_NO_LPM)
+   return 0;
+
/* USB 2.1 (and greater) devices indicate LPM support through
 * their USB 2.0 Extended Capabilities BOS descriptor.
 */
@@ -4503,6 +4507,8 @@ hub_port_init (struct usb_hub *hub, stru
goto fail;
}
 
+   usb_detect_quirks(udev);
+
if (udev->wusb == 0 && le16_to_cpu(udev->descriptor.bcdUSB) >= 0x0201) {
retval = usb_get_bos_descriptor(udev);
if (!retval) {
@@ -4701,7 +4707,6 @@ static void hub_port_connect(struct usb_
if (status < 0)
goto loop;
 
-   usb_detect_quirks(udev);
if (udev->quirks & USB_QUIRK_DELAY_INIT)
msleep(1000);
 
Index: usb-4.3/drivers/usb/core/quirks.c
===
--- usb-4.3.orig/drivers/usb/core/quirks.c
+++ usb-4.3/drivers/usb/core/quirks.c
@@ -186,6 +186,12 @@ static const struct usb_device_id usb_qu
{ USB_DEVICE(0x1a0a, 0x0200), .driver_info =
USB_QUIRK_LINEAR_UFRAME_INTR_BINTERVAL },
 
+   /* Blackmagic Design Intensity Shuttle */
+   { USB_DEVICE(0x1edb, 0xbd3b), .driver_info = USB_QUIRK_NO_LPM },
+
+   /* Blackmagic Design UltraStudio SDI */
+   { USB_DEVICE(0x1edb, 0xbd4f), .driver_info = USB_QUIRK_NO_LPM },
+
{ }  /* terminating entry must be last */
 };
 

--
To unsubscribe from this list: send the line "unsubscribe linux-usb" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Overly conservative xHCI bandwidth estimation

[Alan Stern]

On Mon, 2 Nov 2015, Steinar H. Gunderson wrote:

> On Mon, Nov 02, 2015 at 03:32:46PM -0500, Alan Stern wrote:
> > You don't need 4.3.  Anything released within the last few years 
> > will contain the LPM code.
> 
> According to the bisect, the patch that broke it was from June this year
> (2d2a316765d956bc5cb6bb367b2ec52ca59ab8e9), so it would really seem I need
> something quite new.

That commit was included in (approximately) the 4.1.5 or later stable 
kernel, and it is included in 4.2.  You should be able to put one of 
those on a bootable USB stick.

Even with older kernels, you should be able to back-port that
troublesome commit easily enough.

> The kernel on the other machine is Ubuntu's 3.19.0-28-generic, as far as I
> can see.

Alan Stern

--
To unsubscribe from this list: send the line "unsubscribe linux-usb" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Overly conservative xHCI bandwidth estimation

[Steinar H. Gunderson]

On Wed, Oct 21, 2015 at 09:49:16AM +0800, Lu, Baolu wrote:
> I could spend some time on this issue a week later.
> I'd like to check whether there is any bugs in the driver itself.
> Otherwise, blacklist this specific device for LPM.

I don't know if anything happened here, but if you need information for a
blacklist, the two devices I've tested (which both have this issue) are:

 - 1edb:bd3b (Blackmagic Design Intensity Shuttle)
 - 1edb:bd4f (Blackmagic Design UltraStudio SDI)

/* Steinar */
-- 
Homepage: http://www.sesse.net/
--
To unsubscribe from this list: send the line "unsubscribe linux-usb" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Overly conservative xHCI bandwidth estimation

[Alan Stern]

On Wed, 21 Oct 2015, Steinar H. Gunderson wrote:

> On Wed, Oct 21, 2015 at 10:29:11AM -0400, Alan Stern wrote:
> >> Is there such a blacklist already, where I can add the devices (there are 
> >> two
> >> distinct IDs behaving identically) and see if it helps?
> > There is a blacklist, but it does not affect LPM.  See 
> > drivers/usb/core/quirks.c and include/linux/usb/quirks.h.
> 
> OK, so to blacklist the device for LPM, some new code would need to get
> written.

Yes.  The difficulty is that usb_device_supports_lpm() and
usb_set_lpm_parameters() in hub.c are called before
usb_detect_quirks().

It would be easier to move the usb_detect_quirks() call into
hub_port_init(), before the usb_get_bos_descriptor() and LPM stuff.  
Then you could simply skip the LPM setup if your quirk was set.

Alan Stern

--
To unsubscribe from this list: send the line "unsubscribe linux-usb" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Overly conservative xHCI bandwidth estimation

[Alan Stern]

On Wed, 21 Oct 2015, Steinar H. Gunderson wrote:

> On Wed, Oct 21, 2015 at 09:49:16AM +0800, Lu, Baolu wrote:
> > I could spend some time on this issue a week later.
> > I'd like to check whether there is any bugs in the driver itself.
> > Otherwise, blacklist this specific device for LPM.
> 
> Is there such a blacklist already, where I can add the devices (there are two
> distinct IDs behaving identically) and see if it helps?

There is a blacklist, but it does not affect LPM.  See 
drivers/usb/core/quirks.c and include/linux/usb/quirks.h.

Alan Stern

--
To unsubscribe from this list: send the line "unsubscribe linux-usb" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Overly conservative xHCI bandwidth estimation

[Steinar H. Gunderson]

On Wed, Oct 21, 2015 at 10:29:11AM -0400, Alan Stern wrote:
>> Is there such a blacklist already, where I can add the devices (there are two
>> distinct IDs behaving identically) and see if it helps?
> There is a blacklist, but it does not affect LPM.  See 
> drivers/usb/core/quirks.c and include/linux/usb/quirks.h.

OK, so to blacklist the device for LPM, some new code would need to get
written.

/* Steinar */
-- 
Homepage: http://www.sesse.net/
--
To unsubscribe from this list: send the line "unsubscribe linux-usb" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Overly conservative xHCI bandwidth estimation

[Steinar H. Gunderson]

On Wed, Oct 21, 2015 at 09:49:16AM +0800, Lu, Baolu wrote:
> I could spend some time on this issue a week later.
> I'd like to check whether there is any bugs in the driver itself.
> Otherwise, blacklist this specific device for LPM.

Is there such a blacklist already, where I can add the devices (there are two
distinct IDs behaving identically) and see if it helps?

/* Steinar */
-- 
Homepage: http://www.sesse.net/
--
To unsubscribe from this list: send the line "unsubscribe linux-usb" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Overly conservative xHCI bandwidth estimation

[Steinar H. Gunderson]

On Mon, Sep 28, 2015 at 02:32:13PM +0200, Steinar H. Gunderson wrote:
> Just so that it doesn't get lost: I've reported issues with this specific
> device and LPM not too long ago. It's entirely possible that the device
> somehow is broken, although it works in Windows 7/8/10 and OS X, from what I
> know, and at least Windows 10 uses USB3 LPM.
> 
> Perhaps figuring out what's wrong with the ping timeouts would also fix this
> issue? I'm afraid I don't have any fancy USB analyzer equipment to debug
> with, though.
> 
> In any case, I'll reiterate my request to turn off LPM for a specific device,
> no matter whose fault these specific issues are. :-)

Is there anything else I can do to help fixing these issues? (U1/U2 timeouts
causing disconnects, and xHCI bandwidth estimation being off.) I've been
running with CONFIG_PM=n for a while now, but it is obviously not a good
place to be for a laptop.

/* Steinar */
-- 
Homepage: http://www.sesse.net/
--
To unsubscribe from this list: send the line "unsubscribe linux-usb" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Overly conservative xHCI bandwidth estimation

[Lu, Baolu]

I could spend some time on this issue a week later.
I'd like to check whether there is any bugs in the driver itself.
Otherwise, blacklist this specific device for LPM.

Thanks,
Baolu

On 10/21/2015 07:05 AM, Steinar H. Gunderson wrote:

On Mon, Sep 28, 2015 at 02:32:13PM +0200, Steinar H. Gunderson wrote:

Just so that it doesn't get lost: I've reported issues with this specific
device and LPM not too long ago. It's entirely possible that the device
somehow is broken, although it works in Windows 7/8/10 and OS X, from what I
know, and at least Windows 10 uses USB3 LPM.

Perhaps figuring out what's wrong with the ping timeouts would also fix this
issue? I'm afraid I don't have any fancy USB analyzer equipment to debug
with, though.

In any case, I'll reiterate my request to turn off LPM for a specific device,
no matter whose fault these specific issues are. :-)

Is there anything else I can do to help fixing these issues? (U1/U2 timeouts
causing disconnects, and xHCI bandwidth estimation being off.) I've been
running with CONFIG_PM=n for a while now, but it is obviously not a good
place to be for a laptop.

/* Steinar */


--
To unsubscribe from this list: send the line "unsubscribe linux-usb" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Overly conservative xHCI bandwidth estimation

[Mathias Nyman]

On 26.09.2015 23:13, Steinar H. Gunderson wrote:

On Fri, Sep 25, 2015 at 10:12:51PM +0200, Steinar H. Gunderson wrote:

I have no idea whatsoever how this breaks bandwidth management, but seemingly
it does.


To verify; I built 4.3rc-2 with CONFIG_PM=n, and it can drive both cards
without problems whatwoever. So there's something about power management
(LPM?) in newer kernels that causes this issue.



Hi

Thanks again for taking the time to narrow down the issue.

Link power management (LPM) enabled for usb3 devices could be relevant because 
with LPM enabled the driver will
calculate a max exit latency time, the time it will will take to "wake up" a 
link from U1 or U2 to a fully running U0

Driver will tell the latency value to the host controller, when a exit latency 
time is set the host will know that
the link is power managed, and host will start to schedule additional PING TP 
transfers to the device to wake the link up to U0.
the PING TP tranfers are sent Max-exit-latency microeseconds before the actual 
transfer to make sure link is woken up in time.
It's possible that this impacts on the bandwidth.

It's also very possible that the max exit latency is calculated wrong, or that 
one of the other parameters set with the same
"evaluate context" command is wrong

-Mathias
--
To unsubscribe from this list: send the line "unsubscribe linux-usb" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Overly conservative xHCI bandwidth estimation

[Steinar H. Gunderson]

On Mon, Sep 28, 2015 at 03:24:04PM +0300, Mathias Nyman wrote:
> Driver will tell the latency value to the host controller, when a exit
> latency time is set the host will know that the link is power managed, and
> host will start to schedule additional PING TP transfers to the device to
> wake the link up to U0.  the PING TP tranfers are sent Max-exit-latency
> microeseconds before the actual transfer to make sure link is woken up in
> time.  It's possible that this impacts on the bandwidth.
> 
> It's also very possible that the max exit latency is calculated wrong, or
> that one of the other parameters set with the same "evaluate context"
> command is wrong

Just so that it doesn't get lost: I've reported issues with this specific
device and LPM not too long ago. It's entirely possible that the device
somehow is broken, although it works in Windows 7/8/10 and OS X, from what I
know, and at least Windows 10 uses USB3 LPM.

Perhaps figuring out what's wrong with the ping timeouts would also fix this
issue? I'm afraid I don't have any fancy USB analyzer equipment to debug
with, though.

In any case, I'll reiterate my request to turn off LPM for a specific device,
no matter whose fault these specific issues are. :-)

/* Steinar */
-- 
Homepage: http://www.sesse.net/
--
To unsubscribe from this list: send the line "unsubscribe linux-usb" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Overly conservative xHCI bandwidth estimation

[Steinar H. Gunderson]

On Fri, Sep 25, 2015 at 10:12:51PM +0200, Steinar H. Gunderson wrote:
> I have no idea whatsoever how this breaks bandwidth management, but seemingly
> it does.

To verify; I built 4.3rc-2 with CONFIG_PM=n, and it can drive both cards
without problems whatwoever. So there's something about power management
(LPM?) in newer kernels that causes this issue.

/* Steinar */
-- 
Homepage: http://www.sesse.net/
--
To unsubscribe from this list: send the line "unsubscribe linux-usb" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Overly conservative xHCI bandwidth estimation

[Steinar H. Gunderson]

On Fri, Sep 25, 2015 at 10:26:23AM -0400, Alan Stern wrote:
>> Does this mean I have a long bisect ahead of me?
> In the absence of any better suggestions, that's the most 
> straightforward way to get an answer.

Three hours of bisecting (actually three bisects; I found a git bug along the
way :-/) later shows the culprit:

  commit 2d2a316765d956bc5cb6bb367b2ec52ca59ab8e9
  Author: Lu Baolu 
  Date:   Tue Jun 16 09:08:26 2015 +0800
  
  usb: core: lpm: set lpm_capable for root hub device

I have no idea whatsoever how this breaks bandwidth management, but seemingly
it does.

/* Steinar */
-- 
Homepage: http://www.sesse.net/
--
To unsubscribe from this list: send the line "unsubscribe linux-usb" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Overly conservative xHCI bandwidth estimation

[Alan Stern]

On Fri, 25 Sep 2015, Steinar H. Gunderson wrote:

> On Fri, Sep 25, 2015 at 12:42:22AM +0200, Steinar H. Gunderson wrote:
> > I downgraded to Debian's 3.16 kernel. Both cards came up without a hitch.
> > But I only seem to get frames back from the second one.
> 
> ...and after a quick app fix, I have capture from both cards.
> 
> Does this mean I have a long bisect ahead of me?

In the absence of any better suggestions, that's the most 
straightforward way to get an answer.

Alan Stern

--
To unsubscribe from this list: send the line "unsubscribe linux-usb" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Overly conservative xHCI bandwidth estimation

[Krzysztof Opasiak]

On 09/23/2015 10:08 PM, Steinar H. Gunderson wrote:

Hi again,

I'm trying to figure out why my xHCI controller refuses me to run two very
similar video cards at the same time. I'm not sure if this is a bug or if I'm
just misunderstanding, so let me see if I understand this right:

The interface of each card has two relevant alternates, 2 and 4. Number 2 has
bInterval=1, bMaxBurst=11, Mult=2, which from what I can see should be
11*(2+1)*1024 byte / 125µs, or 2.16 Gbit/sec. Number 4 has bInterval=1,
bMaxBurst=8, Mult=1, which should be 8*(1+1)*1024 byte / 125 µs, or 1.04 
Gbit/sec.

I can run one card in alternates 2 or 4, no problem. But if I put another
card in the other USB port (my Lenovo X240 has one on each side) and try to
activate that too, I get

   [  287.919173] usb 2-2: Not enough bandwidth for altsetting 4

I've tried on another machine and it works fine there, so my code should,
at least on the surface of it, be fine. I've verified that the error is
COMP_BW_ERR (not COMP_2ND_BW_ERR).

My xHCI controller is 8086:9c31 (I believe this is Intel). lsusb -v with both
devices plugged in is attached. I'm running 4.3-rc2. There are no external
USB hubs in the mix.

So this is my question: Why can I run one card at 2.16 Gbit/sec but not two
at 1.04 Gbit/sec each? (Shouldn't the USB3 bus be 5 Gbit/sec?) Is there any
way to debug these issues, or is the controller just deciding and unwilling
to give out any information? Is the information of the bandwidth domains/
bus instances available anywhere? Am I misunderstanding something
fundamentally?


Well it's not so simple. I'm not sure if this is exactly your problem 
but it may help you.


Let's start from the beginning. Your device use ISO endpoints which 
means that host allocates specific amount of bandwidth on the bus. More 
over, interfaces in your devices has many alternate settings. Probably 
each of them reserves different amount of bandwidth. When you connect 
your first device, driver selects the highest possible quality and 
allocate maximum bandwidth. When you connect second device driver tries 
to select altsetting for which there is enough bandwidth left but as you 
see this fail. So we get:


1st card - 2.16 bandwidth
2nd card - not enough bandwidth left even for lowest quality

what you should try is to force driver of the first card to use lower 
quality and allocate less bandwidth (this means choose other altsetting 
of interface) then you will get:


1st card - x bandwidth (x < 2.16)
2nd card - y bandwidth (I think that USB 3.0 should provide enough 
bandwidth for those 2 devices at lowest quality)


That's just my guess, no guarantee bud it may be worth to try.

--
Krzysztof Opasiak
Samsung R Institute Poland
Samsung Electronics
--
To unsubscribe from this list: send the line "unsubscribe linux-usb" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Overly conservative xHCI bandwidth estimation

[Krzysztof Opasiak]

On 09/24/2015 03:14 PM, Steinar H. Gunderson wrote:

On Thu, Sep 24, 2015 at 03:09:55PM +0200, Krzysztof Opasiak wrote:

But still the problem may exist. Is the 2.16 GBit bandwidth for 4th altset?


No. The 2nd altset is 2.16, the 4th altset is 1.04. Unless my calculations
are wrong.


How do you do your calculations?




Remember that according to USB spec not whole bandwidth can be used by iso
transfers (in usb 2.0 the limit is 80%, don't know how it looks in 3.0). So
if 2.16 is lowest possible bandwidth for your device you have 2.16 x 2 =
4.32 but 80% * 5 = 4...


But total bandwidth shouldn't be the issue as long as the 2nd altset
(by itself) works fine, no? It's _more_ than twice that of the 4th.



Definitely agree. Didn't get this from your previous messages.
--
Krzysztof Opasiak
Samsung R Institute Poland
Samsung Electronics
--
To unsubscribe from this list: send the line "unsubscribe linux-usb" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Overly conservative xHCI bandwidth estimation

[Steinar H. Gunderson]

On Thu, Sep 24, 2015 at 03:35:10PM +0200, Krzysztof Opasiak wrote:
>> No. The 2nd altset is 2.16, the 4th altset is 1.04. Unless my calculations
>> are wrong.
> How do you do your calculations?

Like I said in my initial email:

>>> The interface of each card has two relevant alternates, 2 and 4. Number 2 
>>> has
>>> bInterval=1, bMaxBurst=11, Mult=2, which from what I can see should be
>>> 11*(2+1)*1024 byte / 125µs, or 2.16 Gbit/sec. Number 4 has bInterval=1,
>>> bMaxBurst=8, Mult=1, which should be 8*(1+1)*1024 byte / 125 µs, or 1.04 
>>> Gbit/sec.

/* Steinar */
-- 
Homepage: http://www.sesse.net/
--
To unsubscribe from this list: send the line "unsubscribe linux-usb" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Overly conservative xHCI bandwidth estimation

[Steinar H. Gunderson]

On Thu, Sep 24, 2015 at 03:09:55PM +0200, Krzysztof Opasiak wrote:
> But still the problem may exist. Is the 2.16 GBit bandwidth for 4th altset?

No. The 2nd altset is 2.16, the 4th altset is 1.04. Unless my calculations
are wrong.

> Remember that according to USB spec not whole bandwidth can be used by iso
> transfers (in usb 2.0 the limit is 80%, don't know how it looks in 3.0). So
> if 2.16 is lowest possible bandwidth for your device you have 2.16 x 2 =
> 4.32 but 80% * 5 = 4...

But total bandwidth shouldn't be the issue as long as the 2nd altset
(by itself) works fine, no? It's _more_ than twice that of the 4th.

/* Steinar */
-- 
Homepage: http://www.sesse.net/
--
To unsubscribe from this list: send the line "unsubscribe linux-usb" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Overly conservative xHCI bandwidth estimation

[Steinar H. Gunderson]

On Thu, Sep 24, 2015 at 02:09:41PM +0200, Krzysztof Opasiak wrote:
> Let's start from the beginning. Your device use ISO endpoints which means
> that host allocates specific amount of bandwidth on the bus. More over,
> interfaces in your devices has many alternate settings. Probably each of
> them reserves different amount of bandwidth. When you connect your first
> device, driver selects the highest possible quality and allocate maximum
> bandwidth. When you connect second device driver tries to select altsetting
> for which there is enough bandwidth left but as you see this fail. So we
> get:

I'm sorry, I should have mentioned an important detail here. There is no
kernel driver for my card; I drive it from userspace with libusb-1.0. So I'm
the one setting the alternates. The cards start out in alternate 0, so the
scenario you're outlining shouldn't exist; and indeed, the setting the first
card to 4 (the low-bandwidth alternate) works just fine, it's setting the
second one that's the problem.

/* Steinar */
-- 
Homepage: http://www.sesse.net/
--
To unsubscribe from this list: send the line "unsubscribe linux-usb" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Overly conservative xHCI bandwidth estimation

[Krzysztof Opasiak]

On 09/24/2015 02:50 PM, Steinar H. Gunderson wrote:

On Thu, Sep 24, 2015 at 02:09:41PM +0200, Krzysztof Opasiak wrote:

Let's start from the beginning. Your device use ISO endpoints which means
that host allocates specific amount of bandwidth on the bus. More over,
interfaces in your devices has many alternate settings. Probably each of
them reserves different amount of bandwidth. When you connect your first
device, driver selects the highest possible quality and allocate maximum
bandwidth. When you connect second device driver tries to select altsetting
for which there is enough bandwidth left but as you see this fail. So we
get:


I'm sorry, I should have mentioned an important detail here. There is no
kernel driver for my card; I drive it from userspace with libusb-1.0. So I'm
the one setting the alternates. The cards start out in alternate 0, so the
scenario you're outlining shouldn't exist; and indeed, the setting the first
card to 4 (the low-bandwidth alternate) works just fine, it's setting the
second one that's the problem.



But still the problem may exist. Is the 2.16 GBit bandwidth for 4th altset?

Remember that according to USB spec not whole bandwidth can be used by 
iso transfers (in usb 2.0 the limit is 80%, don't know how it looks in 
3.0). So if 2.16 is lowest possible bandwidth for your device you have 
2.16 x 2 = 4.32 but 80% * 5 = 4...


--
Krzysztof Opasiak
Samsung R Institute Poland
Samsung Electronics
--
To unsubscribe from this list: send the line "unsubscribe linux-usb" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Overly conservative xHCI bandwidth estimation

[Steinar H. Gunderson]

On Thu, Sep 24, 2015 at 10:17:19AM -0400, Alan Stern wrote:
> However, none of this answers the question of why you can use both
> cards on a different machine but not on yours.  It comes down to the
> implementations of the xHCI controller chips.  In USB-3, bandwidth
> allocation is handled by firmware running on the controller, not by the
> operating system's driver.  The driver presents a series of endpoints
> with all their bandwidth requirements to the controller, and the
> controller either accepts it or rejects it.

OK, I feared as much. The other machine also has an Intel controller,
but as far as I know, a newer one (and the PCI ID is different -- 8086:9cb1).

> It doesn't give any explanation for its decision, and as far as I know, it
> doesn't provide any information about the details of how it allocates the
> bandwidth.

I thought I saw something in the xHCI spec about enumerating the bandwidth
domains to try to get some more insight in what the topology looks like,
but I guess I misunderstood it? (It all wasn't very clear to me.)

I assume there's no way I can lie to the chip? Like, if I know for a fact
that the card will send less data than the alternate claims (like,
I'm using a video mode that will require only a few hundred megabits/second
in practice, but even the lowest alternate claims >1 Gbit/sec).

/* Steinar */
-- 
Homepage: http://www.sesse.net/
--
To unsubscribe from this list: send the line "unsubscribe linux-usb" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Overly conservative xHCI bandwidth estimation

[Alan Stern]

On Wed, 23 Sep 2015, Steinar H. Gunderson wrote:

> Hi again,
> 
> I'm trying to figure out why my xHCI controller refuses me to run two very
> similar video cards at the same time. I'm not sure if this is a bug or if I'm
> just misunderstanding, so let me see if I understand this right:
> 
> The interface of each card has two relevant alternates, 2 and 4. Number 2 has 
> bInterval=1, bMaxBurst=11, Mult=2, which from what I can see should be
> 11*(2+1)*1024 byte / 125µs, or 2.16 Gbit/sec. Number 4 has bInterval=1,
> bMaxBurst=8, Mult=1, which should be 8*(1+1)*1024 byte / 125 µs, or 1.04 
> Gbit/sec.
> 
> I can run one card in alternates 2 or 4, no problem. But if I put another
> card in the other USB port (my Lenovo X240 has one on each side) and try to
> activate that too, I get
> 
>   [  287.919173] usb 2-2: Not enough bandwidth for altsetting 4
> 
> I've tried on another machine and it works fine there, so my code should,
> at least on the surface of it, be fine. I've verified that the error is
> COMP_BW_ERR (not COMP_2ND_BW_ERR).
> 
> My xHCI controller is 8086:9c31 (I believe this is Intel). lsusb -v with both
> devices plugged in is attached. I'm running 4.3-rc2. There are no external
> USB hubs in the mix.
> 
> So this is my question: Why can I run one card at 2.16 Gbit/sec but not two
> at 1.04 Gbit/sec each? (Shouldn't the USB3 bus be 5 Gbit/sec?) Is there any
> way to debug these issues, or is the controller just deciding and unwilling
> to give out any information? Is the information of the bandwidth domains/
> bus instances available anywhere? Am I misunderstanding something
> fundamentally?

Like Krzysztof said, it's more complicated than this.  No more than 80% 
of the total bus capacity can be allocated for periodic transfers.  In 
addition, USB-3.0 doesn't really handle 5 Gb/s.  That's the bit rate of 
the bus itself, but data on the bus gets encoded in way that requires 
10 bits of bus traffic for every 8 bits of data transferred.  Thus the 
available bandwidth is really 4 Gb/s -- and part of that is used up by 
overhead.

However, none of this answers the question of why you can use both
cards on a different machine but not on yours.  It comes down to the
implementations of the xHCI controller chips.  In USB-3, bandwidth
allocation is handled by firmware running on the controller, not by the
operating system's driver.  The driver presents a series of endpoints
with all their bandwidth requirements to the controller, and the
controller either accepts it or rejects it.  It doesn't give any
explanation for its decision, and as far as I know, it doesn't provide 
any information about the details of how it allocates the bandwidth.

Alan Stern

--
To unsubscribe from this list: send the line "unsubscribe linux-usb" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Overly conservative xHCI bandwidth estimation

[Krzysztof Opasiak]

On 09/24/2015 03:50 PM, Steinar H. Gunderson wrote:

On Thu, Sep 24, 2015 at 03:35:10PM +0200, Krzysztof Opasiak wrote:

No. The 2nd altset is 2.16, the 4th altset is 1.04. Unless my calculations
are wrong.

How do you do your calculations?


Like I said in my initial email:


The interface of each card has two relevant alternates, 2 and 4. Number 2 has
bInterval=1, bMaxBurst=11, Mult=2, which from what I can see should be
11*(2+1)*1024 byte / 125µs, or 2.16 Gbit/sec. Number 4 has bInterval=1,
bMaxBurst=8, Mult=1, which should be 8*(1+1)*1024 byte / 125 µs, or 1.04 
Gbit/sec.


As far I can see in tutorials the correct equation is:

(bMaxBurst + 1)*(Mult + 1)*wMaxPacketSize

So for 2nd you should get:

12*3*1024 = 36864 /125µs what gives 2.95 GBit/sec.

and for 4th one:

9*2*1024 = 18432 /125µs what gives 1.47 GBit/sec.

But this should still fit.

Sorry... no other ideas...

--
Krzysztof Opasiak
Samsung R Institute Poland
Samsung Electronics
--
To unsubscribe from this list: send the line "unsubscribe linux-usb" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Overly conservative xHCI bandwidth estimation

[Alan Stern]

On Thu, 24 Sep 2015, Steinar H. Gunderson wrote:

> On Thu, Sep 24, 2015 at 10:17:19AM -0400, Alan Stern wrote:
> > However, none of this answers the question of why you can use both
> > cards on a different machine but not on yours.  It comes down to the
> > implementations of the xHCI controller chips.  In USB-3, bandwidth
> > allocation is handled by firmware running on the controller, not by the
> > operating system's driver.  The driver presents a series of endpoints
> > with all their bandwidth requirements to the controller, and the
> > controller either accepts it or rejects it.
> 
> OK, I feared as much. The other machine also has an Intel controller,
> but as far as I know, a newer one (and the PCI ID is different -- 8086:9cb1).
> 
> > It doesn't give any explanation for its decision, and as far as I know, it
> > doesn't provide any information about the details of how it allocates the
> > bandwidth.
> 
> I thought I saw something in the xHCI spec about enumerating the bandwidth
> domains to try to get some more insight in what the topology looks like,
> but I guess I misunderstood it? (It all wasn't very clear to me.)

Ah, you're referring to the Get Port Bandwidth command, documented in
section 4.6.15 of the xHCI-1.1 spec.  Yes, that does provide some
information on what bandwidth remains available.  Apparently the
xhci-hcd driver doesn't support it.  Adding it would be a good way to
improve the driver's debugging facilities.

> I assume there's no way I can lie to the chip? Like, if I know for a fact
> that the card will send less data than the alternate claims (like,
> I'm using a video mode that will require only a few hundred megabits/second
> in practice, but even the lowest alternate claims >1 Gbit/sec).

You can always patch the kernel to do whatever you want.  For instance, 
you could reduce the bMaxBurst value in the descriptor after the kernel 
gets it from the card.  Short of that, there is no way to affect the 
outcome.

Alan Stern

--
To unsubscribe from this list: send the line "unsubscribe linux-usb" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Overly conservative xHCI bandwidth estimation

[Steinar H. Gunderson]

On Thu, Sep 24, 2015 at 11:22:57AM -0400, Alan Stern wrote:
>> I assume there's no way I can lie to the chip? Like, if I know for a fact
>> that the card will send less data than the alternate claims (like,
>> I'm using a video mode that will require only a few hundred megabits/second
>> in practice, but even the lowest alternate claims >1 Gbit/sec).
> You can always patch the kernel to do whatever you want.  For instance, 
> you could reduce the bMaxBurst value in the descriptor after the kernel 
> gets it from the card.  Short of that, there is no way to affect the 
> outcome.

I tried, but it didn't work; I bMaxBurst to 4 instead of to 15
(in usb_parse_ss_endpoint_companion). It didn't change the outcome.
Perhaps this doesn't actually change what the xHCI controller sees,
though.

/* Steinar */
-- 
Homepage: http://www.sesse.net/
--
To unsubscribe from this list: send the line "unsubscribe linux-usb" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Overly conservative xHCI bandwidth estimation

[Steinar H. Gunderson]

On Thu, Sep 24, 2015 at 02:33:06PM -0700, Paul Zimmerman wrote:
> IIRC, at least some of the Intel controllers require the bandwidth
> calculations to be done by the xHCI driver, instead of doing it
> themselves in hardware. Perhaps you're tripping over a bug in the xHCI
> driver? Mathias is probably best the one to comment on this.

FWIW, I've checked that the XHCI_SW_BW_CHECKING quirk isn't set for my
device. (I suppose that's the one you're thinking about?)

/* Steinar */
-- 
Homepage: http://www.sesse.net/
--
To unsubscribe from this list: send the line "unsubscribe linux-usb" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Overly conservative xHCI bandwidth estimation

[Paul Zimmerman]

IIRC, at least some of the Intel controllers require the bandwidth
calculations to be done by the xHCI driver, instead of doing it
themselves in hardware. Perhaps you're tripping over a bug in the xHCI
driver? Mathias is probably best the one to comment on this.

-- 
Paul
--
To unsubscribe from this list: send the line "unsubscribe linux-usb" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Overly conservative xHCI bandwidth estimation

[Paul Zimmerman]

Yep, that's the one.

-- 
Paul

On Thu, Sep 24, 2015 at 2:37 PM, Steinar H. Gunderson
 wrote:
> On Thu, Sep 24, 2015 at 02:33:06PM -0700, Paul Zimmerman wrote:
>> IIRC, at least some of the Intel controllers require the bandwidth
>> calculations to be done by the xHCI driver, instead of doing it
>> themselves in hardware. Perhaps you're tripping over a bug in the xHCI
>> driver? Mathias is probably best the one to comment on this.
>
> FWIW, I've checked that the XHCI_SW_BW_CHECKING quirk isn't set for my
> device. (I suppose that's the one you're thinking about?)
>
> /* Steinar */
> --
> Homepage: http://www.sesse.net/
--
To unsubscribe from this list: send the line "unsubscribe linux-usb" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Overly conservative xHCI bandwidth estimation

[Steinar H. Gunderson]

On Fri, Sep 25, 2015 at 12:42:22AM +0200, Steinar H. Gunderson wrote:
> I downgraded to Debian's 3.16 kernel. Both cards came up without a hitch.
> But I only seem to get frames back from the second one.

...and after a quick app fix, I have capture from both cards.

Does this mean I have a long bisect ahead of me?

/* Steinar */
-- 
Homepage: http://www.sesse.net/
--
To unsubscribe from this list: send the line "unsubscribe linux-usb" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Overly conservative xHCI bandwidth estimation

[Alan Stern]

On Thu, 24 Sep 2015, Steinar H. Gunderson wrote:

> On Thu, Sep 24, 2015 at 11:22:57AM -0400, Alan Stern wrote:
> >> I assume there's no way I can lie to the chip? Like, if I know for a fact
> >> that the card will send less data than the alternate claims (like,
> >> I'm using a video mode that will require only a few hundred megabits/second
> >> in practice, but even the lowest alternate claims >1 Gbit/sec).
> > You can always patch the kernel to do whatever you want.  For instance, 
> > you could reduce the bMaxBurst value in the descriptor after the kernel 
> > gets it from the card.  Short of that, there is no way to affect the 
> > outcome.
> 
> I tried, but it didn't work; I bMaxBurst to 4 instead of to 15
> (in usb_parse_ss_endpoint_companion). It didn't change the outcome.
> Perhaps this doesn't actually change what the xHCI controller sees,
> though.

It does.  Grep for max_burst in drivers/usb/host/x*.c to see where it 
gets used.  (Note that in a couple of places involving USB-2 devices, 
the code uses max_burst where it really means multiplicity.)

Alan Stern

--
To unsubscribe from this list: send the line "unsubscribe linux-usb" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Overly conservative xHCI bandwidth estimation

[Steinar H. Gunderson]

On Wed, Sep 23, 2015 at 10:08:05PM +0200, Steinar H. Gunderson wrote:
> I've tried on another machine and it works fine there, so my code should,
> at least on the surface of it, be fine.

PLOT TWIST:

I downgraded to Debian's 3.16 kernel. Both cards came up without a hitch.
But I only seem to get frames back from the second one.

???

/* Steinar */
-- 
Homepage: http://www.sesse.net/
--
To unsubscribe from this list: send the line "unsubscribe linux-usb" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Overly conservative xHCI bandwidth estimation

[Steinar H. Gunderson]

On Thu, Sep 24, 2015 at 04:46:22PM -0400, Alan Stern wrote:
> It does.  Grep for max_burst in drivers/usb/host/x*.c to see where it 
> gets used.  (Note that in a couple of places involving USB-2 devices, 
> the code uses max_burst where it really means multiplicity.)

OK, so this is very curious. I hacked the endpoint descriptors to set
max_burst=1, max_esit_payload=1024, mult=0 (in xhci_endpoint_init, which
seems to be the thing that actually produces the structures the device 
consumed, so I'm reasonably sure nothing else is modifying it on the way).
Still it says COMP_BW_ERR. Setting interval=3 (it used to be 0) has no
effect. Either my override isn't working, or this just isn't about bandwidth,
but something else.

I also tried ignoring the three other endpoints (early-out in
xhci_add_endpoint) just to check that they weren't taking up some resource
I was needing, and it didn't change anything. If I also ignore the video
endpoint itself (0x83), the altsetting _does_ go through, but of course then
I can't get any data from it.

/* Steinar */
-- 
Homepage: http://www.sesse.net/
--
To unsubscribe from this list: send the line "unsubscribe linux-usb" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html