Re: [E1000-devel] 82571EB: Detected Hardware Unit Hang
Hi Yijing, Thanks for your reference, the patch looks good for me, but I have no chance to test it on customer's env. Best Regards, Joe On 12/19/12 13:52, Yijing Wang wrote: > On 2012/12/19 11:04, Joe Jin wrote: >> Hi all, >> >> I backported mps commits and ask customer pass "pci=pcie_bus_peer2pee" to >> kernel >> to limited MPS to 128 and issue disappeared, sound like this is a BIOS bug. >> > > Hi Joe, >I found similar problem when I do pci hotplug, discussion is > here:http://marc.info/?l=linux-pci=134810569924220=2. > We try to improve Linux kernel to debug this problem easily based Bjorn's > suggestion. Jon sent out the first version patch > http://marc.info/?l=linux-pci=135002016005274=2. > I think we can do further here, > http://marc.info/?l=linux-pci=135115581307869=2. I hope this information > can help you. > > Thanks! > Yijing. > >> Thanks all of your help. >> >> Best Regards, >> Joe >> >> On 11/29/12 23:52, Fujinaka, Todd wrote: >>> Someone else pointed this out to me locally. If you have a non-client BIOS, >>> you should be able to set the MaxPayloadSize using setpci. You have to make >>> sure that you're being consistent throughout all the associated links. >>> >>> Todd Fujinaka >>> Technical Marketing Engineer >>> LAN Access Division (LAD) >>> Intel Corporation >>> todd.fujin...@intel.com >>> (503) 712-4565 >>> >>> >>> -Original Message- >>> From: Ethan Zhao [mailto:ethan.ker...@gmail.com] >>> Sent: Wednesday, November 28, 2012 7:10 PM >>> To: Fujinaka, Todd >>> Cc: Joe Jin; Ben Hutchings; Mary Mcgrath; net...@vger.kernel.org; >>> e1000-de...@lists.sf.net; linux-kernel@vger.kernel.org; linux-pci >>> Subject: Re: [E1000-devel] 82571EB: Detected Hardware Unit Hang >>> >>> Joe, >>> Possibly your customer is running a kernel without source code on a >>> platform whose vendor wouldn't like to fix BIOS issue( Is that a HP/Dell >>> server ?). >>> Anyway, to see if is a payload issue or, you could change the payload >>> size with setpci tool to those devices and set the link retrain bit to >>> trigger the link retraining to debug the issue and identity the root cause. >>> I thinks it is much easier than modify the BIOS or eeprom of NIC. >>> >>> e.g. >>>set device control register to 0f 00 (128 bytes payload size) >>># setpci -v -s 00:02.0 98.w=000f >>>set device link control register to 60h (retrain the link) >>># setpci -v -s 00:02.0 a0.b=60 >>> >>> Hope it works, Just my 2 cents. >>> >>> ethan.z...@oracle.com >>> >>> On Wed, Nov 28, 2012 at 11:53 PM, Fujinaka, Todd >>> wrote: >>>> The only EEPROM I know about or can speak to is the one attached to the >>>> 82571 and it doesn't set the MaxPayloadSize. That's done by the BIOS. >>>> >>>> Todd Fujinaka >>>> Technical Marketing Engineer >>>> LAN Access Division (LAD) >>>> Intel Corporation >>>> todd.fujin...@intel.com >>>> (503) 712-4565 >>>> >>>> >>>> -Original Message- >>>> From: Joe Jin [mailto:joe@oracle.com] >>>> Sent: Wednesday, November 28, 2012 12:31 AM >>>> To: Ben Hutchings >>>> Cc: Fujinaka, Todd; Mary Mcgrath; net...@vger.kernel.org; >>>> e1000-de...@lists.sf.net; linux-kernel@vger.kernel.org; linux-pci >>>> Subject: Re: [E1000-devel] 82571EB: Detected Hardware Unit Hang >>>> >>>> On 11/28/12 02:10, Ben Hutchings wrote: >>>>> On Tue, 2012-11-27 at 17:32 +, Fujinaka, Todd wrote: >>>>>> Forgive me if I'm being too repetitious as I think some of this has >>>>>> been mentioned in the past. >>>>>> >>>>>> We (and by we I mean the Ethernet part and driver) can only change >>>>>> the advertised availability of a larger MaxPayloadSize. The size is >>>>>> negotiated by both sides of the link when the link is established. >>>>>> The driver should not change the size of the link as it would be >>>>>> poking at registers outside of its scope and is controlled by the >>>>>> upstream bridge (not us). >>>>> [...] >>>>> >>>>> MaxPayloadSize (MPS) is not negotiated between devices but is >>>>> programm
Re: [E1000-devel] 82571EB: Detected Hardware Unit Hang
On 2012/12/19 11:04, Joe Jin wrote: > Hi all, > > I backported mps commits and ask customer pass "pci=pcie_bus_peer2pee" to > kernel > to limited MPS to 128 and issue disappeared, sound like this is a BIOS bug. > Hi Joe, I found similar problem when I do pci hotplug, discussion is here:http://marc.info/?l=linux-pci=134810569924220=2. We try to improve Linux kernel to debug this problem easily based Bjorn's suggestion. Jon sent out the first version patch http://marc.info/?l=linux-pci=135002016005274=2. I think we can do further here, http://marc.info/?l=linux-pci=135115581307869=2. I hope this information can help you. Thanks! Yijing. > Thanks all of your help. > > Best Regards, > Joe > > On 11/29/12 23:52, Fujinaka, Todd wrote: >> Someone else pointed this out to me locally. If you have a non-client BIOS, >> you should be able to set the MaxPayloadSize using setpci. You have to make >> sure that you're being consistent throughout all the associated links. >> >> Todd Fujinaka >> Technical Marketing Engineer >> LAN Access Division (LAD) >> Intel Corporation >> todd.fujin...@intel.com >> (503) 712-4565 >> >> >> -Original Message- >> From: Ethan Zhao [mailto:ethan.ker...@gmail.com] >> Sent: Wednesday, November 28, 2012 7:10 PM >> To: Fujinaka, Todd >> Cc: Joe Jin; Ben Hutchings; Mary Mcgrath; net...@vger.kernel.org; >> e1000-de...@lists.sf.net; linux-kernel@vger.kernel.org; linux-pci >> Subject: Re: [E1000-devel] 82571EB: Detected Hardware Unit Hang >> >> Joe, >> Possibly your customer is running a kernel without source code on a >> platform whose vendor wouldn't like to fix BIOS issue( Is that a HP/Dell >> server ?). >> Anyway, to see if is a payload issue or, you could change the payload >> size with setpci tool to those devices and set the link retrain bit to >> trigger the link retraining to debug the issue and identity the root cause. >> I thinks it is much easier than modify the BIOS or eeprom of NIC. >> >> e.g. >>set device control register to 0f 00 (128 bytes payload size) >># setpci -v -s 00:02.0 98.w=000f >>set device link control register to 60h (retrain the link) >># setpci -v -s 00:02.0 a0.b=60 >> >> Hope it works, Just my 2 cents. >> >> ethan.z...@oracle.com >> >> On Wed, Nov 28, 2012 at 11:53 PM, Fujinaka, Todd >> wrote: >>> The only EEPROM I know about or can speak to is the one attached to the >>> 82571 and it doesn't set the MaxPayloadSize. That's done by the BIOS. >>> >>> Todd Fujinaka >>> Technical Marketing Engineer >>> LAN Access Division (LAD) >>> Intel Corporation >>> todd.fujin...@intel.com >>> (503) 712-4565 >>> >>> >>> -Original Message- >>> From: Joe Jin [mailto:joe@oracle.com] >>> Sent: Wednesday, November 28, 2012 12:31 AM >>> To: Ben Hutchings >>> Cc: Fujinaka, Todd; Mary Mcgrath; net...@vger.kernel.org; >>> e1000-de...@lists.sf.net; linux-kernel@vger.kernel.org; linux-pci >>> Subject: Re: [E1000-devel] 82571EB: Detected Hardware Unit Hang >>> >>> On 11/28/12 02:10, Ben Hutchings wrote: >>>> On Tue, 2012-11-27 at 17:32 +, Fujinaka, Todd wrote: >>>>> Forgive me if I'm being too repetitious as I think some of this has >>>>> been mentioned in the past. >>>>> >>>>> We (and by we I mean the Ethernet part and driver) can only change >>>>> the advertised availability of a larger MaxPayloadSize. The size is >>>>> negotiated by both sides of the link when the link is established. >>>>> The driver should not change the size of the link as it would be >>>>> poking at registers outside of its scope and is controlled by the >>>>> upstream bridge (not us). >>>> [...] >>>> >>>> MaxPayloadSize (MPS) is not negotiated between devices but is >>>> programmed by the system firmware (at least for devices present at >>>> boot - the kernel may be responsible in case of hotplug). You can >>>> use the kernel parameter 'pci=pcie_bus_perf' (or one of several >>>> others) to set a policy that overrides this, but no policy will allow >>>> setting MPS above the device's MaxPayloadSizeSupported (MPSS). >>>> >>> >>> Ben, >>> >>> Unfortunately I'm using 3.0.x kernel and this is not included in the kernel. >>> So I'm trying to use ethtool modify it from eeprom to see if help or no. >>> >>> >>> Todd, I'll review all MaxPayload for all devices, but need to say if it >>> mismatch, customer could not modify it from BIOS for there was not entry at >>> there, to test it, we have to find how to verify if this is the root cause, >>> so still need to find the offset in eeprom. >>> >>> Thanks in advance, >>> Joe >>> > > -- Thanks! Yijing -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [E1000-devel] 82571EB: Detected Hardware Unit Hang
Hi all, I backported mps commits and ask customer pass "pci=pcie_bus_peer2pee" to kernel to limited MPS to 128 and issue disappeared, sound like this is a BIOS bug. Thanks all of your help. Best Regards, Joe On 11/29/12 23:52, Fujinaka, Todd wrote: > Someone else pointed this out to me locally. If you have a non-client BIOS, > you should be able to set the MaxPayloadSize using setpci. You have to make > sure that you're being consistent throughout all the associated links. > > Todd Fujinaka > Technical Marketing Engineer > LAN Access Division (LAD) > Intel Corporation > todd.fujin...@intel.com > (503) 712-4565 > > > -Original Message- > From: Ethan Zhao [mailto:ethan.ker...@gmail.com] > Sent: Wednesday, November 28, 2012 7:10 PM > To: Fujinaka, Todd > Cc: Joe Jin; Ben Hutchings; Mary Mcgrath; net...@vger.kernel.org; > e1000-de...@lists.sf.net; linux-kernel@vger.kernel.org; linux-pci > Subject: Re: [E1000-devel] 82571EB: Detected Hardware Unit Hang > > Joe, > Possibly your customer is running a kernel without source code on a > platform whose vendor wouldn't like to fix BIOS issue( Is that a HP/Dell > server ?). > Anyway, to see if is a payload issue or, you could change the payload > size with setpci tool to those devices and set the link retrain bit to > trigger the link retraining to debug the issue and identity the root cause. > I thinks it is much easier than modify the BIOS or eeprom of NIC. > > e.g. >set device control register to 0f 00 (128 bytes payload size) ># setpci -v -s 00:02.0 98.w=000f >set device link control register to 60h (retrain the link) ># setpci -v -s 00:02.0 a0.b=60 > > Hope it works, Just my 2 cents. > > ethan.z...@oracle.com > > On Wed, Nov 28, 2012 at 11:53 PM, Fujinaka, Todd > wrote: >> The only EEPROM I know about or can speak to is the one attached to the >> 82571 and it doesn't set the MaxPayloadSize. That's done by the BIOS. >> >> Todd Fujinaka >> Technical Marketing Engineer >> LAN Access Division (LAD) >> Intel Corporation >> todd.fujin...@intel.com >> (503) 712-4565 >> >> >> -Original Message- >> From: Joe Jin [mailto:joe@oracle.com] >> Sent: Wednesday, November 28, 2012 12:31 AM >> To: Ben Hutchings >> Cc: Fujinaka, Todd; Mary Mcgrath; net...@vger.kernel.org; >> e1000-de...@lists.sf.net; linux-kernel@vger.kernel.org; linux-pci >> Subject: Re: [E1000-devel] 82571EB: Detected Hardware Unit Hang >> >> On 11/28/12 02:10, Ben Hutchings wrote: >>> On Tue, 2012-11-27 at 17:32 +, Fujinaka, Todd wrote: >>>> Forgive me if I'm being too repetitious as I think some of this has >>>> been mentioned in the past. >>>> >>>> We (and by we I mean the Ethernet part and driver) can only change >>>> the advertised availability of a larger MaxPayloadSize. The size is >>>> negotiated by both sides of the link when the link is established. >>>> The driver should not change the size of the link as it would be >>>> poking at registers outside of its scope and is controlled by the >>>> upstream bridge (not us). >>> [...] >>> >>> MaxPayloadSize (MPS) is not negotiated between devices but is >>> programmed by the system firmware (at least for devices present at >>> boot - the kernel may be responsible in case of hotplug). You can >>> use the kernel parameter 'pci=pcie_bus_perf' (or one of several >>> others) to set a policy that overrides this, but no policy will allow >>> setting MPS above the device's MaxPayloadSizeSupported (MPSS). >>> >> >> Ben, >> >> Unfortunately I'm using 3.0.x kernel and this is not included in the kernel. >> So I'm trying to use ethtool modify it from eeprom to see if help or no. >> >> >> Todd, I'll review all MaxPayload for all devices, but need to say if it >> mismatch, customer could not modify it from BIOS for there was not entry at >> there, to test it, we have to find how to verify if this is the root cause, >> so still need to find the offset in eeprom. >> >> Thanks in advance, >> Joe >> -- Oracle <http://www.oracle.com> Joe Jin | Software Development Senior Manager | +8610.6106.5624 ORACLE | Linux and Virtualization No. 24 Zhongguancun Software Park, Haidian District | 100193 Beijing -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [E1000-devel] 82571EB: Detected Hardware Unit Hang
Hi all, I backported mps commits and ask customer pass pci=pcie_bus_peer2pee to kernel to limited MPS to 128 and issue disappeared, sound like this is a BIOS bug. Thanks all of your help. Best Regards, Joe On 11/29/12 23:52, Fujinaka, Todd wrote: Someone else pointed this out to me locally. If you have a non-client BIOS, you should be able to set the MaxPayloadSize using setpci. You have to make sure that you're being consistent throughout all the associated links. Todd Fujinaka Technical Marketing Engineer LAN Access Division (LAD) Intel Corporation todd.fujin...@intel.com (503) 712-4565 -Original Message- From: Ethan Zhao [mailto:ethan.ker...@gmail.com] Sent: Wednesday, November 28, 2012 7:10 PM To: Fujinaka, Todd Cc: Joe Jin; Ben Hutchings; Mary Mcgrath; net...@vger.kernel.org; e1000-de...@lists.sf.net; linux-kernel@vger.kernel.org; linux-pci Subject: Re: [E1000-devel] 82571EB: Detected Hardware Unit Hang Joe, Possibly your customer is running a kernel without source code on a platform whose vendor wouldn't like to fix BIOS issue( Is that a HP/Dell server ?). Anyway, to see if is a payload issue or, you could change the payload size with setpci tool to those devices and set the link retrain bit to trigger the link retraining to debug the issue and identity the root cause. I thinks it is much easier than modify the BIOS or eeprom of NIC. e.g. set device control register to 0f 00 (128 bytes payload size) # setpci -v -s 00:02.0 98.w=000f set device link control register to 60h (retrain the link) # setpci -v -s 00:02.0 a0.b=60 Hope it works, Just my 2 cents. ethan.z...@oracle.com On Wed, Nov 28, 2012 at 11:53 PM, Fujinaka, Todd todd.fujin...@intel.com wrote: The only EEPROM I know about or can speak to is the one attached to the 82571 and it doesn't set the MaxPayloadSize. That's done by the BIOS. Todd Fujinaka Technical Marketing Engineer LAN Access Division (LAD) Intel Corporation todd.fujin...@intel.com (503) 712-4565 -Original Message- From: Joe Jin [mailto:joe@oracle.com] Sent: Wednesday, November 28, 2012 12:31 AM To: Ben Hutchings Cc: Fujinaka, Todd; Mary Mcgrath; net...@vger.kernel.org; e1000-de...@lists.sf.net; linux-kernel@vger.kernel.org; linux-pci Subject: Re: [E1000-devel] 82571EB: Detected Hardware Unit Hang On 11/28/12 02:10, Ben Hutchings wrote: On Tue, 2012-11-27 at 17:32 +, Fujinaka, Todd wrote: Forgive me if I'm being too repetitious as I think some of this has been mentioned in the past. We (and by we I mean the Ethernet part and driver) can only change the advertised availability of a larger MaxPayloadSize. The size is negotiated by both sides of the link when the link is established. The driver should not change the size of the link as it would be poking at registers outside of its scope and is controlled by the upstream bridge (not us). [...] MaxPayloadSize (MPS) is not negotiated between devices but is programmed by the system firmware (at least for devices present at boot - the kernel may be responsible in case of hotplug). You can use the kernel parameter 'pci=pcie_bus_perf' (or one of several others) to set a policy that overrides this, but no policy will allow setting MPS above the device's MaxPayloadSizeSupported (MPSS). Ben, Unfortunately I'm using 3.0.x kernel and this is not included in the kernel. So I'm trying to use ethtool modify it from eeprom to see if help or no. Todd, I'll review all MaxPayload for all devices, but need to say if it mismatch, customer could not modify it from BIOS for there was not entry at there, to test it, we have to find how to verify if this is the root cause, so still need to find the offset in eeprom. Thanks in advance, Joe -- Oracle http://www.oracle.com Joe Jin | Software Development Senior Manager | +8610.6106.5624 ORACLE | Linux and Virtualization No. 24 Zhongguancun Software Park, Haidian District | 100193 Beijing -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [E1000-devel] 82571EB: Detected Hardware Unit Hang
On 2012/12/19 11:04, Joe Jin wrote: Hi all, I backported mps commits and ask customer pass pci=pcie_bus_peer2pee to kernel to limited MPS to 128 and issue disappeared, sound like this is a BIOS bug. Hi Joe, I found similar problem when I do pci hotplug, discussion is here:http://marc.info/?l=linux-pcim=134810569924220w=2. We try to improve Linux kernel to debug this problem easily based Bjorn's suggestion. Jon sent out the first version patch http://marc.info/?l=linux-pcim=135002016005274w=2. I think we can do further here, http://marc.info/?l=linux-pcim=135115581307869w=2. I hope this information can help you. Thanks! Yijing. Thanks all of your help. Best Regards, Joe On 11/29/12 23:52, Fujinaka, Todd wrote: Someone else pointed this out to me locally. If you have a non-client BIOS, you should be able to set the MaxPayloadSize using setpci. You have to make sure that you're being consistent throughout all the associated links. Todd Fujinaka Technical Marketing Engineer LAN Access Division (LAD) Intel Corporation todd.fujin...@intel.com (503) 712-4565 -Original Message- From: Ethan Zhao [mailto:ethan.ker...@gmail.com] Sent: Wednesday, November 28, 2012 7:10 PM To: Fujinaka, Todd Cc: Joe Jin; Ben Hutchings; Mary Mcgrath; net...@vger.kernel.org; e1000-de...@lists.sf.net; linux-kernel@vger.kernel.org; linux-pci Subject: Re: [E1000-devel] 82571EB: Detected Hardware Unit Hang Joe, Possibly your customer is running a kernel without source code on a platform whose vendor wouldn't like to fix BIOS issue( Is that a HP/Dell server ?). Anyway, to see if is a payload issue or, you could change the payload size with setpci tool to those devices and set the link retrain bit to trigger the link retraining to debug the issue and identity the root cause. I thinks it is much easier than modify the BIOS or eeprom of NIC. e.g. set device control register to 0f 00 (128 bytes payload size) # setpci -v -s 00:02.0 98.w=000f set device link control register to 60h (retrain the link) # setpci -v -s 00:02.0 a0.b=60 Hope it works, Just my 2 cents. ethan.z...@oracle.com On Wed, Nov 28, 2012 at 11:53 PM, Fujinaka, Todd todd.fujin...@intel.com wrote: The only EEPROM I know about or can speak to is the one attached to the 82571 and it doesn't set the MaxPayloadSize. That's done by the BIOS. Todd Fujinaka Technical Marketing Engineer LAN Access Division (LAD) Intel Corporation todd.fujin...@intel.com (503) 712-4565 -Original Message- From: Joe Jin [mailto:joe@oracle.com] Sent: Wednesday, November 28, 2012 12:31 AM To: Ben Hutchings Cc: Fujinaka, Todd; Mary Mcgrath; net...@vger.kernel.org; e1000-de...@lists.sf.net; linux-kernel@vger.kernel.org; linux-pci Subject: Re: [E1000-devel] 82571EB: Detected Hardware Unit Hang On 11/28/12 02:10, Ben Hutchings wrote: On Tue, 2012-11-27 at 17:32 +, Fujinaka, Todd wrote: Forgive me if I'm being too repetitious as I think some of this has been mentioned in the past. We (and by we I mean the Ethernet part and driver) can only change the advertised availability of a larger MaxPayloadSize. The size is negotiated by both sides of the link when the link is established. The driver should not change the size of the link as it would be poking at registers outside of its scope and is controlled by the upstream bridge (not us). [...] MaxPayloadSize (MPS) is not negotiated between devices but is programmed by the system firmware (at least for devices present at boot - the kernel may be responsible in case of hotplug). You can use the kernel parameter 'pci=pcie_bus_perf' (or one of several others) to set a policy that overrides this, but no policy will allow setting MPS above the device's MaxPayloadSizeSupported (MPSS). Ben, Unfortunately I'm using 3.0.x kernel and this is not included in the kernel. So I'm trying to use ethtool modify it from eeprom to see if help or no. Todd, I'll review all MaxPayload for all devices, but need to say if it mismatch, customer could not modify it from BIOS for there was not entry at there, to test it, we have to find how to verify if this is the root cause, so still need to find the offset in eeprom. Thanks in advance, Joe -- Thanks! Yijing -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [E1000-devel] 82571EB: Detected Hardware Unit Hang
Hi Yijing, Thanks for your reference, the patch looks good for me, but I have no chance to test it on customer's env. Best Regards, Joe On 12/19/12 13:52, Yijing Wang wrote: On 2012/12/19 11:04, Joe Jin wrote: Hi all, I backported mps commits and ask customer pass pci=pcie_bus_peer2pee to kernel to limited MPS to 128 and issue disappeared, sound like this is a BIOS bug. Hi Joe, I found similar problem when I do pci hotplug, discussion is here:http://marc.info/?l=linux-pcim=134810569924220w=2. We try to improve Linux kernel to debug this problem easily based Bjorn's suggestion. Jon sent out the first version patch http://marc.info/?l=linux-pcim=135002016005274w=2. I think we can do further here, http://marc.info/?l=linux-pcim=135115581307869w=2. I hope this information can help you. Thanks! Yijing. Thanks all of your help. Best Regards, Joe On 11/29/12 23:52, Fujinaka, Todd wrote: Someone else pointed this out to me locally. If you have a non-client BIOS, you should be able to set the MaxPayloadSize using setpci. You have to make sure that you're being consistent throughout all the associated links. Todd Fujinaka Technical Marketing Engineer LAN Access Division (LAD) Intel Corporation todd.fujin...@intel.com (503) 712-4565 -Original Message- From: Ethan Zhao [mailto:ethan.ker...@gmail.com] Sent: Wednesday, November 28, 2012 7:10 PM To: Fujinaka, Todd Cc: Joe Jin; Ben Hutchings; Mary Mcgrath; net...@vger.kernel.org; e1000-de...@lists.sf.net; linux-kernel@vger.kernel.org; linux-pci Subject: Re: [E1000-devel] 82571EB: Detected Hardware Unit Hang Joe, Possibly your customer is running a kernel without source code on a platform whose vendor wouldn't like to fix BIOS issue( Is that a HP/Dell server ?). Anyway, to see if is a payload issue or, you could change the payload size with setpci tool to those devices and set the link retrain bit to trigger the link retraining to debug the issue and identity the root cause. I thinks it is much easier than modify the BIOS or eeprom of NIC. e.g. set device control register to 0f 00 (128 bytes payload size) # setpci -v -s 00:02.0 98.w=000f set device link control register to 60h (retrain the link) # setpci -v -s 00:02.0 a0.b=60 Hope it works, Just my 2 cents. ethan.z...@oracle.com On Wed, Nov 28, 2012 at 11:53 PM, Fujinaka, Todd todd.fujin...@intel.com wrote: The only EEPROM I know about or can speak to is the one attached to the 82571 and it doesn't set the MaxPayloadSize. That's done by the BIOS. Todd Fujinaka Technical Marketing Engineer LAN Access Division (LAD) Intel Corporation todd.fujin...@intel.com (503) 712-4565 -Original Message- From: Joe Jin [mailto:joe@oracle.com] Sent: Wednesday, November 28, 2012 12:31 AM To: Ben Hutchings Cc: Fujinaka, Todd; Mary Mcgrath; net...@vger.kernel.org; e1000-de...@lists.sf.net; linux-kernel@vger.kernel.org; linux-pci Subject: Re: [E1000-devel] 82571EB: Detected Hardware Unit Hang On 11/28/12 02:10, Ben Hutchings wrote: On Tue, 2012-11-27 at 17:32 +, Fujinaka, Todd wrote: Forgive me if I'm being too repetitious as I think some of this has been mentioned in the past. We (and by we I mean the Ethernet part and driver) can only change the advertised availability of a larger MaxPayloadSize. The size is negotiated by both sides of the link when the link is established. The driver should not change the size of the link as it would be poking at registers outside of its scope and is controlled by the upstream bridge (not us). [...] MaxPayloadSize (MPS) is not negotiated between devices but is programmed by the system firmware (at least for devices present at boot - the kernel may be responsible in case of hotplug). You can use the kernel parameter 'pci=pcie_bus_perf' (or one of several others) to set a policy that overrides this, but no policy will allow setting MPS above the device's MaxPayloadSizeSupported (MPSS). Ben, Unfortunately I'm using 3.0.x kernel and this is not included in the kernel. So I'm trying to use ethtool modify it from eeprom to see if help or no. Todd, I'll review all MaxPayload for all devices, but need to say if it mismatch, customer could not modify it from BIOS for there was not entry at there, to test it, we have to find how to verify if this is the root cause, so still need to find the offset in eeprom. Thanks in advance, Joe -- Oracle http://www.oracle.com Joe Jin | Software Development Senior Manager | +8610.6106.5624 ORACLE | Linux and Virtualization No. 24 Zhongguancun Software Park, Haidian District | 100193 Beijing -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
RE: [E1000-devel] 82571EB: Detected Hardware Unit Hang
Someone else pointed this out to me locally. If you have a non-client BIOS, you should be able to set the MaxPayloadSize using setpci. You have to make sure that you're being consistent throughout all the associated links. Todd Fujinaka Technical Marketing Engineer LAN Access Division (LAD) Intel Corporation todd.fujin...@intel.com (503) 712-4565 -Original Message- From: Ethan Zhao [mailto:ethan.ker...@gmail.com] Sent: Wednesday, November 28, 2012 7:10 PM To: Fujinaka, Todd Cc: Joe Jin; Ben Hutchings; Mary Mcgrath; net...@vger.kernel.org; e1000-de...@lists.sf.net; linux-kernel@vger.kernel.org; linux-pci Subject: Re: [E1000-devel] 82571EB: Detected Hardware Unit Hang Joe, Possibly your customer is running a kernel without source code on a platform whose vendor wouldn't like to fix BIOS issue( Is that a HP/Dell server ?). Anyway, to see if is a payload issue or, you could change the payload size with setpci tool to those devices and set the link retrain bit to trigger the link retraining to debug the issue and identity the root cause. I thinks it is much easier than modify the BIOS or eeprom of NIC. e.g. set device control register to 0f 00 (128 bytes payload size) # setpci -v -s 00:02.0 98.w=000f set device link control register to 60h (retrain the link) # setpci -v -s 00:02.0 a0.b=60 Hope it works, Just my 2 cents. ethan.z...@oracle.com On Wed, Nov 28, 2012 at 11:53 PM, Fujinaka, Todd wrote: > The only EEPROM I know about or can speak to is the one attached to the 82571 > and it doesn't set the MaxPayloadSize. That's done by the BIOS. > > Todd Fujinaka > Technical Marketing Engineer > LAN Access Division (LAD) > Intel Corporation > todd.fujin...@intel.com > (503) 712-4565 > > > -Original Message- > From: Joe Jin [mailto:joe@oracle.com] > Sent: Wednesday, November 28, 2012 12:31 AM > To: Ben Hutchings > Cc: Fujinaka, Todd; Mary Mcgrath; net...@vger.kernel.org; > e1000-de...@lists.sf.net; linux-kernel@vger.kernel.org; linux-pci > Subject: Re: [E1000-devel] 82571EB: Detected Hardware Unit Hang > > On 11/28/12 02:10, Ben Hutchings wrote: >> On Tue, 2012-11-27 at 17:32 +, Fujinaka, Todd wrote: >>> Forgive me if I'm being too repetitious as I think some of this has >>> been mentioned in the past. >>> >>> We (and by we I mean the Ethernet part and driver) can only change >>> the advertised availability of a larger MaxPayloadSize. The size is >>> negotiated by both sides of the link when the link is established. >>> The driver should not change the size of the link as it would be >>> poking at registers outside of its scope and is controlled by the >>> upstream bridge (not us). >> [...] >> >> MaxPayloadSize (MPS) is not negotiated between devices but is >> programmed by the system firmware (at least for devices present at >> boot - the kernel may be responsible in case of hotplug). You can >> use the kernel parameter 'pci=pcie_bus_perf' (or one of several >> others) to set a policy that overrides this, but no policy will allow >> setting MPS above the device's MaxPayloadSizeSupported (MPSS). >> > > Ben, > > Unfortunately I'm using 3.0.x kernel and this is not included in the kernel. > So I'm trying to use ethtool modify it from eeprom to see if help or no. > > > Todd, I'll review all MaxPayload for all devices, but need to say if it > mismatch, customer could not modify it from BIOS for there was not entry at > there, to test it, we have to find how to verify if this is the root cause, > so still need to find the offset in eeprom. > > Thanks in advance, > Joe > -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
RE: [E1000-devel] 82571EB: Detected Hardware Unit Hang
Someone else pointed this out to me locally. If you have a non-client BIOS, you should be able to set the MaxPayloadSize using setpci. You have to make sure that you're being consistent throughout all the associated links. Todd Fujinaka Technical Marketing Engineer LAN Access Division (LAD) Intel Corporation todd.fujin...@intel.com (503) 712-4565 -Original Message- From: Ethan Zhao [mailto:ethan.ker...@gmail.com] Sent: Wednesday, November 28, 2012 7:10 PM To: Fujinaka, Todd Cc: Joe Jin; Ben Hutchings; Mary Mcgrath; net...@vger.kernel.org; e1000-de...@lists.sf.net; linux-kernel@vger.kernel.org; linux-pci Subject: Re: [E1000-devel] 82571EB: Detected Hardware Unit Hang Joe, Possibly your customer is running a kernel without source code on a platform whose vendor wouldn't like to fix BIOS issue( Is that a HP/Dell server ?). Anyway, to see if is a payload issue or, you could change the payload size with setpci tool to those devices and set the link retrain bit to trigger the link retraining to debug the issue and identity the root cause. I thinks it is much easier than modify the BIOS or eeprom of NIC. e.g. set device control register to 0f 00 (128 bytes payload size) # setpci -v -s 00:02.0 98.w=000f set device link control register to 60h (retrain the link) # setpci -v -s 00:02.0 a0.b=60 Hope it works, Just my 2 cents. ethan.z...@oracle.com On Wed, Nov 28, 2012 at 11:53 PM, Fujinaka, Todd todd.fujin...@intel.com wrote: The only EEPROM I know about or can speak to is the one attached to the 82571 and it doesn't set the MaxPayloadSize. That's done by the BIOS. Todd Fujinaka Technical Marketing Engineer LAN Access Division (LAD) Intel Corporation todd.fujin...@intel.com (503) 712-4565 -Original Message- From: Joe Jin [mailto:joe@oracle.com] Sent: Wednesday, November 28, 2012 12:31 AM To: Ben Hutchings Cc: Fujinaka, Todd; Mary Mcgrath; net...@vger.kernel.org; e1000-de...@lists.sf.net; linux-kernel@vger.kernel.org; linux-pci Subject: Re: [E1000-devel] 82571EB: Detected Hardware Unit Hang On 11/28/12 02:10, Ben Hutchings wrote: On Tue, 2012-11-27 at 17:32 +, Fujinaka, Todd wrote: Forgive me if I'm being too repetitious as I think some of this has been mentioned in the past. We (and by we I mean the Ethernet part and driver) can only change the advertised availability of a larger MaxPayloadSize. The size is negotiated by both sides of the link when the link is established. The driver should not change the size of the link as it would be poking at registers outside of its scope and is controlled by the upstream bridge (not us). [...] MaxPayloadSize (MPS) is not negotiated between devices but is programmed by the system firmware (at least for devices present at boot - the kernel may be responsible in case of hotplug). You can use the kernel parameter 'pci=pcie_bus_perf' (or one of several others) to set a policy that overrides this, but no policy will allow setting MPS above the device's MaxPayloadSizeSupported (MPSS). Ben, Unfortunately I'm using 3.0.x kernel and this is not included in the kernel. So I'm trying to use ethtool modify it from eeprom to see if help or no. Todd, I'll review all MaxPayload for all devices, but need to say if it mismatch, customer could not modify it from BIOS for there was not entry at there, to test it, we have to find how to verify if this is the root cause, so still need to find the offset in eeprom. Thanks in advance, Joe -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [E1000-devel] 82571EB: Detected Hardware Unit Hang
Joe, Possibly your customer is running a kernel without source code on a platform whose vendor wouldn't like to fix BIOS issue( Is that a HP/Dell server ?). Anyway, to see if is a payload issue or, you could change the payload size with setpci tool to those devices and set the link retrain bit to trigger the link retraining to debug the issue and identity the root cause. I thinks it is much easier than modify the BIOS or eeprom of NIC. e.g. set device control register to 0f 00 (128 bytes payload size) # setpci -v -s 00:02.0 98.w=000f set device link control register to 60h (retrain the link) # setpci -v -s 00:02.0 a0.b=60 Hope it works, Just my 2 cents. ethan.z...@oracle.com On Wed, Nov 28, 2012 at 11:53 PM, Fujinaka, Todd wrote: > The only EEPROM I know about or can speak to is the one attached to the 82571 > and it doesn't set the MaxPayloadSize. That's done by the BIOS. > > Todd Fujinaka > Technical Marketing Engineer > LAN Access Division (LAD) > Intel Corporation > todd.fujin...@intel.com > (503) 712-4565 > > > -Original Message- > From: Joe Jin [mailto:joe@oracle.com] > Sent: Wednesday, November 28, 2012 12:31 AM > To: Ben Hutchings > Cc: Fujinaka, Todd; Mary Mcgrath; net...@vger.kernel.org; > e1000-de...@lists.sf.net; linux-kernel@vger.kernel.org; linux-pci > Subject: Re: [E1000-devel] 82571EB: Detected Hardware Unit Hang > > On 11/28/12 02:10, Ben Hutchings wrote: >> On Tue, 2012-11-27 at 17:32 +, Fujinaka, Todd wrote: >>> Forgive me if I'm being too repetitious as I think some of this has >>> been mentioned in the past. >>> >>> We (and by we I mean the Ethernet part and driver) can only change >>> the advertised availability of a larger MaxPayloadSize. The size is >>> negotiated by both sides of the link when the link is established. >>> The driver should not change the size of the link as it would be >>> poking at registers outside of its scope and is controlled by the >>> upstream bridge (not us). >> [...] >> >> MaxPayloadSize (MPS) is not negotiated between devices but is >> programmed by the system firmware (at least for devices present at >> boot - the kernel may be responsible in case of hotplug). You can use >> the kernel parameter 'pci=pcie_bus_perf' (or one of several others) to >> set a policy that overrides this, but no policy will allow setting MPS >> above the device's MaxPayloadSizeSupported (MPSS). >> > > Ben, > > Unfortunately I'm using 3.0.x kernel and this is not included in the kernel. > So I'm trying to use ethtool modify it from eeprom to see if help or no. > > > Todd, I'll review all MaxPayload for all devices, but need to say if it > mismatch, customer could not modify it from BIOS for there was not entry at > there, to test it, we have to find how to verify if this is the root cause, > so still need to find the offset in eeprom. > > Thanks in advance, > Joe > -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
RE: [E1000-devel] 82571EB: Detected Hardware Unit Hang
The only EEPROM I know about or can speak to is the one attached to the 82571 and it doesn't set the MaxPayloadSize. That's done by the BIOS. Todd Fujinaka Technical Marketing Engineer LAN Access Division (LAD) Intel Corporation todd.fujin...@intel.com (503) 712-4565 -Original Message- From: Joe Jin [mailto:joe@oracle.com] Sent: Wednesday, November 28, 2012 12:31 AM To: Ben Hutchings Cc: Fujinaka, Todd; Mary Mcgrath; net...@vger.kernel.org; e1000-de...@lists.sf.net; linux-kernel@vger.kernel.org; linux-pci Subject: Re: [E1000-devel] 82571EB: Detected Hardware Unit Hang On 11/28/12 02:10, Ben Hutchings wrote: > On Tue, 2012-11-27 at 17:32 +, Fujinaka, Todd wrote: >> Forgive me if I'm being too repetitious as I think some of this has >> been mentioned in the past. >> >> We (and by we I mean the Ethernet part and driver) can only change >> the advertised availability of a larger MaxPayloadSize. The size is >> negotiated by both sides of the link when the link is established. >> The driver should not change the size of the link as it would be >> poking at registers outside of its scope and is controlled by the >> upstream bridge (not us). > [...] > > MaxPayloadSize (MPS) is not negotiated between devices but is > programmed by the system firmware (at least for devices present at > boot - the kernel may be responsible in case of hotplug). You can use > the kernel parameter 'pci=pcie_bus_perf' (or one of several others) to > set a policy that overrides this, but no policy will allow setting MPS > above the device's MaxPayloadSizeSupported (MPSS). > Ben, Unfortunately I'm using 3.0.x kernel and this is not included in the kernel. So I'm trying to use ethtool modify it from eeprom to see if help or no. Todd, I'll review all MaxPayload for all devices, but need to say if it mismatch, customer could not modify it from BIOS for there was not entry at there, to test it, we have to find how to verify if this is the root cause, so still need to find the offset in eeprom. Thanks in advance, Joe
Re: [E1000-devel] 82571EB: Detected Hardware Unit Hang
On 11/28/12 02:10, Ben Hutchings wrote: > On Tue, 2012-11-27 at 17:32 +, Fujinaka, Todd wrote: >> Forgive me if I'm being too repetitious as I think some of this has >> been mentioned in the past. >> >> We (and by we I mean the Ethernet part and driver) can only change the >> advertised availability of a larger MaxPayloadSize. The size is >> negotiated by both sides of the link when the link is established. The >> driver should not change the size of the link as it would be poking at >> registers outside of its scope and is controlled by the upstream >> bridge (not us). > [...] > > MaxPayloadSize (MPS) is not negotiated between devices but is programmed > by the system firmware (at least for devices present at boot - the > kernel may be responsible in case of hotplug). You can use the kernel > parameter 'pci=pcie_bus_perf' (or one of several others) to set a policy > that overrides this, but no policy will allow setting MPS above the > device's MaxPayloadSizeSupported (MPSS). > Ben, Unfortunately I'm using 3.0.x kernel and this is not included in the kernel. So I'm trying to use ethtool modify it from eeprom to see if help or no. Todd, I'll review all MaxPayload for all devices, but need to say if it mismatch, customer could not modify it from BIOS for there was not entry at there, to test it, we have to find how to verify if this is the root cause, so still need to find the offset in eeprom. Thanks in advance, Joe -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [E1000-devel] 82571EB: Detected Hardware Unit Hang
On 11/28/12 02:10, Ben Hutchings wrote: On Tue, 2012-11-27 at 17:32 +, Fujinaka, Todd wrote: Forgive me if I'm being too repetitious as I think some of this has been mentioned in the past. We (and by we I mean the Ethernet part and driver) can only change the advertised availability of a larger MaxPayloadSize. The size is negotiated by both sides of the link when the link is established. The driver should not change the size of the link as it would be poking at registers outside of its scope and is controlled by the upstream bridge (not us). [...] MaxPayloadSize (MPS) is not negotiated between devices but is programmed by the system firmware (at least for devices present at boot - the kernel may be responsible in case of hotplug). You can use the kernel parameter 'pci=pcie_bus_perf' (or one of several others) to set a policy that overrides this, but no policy will allow setting MPS above the device's MaxPayloadSizeSupported (MPSS). Ben, Unfortunately I'm using 3.0.x kernel and this is not included in the kernel. So I'm trying to use ethtool modify it from eeprom to see if help or no. Todd, I'll review all MaxPayload for all devices, but need to say if it mismatch, customer could not modify it from BIOS for there was not entry at there, to test it, we have to find how to verify if this is the root cause, so still need to find the offset in eeprom. Thanks in advance, Joe -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
RE: [E1000-devel] 82571EB: Detected Hardware Unit Hang
The only EEPROM I know about or can speak to is the one attached to the 82571 and it doesn't set the MaxPayloadSize. That's done by the BIOS. Todd Fujinaka Technical Marketing Engineer LAN Access Division (LAD) Intel Corporation todd.fujin...@intel.com (503) 712-4565 -Original Message- From: Joe Jin [mailto:joe@oracle.com] Sent: Wednesday, November 28, 2012 12:31 AM To: Ben Hutchings Cc: Fujinaka, Todd; Mary Mcgrath; net...@vger.kernel.org; e1000-de...@lists.sf.net; linux-kernel@vger.kernel.org; linux-pci Subject: Re: [E1000-devel] 82571EB: Detected Hardware Unit Hang On 11/28/12 02:10, Ben Hutchings wrote: On Tue, 2012-11-27 at 17:32 +, Fujinaka, Todd wrote: Forgive me if I'm being too repetitious as I think some of this has been mentioned in the past. We (and by we I mean the Ethernet part and driver) can only change the advertised availability of a larger MaxPayloadSize. The size is negotiated by both sides of the link when the link is established. The driver should not change the size of the link as it would be poking at registers outside of its scope and is controlled by the upstream bridge (not us). [...] MaxPayloadSize (MPS) is not negotiated between devices but is programmed by the system firmware (at least for devices present at boot - the kernel may be responsible in case of hotplug). You can use the kernel parameter 'pci=pcie_bus_perf' (or one of several others) to set a policy that overrides this, but no policy will allow setting MPS above the device's MaxPayloadSizeSupported (MPSS). Ben, Unfortunately I'm using 3.0.x kernel and this is not included in the kernel. So I'm trying to use ethtool modify it from eeprom to see if help or no. Todd, I'll review all MaxPayload for all devices, but need to say if it mismatch, customer could not modify it from BIOS for there was not entry at there, to test it, we have to find how to verify if this is the root cause, so still need to find the offset in eeprom. Thanks in advance, Joe
Re: [E1000-devel] 82571EB: Detected Hardware Unit Hang
Joe, Possibly your customer is running a kernel without source code on a platform whose vendor wouldn't like to fix BIOS issue( Is that a HP/Dell server ?). Anyway, to see if is a payload issue or, you could change the payload size with setpci tool to those devices and set the link retrain bit to trigger the link retraining to debug the issue and identity the root cause. I thinks it is much easier than modify the BIOS or eeprom of NIC. e.g. set device control register to 0f 00 (128 bytes payload size) # setpci -v -s 00:02.0 98.w=000f set device link control register to 60h (retrain the link) # setpci -v -s 00:02.0 a0.b=60 Hope it works, Just my 2 cents. ethan.z...@oracle.com On Wed, Nov 28, 2012 at 11:53 PM, Fujinaka, Todd todd.fujin...@intel.com wrote: The only EEPROM I know about or can speak to is the one attached to the 82571 and it doesn't set the MaxPayloadSize. That's done by the BIOS. Todd Fujinaka Technical Marketing Engineer LAN Access Division (LAD) Intel Corporation todd.fujin...@intel.com (503) 712-4565 -Original Message- From: Joe Jin [mailto:joe@oracle.com] Sent: Wednesday, November 28, 2012 12:31 AM To: Ben Hutchings Cc: Fujinaka, Todd; Mary Mcgrath; net...@vger.kernel.org; e1000-de...@lists.sf.net; linux-kernel@vger.kernel.org; linux-pci Subject: Re: [E1000-devel] 82571EB: Detected Hardware Unit Hang On 11/28/12 02:10, Ben Hutchings wrote: On Tue, 2012-11-27 at 17:32 +, Fujinaka, Todd wrote: Forgive me if I'm being too repetitious as I think some of this has been mentioned in the past. We (and by we I mean the Ethernet part and driver) can only change the advertised availability of a larger MaxPayloadSize. The size is negotiated by both sides of the link when the link is established. The driver should not change the size of the link as it would be poking at registers outside of its scope and is controlled by the upstream bridge (not us). [...] MaxPayloadSize (MPS) is not negotiated between devices but is programmed by the system firmware (at least for devices present at boot - the kernel may be responsible in case of hotplug). You can use the kernel parameter 'pci=pcie_bus_perf' (or one of several others) to set a policy that overrides this, but no policy will allow setting MPS above the device's MaxPayloadSizeSupported (MPSS). Ben, Unfortunately I'm using 3.0.x kernel and this is not included in the kernel. So I'm trying to use ethtool modify it from eeprom to see if help or no. Todd, I'll review all MaxPayload for all devices, but need to say if it mismatch, customer could not modify it from BIOS for there was not entry at there, to test it, we have to find how to verify if this is the root cause, so still need to find the offset in eeprom. Thanks in advance, Joe -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
RE: [E1000-devel] 82571EB: Detected Hardware Unit Hang
Thanks for the clarification. I was just going by the PCIe spec, which says the lowest value of both ends is used, and I figured SOMETHING had to be looking at that and doing some sort of negotiation. I'm no BIOS guy, so I'm not sure what's actually going on, whether something walks the PCIe tree or if the BIOS just sets all the values to the minimum. Todd Fujinaka Technical Marketing Engineer LAN Access Division (LAD) Intel Corporation todd.fujin...@intel.com (503) 712-4565 -Original Message- From: Ben Hutchings [mailto:bhutchi...@solarflare.com] Sent: Tuesday, November 27, 2012 10:11 AM To: Fujinaka, Todd; Mary Mcgrath Cc: Joe Jin; net...@vger.kernel.org; e1000-de...@lists.sf.net; linux-kernel@vger.kernel.org; linux-pci Subject: RE: [E1000-devel] 82571EB: Detected Hardware Unit Hang On Tue, 2012-11-27 at 17:32 +, Fujinaka, Todd wrote: > Forgive me if I'm being too repetitious as I think some of this has > been mentioned in the past. > > We (and by we I mean the Ethernet part and driver) can only change the > advertised availability of a larger MaxPayloadSize. The size is > negotiated by both sides of the link when the link is established. The > driver should not change the size of the link as it would be poking at > registers outside of its scope and is controlled by the upstream > bridge (not us). [...] MaxPayloadSize (MPS) is not negotiated between devices but is programmed by the system firmware (at least for devices present at boot - the kernel may be responsible in case of hotplug). You can use the kernel parameter 'pci=pcie_bus_perf' (or one of several others) to set a policy that overrides this, but no policy will allow setting MPS above the device's MaxPayloadSizeSupported (MPSS). (These parameters are not documented in Documentation/kernel-parameters.txt! Someone ought to fix that.) Ben. -- Ben Hutchings, Staff Engineer, Solarflare Not speaking for my employer; that's the marketing department's job. They asked us to note that Solarflare product names are trademarked.
RE: [E1000-devel] 82571EB: Detected Hardware Unit Hang
On Tue, 2012-11-27 at 17:32 +, Fujinaka, Todd wrote: > Forgive me if I'm being too repetitious as I think some of this has > been mentioned in the past. > > We (and by we I mean the Ethernet part and driver) can only change the > advertised availability of a larger MaxPayloadSize. The size is > negotiated by both sides of the link when the link is established. The > driver should not change the size of the link as it would be poking at > registers outside of its scope and is controlled by the upstream > bridge (not us). [...] MaxPayloadSize (MPS) is not negotiated between devices but is programmed by the system firmware (at least for devices present at boot - the kernel may be responsible in case of hotplug). You can use the kernel parameter 'pci=pcie_bus_perf' (or one of several others) to set a policy that overrides this, but no policy will allow setting MPS above the device's MaxPayloadSizeSupported (MPSS). (These parameters are not documented in Documentation/kernel-parameters.txt! Someone ought to fix that.) Ben. -- Ben Hutchings, Staff Engineer, Solarflare Not speaking for my employer; that's the marketing department's job. They asked us to note that Solarflare product names are trademarked. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
RE: [E1000-devel] 82571EB: Detected Hardware Unit Hang
Forgive me if I'm being too repetitious as I think some of this has been mentioned in the past. We (and by we I mean the Ethernet part and driver) can only change the advertised availability of a larger MaxPayloadSize. The size is negotiated by both sides of the link when the link is established. The driver should not change the size of the link as it would be poking at registers outside of its scope and is controlled by the upstream bridge (not us). You also need to check all the PCIe links to get to the device. There can be several to get from the root complex, through bridges, to the endpoint Ethernet controller. The Ethernet part and driver has no control over any other links. You'll have to talk to the motherboard manufacturer about those links. Your original problem appears to be hangs and Tushar asked you to the entire path of PCIe connections from the root complex to the endpoint. Any mismatches in payload can cause hangs and I believe you have had the problem in the past. I'm sure you remember all the lspci commands to list the tree view and to dump all the details from each of the links and I would suggest you do that to check to see that the payload sizes match. What I do is "lspci -tvvv" to see what's connected, then "lspci -s xx:xx.x -vvv" to check the devices on the link. Thanks. Todd Fujinaka Technical Marketing Engineer LAN Access Division (LAD) Intel Corporation todd.fujin...@intel.com (503) 712-4565 -Original Message- From: Mary Mcgrath [mailto:mary.mcgr...@oracle.com] Sent: Monday, November 26, 2012 6:07 PM To: Joe Jin Cc: net...@vger.kernel.org; e1000-de...@lists.sf.net; linux-kernel@vger.kernel.org Subject: Re: [E1000-devel] 82571EB: Detected Hardware Unit Hang Joe Thank you for working this. I would love to find out how they expect a customer to make the modification To "word 0x1A, and see if the 8th bit is 0 or 1, and to change to 0." I have in turn asked the ct for the lspci command on eth3, maybe the incorrect setting is upstream. Again, thank you. Regards Mary -Original Message- From: Joe Jin Sent: Monday, November 26, 2012 8:00 PM To: Fujinaka, Todd Cc: Dave, Tushar N; net...@vger.kernel.org; e1000-de...@lists.sf.net; linux-kernel@vger.kernel.org; Mary Mcgrath Subject: Re: [E1000-devel] 82571EB: Detected Hardware Unit Hang On 11/27/12 00:23, Fujinaka, Todd wrote: > If you look at the previous section, DevCap, you'll see that it's > correctly advertising 256 bytes but the system is negotiating 128 for > the link to the Ethernet controller. Things on the "other" side of the > link are controlled outside of the e1000 driver. > > Tushar's first suggestion was to check the PCIe payload settings in > the entire chain. Have you done that? Mismatches will cause hangs. Hi Todd, So far I had to know how to modify the maxpayload size, since BIOS have not entry to change this, so I had to use ethtool, now I need to get the offset of MaxPayload size in eeprom, I ever tried to find from Intel online document but failed, any idea? Thanks in advance, Joe -- Monitor your physical, virtual and cloud infrastructure from a single web console. Get in-depth insight into apps, servers, databases, vmware, SAP, cloud infrastructure, etc. Download 30-day Free Trial. Pricing starts from $795 for 25 servers or applications! http://p.sf.net/sfu/zoho_dev2dev_nov ___ E1000-devel mailing list e1000-de...@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/e1000-devel To learn more about Intel Ethernet, visit http://communities.intel.com/community/wired -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
RE: [E1000-devel] 82571EB: Detected Hardware Unit Hang
Forgive me if I'm being too repetitious as I think some of this has been mentioned in the past. We (and by we I mean the Ethernet part and driver) can only change the advertised availability of a larger MaxPayloadSize. The size is negotiated by both sides of the link when the link is established. The driver should not change the size of the link as it would be poking at registers outside of its scope and is controlled by the upstream bridge (not us). You also need to check all the PCIe links to get to the device. There can be several to get from the root complex, through bridges, to the endpoint Ethernet controller. The Ethernet part and driver has no control over any other links. You'll have to talk to the motherboard manufacturer about those links. Your original problem appears to be hangs and Tushar asked you to the entire path of PCIe connections from the root complex to the endpoint. Any mismatches in payload can cause hangs and I believe you have had the problem in the past. I'm sure you remember all the lspci commands to list the tree view and to dump all the details from each of the links and I would suggest you do that to check to see that the payload sizes match. What I do is lspci -tvvv to see what's connected, then lspci -s xx:xx.x -vvv to check the devices on the link. Thanks. Todd Fujinaka Technical Marketing Engineer LAN Access Division (LAD) Intel Corporation todd.fujin...@intel.com (503) 712-4565 -Original Message- From: Mary Mcgrath [mailto:mary.mcgr...@oracle.com] Sent: Monday, November 26, 2012 6:07 PM To: Joe Jin Cc: net...@vger.kernel.org; e1000-de...@lists.sf.net; linux-kernel@vger.kernel.org Subject: Re: [E1000-devel] 82571EB: Detected Hardware Unit Hang Joe Thank you for working this. I would love to find out how they expect a customer to make the modification To word 0x1A, and see if the 8th bit is 0 or 1, and to change to 0. I have in turn asked the ct for the lspci command on eth3, maybe the incorrect setting is upstream. Again, thank you. Regards Mary -Original Message- From: Joe Jin Sent: Monday, November 26, 2012 8:00 PM To: Fujinaka, Todd Cc: Dave, Tushar N; net...@vger.kernel.org; e1000-de...@lists.sf.net; linux-kernel@vger.kernel.org; Mary Mcgrath Subject: Re: [E1000-devel] 82571EB: Detected Hardware Unit Hang On 11/27/12 00:23, Fujinaka, Todd wrote: If you look at the previous section, DevCap, you'll see that it's correctly advertising 256 bytes but the system is negotiating 128 for the link to the Ethernet controller. Things on the other side of the link are controlled outside of the e1000 driver. Tushar's first suggestion was to check the PCIe payload settings in the entire chain. Have you done that? Mismatches will cause hangs. Hi Todd, So far I had to know how to modify the maxpayload size, since BIOS have not entry to change this, so I had to use ethtool, now I need to get the offset of MaxPayload size in eeprom, I ever tried to find from Intel online document but failed, any idea? Thanks in advance, Joe -- Monitor your physical, virtual and cloud infrastructure from a single web console. Get in-depth insight into apps, servers, databases, vmware, SAP, cloud infrastructure, etc. Download 30-day Free Trial. Pricing starts from $795 for 25 servers or applications! http://p.sf.net/sfu/zoho_dev2dev_nov ___ E1000-devel mailing list e1000-de...@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/e1000-devel To learn more about Intel#174; Ethernet, visit http://communities.intel.com/community/wired -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
RE: [E1000-devel] 82571EB: Detected Hardware Unit Hang
On Tue, 2012-11-27 at 17:32 +, Fujinaka, Todd wrote: Forgive me if I'm being too repetitious as I think some of this has been mentioned in the past. We (and by we I mean the Ethernet part and driver) can only change the advertised availability of a larger MaxPayloadSize. The size is negotiated by both sides of the link when the link is established. The driver should not change the size of the link as it would be poking at registers outside of its scope and is controlled by the upstream bridge (not us). [...] MaxPayloadSize (MPS) is not negotiated between devices but is programmed by the system firmware (at least for devices present at boot - the kernel may be responsible in case of hotplug). You can use the kernel parameter 'pci=pcie_bus_perf' (or one of several others) to set a policy that overrides this, but no policy will allow setting MPS above the device's MaxPayloadSizeSupported (MPSS). (These parameters are not documented in Documentation/kernel-parameters.txt! Someone ought to fix that.) Ben. -- Ben Hutchings, Staff Engineer, Solarflare Not speaking for my employer; that's the marketing department's job. They asked us to note that Solarflare product names are trademarked. -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
RE: [E1000-devel] 82571EB: Detected Hardware Unit Hang
Thanks for the clarification. I was just going by the PCIe spec, which says the lowest value of both ends is used, and I figured SOMETHING had to be looking at that and doing some sort of negotiation. I'm no BIOS guy, so I'm not sure what's actually going on, whether something walks the PCIe tree or if the BIOS just sets all the values to the minimum. Todd Fujinaka Technical Marketing Engineer LAN Access Division (LAD) Intel Corporation todd.fujin...@intel.com (503) 712-4565 -Original Message- From: Ben Hutchings [mailto:bhutchi...@solarflare.com] Sent: Tuesday, November 27, 2012 10:11 AM To: Fujinaka, Todd; Mary Mcgrath Cc: Joe Jin; net...@vger.kernel.org; e1000-de...@lists.sf.net; linux-kernel@vger.kernel.org; linux-pci Subject: RE: [E1000-devel] 82571EB: Detected Hardware Unit Hang On Tue, 2012-11-27 at 17:32 +, Fujinaka, Todd wrote: Forgive me if I'm being too repetitious as I think some of this has been mentioned in the past. We (and by we I mean the Ethernet part and driver) can only change the advertised availability of a larger MaxPayloadSize. The size is negotiated by both sides of the link when the link is established. The driver should not change the size of the link as it would be poking at registers outside of its scope and is controlled by the upstream bridge (not us). [...] MaxPayloadSize (MPS) is not negotiated between devices but is programmed by the system firmware (at least for devices present at boot - the kernel may be responsible in case of hotplug). You can use the kernel parameter 'pci=pcie_bus_perf' (or one of several others) to set a policy that overrides this, but no policy will allow setting MPS above the device's MaxPayloadSizeSupported (MPSS). (These parameters are not documented in Documentation/kernel-parameters.txt! Someone ought to fix that.) Ben. -- Ben Hutchings, Staff Engineer, Solarflare Not speaking for my employer; that's the marketing department's job. They asked us to note that Solarflare product names are trademarked.
RE: [E1000-devel] 82571EB: Detected Hardware Unit Hang
Joe Thank you for working this. I would love to find out how they expect a customer to make the modification To "word 0x1A, and see if the 8th bit is 0 or 1, and to change to 0." I have in turn asked the ct for the lspci command on eth3, maybe the incorrect setting is upstream. Again, thank you. Regards Mary -Original Message- From: Joe Jin Sent: Monday, November 26, 2012 8:00 PM To: Fujinaka, Todd Cc: Dave, Tushar N; net...@vger.kernel.org; e1000-de...@lists.sf.net; linux-kernel@vger.kernel.org; Mary Mcgrath Subject: Re: [E1000-devel] 82571EB: Detected Hardware Unit Hang On 11/27/12 00:23, Fujinaka, Todd wrote: > If you look at the previous section, DevCap, you'll see that it's > correctly advertising 256 bytes but the system is negotiating 128 for > the link to the Ethernet controller. Things on the "other" side of the > link are controlled outside of the e1000 driver. > > Tushar's first suggestion was to check the PCIe payload settings in > the entire chain. Have you done that? Mismatches will cause hangs. Hi Todd, So far I had to know how to modify the maxpayload size, since BIOS have not entry to change this, so I had to use ethtool, now I need to get the offset of MaxPayload size in eeprom, I ever tried to find from Intel online document but failed, any idea? Thanks in advance, Joe -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [E1000-devel] 82571EB: Detected Hardware Unit Hang
On 11/27/12 00:23, Fujinaka, Todd wrote: > If you look at the previous section, DevCap, you'll see that it's > correctly advertising 256 bytes but the system is negotiating 128 for > the link to the Ethernet controller. Things on the "other" side of the > link are controlled outside of the e1000 driver. > > Tushar's first suggestion was to check the PCIe payload settings in the > entire chain. Have you done that? Mismatches will cause hangs. Hi Todd, So far I had to know how to modify the maxpayload size, since BIOS have not entry to change this, so I had to use ethtool, now I need to get the offset of MaxPayload size in eeprom, I ever tried to find from Intel online document but failed, any idea? Thanks in advance, Joe -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
RE: [E1000-devel] 82571EB: Detected Hardware Unit Hang
On Tue, 20 Nov 2012, Joe Jin wrote: > On 11/20/12 16:59, Dave, Tushar N wrote: >> Have you power off the system completely after modifying eeprom? If not >> please do so. > > Hi Tushar, > > Seems not works for me, would you please help to check what is wrong of my > operations? ... > # lspci -s :52:00.1 -vvv > 52:00.1 Ethernet controller: Intel Corporation 82571EB Gigabit Ethernet > Controller (rev 06) > <--snip--> > Capabilities: [e0] Express (v1) Endpoint, MSI 00 > DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s <512ns, > L1 <64us > ExtTag- AttnBtn- AttnInd- PwrInd- RBE- FLReset- > DevCtl: Report errors: Correctable+ Non-Fatal+ Fatal+ > Unsupported+ > RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+ > MaxPayload 128 bytes, MaxReadReq 4096 bytes > ^ > <--snip--> If you look at the previous section, DevCap, you'll see that it's correctly advertising 256 bytes but the system is negotiating 128 for the link to the Ethernet controller. Things on the "other" side of the link are controlled outside of the e1000 driver. Tushar's first suggestion was to check the PCIe payload settings in the entire chain. Have you done that? Mismatches will cause hangs. Todd Fujinaka Technical Marketing Engineer LAN Access Division (LAD) Intel Corporation todd.fujin...@intel.com (503) 712-4565 -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
RE: [E1000-devel] 82571EB: Detected Hardware Unit Hang
On Tue, 20 Nov 2012, Joe Jin wrote: On 11/20/12 16:59, Dave, Tushar N wrote: Have you power off the system completely after modifying eeprom? If not please do so. Hi Tushar, Seems not works for me, would you please help to check what is wrong of my operations? ... # lspci -s :52:00.1 -vvv 52:00.1 Ethernet controller: Intel Corporation 82571EB Gigabit Ethernet Controller (rev 06) --snip-- Capabilities: [e0] Express (v1) Endpoint, MSI 00 DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s 512ns, L1 64us ExtTag- AttnBtn- AttnInd- PwrInd- RBE- FLReset- DevCtl: Report errors: Correctable+ Non-Fatal+ Fatal+ Unsupported+ RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+ MaxPayload 128 bytes, MaxReadReq 4096 bytes ^ --snip-- If you look at the previous section, DevCap, you'll see that it's correctly advertising 256 bytes but the system is negotiating 128 for the link to the Ethernet controller. Things on the other side of the link are controlled outside of the e1000 driver. Tushar's first suggestion was to check the PCIe payload settings in the entire chain. Have you done that? Mismatches will cause hangs. Todd Fujinaka Technical Marketing Engineer LAN Access Division (LAD) Intel Corporation todd.fujin...@intel.com (503) 712-4565 -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [E1000-devel] 82571EB: Detected Hardware Unit Hang
On 11/27/12 00:23, Fujinaka, Todd wrote: If you look at the previous section, DevCap, you'll see that it's correctly advertising 256 bytes but the system is negotiating 128 for the link to the Ethernet controller. Things on the other side of the link are controlled outside of the e1000 driver. Tushar's first suggestion was to check the PCIe payload settings in the entire chain. Have you done that? Mismatches will cause hangs. Hi Todd, So far I had to know how to modify the maxpayload size, since BIOS have not entry to change this, so I had to use ethtool, now I need to get the offset of MaxPayload size in eeprom, I ever tried to find from Intel online document but failed, any idea? Thanks in advance, Joe -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
RE: [E1000-devel] 82571EB: Detected Hardware Unit Hang
Joe Thank you for working this. I would love to find out how they expect a customer to make the modification To word 0x1A, and see if the 8th bit is 0 or 1, and to change to 0. I have in turn asked the ct for the lspci command on eth3, maybe the incorrect setting is upstream. Again, thank you. Regards Mary -Original Message- From: Joe Jin Sent: Monday, November 26, 2012 8:00 PM To: Fujinaka, Todd Cc: Dave, Tushar N; net...@vger.kernel.org; e1000-de...@lists.sf.net; linux-kernel@vger.kernel.org; Mary Mcgrath Subject: Re: [E1000-devel] 82571EB: Detected Hardware Unit Hang On 11/27/12 00:23, Fujinaka, Todd wrote: If you look at the previous section, DevCap, you'll see that it's correctly advertising 256 bytes but the system is negotiating 128 for the link to the Ethernet controller. Things on the other side of the link are controlled outside of the e1000 driver. Tushar's first suggestion was to check the PCIe payload settings in the entire chain. Have you done that? Mismatches will cause hangs. Hi Todd, So far I had to know how to modify the maxpayload size, since BIOS have not entry to change this, so I had to use ethtool, now I need to get the offset of MaxPayload size in eeprom, I ever tried to find from Intel online document but failed, any idea? Thanks in advance, Joe -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
