RE: /sys/module/pcie_aspm/parameters/policy not writable?
> -Original Message- > From: Pavel Machek [mailto:pa...@ucw.cz] > Sent: Thursday, August 01, 2013 5:40 PM > To: Wyborny, Carolyn > Cc: Bjorn Helgaas; Greg KH; kernel list; Joe Lawrence; Myron Stowe; Kirsher, > Jeffrey T; Brandeburg, Jesse; Allan, Bruce W; Skidmore, Donald C; Rose, > Gregory > V; Waskiewicz Jr, Peter P; Duyck, Alexander H; Ronciak, John; Dave, Tushar N; > e1000-de...@lists.sourceforge.net > Subject: Re: /sys/module/pcie_aspm/parameters/policy not writable? > > Hi! > > > > > If there's patch, I'll gladly try it :-). Thanks, > > > > > > > Pavel > > > Thanks Pavel, > > > > Minor fix. Missed a bracket removal Please use this for your testing > > instead. > > > > Seems to work here. (I did testing on 3.11-rc, but that should not change > things.) > Latencies are in expected range. > > Tested-by: Pavel Machek Thanks Pavel, I'll submit a version of this to our internal queue. Carolyn -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: /sys/module/pcie_aspm/parameters/policy not writable?
On Thu, Aug 1, 2013 at 8:13 PM, Pavel Machek wrote: > Hmm. So I've update bios using 7buj28uc.iso . Reverted the patches and > yes, ping latencies are still bad: Wow. I can hardly believe how bad this is (assuming Windows has the same problem). Thanks a lot for checking this out. >> >> Carolyn's patch will likely work, at least most of the time, but I >> >> think there's a small possibility that it could cause a conflict >> >> between the BIOS and the OS over ASPM control, so I'm not 100% in >> >> support of that approach. A conflict may not happen on your >> >> machine, >> > >> > Can we base it on DMI whitelist? >> >> I don't think we can know a priori whether a machine (even your >> machine) is susceptible to a conflict. But if Carolyn forcibly > > We don't apriori now how broken machines are, true. There are 1000 > ways BIOS can break things. And yes, it will be us breaking the specs > here. But "useful machine with OS breaking specs" is better than > "machine useless for ssh". > > _If_ there's a conflict, we can try something else. > > (Has someone really seen a conflict, or is it just theoretical thing?) Completely theoretical, as far as I know. I don't even know what a conflict would look like. Maybe some unexpected ASPM enable/disable from SMM during suspend/resume or something. Things like that would likely go unnoticed anyway. Bjorn -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: /sys/module/pcie_aspm/parameters/policy not writable?
Hi! > >> > But since the problem also occurs with Windows, it's pretty likely > >> > that there's a BIOS update to fix it. I notice on the X60 support > >> > page that there are several versions newer than what you're running. > >> > >> Do you have any interest in trying a newer BIOS to see if it's fixed > >> there? If not, I understand; BIOS updates are a hassle at best. > >> You're running BIOS 2.14, and it looks like the current BIOS for an > >> X60 1709 7HU is 2.19 (from http://support.lenovo.com). > > > > I'm lost at the lenovo pages :-(. And frankly I'd prefer not to touch > > the BIOS. > > I tried to provide a complete link, but the web site isn't amenable to > that. I found it by selecting the "Drivers & Software" section, > entering "X60" as the machine type, selecting "ThinkPad X60 (1709)" > and going to the "BIOS" section. I was surprised how easy it was :) Hmm. So I've update bios using 7buj28uc.iso . Reverted the patches and yes, ping latencies are still bad: pavel@amd:~$ ping 10.0.0.250 PING 10.0.0.250 (10.0.0.250) 56(84) bytes of data. 64 bytes from 10.0.0.250: icmp_req=1 ttl=127 time=1.05 ms 64 bytes from 10.0.0.250: icmp_req=2 ttl=127 time=0.820 ms 64 bytes from 10.0.0.250: icmp_req=3 ttl=127 time=73.5 ms 64 bytes from 10.0.0.250: icmp_req=4 ttl=127 time=7.41 ms 64 bytes from 10.0.0.250: icmp_req=5 ttl=127 time=5.25 ms 64 bytes from 10.0.0.250: icmp_req=6 ttl=127 time=1.16 ms 64 bytes from 10.0.0.250: icmp_req=7 ttl=127 time=36.3 ms 64 bytes from 10.0.0.250: icmp_req=8 ttl=127 time=1.64 ms > >> Carolyn's patch will likely work, at least most of the time, but I > >> think there's a small possibility that it could cause a conflict > >> between the BIOS and the OS over ASPM control, so I'm not 100% in > >> support of that approach. A conflict may not happen on your > >> machine, > > > > Can we base it on DMI whitelist? > > I don't think we can know a priori whether a machine (even your > machine) is susceptible to a conflict. But if Carolyn forcibly We don't apriori now how broken machines are, true. There are 1000 ways BIOS can break things. And yes, it will be us breaking the specs here. But "useful machine with OS breaking specs" is better than "machine useless for ssh". _If_ there's a conflict, we can try something else. (Has someone really seen a conflict, or is it just theoretical thing?) Pavel -- (english) http://www.livejournal.com/~pavelmachek (cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: /sys/module/pcie_aspm/parameters/policy not writable?
Hi! > >> > But since the problem also occurs with Windows, it's pretty likely > >> > that there's a BIOS update to fix it. I notice on the X60 support > >> > page that there are several versions newer than what you're running. > >> > >> Do you have any interest in trying a newer BIOS to see if it's fixed > >> there? If not, I understand; BIOS updates are a hassle at best. > >> You're running BIOS 2.14, and it looks like the current BIOS for an > >> X60 1709 7HU is 2.19 (from http://support.lenovo.com). > > > > I'm lost at the lenovo pages :-(. And frankly I'd prefer not to touch > > the BIOS. > > I tried to provide a complete link, but the web site isn't amenable to > that. I found it by selecting the "Drivers & Software" section, > entering "X60" as the machine type, selecting "ThinkPad X60 (1709)" > and going to the "BIOS" section. I was surprised how easy it was :) Ok, got it, the web pages are horible. > >> Carolyn's patch will likely work, at least most of the time, but I > >> think there's a small possibility that it could cause a conflict > >> between the BIOS and the OS over ASPM control, so I'm not 100% in > >> support of that approach. A conflict may not happen on your > >> machine, > > > > Can we base it on DMI whitelist? > > I don't think we can know a priori whether a machine (even your > machine) is susceptible to a conflict. But if Carolyn forcibly > disables it in the driver, she can structure that with a whitelist or > whatever she thinks is best. Well, there's only one way to find out: try it. I was running with the "enable ASPM so we can disable it" one-liner for a while, and seen no ill effects. Anyway, if you don't hear from me again, it means that I have a brick (and not a notebook) and am desperately trying to get replacement X60. Will try to flash it in few minutes. Pavel -- (english) http://www.livejournal.com/~pavelmachek (cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: /sys/module/pcie_aspm/parameters/policy not writable?
Hi! > > > If there's patch, I'll gladly try it :-). Thanks, > > > > > Pavel > > Thanks Pavel, > > Minor fix. Missed a bracket removal Please use this for your testing > instead. > Seems to work here. (I did testing on 3.11-rc, but that should not change things.) Latencies are in expected range. Tested-by: Pavel Machek Thanks, Pavel -- (english) http://www.livejournal.com/~pavelmachek (cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: /sys/module/pcie_aspm/parameters/policy not writable?
On Thu, Aug 1, 2013 at 2:00 PM, Pavel Machek wrote: > Hi! > >> > But since the problem also occurs with Windows, it's pretty likely >> > that there's a BIOS update to fix it. I notice on the X60 support >> > page that there are several versions newer than what you're running. >> >> Do you have any interest in trying a newer BIOS to see if it's fixed >> there? If not, I understand; BIOS updates are a hassle at best. >> You're running BIOS 2.14, and it looks like the current BIOS for an >> X60 1709 7HU is 2.19 (from http://support.lenovo.com). > > I'm lost at the lenovo pages :-(. And frankly I'd prefer not to touch > the BIOS. I tried to provide a complete link, but the web site isn't amenable to that. I found it by selecting the "Drivers & Software" section, entering "X60" as the machine type, selecting "ThinkPad X60 (1709)" and going to the "BIOS" section. I was surprised how easy it was :) >> Carolyn's patch will likely work, at least most of the time, but I >> think there's a small possibility that it could cause a conflict >> between the BIOS and the OS over ASPM control, so I'm not 100% in >> support of that approach. A conflict may not happen on your >> machine, > > Can we base it on DMI whitelist? I don't think we can know a priori whether a machine (even your machine) is susceptible to a conflict. But if Carolyn forcibly disables it in the driver, she can structure that with a whitelist or whatever she thinks is best. Bjorn -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: /sys/module/pcie_aspm/parameters/policy not writable?
Hi! > > But since the problem also occurs with Windows, it's pretty likely > > that there's a BIOS update to fix it. I notice on the X60 support > > page that there are several versions newer than what you're running. > > Do you have any interest in trying a newer BIOS to see if it's fixed > there? If not, I understand; BIOS updates are a hassle at best. > You're running BIOS 2.14, and it looks like the current BIOS for an > X60 1709 7HU is 2.19 (from http://support.lenovo.com). I'm lost at the lenovo pages :-(. And frankly I'd prefer not to touch the BIOS. > Carolyn's patch will likely work, at least most of the time, but I > think there's a small possibility that it could cause a conflict > between the BIOS and the OS over ASPM control, so I'm not 100% in > support of that approach. A conflict may not happen on your > machine, Can we base it on DMI whitelist? Pavel -- (english) http://www.livejournal.com/~pavelmachek (cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: /sys/module/pcie_aspm/parameters/policy not writable?
On Thu, Aug 1, 2013 at 8:57 AM, Wyborny, Carolyn wrote: >> -Original Message- >> From: Bjorn Helgaas [mailto:bhelg...@google.com] >> Sent: Wednesday, July 31, 2013 4:53 PM >> To: Pavel Machek >> Cc: Greg KH; kernel list; Joe Lawrence; Myron Stowe; Kirsher, Jeffrey T; >> Brandeburg, Jesse; Allan, Bruce W; Wyborny, Carolyn; Skidmore, Donald C; >> Rose, Gregory V; Waskiewicz Jr, Peter P; Duyck, Alexander H; Ronciak, John; >> Dave, Tushar N; e1000-de...@lists.sourceforge.net >> Subject: Re: /sys/module/pcie_aspm/parameters/policy not writable? >> >> On Fri, Jul 19, 2013 at 11:46 AM, Bjorn Helgaas wrote: >> >> > Just to make sure I understand you correctly: I think you are saying >> > that the NIC has the same problem under Windows. >> >> Can you confirm or deny that the same problem occurs with Windows? > > In our testing here we saw the same problem with Windows. >> >> > But since the problem also occurs with Windows, it's pretty likely >> > that there's a BIOS update to fix it. I notice on the X60 support >> > page that there are several versions newer than what you're running. >> >> Do you have any interest in trying a newer BIOS to see if it's fixed there? >> If not, I >> understand; BIOS updates are a hassle at best. >> You're running BIOS 2.14, and it looks like the current BIOS for an >> X60 1709 7HU is 2.19 (from http://support.lenovo.com). >> >> Carolyn's patch will likely work, at least most of the time, but I think >> there's a >> small possibility that it could cause a conflict between the BIOS and the OS >> over >> ASPM control, so I'm not 100% in support of that approach. A conflict may >> not >> happen on your machine, and not on my machine, but it may happen >> somewhere, and if it does happen, it's likely to be rare and difficult to >> debug. >> > I've proposed a patch for Pavel to test. I understand your position, but do > you have any other proposal? We have a severe hw problem we are trying > alleviate as best we can. I think the safest route is to change the BIOS so it either leaves ASPM disabled, or enables it but grants control to the OS. It seems way too risky in general for the BIOS to enable ASPM and tell the OS "you can't change this." There's no way the BIOS can know about device errata that may make ASPM unsafe. If the same problem happens on Windows, I can hardly believe the BIOS would pass Windows logo testing or that customers would tolerate it, so I would think vendors would step up and update the BIOS. This is a built-in device, for goodness sake! It's not like this is some random plug-in device that the vendor might not have tested. At a minimum, I think it would be good if the driver logged a warning about "ASPM enabled and device may not work; check for BIOS update" so future issues would be easier to debug. For the reason above, I'm not comfortable forcing the disable in the PCI core. It *might* make sense to forcibly disable ASPM in the driver if you're willing to take the risk of conflicts, but I think you should also log a warning in that case so you have a clue if a conflict does turn up somewhere. Bjorn -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
RE: /sys/module/pcie_aspm/parameters/policy not writable?
> -Original Message-
> From: Wyborny, Carolyn
> Sent: Thursday, August 01, 2013 7:56 AM
> To: 'Pavel Machek'
> Cc: Bjorn Helgaas; Greg KH; kernel list; Joe Lawrence; Myron Stowe; Kirsher,
> Jeffrey T; Brandeburg, Jesse; Allan, Bruce W; Skidmore, Donald C; Rose,
> Gregory
> V; Waskiewicz Jr, Peter P; Duyck, Alexander H; Ronciak, John; Dave, Tushar N;
> e1000-de...@lists.sourceforge.net
> Subject: RE: /sys/module/pcie_aspm/parameters/policy not writable?
>
> [..]
> > If there's patch, I'll gladly try it :-). Thanks,
> >
> Pavel
> Thanks Pavel,
Minor fix. Missed a bracket removal Please use this for your testing instead.
Thanks,
Carolyn
>
> It ended up taking more time than I thought. The checking of whether ASPM is
> really enabled or not is, unfortunately, not as straightforward as I thought
> initially, so we ended up rewriting the function a bit. In this patch we try
> to use
> the pci_disable_link_state_locked() call, but if it fails, we then write to
> pci config
> space of the device to disable it.
>
> Please let me know if this actually disables the ASPM for your 82574 parts or
> not. We can still continue the discussion on whether this is the best
> approach or
> not, or what else could be done.
>
> This patch attempts to work around a problem found with some systems where
> the call to pci_diable_link_state_locked() fails. As a result, ASPM is not,
> in fact,
> disabled. Changing disable aspm code to check if state actually is disabled
> after
> the call and, if not, try another way to disable it.
>
> Signed-off-by: Carolyn Wyborny
> ---
>
> drivers/net/ethernet/intel/e1000e/netdev.c | 86 --
> --
> 1 files changed, 60 insertions(+), 26 deletions(-)
>
> diff --git a/drivers/net/ethernet/intel/e1000e/netdev.c
> b/drivers/net/ethernet/intel/e1000e/netdev.c
> index e6d2c0f..bc3025b 100644
> --- a/drivers/net/ethernet/intel/e1000e/netdev.c
> +++ b/drivers/net/ethernet/intel/e1000e/netdev.c
> @@ -64,8 +64,6 @@ static int debug = -1; module_param(debug, int, 0);
> MODULE_PARM_DESC(debug, "Debug level (0=none,...,16=all)");
>
> -static void e1000e_disable_aspm(struct pci_dev *pdev, u16 state);
> -
> static const struct e1000_info *e1000_info_tbl[] = {
> [board_82571] = &e1000_82571_info,
> [board_82572] = &e1000_82572_info,
> @@ -6019,38 +6017,74 @@ static int __e1000_shutdown(struct pci_dev *pdev,
> bool runtime)
> return 0;
> }
>
> -#ifdef CONFIG_PCIEASPM
> -static void __e1000e_disable_aspm(struct pci_dev *pdev, u16 state)
> +/**
> + * e1000e_disable_aspm - Disable ASPM states
> + * @pdev: pointer to PCI device struct
> + * @state: bit-mask of ASPM states to disable
> + *
> + * Some devices *must* have certain ASPM states disabled per hardware
> errata.
> + **/
> +static void e1000e_disable_aspm(struct pci_dev *pdev, u16 state)
> {
> + struct pci_dev *parent = pdev->bus->self;
> + u16 aspm_dis_mask = 0;
> + u16 pdev_aspmc, parent_aspmc;
> +
> + switch (state) {
> + case PCIE_LINK_STATE_L0S:
> + case PCIE_LINK_STATE_L0S + PCIE_LINK_STATE_L1:
> + aspm_dis_mask |= PCI_EXP_LNKCTL_ASPM_L0S;
> + /* fall-through - can't have L1 without L0s */
> + case PCIE_LINK_STATE_L1:
> + aspm_dis_mask |= PCI_EXP_LNKCTL_ASPM_L1;
> + break;
> + default:
> + return;
> + }
> +
> + pcie_capability_read_word(pdev, PCI_EXP_LNKCTL, &pdev_aspmc);
> + pdev_aspmc &= PCI_EXP_LNKCTL_ASPMC;
> +
> + if (parent) {
> + pcie_capability_read_word(parent, PCI_EXP_LNKCTL,
> + &parent_aspmc);
> + parent_aspmc &= PCI_EXP_LNKCTL_ASPMC;
> + }
> +
> + /* Nothing to do if the ASPM states to be disabled already are */
> + if (!(pdev_aspmc & aspm_dis_mask) &&
> + (!parent || !(parent_aspmc & aspm_dis_mask)))
> + return;
> +
> + dev_info(&pdev->dev, "Disabling ASPM %s %s\n",
> + (aspm_dis_mask & pdev_aspmc &
> PCI_EXP_LNKCTL_ASPM_L0S) ?
> + "L0s" : "",
> + (aspm_dis_mask & pdev_aspmc & PCI_EXP_LNKCTL_ASPM_L1)
> ?
> + "L1" : "");
> +
> +#ifdef CONFIG_PCIEASPM
> pci_disable_link_state_locked(pdev, state); -} -#else -static void
> __e1000e_disable_aspm(struct pci_dev *pdev, u16 state) -{
> - u16 aspm_ctl = 0;
RE: /sys/module/pcie_aspm/parameters/policy not writable?
> -Original Message- > From: Bjorn Helgaas [mailto:bhelg...@google.com] > Sent: Wednesday, July 31, 2013 4:53 PM > To: Pavel Machek > Cc: Greg KH; kernel list; Joe Lawrence; Myron Stowe; Kirsher, Jeffrey T; > Brandeburg, Jesse; Allan, Bruce W; Wyborny, Carolyn; Skidmore, Donald C; > Rose, Gregory V; Waskiewicz Jr, Peter P; Duyck, Alexander H; Ronciak, John; > Dave, Tushar N; e1000-de...@lists.sourceforge.net > Subject: Re: /sys/module/pcie_aspm/parameters/policy not writable? > > On Fri, Jul 19, 2013 at 11:46 AM, Bjorn Helgaas wrote: > > > Just to make sure I understand you correctly: I think you are saying > > that the NIC has the same problem under Windows. > > Can you confirm or deny that the same problem occurs with Windows? In our testing here we saw the same problem with Windows. > > > But since the problem also occurs with Windows, it's pretty likely > > that there's a BIOS update to fix it. I notice on the X60 support > > page that there are several versions newer than what you're running. > > Do you have any interest in trying a newer BIOS to see if it's fixed there? > If not, I > understand; BIOS updates are a hassle at best. > You're running BIOS 2.14, and it looks like the current BIOS for an > X60 1709 7HU is 2.19 (from http://support.lenovo.com). > > Carolyn's patch will likely work, at least most of the time, but I think > there's a > small possibility that it could cause a conflict between the BIOS and the OS > over > ASPM control, so I'm not 100% in support of that approach. A conflict may not > happen on your machine, and not on my machine, but it may happen > somewhere, and if it does happen, it's likely to be rare and difficult to > debug. > I've proposed a patch for Pavel to test. I understand your position, but do you have any other proposal? We have a severe hw problem we are trying alleviate as best we can. Pavel, were you also able to try the updated BIOS to see if that resolved the issue without my patch? Thanks, Carolyn -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
RE: /sys/module/pcie_aspm/parameters/policy not writable?
[..]
> If there's patch, I'll gladly try it :-). Thanks,
>
Pavel
Thanks Pavel,
It ended up taking more time than I thought. The checking of whether ASPM is
really enabled or not is, unfortunately, not as straightforward as I thought
initially, so we ended up rewriting the function a bit. In this patch we try
to use the pci_disable_link_state_locked() call, but if it fails, we then write
to pci config space of the device to disable it.
Please let me know if this actually disables the ASPM for your 82574 parts or
not. We can still continue the discussion on whether this is the best approach
or not, or what else could be done.
This patch attempts to work around a problem found with some systems where the
call to pci_diable_link_state_locked() fails. As a result, ASPM is not, in
fact, disabled. Changing disable aspm code to check if state actually is
disabled after the call and, if not, try another way to disable it.
Signed-off-by: Carolyn Wyborny
---
drivers/net/ethernet/intel/e1000e/netdev.c | 86
1 files changed, 60 insertions(+), 26 deletions(-)
diff --git a/drivers/net/ethernet/intel/e1000e/netdev.c
b/drivers/net/ethernet/intel/e1000e/netdev.c
index e6d2c0f..bc3025b 100644
--- a/drivers/net/ethernet/intel/e1000e/netdev.c
+++ b/drivers/net/ethernet/intel/e1000e/netdev.c
@@ -64,8 +64,6 @@ static int debug = -1; module_param(debug, int, 0);
MODULE_PARM_DESC(debug, "Debug level (0=none,...,16=all)");
-static void e1000e_disable_aspm(struct pci_dev *pdev, u16 state);
-
static const struct e1000_info *e1000_info_tbl[] = {
[board_82571] = &e1000_82571_info,
[board_82572] = &e1000_82572_info,
@@ -6019,38 +6017,74 @@ static int __e1000_shutdown(struct pci_dev *pdev, bool
runtime)
return 0;
}
-#ifdef CONFIG_PCIEASPM
-static void __e1000e_disable_aspm(struct pci_dev *pdev, u16 state)
+/**
+ * e1000e_disable_aspm - Disable ASPM states
+ * @pdev: pointer to PCI device struct
+ * @state: bit-mask of ASPM states to disable
+ *
+ * Some devices *must* have certain ASPM states disabled per hardware errata.
+ **/
+static void e1000e_disable_aspm(struct pci_dev *pdev, u16 state)
{
+ struct pci_dev *parent = pdev->bus->self;
+ u16 aspm_dis_mask = 0;
+ u16 pdev_aspmc, parent_aspmc;
+
+ switch (state) {
+ case PCIE_LINK_STATE_L0S:
+ case PCIE_LINK_STATE_L0S + PCIE_LINK_STATE_L1:
+ aspm_dis_mask |= PCI_EXP_LNKCTL_ASPM_L0S;
+ /* fall-through - can't have L1 without L0s */
+ case PCIE_LINK_STATE_L1:
+ aspm_dis_mask |= PCI_EXP_LNKCTL_ASPM_L1;
+ break;
+ default:
+ return;
+ }
+
+ pcie_capability_read_word(pdev, PCI_EXP_LNKCTL, &pdev_aspmc);
+ pdev_aspmc &= PCI_EXP_LNKCTL_ASPMC;
+
+ if (parent) {
+ pcie_capability_read_word(parent, PCI_EXP_LNKCTL,
+ &parent_aspmc);
+ parent_aspmc &= PCI_EXP_LNKCTL_ASPMC;
+ }
+
+ /* Nothing to do if the ASPM states to be disabled already are */
+ if (!(pdev_aspmc & aspm_dis_mask) &&
+ (!parent || !(parent_aspmc & aspm_dis_mask)))
+ return;
+
+ dev_info(&pdev->dev, "Disabling ASPM %s %s\n",
+(aspm_dis_mask & pdev_aspmc & PCI_EXP_LNKCTL_ASPM_L0S) ?
+"L0s" : "",
+(aspm_dis_mask & pdev_aspmc & PCI_EXP_LNKCTL_ASPM_L1) ?
+"L1" : "");
+
+#ifdef CONFIG_PCIEASPM
pci_disable_link_state_locked(pdev, state); -} -#else -static void
__e1000e_disable_aspm(struct pci_dev *pdev, u16 state) -{
- u16 aspm_ctl = 0;
- if (state & PCIE_LINK_STATE_L0S)
- aspm_ctl |= PCI_EXP_LNKCTL_ASPM_L0S;
- if (state & PCIE_LINK_STATE_L1)
- aspm_ctl |= PCI_EXP_LNKCTL_ASPM_L1;
+ /* Double-check ASPM control. If not disabled by the above, the
+* BIOS is preventing that from happening (or CONFIG_PCIEASPM is
+* not enabled); override by writing PCI config space directly.
+*/
+ pcie_capability_read_word(pdev, PCI_EXP_LNKCTL, &pdev_aspmc);
+ pdev_aspmc &= PCI_EXP_LNKCTL_ASPMC;
+
+ if (!(aspm_dis_mask & pdev_aspmc))
+ return;
+ }
+#endif
/* Both device and parent should have the same ASPM setting.
* Disable ASPM in downstream component first and then upstream.
*/
- pcie_capability_clear_word(pdev, PCI_EXP_LNKCTL, aspm_ctl);
-
- if (pdev->bus->self)
- pcie_capability_clear_word(pdev->bus->self, PCI_EXP_LNKCTL,
- aspm_ctl);
-}
-#endif
-static void e1000e_disable_aspm(struct pci_dev *pdev, u16 state) -{
- dev_info(&pdev->dev, "Disabling ASPM %s %s\n",
-(state & PCIE_LINK_STATE_L0S) ? "L0s" : "",
-(state &
Re: /sys/module/pcie_aspm/parameters/policy not writable?
On Wed, 2013-07-10 at 13:57 -0600, Bjorn Helgaas wrote:
> [+cc Jeff, Jesse, et al, e1000-devel]
>
> Holy cow, you guys have a lot of folks listed in MAINTAINERS for Intel
> drivers :) This is an ASPM question, if that helps narrow down the
> folks interested.
Bruce Allan is the e1000e maintainer, I have trimmed down the
recipients. If you simply add e1000-devel mailing list, that will get
to all of us and we can respond accordingly.
I need to put together a patch to remove PJ because he is no longer in
our group.
>
> On Wed, Jul 10, 2013 at 7:29 AM, Pavel Machek wrote:
> > Hi!
> >
> >> >> But:
> >> >> 1) it should not list unavailable options
> >> >>
> >> >> 2) operation not permitted seems like wrong error code for
> >> >> operation not supported.
> >> >
> >> > So I forcibly enabled ASPM, and now ping latencies are in normal
> >> > range... no matter how I set
> >> > /sys/module/pcie_aspm/parameters/policy. Strange.
> >> >
> >> > Any ideas what correct solution is?
> >> >
> >> > Pavel
> >> > Signed-off-by: Pavel Machek
> >> > (but don't apply)
> > ...
> >> > diff --git a/drivers/pci/pci-acpi.c b/drivers/pci/pci-acpi.c
> >> > index e4b1fb2..9a1b63e 100644
> >> > --- a/drivers/pci/pci-acpi.c
> >> > +++ b/drivers/pci/pci-acpi.c
> >> > @@ -382,7 +382,7 @@ static int __init acpi_pci_init(void)
> >> >
> >> > if (acpi_gbl_FADT.boot_flags & ACPI_FADT_NO_ASPM) {
> >> > printk(KERN_INFO"ACPI FADT declares the system doesn't
> >> > support PCIe ASPM, so disable it\n");
> >> > - pcie_no_aspm();
> >> > +// pcie_no_aspm();
> >> > }
> >> >
> >> > ret = register_acpi_bus_type(&acpi_pci_bus);
> >>
> >> Hi Pavel,
> >>
> >> Interesting. Can you collect dmesg and "lspci -vvv" output for both
> >> cases (high ping latency and normal ping latency)?
> >
> > Will do. Results are in attachment (200KB...)
> >
> >> Also, how much
> >> difference does this make in ping latency?
> >
> > The ping latency goes from 100msec range to <2msec.
> >
> >> If ASPM is enabled for a
> >> device, e.g., your NIC, the link may be put in a low power state when
> >> the device is idle. It takes time to exit that low power state, of
> >> course, but I would expect that time to be in the microsecond time and
> >> probably not observable via ping.
> >
> > I'd hope so. 100msec ping makes ssh unpleasant to use.
>
> Pavel's ThinkPad X60 has two NICs: Intel 82573L and Intel PRO/Wireless
> 3945ABG. I'm pretty sure the problem he's reporting is with the
> 82573L. Ping times are bad (~100msec) when ASPM is enabled, as
> reported by lspci.
>
> On Pavel's system, the FADT says we shouldn't enable OSPM control of
> ASPM (ACPI_FADT_NO_ASPM is set), so we set "aspm_disabled = 1". One
> effect is that we don't blacklist the pre-1.1 82573L device, which I
> think results in it being left with the BIOS configuration, which
> apparently has ASPM enabled. (Pavel, could you confirm the BIOS
> config, e.g., with "pci=earlydump"?)
>
> e1000e claims to disable ASPM, but because aspm_disabled is set, the
> driver's call to pci_disable_link_state_locked() actually does nothing
> [1].
>
> I experimented [2] with Windows and found that when a driver requests
> PciASPMOptOut, Windows will not touch ASPM config if the _OSC method
> fails, i.e., the BIOS declines to grant ASPM control to the OS.
> However, I do not know if Windows similarly ignores PciASPMOptOut when
> the FADT ACPI_FADT_NO_ASPM bit is set.
>
> The PCI core has failed spectacularly at providing useful ASPM
> interfaces. Do you Intel folks have any suggestions about how to
> resolve this? I assume that the Windows driver for the 82573L must
> disable ASPM somehow, even though ACPI_FADT_NO_ASPM is set. Does it
> just use brute-force, as in the version of __e1000e_disable_aspm()
> that's used when CONFIG_PCIEASPM is not set?
>
> Bjorn
>
> [1] We just merged 2add0ec1, which adds a "can't disable ASPM; OS
> doesn't have ASPM control" message in this case, but I don't think
> Pavel's kernel has this change. It doesn't change the behavior
> anyway.
>
> [2] https://bugzilla.kernel.org/show_bug.cgi?id=57331
signature.asc
Description: This is a digitally signed message part
Re: /sys/module/pcie_aspm/parameters/policy not writable?
On Fri, Jul 19, 2013 at 11:46 AM, Bjorn Helgaas wrote: > Just to make sure I understand you correctly: I think you are saying > that the NIC has the same problem under Windows. Can you confirm or deny that the same problem occurs with Windows? > But since the problem also occurs with Windows, it's pretty likely > that there's a BIOS update to fix it. I notice on the X60 support > page that there are several versions newer than what you're running. Do you have any interest in trying a newer BIOS to see if it's fixed there? If not, I understand; BIOS updates are a hassle at best. You're running BIOS 2.14, and it looks like the current BIOS for an X60 1709 7HU is 2.19 (from http://support.lenovo.com). Carolyn's patch will likely work, at least most of the time, but I think there's a small possibility that it could cause a conflict between the BIOS and the OS over ASPM control, so I'm not 100% in support of that approach. A conflict may not happen on your machine, and not on my machine, but it may happen somewhere, and if it does happen, it's likely to be rare and difficult to debug. Bjorn -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: /sys/module/pcie_aspm/parameters/policy not writable?
On Wed 2013-07-24 15:19:55, Wyborny, Carolyn wrote: > > -Original Message- > > From: Bjorn Helgaas [mailto:bhelg...@google.com] > [..} > > > > We *could* consider something like this, since its scope is limited. > > But since the problem also occurs with Windows, it's pretty likely that > > there's a > > BIOS update to fix it. I notice on the X60 support page that there are > > several > > versions newer than what you're running. > > > > Bjorn > > I believe that some BIOS don't allow user control on this feature, thus we > end up unable to disable it from driverspace or from userspace, with the > method we use today in this scenario. > > I have a patch almost ready that attempts to disable using the > pci_disable_link_state() call but then checks again to see if its still > enabled and if not, then changes the pci config space on the devices on > either side of the pcie link. I plan to send it to this list for Pavel's > testing as soon as I have it finished, at worst end of week. > If there's patch, I'll gladly try it :-). Thanks, Pavel -- (english) http://www.livejournal.com/~pavelmachek (cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
RE: /sys/module/pcie_aspm/parameters/policy not writable?
> -Original Message- > From: Bjorn Helgaas [mailto:bhelg...@google.com] [..} > > We *could* consider something like this, since its scope is limited. > But since the problem also occurs with Windows, it's pretty likely that > there's a > BIOS update to fix it. I notice on the X60 support page that there are > several > versions newer than what you're running. > > Bjorn I believe that some BIOS don't allow user control on this feature, thus we end up unable to disable it from driverspace or from userspace, with the method we use today in this scenario. I have a patch almost ready that attempts to disable using the pci_disable_link_state() call but then checks again to see if its still enabled and if not, then changes the pci config space on the devices on either side of the pcie link. I plan to send it to this list for Pavel's testing as soon as I have it finished, at worst end of week. Thanks, Carolyn Carolyn Wyborny Linux Development Networking Division Intel Corporation -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: /sys/module/pcie_aspm/parameters/policy not writable?
On Fri, Jul 12, 2013 at 5:11 AM, Pavel Machek wrote: >> Pavel's ThinkPad X60 has two NICs: Intel 82573L and Intel PRO/Wireless >> 3945ABG. I'm pretty sure the problem he's reporting is with the >> 82573L. Ping times are bad (~100msec) when ASPM is enabled, as >> reported by lspci. > > Yep. Wired ethernet has the problems. > >> On Pavel's system, the FADT says we shouldn't enable OSPM control of >> ASPM (ACPI_FADT_NO_ASPM is set), so we set "aspm_disabled = 1". One >> effect is that we don't blacklist the pre-1.1 82573L device, which I >> think results in it being left with the BIOS configuration, which >> apparently has ASPM enabled. (Pavel, could you confirm the BIOS >> config, e.g., with "pci=earlydump"?) > > Dump sent in another mail. Thanks for the dmesg log. It confirms that the BIOS left ASPM L1 enabled on your 82573L device. The PCIe cap is at 0xe0 per previous lspci output, and the Link Control register at 0xf0 contains 0x0142, which means ASPM L1, Common Clock, and Clock PM are enabled. > Ok, so we get copy of Windows, including the problem :-(. Just to make sure I understand you correctly: I think you are saying that the NIC has the same problem under Windows. > Ok, so we have BIOS on x60 that sets up hardware in a way that does > not work... and then tells us we must not do ASPM, so we don't fix it. > > One option would be "always disable ASPM, even if BIOS tells us it is > not supported". Yes. We considered this route, but it didn't seem safe in general. The problem is that the BIOS isn't actually telling us that ASPM is not supported. Rather, it is telling us that we "must not enable OSPM ASPM control". That means we can't touch ASPM control at all, whether to enable or disable. We have to assume the BIOS itself is managing ASPM, and if the OS tries to manage ASPM it will cause conflicts. > Other option would be "add explicit blacklist for x60, disable ASPM > there.". We *could* consider something like this, since its scope is limited. But since the problem also occurs with Windows, it's pretty likely that there's a BIOS update to fix it. I notice on the X60 support page that there are several versions newer than what you're running. Bjorn -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: /sys/module/pcie_aspm/parameters/policy not writable?
Hi! > > > Pavel's ThinkPad X60 has two NICs: Intel 82573L and Intel PRO/Wireless > > > 3945ABG. I'm pretty sure the problem he's reporting is with the 82573L. > > > Ping > > > times are bad (~100msec) when ASPM is enabled, as reported by lspci. > > > > > > On Pavel's system, the FADT says we shouldn't enable OSPM control of ASPM > > > (ACPI_FADT_NO_ASPM is set), so we set "aspm_disabled = 1". One effect is > > > that > > > we don't blacklist the pre-1.1 82573L device, which I think results in it > > > being left > > > with the BIOS configuration, which apparently has ASPM enabled. (Pavel, > > > could > > > you confirm the BIOS config, e.g., with "pci=earlydump"?) > > > > > > e1000e claims to disable ASPM, but because aspm_disabled is set, the > > > driver's > > > call to pci_disable_link_state_locked() actually does nothing [1]. > > > > Yes, this is the problem we run into. It would help if the call to > > pci_disable_link_state_locked() returned an error if ASPM is not disabled > > as requested so that drivers can then do the brute force disabling of it > > themselves. > > I considered returning an error, but resisted because I think drivers > will just handle the error by doing the brute-force disable themselves, > and then we might as well drop the pci_disable_link_state() interface > completely. What is that? Yes, if function does not perform what it was asked to do, it should return an error. Anything else is antisocial. If drivers will attempt to do something _more_ antisocial (like doing PCI bit banging) we can surely catch it during review. [Actually, the functions seem to have "force" parameter. Should we just let e1000e/iwlwifi set the force? They _know_ hardware will not work properly with the ASPM.] > I proposed a patch [3] a while ago that made pci_disable_link_state() > turn off ASPM unconditionally. That would have the same effect as > returning failure and having drivers disable ASPM themselves. But > Rafael and Matthew thought it was too risky [4] (and I think they're > probably right because it does not match the Windows behavior). Well, I believe "patch might be risky" is trumped by "this ethernet card is not usable". > So by extension, I guess it would also be risky to return an error and > have the driver disable ASPM. It does not seem like clean conclusion to me. Doing force-disabling on selected hardware that needs it (x60+e1000e) is certainly less risky than changing behaviour for all the 1000 notebooks out there. So yes, you should be returning error. At the very least, e1000e and iwlwifi could warn the user loudly and maybe even disable the interface. Pavel -- (english) http://www.livejournal.com/~pavelmachek (cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: /sys/module/pcie_aspm/parameters/policy not writable?
Hi! > [+cc Jeff, Jesse, et al, e1000-devel] > > Holy cow, you guys have a lot of folks listed in MAINTAINERS for Intel > drivers :) This is an ASPM question, if that helps narrow down the > folks interested. > >> If ASPM is enabled for a > >> device, e.g., your NIC, the link may be put in a low power state when > >> the device is idle. It takes time to exit that low power state, of > >> course, but I would expect that time to be in the microsecond time and > >> probably not observable via ping. > > > > I'd hope so. 100msec ping makes ssh unpleasant to use. > > Pavel's ThinkPad X60 has two NICs: Intel 82573L and Intel PRO/Wireless > 3945ABG. I'm pretty sure the problem he's reporting is with the > 82573L. Ping times are bad (~100msec) when ASPM is enabled, as > reported by lspci. Yep. Wired ethernet has the problems. > On Pavel's system, the FADT says we shouldn't enable OSPM control of > ASPM (ACPI_FADT_NO_ASPM is set), so we set "aspm_disabled = 1". One > effect is that we don't blacklist the pre-1.1 82573L device, which I > think results in it being left with the BIOS configuration, which > apparently has ASPM enabled. (Pavel, could you confirm the BIOS > config, e.g., with "pci=earlydump"?) Dump sent in another mail. > e1000e claims to disable ASPM, but because aspm_disabled is set, the > driver's call to pci_disable_link_state_locked() actually does nothing > [1]. > > I experimented [2] with Windows and found that when a driver requests > PciASPMOptOut, Windows will not touch ASPM config if the _OSC method > fails, i.e., the BIOS declines to grant ASPM control to the OS. > However, I do not know if Windows similarly ignores PciASPMOptOut when > the FADT ACPI_FADT_NO_ASPM bit is set. Ok, so we get copy of Windows, including the problem :-(. > [1] We just merged 2add0ec1, which adds a "can't disable ASPM; OS > doesn't have ASPM control" message in this case, but I don't think > Pavel's kernel has this change. It doesn't change the behavior > anyway. Oops. Ok, so we have BIOS on x60 that sets up hardware in a way that does not work... and then tells us we must not do ASPM, so we don't fix it. One option would be "always disable ASPM, even if BIOS tells us it is not supported". Other option would be "add explicit blacklist for x60, disable ASPM there.". Pavel -- (english) http://www.livejournal.com/~pavelmachek (cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
RE: /sys/module/pcie_aspm/parameters/policy not writable?
> -Original Message- > From: Bjorn Helgaas [mailto:bhelg...@google.com] > Sent: Wednesday, July 10, 2013 3:57 PM > To: Wyborny, Carolyn > Cc: Pavel Machek; Greg KH; kernel list; Joe Lawrence; Myron Stowe; Kirsher, > Jeffrey T; Brandeburg, Jesse; Allan, Bruce W; Skidmore, Donald C; Rose, > Gregory > V; Waskiewicz Jr, Peter P; Duyck, Alexander H; Ronciak, John; Dave, Tushar N; > e1000-de...@lists.sourceforge.net; linux-...@vger.kernel.org > Subject: Re: /sys/module/pcie_aspm/parameters/policy not writable? > > [+cc linux-pci] > > On Wed, Jul 10, 2013 at 10:21:32PM +, Wyborny, Carolyn wrote: > > > -Original Message- > > > From: Bjorn Helgaas [mailto:bhelg...@google.com] > > [..] > > > > > Pavel's ThinkPad X60 has two NICs: Intel 82573L and Intel > > > PRO/Wireless 3945ABG. I'm pretty sure the problem he's reporting is > > > with the 82573L. Ping times are bad (~100msec) when ASPM is enabled, as > reported by lspci. > > > > > > On Pavel's system, the FADT says we shouldn't enable OSPM control of > > > ASPM (ACPI_FADT_NO_ASPM is set), so we set "aspm_disabled = 1". One > > > effect is that we don't blacklist the pre-1.1 82573L device, which I > > > think results in it being left with the BIOS configuration, which > > > apparently has ASPM enabled. (Pavel, could you confirm the BIOS > > > config, e.g., with "pci=earlydump"?) > > > > > > e1000e claims to disable ASPM, but because aspm_disabled is set, the > > > driver's call to pci_disable_link_state_locked() actually does nothing > > > [1]. > > > > Yes, this is the problem we run into. It would help if the call to > pci_disable_link_state_locked() returned an error if ASPM is not disabled as > requested so that drivers can then do the brute force disabling of it > themselves. > > I considered returning an error, but resisted because I think drivers will > just > handle the error by doing the brute-force disable themselves, and then we > might > as well drop the pci_disable_link_state() interface completely. > > I proposed a patch [3] a while ago that made pci_disable_link_state() turn off > ASPM unconditionally. That would have the same effect as returning failure > and > having drivers disable ASPM themselves. But Rafael and Matthew thought it was > too risky [4] (and I think they're probably right because it does not match > the > Windows behavior). > > So by extension, I guess it would also be risky to return an error and have > the > driver disable ASPM. > > > > I experimented [2] with Windows and found that when a driver > > > requests PciASPMOptOut, Windows will not touch ASPM config if the > > > _OSC method fails, i.e., the BIOS declines to grant ASPM control to the > > > OS. > > > However, I do not know if Windows similarly ignores PciASPMOptOut > > > when the FADT ACPI_FADT_NO_ASPM bit is set. > > > > > > The PCI core has failed spectacularly at providing useful ASPM > > > interfaces. Do you Intel folks have any suggestions about how to > > > resolve this? I assume that the Windows driver for the 82573L must > > > disable ASPM somehow, even though ACPI_FADT_NO_ASPM is set. Does it > > > just use brute-force, as in the version of > > > __e1000e_disable_aspm() that's used when CONFIG_PCIEASPM is not set? > > > > My friends in our Windows development team told me that the driver doesn't > try to disable ASPM basically because we can't. I'm not sure if the same > issue > presents in Windows the same way or not. > > So the Windows driver *never* disables ASPM, not even with its own register > writes? So on a machine like Pavel's, it would run with ASPM enabled? (I'm > assuming his BIOS leaves ASPM enabled; hopefully Pavel can confirm that. > > If the Windows driver works with ASPM enabled but the Linux driver on the same > hardware requires ASPM to be disabled, it sounds like the Linux driver just > needs > to be fixed. So, to clarify, Windows *does* have a problem with these parts if ASPM cannot be disabled. We tell users to disable ASPM with these parts. There are systems that have BIOS that do not truly disable ASPM even if the user tries, even with Windows and the symptoms are as bad as Linux, there's no big difference there. The difference is that Windows doesn't interact with the BIOS very much and the Windows driver cannot access PCIconfig space as we can with Linux. I would argue for the error message so that drivers are not brute-fo
Re: /sys/module/pcie_aspm/parameters/policy not writable?
[+cc linux-pci] On Wed, Jul 10, 2013 at 10:21:32PM +, Wyborny, Carolyn wrote: > > -Original Message- > > From: Bjorn Helgaas [mailto:bhelg...@google.com] > [..] > > > Pavel's ThinkPad X60 has two NICs: Intel 82573L and Intel PRO/Wireless > > 3945ABG. I'm pretty sure the problem he's reporting is with the 82573L. > > Ping > > times are bad (~100msec) when ASPM is enabled, as reported by lspci. > > > > On Pavel's system, the FADT says we shouldn't enable OSPM control of ASPM > > (ACPI_FADT_NO_ASPM is set), so we set "aspm_disabled = 1". One effect is > > that > > we don't blacklist the pre-1.1 82573L device, which I think results in it > > being left > > with the BIOS configuration, which apparently has ASPM enabled. (Pavel, > > could > > you confirm the BIOS config, e.g., with "pci=earlydump"?) > > > > e1000e claims to disable ASPM, but because aspm_disabled is set, the > > driver's > > call to pci_disable_link_state_locked() actually does nothing [1]. > > Yes, this is the problem we run into. It would help if the call to > pci_disable_link_state_locked() returned an error if ASPM is not disabled as > requested so that drivers can then do the brute force disabling of it > themselves. I considered returning an error, but resisted because I think drivers will just handle the error by doing the brute-force disable themselves, and then we might as well drop the pci_disable_link_state() interface completely. I proposed a patch [3] a while ago that made pci_disable_link_state() turn off ASPM unconditionally. That would have the same effect as returning failure and having drivers disable ASPM themselves. But Rafael and Matthew thought it was too risky [4] (and I think they're probably right because it does not match the Windows behavior). So by extension, I guess it would also be risky to return an error and have the driver disable ASPM. > > I experimented [2] with Windows and found that when a driver requests > > PciASPMOptOut, Windows will not touch ASPM config if the _OSC method fails, > > i.e., the BIOS declines to grant ASPM control to the OS. > > However, I do not know if Windows similarly ignores PciASPMOptOut when the > > FADT ACPI_FADT_NO_ASPM bit is set. > > > > The PCI core has failed spectacularly at providing useful ASPM interfaces. > > Do > > you Intel folks have any suggestions about how to resolve this? I assume > > that > > the Windows driver for the 82573L must disable ASPM somehow, even though > > ACPI_FADT_NO_ASPM is set. Does it just use brute-force, as in the version > > of > > __e1000e_disable_aspm() that's used when CONFIG_PCIEASPM is not set? > > My friends in our Windows development team told me that the driver doesn't > try to disable ASPM basically because we can't. I'm not sure if the same > issue presents in Windows the same way or not. So the Windows driver *never* disables ASPM, not even with its own register writes? So on a machine like Pavel's, it would run with ASPM enabled? (I'm assuming his BIOS leaves ASPM enabled; hopefully Pavel can confirm that.) If the Windows driver works with ASPM enabled but the Linux driver on the same hardware requires ASPM to be disabled, it sounds like the Linux driver just needs to be fixed. [3] https://lkml.kernel.org/r/20130510225257.ga10...@google.com [4] https://lkml.kernel.org/r/20130516225535.ga27...@google.com -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
RE: /sys/module/pcie_aspm/parameters/policy not writable?
> -Original Message- > From: Bjorn Helgaas [mailto:bhelg...@google.com] [..] > Holy cow, you guys have a lot of folks listed in MAINTAINERS for Intel > drivers :) > This is an ASPM question, if that helps narrow down the folks interested. Well, we try to have everyone involved, I guess. I work on the server parts in the e1000e driver which includes 82573/4 and 82583. [..] > Pavel's ThinkPad X60 has two NICs: Intel 82573L and Intel PRO/Wireless > 3945ABG. I'm pretty sure the problem he's reporting is with the 82573L. Ping > times are bad (~100msec) when ASPM is enabled, as reported by lspci. > > On Pavel's system, the FADT says we shouldn't enable OSPM control of ASPM > (ACPI_FADT_NO_ASPM is set), so we set "aspm_disabled = 1". One effect is that > we don't blacklist the pre-1.1 82573L device, which I think results in it > being left > with the BIOS configuration, which apparently has ASPM enabled. (Pavel, could > you confirm the BIOS config, e.g., with "pci=earlydump"?) > > e1000e claims to disable ASPM, but because aspm_disabled is set, the driver's > call to pci_disable_link_state_locked() actually does nothing [1]. Yes, this is the problem we run into. It would help if the call to pci_disable_link_state_locked() returned an error if ASPM is not disabled as requested so that drivers can then do the brute force disabling of it themselves. > > I experimented [2] with Windows and found that when a driver requests > PciASPMOptOut, Windows will not touch ASPM config if the _OSC method fails, > i.e., the BIOS declines to grant ASPM control to the OS. > However, I do not know if Windows similarly ignores PciASPMOptOut when the > FADT ACPI_FADT_NO_ASPM bit is set. > > The PCI core has failed spectacularly at providing useful ASPM interfaces. Do > you Intel folks have any suggestions about how to resolve this? I assume that > the Windows driver for the 82573L must disable ASPM somehow, even though > ACPI_FADT_NO_ASPM is set. Does it just use brute-force, as in the version of > __e1000e_disable_aspm() that's used when CONFIG_PCIEASPM is not set? My friends in our Windows development team told me that the driver doesn't try to disable ASPM basically because we can't. I'm not sure if the same issue presents in Windows the same way or not. Hope this helps. Carolyn Carolyn Wyborny Linux Development Networking Division Intel Corporation > > Bjorn > > [1] We just merged 2add0ec1, which adds a "can't disable ASPM; OS doesn't > have ASPM control" message in this case, but I don't think Pavel's kernel has > this > change. It doesn't change the behavior anyway. > > [2] https://bugzilla.kernel.org/show_bug.cgi?id=57331 -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: /sys/module/pcie_aspm/parameters/policy not writable?
[+cc Jeff, Jesse, et al, e1000-devel]
Holy cow, you guys have a lot of folks listed in MAINTAINERS for Intel
drivers :) This is an ASPM question, if that helps narrow down the
folks interested.
On Wed, Jul 10, 2013 at 7:29 AM, Pavel Machek wrote:
> Hi!
>
>> >> But:
>> >> 1) it should not list unavailable options
>> >>
>> >> 2) operation not permitted seems like wrong error code for
>> >> operation not supported.
>> >
>> > So I forcibly enabled ASPM, and now ping latencies are in normal
>> > range... no matter how I set
>> > /sys/module/pcie_aspm/parameters/policy. Strange.
>> >
>> > Any ideas what correct solution is?
>> >
>> > Pavel
>> > Signed-off-by: Pavel Machek
>> > (but don't apply)
> ...
>> > diff --git a/drivers/pci/pci-acpi.c b/drivers/pci/pci-acpi.c
>> > index e4b1fb2..9a1b63e 100644
>> > --- a/drivers/pci/pci-acpi.c
>> > +++ b/drivers/pci/pci-acpi.c
>> > @@ -382,7 +382,7 @@ static int __init acpi_pci_init(void)
>> >
>> > if (acpi_gbl_FADT.boot_flags & ACPI_FADT_NO_ASPM) {
>> > printk(KERN_INFO"ACPI FADT declares the system doesn't
>> > support PCIe ASPM, so disable it\n");
>> > - pcie_no_aspm();
>> > +// pcie_no_aspm();
>> > }
>> >
>> > ret = register_acpi_bus_type(&acpi_pci_bus);
>>
>> Hi Pavel,
>>
>> Interesting. Can you collect dmesg and "lspci -vvv" output for both
>> cases (high ping latency and normal ping latency)?
>
> Will do. Results are in attachment (200KB...)
>
>> Also, how much
>> difference does this make in ping latency?
>
> The ping latency goes from 100msec range to <2msec.
>
>> If ASPM is enabled for a
>> device, e.g., your NIC, the link may be put in a low power state when
>> the device is idle. It takes time to exit that low power state, of
>> course, but I would expect that time to be in the microsecond time and
>> probably not observable via ping.
>
> I'd hope so. 100msec ping makes ssh unpleasant to use.
Pavel's ThinkPad X60 has two NICs: Intel 82573L and Intel PRO/Wireless
3945ABG. I'm pretty sure the problem he's reporting is with the
82573L. Ping times are bad (~100msec) when ASPM is enabled, as
reported by lspci.
On Pavel's system, the FADT says we shouldn't enable OSPM control of
ASPM (ACPI_FADT_NO_ASPM is set), so we set "aspm_disabled = 1". One
effect is that we don't blacklist the pre-1.1 82573L device, which I
think results in it being left with the BIOS configuration, which
apparently has ASPM enabled. (Pavel, could you confirm the BIOS
config, e.g., with "pci=earlydump"?)
e1000e claims to disable ASPM, but because aspm_disabled is set, the
driver's call to pci_disable_link_state_locked() actually does nothing
[1].
I experimented [2] with Windows and found that when a driver requests
PciASPMOptOut, Windows will not touch ASPM config if the _OSC method
fails, i.e., the BIOS declines to grant ASPM control to the OS.
However, I do not know if Windows similarly ignores PciASPMOptOut when
the FADT ACPI_FADT_NO_ASPM bit is set.
The PCI core has failed spectacularly at providing useful ASPM
interfaces. Do you Intel folks have any suggestions about how to
resolve this? I assume that the Windows driver for the 82573L must
disable ASPM somehow, even though ACPI_FADT_NO_ASPM is set. Does it
just use brute-force, as in the version of __e1000e_disable_aspm()
that's used when CONFIG_PCIEASPM is not set?
Bjorn
[1] We just merged 2add0ec1, which adds a "can't disable ASPM; OS
doesn't have ASPM control" message in this case, but I don't think
Pavel's kernel has this change. It doesn't change the behavior
anyway.
[2] https://bugzilla.kernel.org/show_bug.cgi?id=57331
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: /sys/module/pcie_aspm/parameters/policy not writable?
Hi!
> >> But:
> >> 1) it should not list unavailable options
> >>
> >> 2) operation not permitted seems like wrong error code for
> >> operation not supported.
> >
> > So I forcibly enabled ASPM, and now ping latencies are in normal
> > range... no matter how I set
> > /sys/module/pcie_aspm/parameters/policy. Strange.
> >
> > Any ideas what correct solution is?
> >
> > Pavel
> > Signed-off-by: Pavel Machek
> > (but don't apply)
...
> > diff --git a/drivers/pci/pci-acpi.c b/drivers/pci/pci-acpi.c
> > index e4b1fb2..9a1b63e 100644
> > --- a/drivers/pci/pci-acpi.c
> > +++ b/drivers/pci/pci-acpi.c
> > @@ -382,7 +382,7 @@ static int __init acpi_pci_init(void)
> >
> > if (acpi_gbl_FADT.boot_flags & ACPI_FADT_NO_ASPM) {
> > printk(KERN_INFO"ACPI FADT declares the system doesn't
> > support PCIe ASPM, so disable it\n");
> > - pcie_no_aspm();
> > +// pcie_no_aspm();
> > }
> >
> > ret = register_acpi_bus_type(&acpi_pci_bus);
>
> Hi Pavel,
>
> Interesting. Can you collect dmesg and "lspci -vvv" output for both
> cases (high ping latency and normal ping latency)?
Will do. Results are in attachment (200KB...)
> Also, how much
> difference does this make in ping latency?
The ping latency goes from 100msec range to <2msec.
> If ASPM is enabled for a
> device, e.g., your NIC, the link may be put in a low power state when
> the device is idle. It takes time to exit that low power state, of
> course, but I would expect that time to be in the microsecond time and
> probably not observable via ping.
I'd hope so. 100msec ping makes ssh unpleasant to use.
Thanks,
Pavel
--
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures)
http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
delme.gz
Description: Binary data
Re: /sys/module/pcie_aspm/parameters/policy not writable?
On 07/09/2013 03:49 AM, Pavel Machek wrote: On Mon 2013-07-08 21:13:21, Greg KH wrote: On Tue, Jul 09, 2013 at 03:26:11AM +0200, Pavel Machek wrote: Hi! My thinkpad has rather high ping latencies... and perhaps it is due to PCIE ASPM. Why would that be the problem? The odds that the PCIE bus is the issue seems strange to me. Aha: I guess that's why the file is not writable: pavel@amd:~$ dmesg | grep -i aspm ACPI FADT declares the system doesn't support PCIe ASPM, so disable it IIRC, this message is somewhat misleading. When that FADT flag is set by the BIOS, the kernel doesn't so much disable ASPM as disable the kernel's control over ASPM. I believe this was to match Windows behavior. e1000e :02:00.0: Disabling ASPM L0s L1 And given that, I think this message may also be misleading, as the kernel won't touch the device's ASPM state. Force-enabling ASPM may actually be allowing the driver to disable ASPM on the device. I seem to recall a recent thread on this about another device.. maybe we need to allow drivers to explicitly disable ASPM if it's enabled even if the FADT flag is set? pavel@amd:~$ cat /sys/module/pcie_aspm/parameters/policy [default] performance powersave pavel@amd:~$ root@amd:~# echo -n performance > /sys/module/pcie_aspm/parameters/policy -su: echo: write error: Operation not permitted root@amd:~# But: 1) it should not list unavailable options 2) operation not permitted seems like wrong error code for operation not supported. Pavel -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: /sys/module/pcie_aspm/parameters/policy not writable?
Hi! > > > > My thinkpad has rather high ping latencies... and perhaps it is due to > > > > PCIE ASPM. > > > > > > Why would that be the problem? The odds that the PCIE bus is the issue > > > seems strange to me. > > > > Aha: I guess that's why the file is not writable: > > > > pavel@amd:~$ dmesg | grep -i aspm > > ACPI FADT declares the system doesn't support PCIe ASPM, so disable it > > e1000e :02:00.0: Disabling ASPM L0s L1 > > pavel@amd:~$ cat /sys/module/pcie_aspm/parameters/policy > > [default] performance powersave > > pavel@amd:~$ > > root@amd:~# echo -n performance > > > /sys/module/pcie_aspm/parameters/policy > > -su: echo: write error: Operation not permitted > > root@amd:~# > > > > But: > > 1) it should not list unavailable options > > It's a module parameter, you can't control if they are present or not > dynamically. > > > 2) operation not permitted seems like wrong error code for > > operation not supported. > > Then what should it be? I'd vote for EOPNOTSUPP /* Operation not supported on transport endpoint */ ... its certainly closer. EIO would also be an option. You are right we do not have really suitable errno. Operation not permitted is bad because it made me think that kernel denied the request for some reason, not that hardware does not support it. Thanks, Pavel -- (english) http://www.livejournal.com/~pavelmachek (cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: /sys/module/pcie_aspm/parameters/policy not writable?
On Tue, Jul 09, 2013 at 11:49:06AM +0200, Pavel Machek wrote: > On Mon 2013-07-08 21:13:21, Greg KH wrote: > > On Tue, Jul 09, 2013 at 03:26:11AM +0200, Pavel Machek wrote: > > > Hi! > > > > > > My thinkpad has rather high ping latencies... and perhaps it is due to > > > PCIE ASPM. > > > > Why would that be the problem? The odds that the PCIE bus is the issue > > seems strange to me. > > Aha: I guess that's why the file is not writable: > > pavel@amd:~$ dmesg | grep -i aspm > ACPI FADT declares the system doesn't support PCIe ASPM, so disable it > e1000e :02:00.0: Disabling ASPM L0s L1 > pavel@amd:~$ cat /sys/module/pcie_aspm/parameters/policy > [default] performance powersave > pavel@amd:~$ > root@amd:~# echo -n performance > > /sys/module/pcie_aspm/parameters/policy > -su: echo: write error: Operation not permitted > root@amd:~# > > But: > 1) it should not list unavailable options It's a module parameter, you can't control if they are present or not dynamically. > 2) operation not permitted seems like wrong error code for > operation not supported. Then what should it be? greg k-h -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: /sys/module/pcie_aspm/parameters/policy not writable?
On Tue, Jul 9, 2013 at 4:10 AM, Pavel Machek wrote:
> Hi!
>> > > My thinkpad has rather high ping latencies... and perhaps it is due to
>> > > PCIE ASPM.
>> >
>> > Why would that be the problem? The odds that the PCIE bus is the issue
>> > seems strange to me.
>>
>> Aha: I guess that's why the file is not writable:
>>
>> pavel@amd:~$ dmesg | grep -i aspm
>> ACPI FADT declares the system doesn't support PCIe ASPM, so disable it
>> e1000e :02:00.0: Disabling ASPM L0s L1
>> pavel@amd:~$ cat /sys/module/pcie_aspm/parameters/policy
>> [default] performance powersave
>> pavel@amd:~$
>> root@amd:~# echo -n performance >
>> /sys/module/pcie_aspm/parameters/policy
>> -su: echo: write error: Operation not permitted
>> root@amd:~#
>>
>> But:
>> 1) it should not list unavailable options
>>
>> 2) operation not permitted seems like wrong error code for
>> operation not supported.
>
> So I forcibly enabled ASPM, and now ping latencies are in normal
> range... no matter how I set
> /sys/module/pcie_aspm/parameters/policy. Strange.
>
> Any ideas what correct solution is?
> Pavel
> Signed-off-by: Pavel Machek
> (but don't apply)
>
> diff --git a/.config b/.config
> index 149f713..d7f5a11 100644
> --- a/.config
> +++ b/.config
> @@ -1,6 +1,6 @@
> #
> # Automatically generated file; DO NOT EDIT.
> -# Linux/x86 3.10.0-rc2 Kernel Configuration
> +# Linux/x86 3.10.0 Kernel Configuration
> #
> # CONFIG_64BIT is not set
> CONFIG_X86_32=y
> @@ -559,9 +559,9 @@ CONFIG_PCIEAER=y
> # CONFIG_PCIEAER_INJECT is not set
> CONFIG_PCIEASPM=y
> CONFIG_PCIEASPM_DEBUG=y
> -CONFIG_PCIEASPM_DEFAULT=y
> +# CONFIG_PCIEASPM_DEFAULT is not set
> # CONFIG_PCIEASPM_POWERSAVE is not set
> -# CONFIG_PCIEASPM_PERFORMANCE is not set
> +CONFIG_PCIEASPM_PERFORMANCE=y
> CONFIG_PCIE_PME=y
> CONFIG_ARCH_SUPPORTS_MSI=y
> # CONFIG_PCI_MSI is not set
> @@ -1340,7 +1340,10 @@ CONFIG_MD_RAID1=y
> # CONFIG_MD_RAID456 is not set
> # CONFIG_MD_MULTIPATH is not set
> # CONFIG_MD_FAULTY is not set
> -# CONFIG_BCACHE is not set
> +CONFIG_BCACHE=y
> +# CONFIG_BCACHE_DEBUG is not set
> +# CONFIG_BCACHE_EDEBUG is not set
> +# CONFIG_BCACHE_CLOSURES_DEBUG is not set
> CONFIG_BLK_DEV_DM=y
> # CONFIG_DM_DEBUG is not set
> CONFIG_DM_CRYPT=y
> diff --git a/drivers/pci/pci-acpi.c b/drivers/pci/pci-acpi.c
> index e4b1fb2..9a1b63e 100644
> --- a/drivers/pci/pci-acpi.c
> +++ b/drivers/pci/pci-acpi.c
> @@ -382,7 +382,7 @@ static int __init acpi_pci_init(void)
>
> if (acpi_gbl_FADT.boot_flags & ACPI_FADT_NO_ASPM) {
> printk(KERN_INFO"ACPI FADT declares the system doesn't
> support PCIe ASPM, so disable it\n");
> - pcie_no_aspm();
> +// pcie_no_aspm();
> }
>
> ret = register_acpi_bus_type(&acpi_pci_bus);
Hi Pavel,
Interesting. Can you collect dmesg and "lspci -vvv" output for both
cases (high ping latency and normal ping latency)? Also, how much
difference does this make in ping latency? If ASPM is enabled for a
device, e.g., your NIC, the link may be put in a low power state when
the device is idle. It takes time to exit that low power state, of
course, but I would expect that time to be in the microsecond time and
probably not observable via ping.
Bjorn
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: /sys/module/pcie_aspm/parameters/policy not writable?
Hi!
> > > My thinkpad has rather high ping latencies... and perhaps it is due to
> > > PCIE ASPM.
> >
> > Why would that be the problem? The odds that the PCIE bus is the issue
> > seems strange to me.
>
> Aha: I guess that's why the file is not writable:
>
> pavel@amd:~$ dmesg | grep -i aspm
> ACPI FADT declares the system doesn't support PCIe ASPM, so disable it
> e1000e :02:00.0: Disabling ASPM L0s L1
> pavel@amd:~$ cat /sys/module/pcie_aspm/parameters/policy
> [default] performance powersave
> pavel@amd:~$
> root@amd:~# echo -n performance >
> /sys/module/pcie_aspm/parameters/policy
> -su: echo: write error: Operation not permitted
> root@amd:~#
>
> But:
> 1) it should not list unavailable options
>
> 2) operation not permitted seems like wrong error code for
> operation not supported.
So I forcibly enabled ASPM, and now ping latencies are in normal
range... no matter how I set
/sys/module/pcie_aspm/parameters/policy. Strange.
Any ideas what correct solution is?
Pavel
Signed-off-by: Pavel Machek
(but don't apply)
diff --git a/.config b/.config
index 149f713..d7f5a11 100644
--- a/.config
+++ b/.config
@@ -1,6 +1,6 @@
#
# Automatically generated file; DO NOT EDIT.
-# Linux/x86 3.10.0-rc2 Kernel Configuration
+# Linux/x86 3.10.0 Kernel Configuration
#
# CONFIG_64BIT is not set
CONFIG_X86_32=y
@@ -559,9 +559,9 @@ CONFIG_PCIEAER=y
# CONFIG_PCIEAER_INJECT is not set
CONFIG_PCIEASPM=y
CONFIG_PCIEASPM_DEBUG=y
-CONFIG_PCIEASPM_DEFAULT=y
+# CONFIG_PCIEASPM_DEFAULT is not set
# CONFIG_PCIEASPM_POWERSAVE is not set
-# CONFIG_PCIEASPM_PERFORMANCE is not set
+CONFIG_PCIEASPM_PERFORMANCE=y
CONFIG_PCIE_PME=y
CONFIG_ARCH_SUPPORTS_MSI=y
# CONFIG_PCI_MSI is not set
@@ -1340,7 +1340,10 @@ CONFIG_MD_RAID1=y
# CONFIG_MD_RAID456 is not set
# CONFIG_MD_MULTIPATH is not set
# CONFIG_MD_FAULTY is not set
-# CONFIG_BCACHE is not set
+CONFIG_BCACHE=y
+# CONFIG_BCACHE_DEBUG is not set
+# CONFIG_BCACHE_EDEBUG is not set
+# CONFIG_BCACHE_CLOSURES_DEBUG is not set
CONFIG_BLK_DEV_DM=y
# CONFIG_DM_DEBUG is not set
CONFIG_DM_CRYPT=y
diff --git a/drivers/pci/pci-acpi.c b/drivers/pci/pci-acpi.c
index e4b1fb2..9a1b63e 100644
--- a/drivers/pci/pci-acpi.c
+++ b/drivers/pci/pci-acpi.c
@@ -382,7 +382,7 @@ static int __init acpi_pci_init(void)
if (acpi_gbl_FADT.boot_flags & ACPI_FADT_NO_ASPM) {
printk(KERN_INFO"ACPI FADT declares the system doesn't support
PCIe ASPM, so disable it\n");
- pcie_no_aspm();
+// pcie_no_aspm();
}
ret = register_acpi_bus_type(&acpi_pci_bus);
--
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures)
http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: /sys/module/pcie_aspm/parameters/policy not writable?
On Mon 2013-07-08 21:13:21, Greg KH wrote: > On Tue, Jul 09, 2013 at 03:26:11AM +0200, Pavel Machek wrote: > > Hi! > > > > My thinkpad has rather high ping latencies... and perhaps it is due to > > PCIE ASPM. > > Why would that be the problem? The odds that the PCIE bus is the issue > seems strange to me. Aha: I guess that's why the file is not writable: pavel@amd:~$ dmesg | grep -i aspm ACPI FADT declares the system doesn't support PCIe ASPM, so disable it e1000e :02:00.0: Disabling ASPM L0s L1 pavel@amd:~$ cat /sys/module/pcie_aspm/parameters/policy [default] performance powersave pavel@amd:~$ root@amd:~# echo -n performance > /sys/module/pcie_aspm/parameters/policy -su: echo: write error: Operation not permitted root@amd:~# But: 1) it should not list unavailable options 2) operation not permitted seems like wrong error code for operation not supported. Pavel -- (english) http://www.livejournal.com/~pavelmachek (cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: /sys/module/pcie_aspm/parameters/policy not writable?
Hi! > > My thinkpad has rather high ping latencies... and perhaps it is due to > > PCIE ASPM. > > Why would that be the problem? The odds that the PCIE bus is the issue > seems strange to me. Well, I google a bit. Apparently, the issue is common on x60/t60 machine and was never fixed properly. And indeed some patches point in that direction, in particular: https://launchpadlibrarian.net/17463623/e1000e-0.4.1.7.diff (It is from comment #27 here: https://bugs.launchpad.net/ubuntu/+source/linux-source-2.6.22/+bug/42572 ). If you have any idea what else it could be... Pavel -- (english) http://www.livejournal.com/~pavelmachek (cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: /sys/module/pcie_aspm/parameters/policy not writable?
On Tue, Jul 09, 2013 at 03:26:11AM +0200, Pavel Machek wrote: > Hi! > > My thinkpad has rather high ping latencies... and perhaps it is due to > PCIE ASPM. Why would that be the problem? The odds that the PCIE bus is the issue seems strange to me. greg k-h -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
/sys/module/pcie_aspm/parameters/policy not writable?
Hi! My thinkpad has rather high ping latencies... and perhaps it is due to PCIE ASPM. Its help text says: CONFIG_PCIEASPM: This enables OS control over PCI Express ASPM (Active State Power Management) and Clock Power Management. ASPM supports state L0/L0s/L1. ... ASPM can be disabled or enabled at runtime via /sys/module/pcie_aspm/parameters/policy So I tried setting the parameter, but it seems to be broken :-( root@amd:/data/pavel# cat /sys/module/pcie_aspm/parameters/policy [default] performance powersave root@amd:/data/pavel# echo performance > /sys/module/pcie_aspm/parameters/policy bash: echo: write error: Operation not permitted root@amd:/data/pavel# echo -n performance > /sys/module/pcie_aspm/parameters/policy bash: echo: write error: Operation not permitted root@amd:/data/pavel# ls -al /sys/module/pcie_aspm/parameters/policy -rw-r--r-- 1 root root 4096 Jul 9 03:16 /sys/module/pcie_aspm/parameters/policy root@amd:/data/pavel# cat /sys/module/pcie_aspm/parameters/policy [default] performance powersave root@amd:/data/pavel# echo powersave > /sys/module/pcie_aspm/parameters/policy bash: echo: write error: Operation not permitted root@amd:/data/pavel# echo -n powersave > /sys/module/pcie_aspm/parameters/policy bash: echo: write error: Operation not permitted root@amd:/data/pavel# echo -n default > /sys/module/pcie_aspm/parameters/policy bash: echo: write error: Operation not permitted Hmm: CONFIG_PCIEASPM_DEFAULT=y # CONFIG_PCIEASPM_POWERSAVE is not set # CONFIG_PCIEASPM_PERFORMANCE is not set Should we avoid displaying options that can't be selected? Thanks, Pavel -- (english) http://www.livejournal.com/~pavelmachek (cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
