Re: [PATCH 03/13] sparc/PCI: Reserve legacy mmio after PCI mmio

2017-05-03 Thread Bjorn Helgaas 
On Thu, Apr 20, 2017 at 10:04:50PM -0700, Yinghai Lu wrote:
> On one system found bunch of claim resource fail from pci device.
> pci_sun4v f02b894c: PCI host bridge to bus :00
> pci_bus :00: root bus resource [io  0x2007e-0x2007e0fff] (bus 
> address [0x-0xfff])
> pci_bus :00: root bus resource [mem 0x2-0x27eff] (bus 
> address [0x-0x7eff])
> pci_bus :00: root bus resource [mem 0x20001-0x20007] (bus 
> address [0x1-0x7])
> ...
> PCI: Claiming :00:02.0: Resource 14: 0002..0002004f 
> [200]
> pci :00:02.0: can't claim BAR 14 [mem 0x2-0x2004f]: 
> address conflict with Video RAM area [??? 0x2000a-0x2000b 
> flags 0x8000]
> pci :02:00.0: can't claim BAR 0 [mem 0x2-0x2000f]: no 
> compatible bridge window
> PCI: Claiming :02:00.0: Resource 3: 00020010..000200103fff 
> [200]
> pci :02:00.0: can't claim BAR 3 [mem 0x20010-0x200103fff]: no 
> compatible bridge window
> PCI: Claiming :02:00.1: Resource 0: 00020020..0002002f 
> [200]
> pci :02:00.1: can't claim BAR 0 [mem 0x20020-0x2002f]: no 
> compatible bridge window
> PCI: Claiming :02:00.1: Resource 3: 000200104000..000200107fff 
> [200]
> pci :02:00.1: can't claim BAR 3 [mem 0x200104000-0x200107fff]: no 
> compatible bridge window
> PCI: Claiming :02:00.2: Resource 0: 00020030..0002003f 
> [200]
> pci :02:00.2: can't claim BAR 0 [mem 0x20030-0x2003f]: no 
> compatible bridge window
> PCI: Claiming :02:00.2: Resource 3: 000200108000..00020010bfff 
> [200]
> pci :02:00.2: can't claim BAR 3 [mem 0x200108000-0x20010bfff]: no 
> compatible bridge window
> PCI: Claiming :02:00.3: Resource 0: 00020040..0002004f 
> [200]
> pci :02:00.3: can't claim BAR 0 [mem 0x20040-0x2004f]: no 
> compatible bridge window
> PCI: Claiming :02:00.3: Resource 3: 00020010c000..00020010 
> [200]
> pci :02:00.3: can't claim BAR 3 [mem 0x20010c000-0x20010]: no 
> compatible bridge window
> 
> The bridge 00:02.0 resource does not get reserved as Video RAM take the 
> position early,
> and following children resources reservation all fail.
> 
> Move down Video RAM area reservation after pci mmio get reserved,
> so we leave pci driver to use those regions.

I think this patch contains two changes:

  1) Factor pci_register_region() out of pci_register_legacy_regions()
  and do the correct bus-to-resource conversion, and

  2) Call pci_register_legacy_regions() from pci_scan_one_pbm() after
  scanning the bus instead of from pci_determine_mem_io_space()
  before.

Can you split this into two patches, one for each change?

pci_register_legacy_regions() reserves these bus address regions:

  [0xa-0xb] Video RAM
  [0xc-0xc7fff] Video ROM
  [0xf-0xf] System ROM

>From a PCI point of view, address space usage is normally reported by
BARs, and we register legacy regions to account for address space
that's not reported by a BAR.

For a legacy VGA device, I think that's only [0xa-0xb] (plus
some I/O ports).  I don't think a PCI device should ever use the ROM
regions mentioned above unless it has a BAR programmed to those
addresses, so I'm not sure we need to reserve them as PCI "legacy"
regions.

I suspect those ROM areas might have been copied from x86 BIOS stuff
and might not be applicable on sparc.  If you want to reserve them to
preserve the previous behavior, that's OK, but maybe we could do it
with a separate "sparc_register_legacy_regions()" or something?

I'm thinking that the VGA part of this is not really arch-specific, so
we could someday factor that out into the PCI core so you wouldn't
have to do that part in the sparc code, and splitting out the non-PCI
stuff would make that easier.

I guess we reserve the VGA RAM either for a VGA device already in the
system or to reserve space for one that might be hotplugged later.  In
your example above, there's a non-VGA device using that area, so I
suppose this patch means the 02:00.0 BAR 0 reservation works
correctly, but the subsequent VGA reservation complains about a
conflict with 02:00.0?  That "address conflict" message isn't really
accurate: there's no actual conflict because there's no VGA device.
The failure just means we won't be able to hot-add a VGA device later,
right?

> -v5: merge simplify one and use pcibios_bus_to_resource()
> 
> -v6: use pci_find_bus_resource()
> 
> Signed-off-by: Yinghai Lu 
> Tested-by: Khalid Aziz 
> Cc: sparcli...@vger.kernel.org
> ---
>  arch/sparc/kernel/pci.c|  1 +
>  arch/sparc/kernel/pci_common.c | 59 
> ++
>  arch/sparc/kernel/pci_impl.h   |  1 +
>  3 files changed, 33

Re: [PATCH 03/13] sparc/PCI: Reserve legacy mmio after PCI mmio

2017-05-03 Thread Bjorn Helgaas 
On Thu, Apr 20, 2017 at 10:04:50PM -0700, Yinghai Lu wrote:
> On one system found bunch of claim resource fail from pci device.
> pci_sun4v f02b894c: PCI host bridge to bus :00
> pci_bus :00: root bus resource [io  0x2007e-0x2007e0fff] (bus 
> address [0x-0xfff])
> pci_bus :00: root bus resource [mem 0x2-0x27eff] (bus 
> address [0x-0x7eff])
> pci_bus :00: root bus resource [mem 0x20001-0x20007] (bus 
> address [0x1-0x7])
> ...
> PCI: Claiming :00:02.0: Resource 14: 0002..0002004f 
> [200]
> pci :00:02.0: can't claim BAR 14 [mem 0x2-0x2004f]: 
> address conflict with Video RAM area [??? 0x2000a-0x2000b 
> flags 0x8000]
> pci :02:00.0: can't claim BAR 0 [mem 0x2-0x2000f]: no 
> compatible bridge window
> PCI: Claiming :02:00.0: Resource 3: 00020010..000200103fff 
> [200]
> pci :02:00.0: can't claim BAR 3 [mem 0x20010-0x200103fff]: no 
> compatible bridge window
> PCI: Claiming :02:00.1: Resource 0: 00020020..0002002f 
> [200]
> pci :02:00.1: can't claim BAR 0 [mem 0x20020-0x2002f]: no 
> compatible bridge window
> PCI: Claiming :02:00.1: Resource 3: 000200104000..000200107fff 
> [200]
> pci :02:00.1: can't claim BAR 3 [mem 0x200104000-0x200107fff]: no 
> compatible bridge window
> PCI: Claiming :02:00.2: Resource 0: 00020030..0002003f 
> [200]
> pci :02:00.2: can't claim BAR 0 [mem 0x20030-0x2003f]: no 
> compatible bridge window
> PCI: Claiming :02:00.2: Resource 3: 000200108000..00020010bfff 
> [200]
> pci :02:00.2: can't claim BAR 3 [mem 0x200108000-0x20010bfff]: no 
> compatible bridge window
> PCI: Claiming :02:00.3: Resource 0: 00020040..0002004f 
> [200]
> pci :02:00.3: can't claim BAR 0 [mem 0x20040-0x2004f]: no 
> compatible bridge window
> PCI: Claiming :02:00.3: Resource 3: 00020010c000..00020010 
> [200]
> pci :02:00.3: can't claim BAR 3 [mem 0x20010c000-0x20010]: no 
> compatible bridge window
> 
> The bridge 00:02.0 resource does not get reserved as Video RAM take the 
> position early,
> and following children resources reservation all fail.
> 
> Move down Video RAM area reservation after pci mmio get reserved,
> so we leave pci driver to use those regions.

I think this patch contains two changes:

  1) Factor pci_register_region() out of pci_register_legacy_regions()
  and do the correct bus-to-resource conversion, and

  2) Call pci_register_legacy_regions() from pci_scan_one_pbm() after
  scanning the bus instead of from pci_determine_mem_io_space()
  before.

Can you split this into two patches, one for each change?

pci_register_legacy_regions() reserves these bus address regions:

  [0xa-0xb] Video RAM
  [0xc-0xc7fff] Video ROM
  [0xf-0xf] System ROM

>From a PCI point of view, address space usage is normally reported by
BARs, and we register legacy regions to account for address space
that's not reported by a BAR.

For a legacy VGA device, I think that's only [0xa-0xb] (plus
some I/O ports).  I don't think a PCI device should ever use the ROM
regions mentioned above unless it has a BAR programmed to those
addresses, so I'm not sure we need to reserve them as PCI "legacy"
regions.

I suspect those ROM areas might have been copied from x86 BIOS stuff
and might not be applicable on sparc.  If you want to reserve them to
preserve the previous behavior, that's OK, but maybe we could do it
with a separate "sparc_register_legacy_regions()" or something?

I'm thinking that the VGA part of this is not really arch-specific, so
we could someday factor that out into the PCI core so you wouldn't
have to do that part in the sparc code, and splitting out the non-PCI
stuff would make that easier.

I guess we reserve the VGA RAM either for a VGA device already in the
system or to reserve space for one that might be hotplugged later.  In
your example above, there's a non-VGA device using that area, so I
suppose this patch means the 02:00.0 BAR 0 reservation works
correctly, but the subsequent VGA reservation complains about a
conflict with 02:00.0?  That "address conflict" message isn't really
accurate: there's no actual conflict because there's no VGA device.
The failure just means we won't be able to hot-add a VGA device later,
right?

> -v5: merge simplify one and use pcibios_bus_to_resource()
> 
> -v6: use pci_find_bus_resource()
> 
> Signed-off-by: Yinghai Lu 
> Tested-by: Khalid Aziz 
> Cc: sparcli...@vger.kernel.org
> ---
>  arch/sparc/kernel/pci.c|  1 +
>  arch/sparc/kernel/pci_common.c | 59 
> ++
>  arch/sparc/kernel/pci_impl.h   |  1 +
>  3 files changed, 33 insertions(+), 28 deletions(-)
> 
> diff

[PATCH 03/13] sparc/PCI: Reserve legacy mmio after PCI mmio

2017-04-20 Thread Yinghai Lu 
On one system found bunch of claim resource fail from pci device.
pci_sun4v f02b894c: PCI host bridge to bus :00
pci_bus :00: root bus resource [io  0x2007e-0x2007e0fff] (bus 
address [0x-0xfff])
pci_bus :00: root bus resource [mem 0x2-0x27eff] (bus 
address [0x-0x7eff])
pci_bus :00: root bus resource [mem 0x20001-0x20007] (bus 
address [0x1-0x7])
...
PCI: Claiming :00:02.0: Resource 14: 0002..0002004f 
[200]
pci :00:02.0: can't claim BAR 14 [mem 0x2-0x2004f]: 
address conflict with Video RAM area [??? 0x2000a-0x2000b flags 
0x8000]
pci :02:00.0: can't claim BAR 0 [mem 0x2-0x2000f]: no 
compatible bridge window
PCI: Claiming :02:00.0: Resource 3: 00020010..000200103fff [200]
pci :02:00.0: can't claim BAR 3 [mem 0x20010-0x200103fff]: no 
compatible bridge window
PCI: Claiming :02:00.1: Resource 0: 00020020..0002002f [200]
pci :02:00.1: can't claim BAR 0 [mem 0x20020-0x2002f]: no 
compatible bridge window
PCI: Claiming :02:00.1: Resource 3: 000200104000..000200107fff [200]
pci :02:00.1: can't claim BAR 3 [mem 0x200104000-0x200107fff]: no 
compatible bridge window
PCI: Claiming :02:00.2: Resource 0: 00020030..0002003f [200]
pci :02:00.2: can't claim BAR 0 [mem 0x20030-0x2003f]: no 
compatible bridge window
PCI: Claiming :02:00.2: Resource 3: 000200108000..00020010bfff [200]
pci :02:00.2: can't claim BAR 3 [mem 0x200108000-0x20010bfff]: no 
compatible bridge window
PCI: Claiming :02:00.3: Resource 0: 00020040..0002004f [200]
pci :02:00.3: can't claim BAR 0 [mem 0x20040-0x2004f]: no 
compatible bridge window
PCI: Claiming :02:00.3: Resource 3: 00020010c000..00020010 [200]
pci :02:00.3: can't claim BAR 3 [mem 0x20010c000-0x20010]: no 
compatible bridge window

The bridge 00:02.0 resource does not get reserved as Video RAM take the 
position early,
and following children resources reservation all fail.

Move down Video RAM area reservation after pci mmio get reserved,
so we leave pci driver to use those regions.

-v5: merge simplify one and use pcibios_bus_to_resource()

-v6: use pci_find_bus_resource()

Signed-off-by: Yinghai Lu 
Tested-by: Khalid Aziz 
Cc: sparcli...@vger.kernel.org
---
 arch/sparc/kernel/pci.c|  1 +
 arch/sparc/kernel/pci_common.c | 59 ++
 arch/sparc/kernel/pci_impl.h   |  1 +
 3 files changed, 33 insertions(+), 28 deletions(-)

diff --git a/arch/sparc/kernel/pci.c b/arch/sparc/kernel/pci.c
index c5cf813..adb9653 100644
--- a/arch/sparc/kernel/pci.c
+++ b/arch/sparc/kernel/pci.c
@@ -686,6 +686,7 @@ struct pci_bus *pci_scan_one_pbm(struct pci_pbm_info *pbm,
pci_bus_register_of_sysfs(bus);
 
pci_claim_bus_resources(bus);
+   pci_register_legacy_regions(bus);
pci_bus_add_devices(bus);
return bus;
 }
diff --git a/arch/sparc/kernel/pci_common.c b/arch/sparc/kernel/pci_common.c
index 76998f8..1ebc7ff 100644
--- a/arch/sparc/kernel/pci_common.c
+++ b/arch/sparc/kernel/pci_common.c
@@ -328,41 +328,46 @@ void pci_get_pbm_props(struct pci_pbm_info *pbm)
}
 }
 
-static void pci_register_legacy_regions(struct resource *io_res,
-   struct resource *mem_res)
+static void pci_register_region(struct pci_bus *bus, const char *name,
+   resource_size_t rstart, resource_size_t size)
 {
-   struct resource *p;
+   struct resource *res, *conflict, *bus_res;
+   struct pci_bus_region region;
 
-   /* VGA Video RAM. */
-   p = kzalloc(sizeof(*p), GFP_KERNEL);
-   if (!p)
+   res = kzalloc(sizeof(*res), GFP_KERNEL);
+   if (!res)
return;
 
-   p->name = "Video RAM area";
-   p->start = mem_res->start + 0xaUL;
-   p->end = p->start + 0x1UL;
-   p->flags = IORESOURCE_BUSY;
-   request_resource(mem_res, p);
+   res->flags = IORESOURCE_MEM;
 
-   p = kzalloc(sizeof(*p), GFP_KERNEL);
-   if (!p)
+   region.start = rstart;
+   region.end = rstart + size - 1UL;
+   pcibios_bus_to_resource(bus, res, );
+   bus_res = pci_find_bus_resource(bus, res);
+   if (!bus_res) {
+   kfree(res);
return;
+   }
+
+   res->name = name;
+   res->flags |= IORESOURCE_BUSY;
+   conflict = request_resource_conflict(bus_res, res);
+   if (conflict) {
+   dev_printk(KERN_DEBUG, >dev,
+   " can't claim %s %pR: address conflict with %s %pR\n",
+   res->name, res, conflict->name, conflict);
+   kfree(res);
+   }
+}

[PATCH 03/13] sparc/PCI: Reserve legacy mmio after PCI mmio

2017-04-20 Thread Yinghai Lu 
On one system found bunch of claim resource fail from pci device.
pci_sun4v f02b894c: PCI host bridge to bus :00
pci_bus :00: root bus resource [io  0x2007e-0x2007e0fff] (bus 
address [0x-0xfff])
pci_bus :00: root bus resource [mem 0x2-0x27eff] (bus 
address [0x-0x7eff])
pci_bus :00: root bus resource [mem 0x20001-0x20007] (bus 
address [0x1-0x7])
...
PCI: Claiming :00:02.0: Resource 14: 0002..0002004f 
[200]
pci :00:02.0: can't claim BAR 14 [mem 0x2-0x2004f]: 
address conflict with Video RAM area [??? 0x2000a-0x2000b flags 
0x8000]
pci :02:00.0: can't claim BAR 0 [mem 0x2-0x2000f]: no 
compatible bridge window
PCI: Claiming :02:00.0: Resource 3: 00020010..000200103fff [200]
pci :02:00.0: can't claim BAR 3 [mem 0x20010-0x200103fff]: no 
compatible bridge window
PCI: Claiming :02:00.1: Resource 0: 00020020..0002002f [200]
pci :02:00.1: can't claim BAR 0 [mem 0x20020-0x2002f]: no 
compatible bridge window
PCI: Claiming :02:00.1: Resource 3: 000200104000..000200107fff [200]
pci :02:00.1: can't claim BAR 3 [mem 0x200104000-0x200107fff]: no 
compatible bridge window
PCI: Claiming :02:00.2: Resource 0: 00020030..0002003f [200]
pci :02:00.2: can't claim BAR 0 [mem 0x20030-0x2003f]: no 
compatible bridge window
PCI: Claiming :02:00.2: Resource 3: 000200108000..00020010bfff [200]
pci :02:00.2: can't claim BAR 3 [mem 0x200108000-0x20010bfff]: no 
compatible bridge window
PCI: Claiming :02:00.3: Resource 0: 00020040..0002004f [200]
pci :02:00.3: can't claim BAR 0 [mem 0x20040-0x2004f]: no 
compatible bridge window
PCI: Claiming :02:00.3: Resource 3: 00020010c000..00020010 [200]
pci :02:00.3: can't claim BAR 3 [mem 0x20010c000-0x20010]: no 
compatible bridge window

The bridge 00:02.0 resource does not get reserved as Video RAM take the 
position early,
and following children resources reservation all fail.

Move down Video RAM area reservation after pci mmio get reserved,
so we leave pci driver to use those regions.

-v5: merge simplify one and use pcibios_bus_to_resource()

-v6: use pci_find_bus_resource()

Signed-off-by: Yinghai Lu 
Tested-by: Khalid Aziz 
Cc: sparcli...@vger.kernel.org
---
 arch/sparc/kernel/pci.c|  1 +
 arch/sparc/kernel/pci_common.c | 59 ++
 arch/sparc/kernel/pci_impl.h   |  1 +
 3 files changed, 33 insertions(+), 28 deletions(-)

diff --git a/arch/sparc/kernel/pci.c b/arch/sparc/kernel/pci.c
index c5cf813..adb9653 100644
--- a/arch/sparc/kernel/pci.c
+++ b/arch/sparc/kernel/pci.c
@@ -686,6 +686,7 @@ struct pci_bus *pci_scan_one_pbm(struct pci_pbm_info *pbm,
pci_bus_register_of_sysfs(bus);
 
pci_claim_bus_resources(bus);
+   pci_register_legacy_regions(bus);
pci_bus_add_devices(bus);
return bus;
 }
diff --git a/arch/sparc/kernel/pci_common.c b/arch/sparc/kernel/pci_common.c
index 76998f8..1ebc7ff 100644
--- a/arch/sparc/kernel/pci_common.c
+++ b/arch/sparc/kernel/pci_common.c
@@ -328,41 +328,46 @@ void pci_get_pbm_props(struct pci_pbm_info *pbm)
}
 }
 
-static void pci_register_legacy_regions(struct resource *io_res,
-   struct resource *mem_res)
+static void pci_register_region(struct pci_bus *bus, const char *name,
+   resource_size_t rstart, resource_size_t size)
 {
-   struct resource *p;
+   struct resource *res, *conflict, *bus_res;
+   struct pci_bus_region region;
 
-   /* VGA Video RAM. */
-   p = kzalloc(sizeof(*p), GFP_KERNEL);
-   if (!p)
+   res = kzalloc(sizeof(*res), GFP_KERNEL);
+   if (!res)
return;
 
-   p->name = "Video RAM area";
-   p->start = mem_res->start + 0xaUL;
-   p->end = p->start + 0x1UL;
-   p->flags = IORESOURCE_BUSY;
-   request_resource(mem_res, p);
+   res->flags = IORESOURCE_MEM;
 
-   p = kzalloc(sizeof(*p), GFP_KERNEL);
-   if (!p)
+   region.start = rstart;
+   region.end = rstart + size - 1UL;
+   pcibios_bus_to_resource(bus, res, );
+   bus_res = pci_find_bus_resource(bus, res);
+   if (!bus_res) {
+   kfree(res);
return;
+   }
+
+   res->name = name;
+   res->flags |= IORESOURCE_BUSY;
+   conflict = request_resource_conflict(bus_res, res);
+   if (conflict) {
+   dev_printk(KERN_DEBUG, >dev,
+   " can't claim %s %pR: address conflict with %s %pR\n",
+   res->name, res, conflict->name, conflict);
+   kfree(res);
+   }
+}
 
-   p->name = "System ROM";
-