Re: Problem with PCI bus rescan on 460EX
Yinghai Lu wrote: On Sun, Mar 28, 2010 at 2:13 AM, Felix Radensky fe...@embedded-sol.com wrote: Hello, Kenji-san I've tried Jesse's tree with Yinghai's patches, but they don't seem to help. Memory for bridge is not allocated after insertion of hotplug device and bus rescan. Attached dmesg output in case of success and failure. that patches only take care of pcie hotplug path... and it calls pci_assign_unassigned_bridge_resources(bridge) instead of pci_bus_assign_resources(bus) so it doesn't check pci_is_enabled() maybe We can update rescan path to check if it is safe to skip the pci_is_enabled() too. - no driver for those devices under that bridge are loaded. Yinghai, Felix, Thank you very much for the explanation, Yinghai. The pci_assign_unassigned_bridge_resources() is what I wanted to suggest to Felix. Felix, I think assigning hpXXsize at boot time is the simpler solution, if it is acceptable workaround for you. And as Yinghai suggested in the another email, removing slot's parent bridge first should be one of the workaround. But the problem is whether it is acceptable for you, because your hotplug problem seems a regression. Thanks, Kenji Kaneshige YH -- To unsubscribe from this list: send the line unsubscribe linux-pci in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html ___ Linuxppc-dev mailing list Linuxppc-dev@lists.ozlabs.org https://lists.ozlabs.org/listinfo/linuxppc-dev
Re: Problem with PCI bus rescan on 460EX
Hello Kenji-san, Kenji Kaneshige wrote: Felix, I think assigning hpXXsize at boot time is the simpler solution, if it is acceptable workaround for you. And as Yinghai suggested in the another email, removing slot's parent bridge first should be one of the workaround. But the problem is whether it is acceptable for you, because your hotplug problem seems a regression. Thanks, Kenji Kaneshige Both workarounds work for me, and will use hpXXsize as a short term solution. In the long run I'd like to see the problem fixed properly and I'm willing to test patches on my hardware, as I also think it's a regression. Thanks Kenji-san and Yinghai for your help. Felix. ___ Linuxppc-dev mailing list Linuxppc-dev@lists.ozlabs.org https://lists.ozlabs.org/listinfo/linuxppc-dev
Re: Problem with PCI bus rescan on 460EX
Hello, Kenji-san Kenji Kaneshige wrote: Felix Radensky wrote: Hello, Kenji-san Kenji Kaneshige wrote: I misunderstood the problem. My understanding was memory resource was not enabled even though Linux set the Memory Space bit in the command register. But it was not correct. The bridge memory window was marked unused and Linux didn't try to set Memory Space bit in the command register. Current my understanding is as follows. Please correct me if I'm still misunderstanding something. 1) Your BIOS doesn't assign any resource to the bridge if its child PCI hot-plug slot is not occupied. 2) At the boot time, pci_assign_unassigned_resources() try to assign memory resouces to the bridge using pci_bus_assign_resource(), but it was disabled because there are no devices require memory resource. 3) And then pci_assign_unassigned_resouces() calls pci_enable_bridge(), but Memory Space bit in the command register was not set because no memory resource are assigned to the bridge. At the same time, pci_dev-enable_cnt was incremented. 4) At the rescan time, pci_setup_bridge() and pci_enable_bridge() doesn't work because the bridge is already marked enabled (i.e. pci_dev-enable_cnt is not zero). I don't have any concrete idea how to fix that so far, but I can say my idea (pcibios_enable_device() should return an error) was wrong. I was wandering if setting is_hotplug_bridge property for this bridge (e.g. via header quirk) can be an acceptable solution. This will allow passing hpmemsize kernel parameter, to specify the amount of memory to assign to the bridge. I've tested this approach and it seems to work. Looks good to me. By the way, I think Yinghai's bridge resource reallocation patch series might help you. It is in Jesse's PCI tree. Please take a look. Thanks, Kenji Kaneshige I've tried Jesse's tree with Yinghai's patches, but they don't seem to help. Memory for bridge is not allocated after insertion of hotplug device and bus rescan. Attached dmesg output in case of success and failure. Thanks. Felix. Linux version 2.6.33 (fe...@felix-laptop.lan) (gcc version 4.2.2) #8 Sun Mar 28 12:04:25 IDT 2010 Found legacy serial port 0 for /plb/opb/ser...@ef600300 mem=4ef600300, taddr=4ef600300, irq=0, clk=7407407, speed=0 Found legacy serial port 1 for /plb/opb/ser...@ef600400 mem=4ef600400, taddr=4ef600400, irq=0, clk=7407407, speed=0 Found legacy serial port 2 for /plb/opb/ser...@ef600500 mem=4ef600500, taddr=4ef600500, irq=0, clk=7407407, speed=0 Found legacy serial port 3 for /plb/opb/ser...@ef600600 mem=4ef600600, taddr=4ef600600, irq=0, clk=7407407, speed=0 Top of RAM: 0x3000, Total RAM: 0x3000 Memory hole size: 0MB Zone PFN ranges: DMA 0x - 0x0003 Normal 0x0003 - 0x0003 Movable zone start PFN for each node early_node_map[1] active PFN ranges 0: 0x - 0x0003 On node 0 totalpages: 196608 free_area_init_node: node 0, pgdat c02a84d0, node_mem_map c02e1000 DMA zone: 1536 pages used for memmap DMA zone: 0 pages reserved DMA zone: 195072 pages, LIFO batch:31 MMU: Allocated 1088 bytes of context maps for 255 contexts Built 1 zonelists in Zone order, mobility grouping on. Total pages: 195072 Kernel command line: root=/dev/nfs rw nfsroot=10.0.0.10:/opt/eldk/ppc_4xxFP ip=10.0.0.30:10.0.0.10:10.0.0.138:255.0.0.0:smbe460:eth0:off panic=1 console=ttyS0,115200 PID hash table entries: 4096 (order: 2, 16384 bytes) Dentry cache hash table entries: 131072 (order: 7, 524288 bytes) Inode-cache hash table entries: 65536 (order: 6, 262144 bytes) Memory: 776500k/786432k available (2656k kernel code, 9932k reserved, 100k data, 184k bss, 132k init) Kernel virtual memory layout: * 0xfffdf000..0xf000 : fixmap * 0xfde0..0xfe00 : consistent mem * 0xfde0..0xfde0 : early ioremap * 0xf100..0xfde0 : vmalloc ioremap SLUB: Genslabs=11, HWalign=32, Order=0-3, MinObjects=0, CPUs=1, Nodes=1 Hierarchical RCU implementation. NR_IRQS:512 nr_irqs:512 UIC0 (32 IRQ sources) at DCR 0xc0 UIC1 (32 IRQ sources) at DCR 0xd0 alloc irq_desc for 30 on node 0 alloc kstat_irqs on node 0 irq: irq 30 on host /interrupt-controller0 mapped to virtual irq 30 UIC2 (32 IRQ sources) at DCR 0xe0 alloc irq_desc for 16 on node 0 alloc kstat_irqs on node 0 irq: irq 10 on host /interrupt-controller0 mapped to virtual irq 16 UIC3 (32 IRQ sources) at DCR 0xf0 alloc irq_desc for 17 on node 0 alloc kstat_irqs on node 0 irq: irq 16 on host /interrupt-controller0 mapped to virtual irq 17 time_init: decrementer frequency = 1000.12 MHz time_init: processor frequency = 1000.12 MHz clocksource: timebase mult[40] shift[22] registered clockevent: decrementer mult[8019] shift[31] cpu[0] Mount-cache hash table entries: 512 NET: Registered protocol family 16 alloc irq_desc for 18 on node 0 alloc kstat_irqs on node 0 irq: irq 11 on host /interrupt-controller1 mapped to virtual irq 18 256k L2-cache enabled
Re: Problem with PCI bus rescan on 460EX
On Sun, 2010-03-28 at 12:13 +0300, Felix Radensky wrote: I've tried Jesse's tree with Yinghai's patches, but they don't seem to help. Memory for bridge is not allocated after insertion of hotplug device and bus rescan. Attached dmesg output in case of success and failure. I'd recommend that for now, your platform uses a quirk to manually resize the bridge resource to make room for future hotplug on it. Cheers, Ben. Thanks. Felix. plain text document attachment (success.txt) Linux version 2.6.33 (fe...@felix-laptop.lan) (gcc version 4.2.2) #8 Sun Mar 28 12:04:25 IDT 2010 Found legacy serial port 0 for /plb/opb/ser...@ef600300 mem=4ef600300, taddr=4ef600300, irq=0, clk=7407407, speed=0 Found legacy serial port 1 for /plb/opb/ser...@ef600400 mem=4ef600400, taddr=4ef600400, irq=0, clk=7407407, speed=0 Found legacy serial port 2 for /plb/opb/ser...@ef600500 mem=4ef600500, taddr=4ef600500, irq=0, clk=7407407, speed=0 Found legacy serial port 3 for /plb/opb/ser...@ef600600 mem=4ef600600, taddr=4ef600600, irq=0, clk=7407407, speed=0 Top of RAM: 0x3000, Total RAM: 0x3000 Memory hole size: 0MB Zone PFN ranges: DMA 0x - 0x0003 Normal 0x0003 - 0x0003 Movable zone start PFN for each node early_node_map[1] active PFN ranges 0: 0x - 0x0003 On node 0 totalpages: 196608 free_area_init_node: node 0, pgdat c02a84d0, node_mem_map c02e1000 DMA zone: 1536 pages used for memmap DMA zone: 0 pages reserved DMA zone: 195072 pages, LIFO batch:31 MMU: Allocated 1088 bytes of context maps for 255 contexts Built 1 zonelists in Zone order, mobility grouping on. Total pages: 195072 Kernel command line: root=/dev/nfs rw nfsroot=10.0.0.10:/opt/eldk/ppc_4xxFP ip=10.0.0.30:10.0.0.10:10.0.0.138:255.0.0.0:smbe460:eth0:off panic=1 console=ttyS0,115200 PID hash table entries: 4096 (order: 2, 16384 bytes) Dentry cache hash table entries: 131072 (order: 7, 524288 bytes) Inode-cache hash table entries: 65536 (order: 6, 262144 bytes) Memory: 776500k/786432k available (2656k kernel code, 9932k reserved, 100k data, 184k bss, 132k init) Kernel virtual memory layout: * 0xfffdf000..0xf000 : fixmap * 0xfde0..0xfe00 : consistent mem * 0xfde0..0xfde0 : early ioremap * 0xf100..0xfde0 : vmalloc ioremap SLUB: Genslabs=11, HWalign=32, Order=0-3, MinObjects=0, CPUs=1, Nodes=1 Hierarchical RCU implementation. NR_IRQS:512 nr_irqs:512 UIC0 (32 IRQ sources) at DCR 0xc0 UIC1 (32 IRQ sources) at DCR 0xd0 alloc irq_desc for 30 on node 0 alloc kstat_irqs on node 0 irq: irq 30 on host /interrupt-controller0 mapped to virtual irq 30 UIC2 (32 IRQ sources) at DCR 0xe0 alloc irq_desc for 16 on node 0 alloc kstat_irqs on node 0 irq: irq 10 on host /interrupt-controller0 mapped to virtual irq 16 UIC3 (32 IRQ sources) at DCR 0xf0 alloc irq_desc for 17 on node 0 alloc kstat_irqs on node 0 irq: irq 16 on host /interrupt-controller0 mapped to virtual irq 17 time_init: decrementer frequency = 1000.12 MHz time_init: processor frequency = 1000.12 MHz clocksource: timebase mult[40] shift[22] registered clockevent: decrementer mult[8019] shift[31] cpu[0] Mount-cache hash table entries: 512 NET: Registered protocol family 16 alloc irq_desc for 18 on node 0 alloc kstat_irqs on node 0 irq: irq 11 on host /interrupt-controller1 mapped to virtual irq 18 256k L2-cache enabled PCI host bridge /plb/p...@c0ec0 (primary) ranges: MEM 0x000d8000..0x000d - 0x8000 MEM 0x000c0ee0..0x000c0eef - 0x IO 0x000c0800..0x000c0800 - 0x Removing ISA hole at 0x000c0ee0 4xx PCI DMA offset set to 0x /plb/p...@c0ec0: Legacy ISA memory support enabled PCI: Probing PCI hardware pci_bus :00: scanning bus pci :00:02.0: found [3388:0020] class 000604 header type 01 pci :00:02.0: calling pcibios_fixup_resources+0x0/0xf4 pci :00:02.0: calling fixup_ppc4xx_pci_bridge+0x0/0x154 pci :00:02.0: calling quirk_resource_alignment+0x0/0x200 pci :00:02.0: supports D1 D2 pci :00:02.0: PME# supported from D0 D1 D2 D3hot pci :00:02.0: PME# disabled pci_bus :00: fixups for bus pci :00:02.0: scanning behind bridge, config 010100, pass 0 pci_bus :01: scanning bus pci :01:00.0: found [1575:0002] class 00ff00 header type 00 pci :01:00.0: reg 10: [mem 0x8000-0x800f] pci :01:00.0: reg 14: [mem 0x8400-0x87ff] pci :01:00.0: reg 18: [mem 0x8800-0x8bff] pci :01:00.0: reg 1c: [mem 0x8c00-0x8c003fff] pci :01:00.0: calling pcibios_fixup_resources+0x0/0xf4 pci :01:00.0: calling fixup_ppc4xx_pci_bridge+0x0/0x154 pci :01:00.0: calling quirk_resource_alignment+0x0/0x200 pci_bus :01: fixups for bus pci :00:02.0: PCI bridge to [bus 01-01] pci :00:02.0: bridge
Re: Problem with PCI bus rescan on 460EX
Hi Ben, Benjamin Herrenschmidt wrote: On Sun, 2010-03-28 at 12:13 +0300, Felix Radensky wrote: I've tried Jesse's tree with Yinghai's patches, but they don't seem to help. Memory for bridge is not allocated after insertion of hotplug device and bus rescan. Attached dmesg output in case of success and failure. I'd recommend that for now, your platform uses a quirk to manually resize the bridge resource to make room for future hotplug on it. Cheers, Ben. I've submitted a patch to do that. See https://patchwork.kernel.org/patch/88769 Felix. ___ Linuxppc-dev mailing list Linuxppc-dev@lists.ozlabs.org https://lists.ozlabs.org/listinfo/linuxppc-dev
Re: Problem with PCI bus rescan on 460EX
On Sun, Mar 28, 2010 at 2:13 AM, Felix Radensky fe...@embedded-sol.com wrote: Hello, Kenji-san I've tried Jesse's tree with Yinghai's patches, but they don't seem to help. Memory for bridge is not allocated after insertion of hotplug device and bus rescan. Attached dmesg output in case of success and failure. that patches only take care of pcie hotplug path... and it calls pci_assign_unassigned_bridge_resources(bridge) instead of pci_bus_assign_resources(bus) so it doesn't check pci_is_enabled() maybe We can update rescan path to check if it is safe to skip the pci_is_enabled() too. - no driver for those devices under that bridge are loaded. YH ___ Linuxppc-dev mailing list Linuxppc-dev@lists.ozlabs.org https://lists.ozlabs.org/listinfo/linuxppc-dev
Re: Problem with PCI bus rescan on 460EX
Hello Kenj-san Kenji Kaneshige wrote: By the way, I think Yinghai's bridge resource reallocation patch series might help you. It is in Jesse's PCI tree. Please take a look. I've tried Jesse's tree on my custom board and on 460EX evaluation board (Canyonlands). In both cases the kernel doesn't boot unless PCI support is disabled. Debugging is difficult as the board just resets before anything is is printed on console. Felix. ___ Linuxppc-dev mailing list Linuxppc-dev@lists.ozlabs.org https://lists.ozlabs.org/listinfo/linuxppc-dev
Re: Problem with PCI bus rescan on 460EX
On Wed, 2010-03-17 at 09:38 +0200, Felix Radensky wrote: Hello Kenj-san Kenji Kaneshige wrote: By the way, I think Yinghai's bridge resource reallocation patch series might help you. It is in Jesse's PCI tree. Please take a look. I've tried Jesse's tree on my custom board and on 460EX evaluation board (Canyonlands). In both cases the kernel doesn't boot unless PCI support is disabled. Debugging is difficult as the board just resets before anything is is printed on console. This is Jesse tree with the default Canyonlands defconfig ? Hrm... I'll have to take a look, somebody mucking with PCI resource allocation is very likely to break something :-) Cheers, Ben. ___ Linuxppc-dev mailing list Linuxppc-dev@lists.ozlabs.org https://lists.ozlabs.org/listinfo/linuxppc-dev
Re: Problem with PCI bus rescan on 460EX
This is Jesse tree with the default Canyonlands defconfig ? Hrm... I'll have to take a look, somebody mucking with PCI resource allocation is very likely to break something :-) Yep, default Canyonlands defconfig and default DTS. Which branch of Jesse tree btw ? master ? Cheers, Ben. ___ Linuxppc-dev mailing list Linuxppc-dev@lists.ozlabs.org https://lists.ozlabs.org/listinfo/linuxppc-dev
Re: Problem with PCI bus rescan on 460EX
This is Jesse tree with the default Canyonlands defconfig ? Hrm... I'll have to take a look, somebody mucking with PCI resource allocation is very likely to break something :-) Yep, default Canyonlands defconfig and default DTS. Ok so Jesse is innocent :-) See the patch I just posted: http://patchwork.ozlabs.org/patch/47976/ That fixes it here. Cheers, Ben. ___ Linuxppc-dev mailing list Linuxppc-dev@lists.ozlabs.org https://lists.ozlabs.org/listinfo/linuxppc-dev
Re: Problem with PCI bus rescan on 460EX
Hello, Kenji-san Kenji Kaneshige wrote: I misunderstood the problem. My understanding was memory resource was not enabled even though Linux set the Memory Space bit in the command register. But it was not correct. The bridge memory window was marked unused and Linux didn't try to set Memory Space bit in the command register. Current my understanding is as follows. Please correct me if I'm still misunderstanding something. 1) Your BIOS doesn't assign any resource to the bridge if its child PCI hot-plug slot is not occupied. 2) At the boot time, pci_assign_unassigned_resources() try to assign memory resouces to the bridge using pci_bus_assign_resource(), but it was disabled because there are no devices require memory resource. 3) And then pci_assign_unassigned_resouces() calls pci_enable_bridge(), but Memory Space bit in the command register was not set because no memory resource are assigned to the bridge. At the same time, pci_dev-enable_cnt was incremented. 4) At the rescan time, pci_setup_bridge() and pci_enable_bridge() doesn't work because the bridge is already marked enabled (i.e. pci_dev-enable_cnt is not zero). I don't have any concrete idea how to fix that so far, but I can say my idea (pcibios_enable_device() should return an error) was wrong. BTW, on my PCI hotplug capable system (SHPC and PCIe), I/O and Memory windows of the bridge are assigned by BIOS regardless of whether hotplug slot(s) behind the bridge is occupied or not. Maybe that is the reason why I have never encountered this problem before. Thanks, Kenji Kaneshige Yes, your understanding of the problem is correct. On this platform BIOS (bootloader, u-boot) is not required to configure PCI, linux is capable of doing all configuration itself. But both u-boot and linux do not assign memory resources to bridge if there's no device behind it. Thanks a lot. Felix. ___ Linuxppc-dev mailing list Linuxppc-dev@lists.ozlabs.org https://lists.ozlabs.org/listinfo/linuxppc-dev
Re: Problem with PCI bus rescan on 460EX
Hello, Kenji-san Kenji Kaneshige wrote: I misunderstood the problem. My understanding was memory resource was not enabled even though Linux set the Memory Space bit in the command register. But it was not correct. The bridge memory window was marked unused and Linux didn't try to set Memory Space bit in the command register. Current my understanding is as follows. Please correct me if I'm still misunderstanding something. 1) Your BIOS doesn't assign any resource to the bridge if its child PCI hot-plug slot is not occupied. 2) At the boot time, pci_assign_unassigned_resources() try to assign memory resouces to the bridge using pci_bus_assign_resource(), but it was disabled because there are no devices require memory resource. 3) And then pci_assign_unassigned_resouces() calls pci_enable_bridge(), but Memory Space bit in the command register was not set because no memory resource are assigned to the bridge. At the same time, pci_dev-enable_cnt was incremented. 4) At the rescan time, pci_setup_bridge() and pci_enable_bridge() doesn't work because the bridge is already marked enabled (i.e. pci_dev-enable_cnt is not zero). I don't have any concrete idea how to fix that so far, but I can say my idea (pcibios_enable_device() should return an error) was wrong. I was wandering if setting is_hotplug_bridge property for this bridge (e.g. via header quirk) can be an acceptable solution. This will allow passing hpmemsize kernel parameter, to specify the amount of memory to assign to the bridge. I've tested this approach and it seems to work. Thanks. Felix. Felix. ___ Linuxppc-dev mailing list Linuxppc-dev@lists.ozlabs.org https://lists.ozlabs.org/listinfo/linuxppc-dev
Re: Problem with PCI bus rescan on 460EX
Felix Radensky wrote: Hello, Kenji-san Kenji Kaneshige wrote: I misunderstood the problem. My understanding was memory resource was not enabled even though Linux set the Memory Space bit in the command register. But it was not correct. The bridge memory window was marked unused and Linux didn't try to set Memory Space bit in the command register. Current my understanding is as follows. Please correct me if I'm still misunderstanding something. 1) Your BIOS doesn't assign any resource to the bridge if its child PCI hot-plug slot is not occupied. 2) At the boot time, pci_assign_unassigned_resources() try to assign memory resouces to the bridge using pci_bus_assign_resource(), but it was disabled because there are no devices require memory resource. 3) And then pci_assign_unassigned_resouces() calls pci_enable_bridge(), but Memory Space bit in the command register was not set because no memory resource are assigned to the bridge. At the same time, pci_dev-enable_cnt was incremented. 4) At the rescan time, pci_setup_bridge() and pci_enable_bridge() doesn't work because the bridge is already marked enabled (i.e. pci_dev-enable_cnt is not zero). I don't have any concrete idea how to fix that so far, but I can say my idea (pcibios_enable_device() should return an error) was wrong. I was wandering if setting is_hotplug_bridge property for this bridge (e.g. via header quirk) can be an acceptable solution. This will allow passing hpmemsize kernel parameter, to specify the amount of memory to assign to the bridge. I've tested this approach and it seems to work. Looks good to me. By the way, I think Yinghai's bridge resource reallocation patch series might help you. It is in Jesse's PCI tree. Please take a look. Thanks, Kenji Kaneshige ___ Linuxppc-dev mailing list Linuxppc-dev@lists.ozlabs.org https://lists.ozlabs.org/listinfo/linuxppc-dev
Re: Problem with PCI bus rescan on 460EX
Hello Kenji-san,
Kenji Kaneshige wrote:
Felix Radensky wrote:
Hello Kenji-san,
Kenji Kaneshige wrote:
I'm not sure, but I guess pci_setup_bridge() didn't update IO
base/limit
and Mem base/limit of the bridge (:00:02.0) because of the
following
lines.
static void pci_setup_bridge(struct pci_bus *bus)
{
struct pci_dev *bridge = bus-self;
struct resource *res;
struct pci_bus_region region;
u32 l, bu, lu, io_upper16;
if (pci_is_enabled(bridge))
===
return;===
dev_info(bridge-dev, PCI bridge to [bus %02x-%02x]\n,
bus-secondary, bus-subordinate);
...
It seems the bridge was already enabled by
pci_assign_unassigned_resources()
at boot time. Does removing those lines make any change?
Yes, with these lines removed bridge memory window is properly
allocated.
These lines are to prevent updating IO or memory windows when there are
some devices working behind the bridge. So please note that removing
these lines is just for debugging.
For some reason bridge memory is disabled, but if I enable it via
setpci, and
also enable device memory, then everything works fine. If the system
is booted
when device behind the bridge is plugged in, bridge memory is enabled.
Maybe because of the following lines, I guess.
void pci_enable_bridges(struct pci_bus *bus)
{
struct pci_dev *dev;
int retval;
list_for_each_entry(dev, bus-devices, bus_list) {
if (dev-subordinate) {
if (!pci_is_enabled(dev)) {===
retval = pci_enable_device(dev);===
pci_set_master(dev);===
}
...
Yes, but removing this check is not enough. The bridge is enabled after
first scan, but it's
memory is disabled. So simply calling pci_enable_device() will not
help, it will return without
enabling memory. I had to call pci_disable_device() before
pci_enable_device() to make things
work.
One possible cause is that pcibios_enable_device() for the bridge didn't
return any error even though it didn't work properly (ex. cannot access
to the command register, and so on) on your platform when there is no
device behind the bridge. But it is just my guess.
Should pcibios_enable_device() return an error if it fails to enable
bridge memory ?
Thanks.
Felix.
___
Linuxppc-dev mailing list
Linuxppc-dev@lists.ozlabs.org
https://lists.ozlabs.org/listinfo/linuxppc-dev
Re: Problem with PCI bus rescan on 460EX
Felix Radensky wrote:
Hello Kenji-san,
Kenji Kaneshige wrote:
Felix Radensky wrote:
Hello Kenji-san,
Kenji Kaneshige wrote:
I'm not sure, but I guess pci_setup_bridge() didn't update IO
base/limit
and Mem base/limit of the bridge (:00:02.0) because of the
following
lines.
static void pci_setup_bridge(struct pci_bus *bus)
{
struct pci_dev *bridge = bus-self;
struct resource *res;
struct pci_bus_region region;
u32 l, bu, lu, io_upper16;
if (pci_is_enabled(bridge))
===
return;===
dev_info(bridge-dev, PCI bridge to [bus %02x-%02x]\n,
bus-secondary, bus-subordinate);
...
It seems the bridge was already enabled by
pci_assign_unassigned_resources()
at boot time. Does removing those lines make any change?
Yes, with these lines removed bridge memory window is properly
allocated.
These lines are to prevent updating IO or memory windows when there are
some devices working behind the bridge. So please note that removing
these lines is just for debugging.
For some reason bridge memory is disabled, but if I enable it via
setpci, and
also enable device memory, then everything works fine. If the system
is booted
when device behind the bridge is plugged in, bridge memory is enabled.
Maybe because of the following lines, I guess.
void pci_enable_bridges(struct pci_bus *bus)
{
struct pci_dev *dev;
int retval;
list_for_each_entry(dev, bus-devices, bus_list) {
if (dev-subordinate) {
if (!pci_is_enabled(dev)) {===
retval = pci_enable_device(dev);===
pci_set_master(dev);===
}
...
Yes, but removing this check is not enough. The bridge is enabled after
first scan, but it's
memory is disabled. So simply calling pci_enable_device() will not
help, it will return without
enabling memory. I had to call pci_disable_device() before
pci_enable_device() to make things
work.
One possible cause is that pcibios_enable_device() for the bridge didn't
return any error even though it didn't work properly (ex. cannot access
to the command register, and so on) on your platform when there is no
device behind the bridge. But it is just my guess.
Should pcibios_enable_device() return an error if it fails to enable
bridge memory ?
I think the device is expected to be ready to work if pci_enable_device()
returns without error. So I think pci_enable_device() should return an
error if it fails to enable the device (device is not ready to work). In
this case, detecting your bridge's failure seems PPC specific to me. So I
thought pcibios_enable_device() was the right to return an error. If
pcibios_enable_device() returned an error, pci_dev-enable_cnt would
decremented by pci_enable_device() (like pci_disable_device() does) and
this problem would not happen.
On the other hand, as Ben suggested, handling this by specific hot-plug
driver would be one of the other candidate to fix the problem.
Thanks,
Kenji Kaneshige
___
Linuxppc-dev mailing list
Linuxppc-dev@lists.ozlabs.org
https://lists.ozlabs.org/listinfo/linuxppc-dev
Re: Problem with PCI bus rescan on 460EX
Hello, Kenji-san I think the device is expected to be ready to work if pci_enable_device() returns without error. So I think pci_enable_device() should return an error if it fails to enable the device (device is not ready to work). In this case, detecting your bridge's failure seems PPC specific to me. So I thought pcibios_enable_device() was the right to return an error. If pcibios_enable_device() returned an error, pci_dev-enable_cnt would decremented by pci_enable_device() (like pci_disable_device() does) and this problem would not happen. As far as I can see on 460EX pcibios_enable_device() just calls pci_enable_resources() which does not return any error for my bridge, although it doesn't find any memory or I/O resource it can enable. Do you think it is correct behavior ? Another question is whether by bridge behaves correctly when no device is connected to it. As you can see in dmesg output I've sent earlier pci :00:02.0: bridge window [mem 0x-0x000f] pci :00:02.0: bridge window [mem 0x-0x000f 64bit pref] and later PCI code disables these memory windows pci :00:02.0: disabling bridge window [mem 0xd-0xd000f pref] to [bus 01-01] (unused) pci :00:02.0: disabling bridge window [mem 0xd-0xd000f] to [bus 01-01] (unused) BTW, there's no problem accessing PCI_COMMAND register, as bus mastering is enabled in the bridge. On the other hand, as Ben suggested, handling this by specific hot-plug driver would be one of the other candidate to fix the problem. I'm not opposed to this idea, it's just that this bridge worked in an older system based on linux-2.6.22 and patched fakephp driver was used for hotplug. There's existing userspace software that I don't really want to modify heavily. But I'll do that if generic PCI rescan cannot be fixed. Thanks a lot for your help. Felix. ___ Linuxppc-dev mailing list Linuxppc-dev@lists.ozlabs.org https://lists.ozlabs.org/listinfo/linuxppc-dev
Re: Problem with PCI bus rescan on 460EX
Felix Radensky wrote: Hello, Kenji-san I think the device is expected to be ready to work if pci_enable_device() returns without error. So I think pci_enable_device() should return an error if it fails to enable the device (device is not ready to work). In this case, detecting your bridge's failure seems PPC specific to me. So I thought pcibios_enable_device() was the right to return an error. If pcibios_enable_device() returned an error, pci_dev-enable_cnt would decremented by pci_enable_device() (like pci_disable_device() does) and this problem would not happen. As far as I can see on 460EX pcibios_enable_device() just calls pci_enable_resources() which does not return any error for my bridge, although it doesn't find any memory or I/O resource it can enable. Do you think it is correct behavior ? Another question is whether by bridge behaves correctly when no device is connected to it. As you can see in dmesg output I've sent earlier pci :00:02.0: bridge window [mem 0x-0x000f] pci :00:02.0: bridge window [mem 0x-0x000f 64bit pref] and later PCI code disables these memory windows pci :00:02.0: disabling bridge window [mem 0xd-0xd000f pref] to [bus 01-01] (unused) pci :00:02.0: disabling bridge window [mem 0xd-0xd000f] to [bus 01-01] (unused) I misunderstood the problem. My understanding was memory resource was not enabled even though Linux set the Memory Space bit in the command register. But it was not correct. The bridge memory window was marked unused and Linux didn't try to set Memory Space bit in the command register. Current my understanding is as follows. Please correct me if I'm still misunderstanding something. 1) Your BIOS doesn't assign any resource to the bridge if its child PCI hot-plug slot is not occupied. 2) At the boot time, pci_assign_unassigned_resources() try to assign memory resouces to the bridge using pci_bus_assign_resource(), but it was disabled because there are no devices require memory resource. 3) And then pci_assign_unassigned_resouces() calls pci_enable_bridge(), but Memory Space bit in the command register was not set because no memory resource are assigned to the bridge. At the same time, pci_dev-enable_cnt was incremented. 4) At the rescan time, pci_setup_bridge() and pci_enable_bridge() doesn't work because the bridge is already marked enabled (i.e. pci_dev-enable_cnt is not zero). I don't have any concrete idea how to fix that so far, but I can say my idea (pcibios_enable_device() should return an error) was wrong. BTW, on my PCI hotplug capable system (SHPC and PCIe), I/O and Memory windows of the bridge are assigned by BIOS regardless of whether hotplug slot(s) behind the bridge is occupied or not. Maybe that is the reason why I have never encountered this problem before. Thanks, Kenji Kaneshige BTW, there's no problem accessing PCI_COMMAND register, as bus mastering is enabled in the bridge. On the other hand, as Ben suggested, handling this by specific hot-plug driver would be one of the other candidate to fix the problem. I'm not opposed to this idea, it's just that this bridge worked in an older system based on linux-2.6.22 and patched fakephp driver was used for hotplug. There's existing userspace software that I don't really want to modify heavily. But I'll do that if generic PCI rescan cannot be fixed. Thanks a lot for your help. Felix. -- To unsubscribe from this list: send the line unsubscribe linux-pci in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html ___ Linuxppc-dev mailing list Linuxppc-dev@lists.ozlabs.org https://lists.ozlabs.org/listinfo/linuxppc-dev
Re: Problem with PCI bus rescan on 460EX
Felix Radensky wrote:
Hello Kenji-san,
Kenji Kaneshige wrote:
I'm not sure, but I guess pci_setup_bridge() didn't update IO base/limit
and Mem base/limit of the bridge (:00:02.0) because of the following
lines.
static void pci_setup_bridge(struct pci_bus *bus)
{
struct pci_dev *bridge = bus-self;
struct resource *res;
struct pci_bus_region region;
u32 l, bu, lu, io_upper16;
if (pci_is_enabled(bridge))
===
return;===
dev_info(bridge-dev, PCI bridge to [bus %02x-%02x]\n,
bus-secondary, bus-subordinate);
...
It seems the bridge was already enabled by
pci_assign_unassigned_resources()
at boot time. Does removing those lines make any change?
Yes, with these lines removed bridge memory window is properly allocated.
These lines are to prevent updating IO or memory windows when there are
some devices working behind the bridge. So please note that removing
these lines is just for debugging.
For some reason bridge memory is disabled, but if I enable it via
setpci, and
also enable device memory, then everything works fine. If the system is
booted
when device behind the bridge is plugged in, bridge memory is enabled.
Maybe because of the following lines, I guess.
void pci_enable_bridges(struct pci_bus *bus)
{
struct pci_dev *dev;
int retval;
list_for_each_entry(dev, bus-devices, bus_list) {
if (dev-subordinate) {
if (!pci_is_enabled(dev)) { ===
retval = pci_enable_device(dev);===
pci_set_master(dev); ===
}
...
One possible cause is that pcibios_enable_device() for the bridge didn't
return any error even though it didn't work properly (ex. cannot access
to the command register, and so on) on your platform when there is no
device behind the bridge. But it is just my guess.
Thanks,
Kenji Kaneshige
Thanks a lot for your help.
Felix.
--
To unsubscribe from this list: send the line unsubscribe linux-pci in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
___
Linuxppc-dev mailing list
Linuxppc-dev@lists.ozlabs.org
https://lists.ozlabs.org/listinfo/linuxppc-dev
Re: Problem with PCI bus rescan on 460EX
On Mon, 2010-03-15 at 14:39 +0900, Kenji Kaneshige wrote: Yes, with these lines removed bridge memory window is properly allocated. These lines are to prevent updating IO or memory windows when there are some devices working behind the bridge. So please note that removing these lines is just for debugging. This is not a very good way to do so though. Many firmwares will leave those enabled even when the bridges -do- have to be reconfigured. Worse... the code means that the kernel -will- have updated its resource tree, probably assigned new resources, etc... to those bridges, and thus to devices underneath them, but will silently fail to actually update the bridge BARs to match. This seems like a major bug to me in fact. Cheers, Ben. ___ Linuxppc-dev mailing list Linuxppc-dev@lists.ozlabs.org https://lists.ozlabs.org/listinfo/linuxppc-dev
Re: Problem with PCI bus rescan on 460EX
On Mon, 2010-03-15 at 16:46 +1100, Benjamin Herrenschmidt wrote: On Mon, 2010-03-15 at 14:39 +0900, Kenji Kaneshige wrote: Yes, with these lines removed bridge memory window is properly allocated. These lines are to prevent updating IO or memory windows when there are some devices working behind the bridge. So please note that removing these lines is just for debugging. This is not a very good way to do so though. Many firmwares will leave those enabled even when the bridges -do- have to be reconfigured. Bah, my bad ... pci_is_enabled() is confusing... At first sight I would have thought it would return whether the device command register MEM or IO enable are set. Instead, it returns whether pci_enable_device() has been called. In any case, Felix bridge looks a tad weird. It should really be handled by a specific hotplug driver imho. Cheers, Ben. ___ Linuxppc-dev mailing list Linuxppc-dev@lists.ozlabs.org https://lists.ozlabs.org/listinfo/linuxppc-dev
Re: Problem with PCI bus rescan on 460EX
I'm not sure, but I guess pci_setup_bridge() didn't update IO base/limit
and Mem base/limit of the bridge (:00:02.0) because of the following
lines.
static void pci_setup_bridge(struct pci_bus *bus)
{
struct pci_dev *bridge = bus-self;
struct resource *res;
struct pci_bus_region region;
u32 l, bu, lu, io_upper16;
if (pci_is_enabled(bridge)) ===
return; ===
dev_info(bridge-dev, PCI bridge to [bus %02x-%02x]\n,
bus-secondary, bus-subordinate);
...
It seems the bridge was already enabled by pci_assign_unassigned_resources()
at boot time. Does removing those lines make any change?
Thanks,
Kenji Kaneshige
Felix Radensky wrote:
Hi Alex,
Resending, previous attempt was erroneously send as HTML.
Thanks a lot for replying.
Alex Chiang wrote:
* Felix Radensky fe...@embedded-sol.com:
The problem arises when device is plugged in after boot. After doing
echo 1 /sys/bus/pci/rescan
the device is identified, but bridge memory window is not allocated,
and reads from device memory regions return 0x. Below is
relevant output:
Do you need firmware support on your platform for hotplug?
I don't think so, but I've added powerpc guys to CC to make sure.
Can you please send full dmesg during successful boot, full dmesg
log during unsuccessful rescan, and contents of /proc/iomem and
/proc/ioports during success and failure cases?
Be sure you have PCI_CONFIG_DEBUG turned on.
Attached. I really appreciate your help. Thanks a lot.
Felix.
___
Linuxppc-dev mailing list
Linuxppc-dev@lists.ozlabs.org
https://lists.ozlabs.org/listinfo/linuxppc-dev
Re: Problem with PCI bus rescan on 460EX
Hello Kenji-san,
Kenji Kaneshige wrote:
I'm not sure, but I guess pci_setup_bridge() didn't update IO base/limit
and Mem base/limit of the bridge (:00:02.0) because of the following
lines.
static void pci_setup_bridge(struct pci_bus *bus)
{
struct pci_dev *bridge = bus-self;
struct resource *res;
struct pci_bus_region region;
u32 l, bu, lu, io_upper16;
if (pci_is_enabled(bridge))
===
return;===
dev_info(bridge-dev, PCI bridge to [bus %02x-%02x]\n,
bus-secondary, bus-subordinate);
...
It seems the bridge was already enabled by
pci_assign_unassigned_resources()
at boot time. Does removing those lines make any change?
Yes, with these lines removed bridge memory window is properly allocated.
For some reason bridge memory is disabled, but if I enable it via
setpci, and
also enable device memory, then everything works fine. If the system is
booted
when device behind the bridge is plugged in, bridge memory is enabled.
Thanks a lot for your help.
Felix.
___
Linuxppc-dev mailing list
Linuxppc-dev@lists.ozlabs.org
https://lists.ozlabs.org/listinfo/linuxppc-dev
Re: Problem with PCI bus rescan on 460EX
Hi Alex, Thanks a lot for replying. Alex Chiang wrote: * Felix Radensky fe...@embedded-sol.com: The problem arises when device is plugged in after boot. After doing echo 1 /sys/bus/pci/rescan the device is identified, but bridge memory window is not allocated, and reads from device memory regions return 0x. Below is relevant output: Do you need firmware support on your platform for hotplug? I don't think so, but I've added powerpc guys to CC to make sure. Can you please send full dmesg during successful boot, full dmesg log during unsuccessful rescan, and contents of /proc/iomem and /proc/ioports during success and failure cases? Be sure you have PCI_CONFIG_DEBUG turned on. Attached. I really appreciate your help. Thanks a lot. Felix. Linux version 2.6.33 (fe...@felix-laptop.lan) (gcc version 4.2.2) #5 Thu Mar 11 09:35:52 IST 2010 Found legacy serial port 0 for /plb/opb/ser...@ef600300 mem=4ef600300, taddr=4ef600300, irq=0, clk=7407407, speed=0 Found legacy serial port 1 for /plb/opb/ser...@ef600400 mem=4ef600400, taddr=4ef600400, irq=0, clk=7407407, speed=0 Found legacy serial port 2 for /plb/opb/ser...@ef600500 mem=4ef600500, taddr=4ef600500, irq=0, clk=7407407, speed=0 Found legacy serial port 3 for /plb/opb/ser...@ef600600 mem=4ef600600, taddr=4ef600600, irq=0, clk=7407407, speed=0 Top of RAM: 0x3000, Total RAM: 0x3000 Memory hole size: 0MB Zone PFN ranges: DMA 0x - 0x0003 Normal 0x0003 - 0x0003 Movable zone start PFN for each node early_node_map[1] active PFN ranges 0: 0x - 0x0003 On node 0 totalpages: 196608 free_area_init_node: node 0, pgdat c035e76c, node_mem_map c0388000 DMA zone: 1536 pages used for memmap DMA zone: 0 pages reserved DMA zone: 195072 pages, LIFO batch:31 MMU: Allocated 1088 bytes of context maps for 255 contexts Built 1 zonelists in Zone order, mobility grouping on. Total pages: 195072 Kernel command line: root=/dev/nfs rw nfsroot=10.0.0.10:/opt/eldk/ppc_4xxFP ip=10.0.0.30:10.0.0.10:10.0.0.138:255.0.0.0:smbe460:eth0:off panic=1 console=ttyS0,115200 PID hash table entries: 4096 (order: 2, 16384 bytes) Dentry cache hash table entries: 131072 (order: 7, 524288 bytes) Inode-cache hash table entries: 65536 (order: 6, 262144 bytes) Memory: 775552k/786432k available (3364k kernel code, 10604k reserved, 116k data, 128k bss, 144k init) Kernel virtual memory layout: * 0xfffdf000..0xf000 : fixmap * 0xfde0..0xfe00 : consistent mem * 0xfde0..0xfde0 : early ioremap * 0xf100..0xfde0 : vmalloc ioremap SLUB: Genslabs=11, HWalign=32, Order=0-3, MinObjects=0, CPUs=1, Nodes=1 Hierarchical RCU implementation. NR_IRQS:512 nr_irqs:512 UIC0 (32 IRQ sources) at DCR 0xc0 UIC1 (32 IRQ sources) at DCR 0xd0 alloc irq_desc for 30 on node 0 alloc kstat_irqs on node 0 irq: irq 30 on host /interrupt-controller0 mapped to virtual irq 30 UIC2 (32 IRQ sources) at DCR 0xe0 alloc irq_desc for 16 on node 0 alloc kstat_irqs on node 0 irq: irq 10 on host /interrupt-controller0 mapped to virtual irq 16 UIC3 (32 IRQ sources) at DCR 0xf0 alloc irq_desc for 17 on node 0 alloc kstat_irqs on node 0 irq: irq 16 on host /interrupt-controller0 mapped to virtual irq 17 time_init: decrementer frequency = 1000.12 MHz time_init: processor frequency = 1000.12 MHz clocksource: timebase mult[40] shift[22] registered clockevent: decrementer mult[8019] shift[31] cpu[0] Mount-cache hash table entries: 512 NET: Registered protocol family 16 alloc irq_desc for 18 on node 0 alloc kstat_irqs on node 0 irq: irq 11 on host /interrupt-controller1 mapped to virtual irq 18 256k L2-cache enabled PCI host bridge /plb/p...@c0ec0 (primary) ranges: MEM 0x000d8000..0x000d - 0x8000 MEM 0x000c0ee0..0x000c0eef - 0x IO 0x000c0800..0x000c0800 - 0x Removing ISA hole at 0x000c0ee0 4xx PCI DMA offset set to 0x /plb/p...@c0ec0: Legacy ISA memory support enabled PCI: Probing PCI hardware pci_bus :00: scanning bus pci :00:02.0: found [3388:0020] class 000604 header type 01 pci :00:02.0: calling pcibios_fixup_resources+0x0/0xf4 pci :00:02.0: calling fixup_ppc4xx_pci_bridge+0x0/0x154 pci :00:02.0: calling quirk_resource_alignment+0x0/0x200 pci :00:02.0: supports D1 D2 pci :00:02.0: PME# supported from D0 D1 D2 D3hot pci :00:02.0: PME# disabled pci_bus :00: fixups for bus pci :00:02.0: scanning behind bridge, config 00, pass 0 pci :00:02.0: bus configuration invalid, reconfiguring pci :00:02.0: scanning behind bridge, config 00, pass 1 pci_bus :01: scanning bus pci_bus :01: fixups for bus pci :00:02.0: PCI bridge to [bus 01-ff] pci :00:02.0: bridge window [io 0x-0x0fff] pci :00:02.0: bridge window [mem 0x-0x000f] pci :00:02.0: bridge window
Re: Problem with PCI bus rescan on 460EX
On Thu, 2010-03-11 at 09:45 +0200, Felix Radensky wrote: Hi Alex, Thanks a lot for replying. Alex Chiang wrote: * Felix Radensky fe...@embedded-sol.com: The problem arises when device is plugged in after boot. After doing echo 1 /sys/bus/pci/rescan the device is identified, but bridge memory window is not allocated, and reads from device memory regions return 0x. Below is relevant output: Do you need firmware support on your platform for hotplug? I don't think so, but I've added powerpc guys to CC to make sure. Can you please send full dmesg during successful boot, full dmesg log during unsuccessful rescan, and contents of /proc/iomem and /proc/ioports during success and failure cases? Be sure you have PCI_CONFIG_DEBUG turned on. Attached. I really appreciate your help. Thanks a lot. Yes, we need to do a resource allocation pass, setup DMA, etc... and that is not done in that manual rescan case I suppose. I have to look. Part of the problem is that there is no proper hooks in the generic PCI code that I know of for that, but I'll have to double check the code, things might have changed. For boot time, we do this after we scan busses and before we add the devices to sysfs. For hotplug, our hotplug drivers do something similar. But that rescan sysfs hook seems to go directly into drivers/pci causing a rescan but without a change to re-allocate resources etc... You may be better off implementing a minimum hotplug driver I suppose... Cheers, Ben. Felix . plain text document attachment (failure.txt) Linux version 2.6.33 (fe...@felix-laptop.lan) (gcc version 4.2.2) #5 Thu Mar 11 09:35:52 IST 2010 Found legacy serial port 0 for /plb/opb/ser...@ef600300 mem=4ef600300, taddr=4ef600300, irq=0, clk=7407407, speed=0 Found legacy serial port 1 for /plb/opb/ser...@ef600400 mem=4ef600400, taddr=4ef600400, irq=0, clk=7407407, speed=0 Found legacy serial port 2 for /plb/opb/ser...@ef600500 mem=4ef600500, taddr=4ef600500, irq=0, clk=7407407, speed=0 Found legacy serial port 3 for /plb/opb/ser...@ef600600 mem=4ef600600, taddr=4ef600600, irq=0, clk=7407407, speed=0 Top of RAM: 0x3000, Total RAM: 0x3000 Memory hole size: 0MB Zone PFN ranges: DMA 0x - 0x0003 Normal 0x0003 - 0x0003 Movable zone start PFN for each node early_node_map[1] active PFN ranges 0: 0x - 0x0003 On node 0 totalpages: 196608 free_area_init_node: node 0, pgdat c035e76c, node_mem_map c0388000 DMA zone: 1536 pages used for memmap DMA zone: 0 pages reserved DMA zone: 195072 pages, LIFO batch:31 MMU: Allocated 1088 bytes of context maps for 255 contexts Built 1 zonelists in Zone order, mobility grouping on. Total pages: 195072 Kernel command line: root=/dev/nfs rw nfsroot=10.0.0.10:/opt/eldk/ppc_4xxFP ip=10.0.0.30:10.0.0.10:10.0.0.138:255.0.0.0:smbe460:eth0:off panic=1 console=ttyS0,115200 PID hash table entries: 4096 (order: 2, 16384 bytes) Dentry cache hash table entries: 131072 (order: 7, 524288 bytes) Inode-cache hash table entries: 65536 (order: 6, 262144 bytes) Memory: 775552k/786432k available (3364k kernel code, 10604k reserved, 116k data, 128k bss, 144k init) Kernel virtual memory layout: * 0xfffdf000..0xf000 : fixmap * 0xfde0..0xfe00 : consistent mem * 0xfde0..0xfde0 : early ioremap * 0xf100..0xfde0 : vmalloc ioremap SLUB: Genslabs=11, HWalign=32, Order=0-3, MinObjects=0, CPUs=1, Nodes=1 Hierarchical RCU implementation. NR_IRQS:512 nr_irqs:512 UIC0 (32 IRQ sources) at DCR 0xc0 UIC1 (32 IRQ sources) at DCR 0xd0 alloc irq_desc for 30 on node 0 alloc kstat_irqs on node 0 irq: irq 30 on host /interrupt-controller0 mapped to virtual irq 30 UIC2 (32 IRQ sources) at DCR 0xe0 alloc irq_desc for 16 on node 0 alloc kstat_irqs on node 0 irq: irq 10 on host /interrupt-controller0 mapped to virtual irq 16 UIC3 (32 IRQ sources) at DCR 0xf0 alloc irq_desc for 17 on node 0 alloc kstat_irqs on node 0 irq: irq 16 on host /interrupt-controller0 mapped to virtual irq 17 time_init: decrementer frequency = 1000.12 MHz time_init: processor frequency = 1000.12 MHz clocksource: timebase mult[40] shift[22] registered clockevent: decrementer mult[8019] shift[31] cpu[0] Mount-cache hash table entries: 512 NET: Registered protocol family 16 alloc irq_desc for 18 on node 0 alloc kstat_irqs on node 0 irq: irq 11 on host /interrupt-controller1 mapped to virtual irq 18 256k L2-cache enabled PCI host bridge /plb/p...@c0ec0 (primary) ranges: MEM 0x000d8000..0x000d - 0x8000 MEM 0x000c0ee0..0x000c0eef - 0x IO 0x000c0800..0x000c0800 - 0x Removing ISA hole at 0x000c0ee0 4xx PCI DMA offset set to 0x /plb/p...@c0ec0: Legacy ISA memory support enabled PCI: Probing PCI
Re: Problem with PCI bus rescan on 460EX
Hi Ben, Benjamin Herrenschmidt wrote: Yes, we need to do a resource allocation pass, setup DMA, etc... and that is not done in that manual rescan case I suppose. I have to look. Part of the problem is that there is no proper hooks in the generic PCI code that I know of for that, but I'll have to double check the code, things might have changed. For boot time, we do this after we scan busses and before we add the devices to sysfs. For hotplug, our hotplug drivers do something similar. But that rescan sysfs hook seems to go directly into drivers/pci causing a rescan but without a change to re-allocate resources etc... You may be better off implementing a minimum hotplug driver I suppose... Cheers, Ben. I'm fine with creating a minimal hotplug driver. The device I'm dealing with partially implements Compact PCI hotplug. It generates ENUM# interrupt, but Hotswap Control register layout does not completely follow the standard. Should I use drivers/pci/hotplug/cpci_hotplug_core.c as a base, or do you have something more simple in mind ? Thanks a lot. Felix. ___ Linuxppc-dev mailing list Linuxppc-dev@lists.ozlabs.org https://lists.ozlabs.org/listinfo/linuxppc-dev
Re: Problem with PCI bus rescan on 460EX
On Thu, 2010-03-11 at 23:41 +0200, Felix Radensky wrote: I'm fine with creating a minimal hotplug driver. The device I'm dealing with partially implements Compact PCI hotplug. It generates ENUM# interrupt, but Hotswap Control register layout does not completely follow the standard. Should I use drivers/pci/hotplug/cpci_hotplug_core.c as a base, or do you have something more simple in mind ? I don't have anything special in mind, but you may need to look at how the pseries hotplug driver does to get those fixups and resource management things done. That driver is mostly a horrible mess, though I did clean quite a bit of it up recently... The main thing of interest to you is pcibios_add_pci_devices() which is called to probe below an existing bridge. It's currently in arch/powerpc/platforms/pseries/pci_dlpar.c but we could move it or some of it to our common code if needed. You also may safely ignore the eeh_* bits on ppc32 Cheers, Ben. ___ Linuxppc-dev mailing list Linuxppc-dev@lists.ozlabs.org https://lists.ozlabs.org/listinfo/linuxppc-dev
Re: Problem with PCI bus rescan on 460EX
Hi Alex, Resending, previous attempt was erroneously send as HTML. Thanks a lot for replying. Alex Chiang wrote: * Felix Radensky fe...@embedded-sol.com: The problem arises when device is plugged in after boot. After doing echo 1 /sys/bus/pci/rescan the device is identified, but bridge memory window is not allocated, and reads from device memory regions return 0x. Below is relevant output: Do you need firmware support on your platform for hotplug? I don't think so, but I've added powerpc guys to CC to make sure. Can you please send full dmesg during successful boot, full dmesg log during unsuccessful rescan, and contents of /proc/iomem and /proc/ioports during success and failure cases? Be sure you have PCI_CONFIG_DEBUG turned on. Attached. I really appreciate your help. Thanks a lot. Felix. Linux version 2.6.33 (fe...@felix-laptop.lan) (gcc version 4.2.2) #5 Thu Mar 11 09:35:52 IST 2010 Found legacy serial port 0 for /plb/opb/ser...@ef600300 mem=4ef600300, taddr=4ef600300, irq=0, clk=7407407, speed=0 Found legacy serial port 1 for /plb/opb/ser...@ef600400 mem=4ef600400, taddr=4ef600400, irq=0, clk=7407407, speed=0 Found legacy serial port 2 for /plb/opb/ser...@ef600500 mem=4ef600500, taddr=4ef600500, irq=0, clk=7407407, speed=0 Found legacy serial port 3 for /plb/opb/ser...@ef600600 mem=4ef600600, taddr=4ef600600, irq=0, clk=7407407, speed=0 Top of RAM: 0x3000, Total RAM: 0x3000 Memory hole size: 0MB Zone PFN ranges: DMA 0x - 0x0003 Normal 0x0003 - 0x0003 Movable zone start PFN for each node early_node_map[1] active PFN ranges 0: 0x - 0x0003 On node 0 totalpages: 196608 free_area_init_node: node 0, pgdat c035e76c, node_mem_map c0388000 DMA zone: 1536 pages used for memmap DMA zone: 0 pages reserved DMA zone: 195072 pages, LIFO batch:31 MMU: Allocated 1088 bytes of context maps for 255 contexts Built 1 zonelists in Zone order, mobility grouping on. Total pages: 195072 Kernel command line: root=/dev/nfs rw nfsroot=10.0.0.10:/opt/eldk/ppc_4xxFP ip=10.0.0.30:10.0.0.10:10.0.0.138:255.0.0.0:smbe460:eth0:off panic=1 console=ttyS0,115200 PID hash table entries: 4096 (order: 2, 16384 bytes) Dentry cache hash table entries: 131072 (order: 7, 524288 bytes) Inode-cache hash table entries: 65536 (order: 6, 262144 bytes) Memory: 775552k/786432k available (3364k kernel code, 10604k reserved, 116k data, 128k bss, 144k init) Kernel virtual memory layout: * 0xfffdf000..0xf000 : fixmap * 0xfde0..0xfe00 : consistent mem * 0xfde0..0xfde0 : early ioremap * 0xf100..0xfde0 : vmalloc ioremap SLUB: Genslabs=11, HWalign=32, Order=0-3, MinObjects=0, CPUs=1, Nodes=1 Hierarchical RCU implementation. NR_IRQS:512 nr_irqs:512 UIC0 (32 IRQ sources) at DCR 0xc0 UIC1 (32 IRQ sources) at DCR 0xd0 alloc irq_desc for 30 on node 0 alloc kstat_irqs on node 0 irq: irq 30 on host /interrupt-controller0 mapped to virtual irq 30 UIC2 (32 IRQ sources) at DCR 0xe0 alloc irq_desc for 16 on node 0 alloc kstat_irqs on node 0 irq: irq 10 on host /interrupt-controller0 mapped to virtual irq 16 UIC3 (32 IRQ sources) at DCR 0xf0 alloc irq_desc for 17 on node 0 alloc kstat_irqs on node 0 irq: irq 16 on host /interrupt-controller0 mapped to virtual irq 17 time_init: decrementer frequency = 1000.12 MHz time_init: processor frequency = 1000.12 MHz clocksource: timebase mult[40] shift[22] registered clockevent: decrementer mult[8019] shift[31] cpu[0] Mount-cache hash table entries: 512 NET: Registered protocol family 16 alloc irq_desc for 18 on node 0 alloc kstat_irqs on node 0 irq: irq 11 on host /interrupt-controller1 mapped to virtual irq 18 256k L2-cache enabled PCI host bridge /plb/p...@c0ec0 (primary) ranges: MEM 0x000d8000..0x000d - 0x8000 MEM 0x000c0ee0..0x000c0eef - 0x IO 0x000c0800..0x000c0800 - 0x Removing ISA hole at 0x000c0ee0 4xx PCI DMA offset set to 0x /plb/p...@c0ec0: Legacy ISA memory support enabled PCI: Probing PCI hardware pci_bus :00: scanning bus pci :00:02.0: found [3388:0020] class 000604 header type 01 pci :00:02.0: calling pcibios_fixup_resources+0x0/0xf4 pci :00:02.0: calling fixup_ppc4xx_pci_bridge+0x0/0x154 pci :00:02.0: calling quirk_resource_alignment+0x0/0x200 pci :00:02.0: supports D1 D2 pci :00:02.0: PME# supported from D0 D1 D2 D3hot pci :00:02.0: PME# disabled pci_bus :00: fixups for bus pci :00:02.0: scanning behind bridge, config 00, pass 0 pci :00:02.0: bus configuration invalid, reconfiguring pci :00:02.0: scanning behind bridge, config 00, pass 1 pci_bus :01: scanning bus pci_bus :01: fixups for bus pci :00:02.0: PCI bridge to [bus 01-ff] pci :00:02.0: bridge window [io 0x-0x0fff] pci :00:02.0: bridge window [mem
Problem with PCI bus rescan on 460EX
Hi, I'm running linux-2.6.33 on a custom board based on 460EX. There's a PCI-PCI bridge on this board, PLX 6254, and a single hot-pluggable device behind the bridge. When this device is plugged in before system boots everything works fine: bridge and device are properly recognized by kernel, resources are allocated and device memory regions are accessible. Below is relevant kernel messages and lspci output: PCI: Probing PCI hardware pci_bus :00: scanning bus pci :00:02.0: found [3388:0020] class 000604 header type 01 pci :00:02.0: calling pcibios_fixup_resources+0x0/0xf4 pci :00:02.0: calling fixup_ppc4xx_pci_bridge+0x0/0x154 pci :00:02.0: calling quirk_resource_alignment+0x0/0x200 pci :00:02.0: supports D1 D2 pci :00:02.0: PME# supported from D0 D1 D2 D3hot pci :00:02.0: PME# disabled pci_bus :00: fixups for bus pci :00:02.0: scanning behind bridge, config 010100, pass 0 pci_bus :01: scanning bus pci :01:00.0: found [1575:0002] class 00ff00 header type 00 pci :01:00.0: reg 10: [mem 0x8000-0x800f] pci :01:00.0: reg 14: [mem 0x8400-0x87ff] pci :01:00.0: reg 18: [mem 0x8800-0x8bff] pci :01:00.0: reg 1c: [mem 0x8c00-0x8c003fff] pci :01:00.0: calling pcibios_fixup_resources+0x0/0xf4 pci :01:00.0: calling fixup_ppc4xx_pci_bridge+0x0/0x154 pci :01:00.0: calling quirk_resource_alignment+0x0/0x200 pci_bus :01: fixups for bus pci :00:02.0: PCI bridge to [bus 01-01] pci :00:02.0: bridge window [mem 0x8000-0x8c0f] pci_bus :01: bus scan returning with max=01 pci :00:02.0: scanning behind bridge, config 010100, pass 1 pci_bus :00: bus scan returning with max=01 pci :00:02.0: BAR 8: assigned [mem 0xd8000-0xd89ff] pci :01:00.0: BAR 1: assigned [mem 0xd8000-0xd83ff] pci :01:00.0: BAR 1: set to [mem 0xd8000-0xd83ff] (PCI address [0x8000-0x83ff] pci :01:00.0: BAR 2: assigned [mem 0xd8400-0xd87ff] pci :01:00.0: BAR 2: set to [mem 0xd8400-0xd87ff] (PCI address [0x8400-0x87ff] pci :01:00.0: BAR 0: assigned [mem 0xd8800-0xd880f] pci :01:00.0: BAR 0: set to [mem 0xd8800-0xd880f] (PCI address [0x8800-0x880f] pci :01:00.0: BAR 3: assigned [mem 0xd8810-0xd88103fff] pci :01:00.0: BAR 3: set to [mem 0xd8810-0xd88103fff] (PCI address [0x8810-0x88103fff] pci :00:02.0: PCI bridge to [bus 01-01] pci :00:02.0: bridge window [io disabled] pci :00:02.0: bridge window [mem 0xd8000-0xd89ff] pci :00:02.0: bridge window [mem pref disabled] pci_bus :00: resource 0 [io 0x-0x] pci_bus :00: resource 1 [mem 0xd8000-0xd] pci_bus :01: resource 1 [mem 0xd8000-0xd89ff] 00:02.0 PCI bridge: Hint Corp HB6 Universal PCI-PCI bridge (transparent mode) (rev 04) 00: 88 33 20 00 87 00 b0 02 04 00 04 06 08 80 01 00 10: 00 00 00 00 00 00 00 00 00 01 01 00 f1 01 a0 02 20: 00 80 f0 89 f1 ff 01 00 00 00 00 00 00 00 00 00 30: 00 00 00 00 dc 00 00 00 00 00 00 00 00 00 00 00 The problem arises when device is plugged in after boot. After doing echo 1 /sys/bus/pci/rescan the device is identified, but bridge memory window is not allocated, and reads from device memory regions return 0x. Below is relevant output: PCI: Probing PCI hardware pci_bus :00: scanning bus pci :00:02.0: found [3388:0020] class 000604 header type 01 pci :00:02.0: calling pcibios_fixup_resources+0x0/0xf4 pci :00:02.0: calling fixup_ppc4xx_pci_bridge+0x0/0x154 pci :00:02.0: calling quirk_resource_alignment+0x0/0x200 pci :00:02.0: supports D1 D2 pci :00:02.0: PME# supported from D0 D1 D2 D3hot pci :00:02.0: PME# disabled pci_bus :00: fixups for bus pci :00:02.0: scanning behind bridge, config 010100, pass 0 pci_bus :01: scanning bus pci_bus :01: fixups for bus pci :00:02.0: PCI bridge to [bus 01-01] pci_bus :01: bus scan returning with max=01 pci :00:02.0: scanning behind bridge, config 010100, pass 1 pci_bus :00: bus scan returning with max=01 pci :00:02.0: PCI bridge to [bus 01-01] pci :00:02.0: bridge window [io disabled] pci :00:02.0: bridge window [mem disabled] pci :00:02.0: bridge window [mem pref disabled] pci_bus :00: resource 0 [io 0x-0x] pci_bus :00: resource 1 [mem 0xd8000-0xd] pci :00:02.0: calling quirk_cardbus_legacy+0x0/0x54 pci :00:02.0: calling quirk_usb_early_handoff+0x0/0x6cc PCI: CLS 32 bytes, default 32 pci_bus :00: scanning bus pci :00:02.0: scanning behind bridge, config 010100, pass 0 pci_bus :01: scanning bus pci :01:00.0: found [1575:0002] class 00ff00 header type 00 pci :01:00.0: reg 10: [mem 0x-0x000f] pci :01:00.0: reg 14: [mem 0x-0x03ff] pci :01:00.0: reg 18: [mem 0x-0x03ff] pci :01:00.0: reg 1c: [mem 0x-0x3fff]
