Re: [PATCH v8 00/45] powerpc/powernv: PCI hotplug support

2016-04-19 Thread Gavin Shan 
On Fri, Apr 15, 2016 at 11:10:21AM -0500, Rob Herring wrote:
>On Wed, Apr 13, 2016 at 8:30 PM, Gavin Shan  wrote:
>> On Thu, Apr 14, 2016 at 09:57:32AM +1000, Alistair Popple wrote:
>>>Hi Gavin,
>>>
>>>
>>>
 >Why exactly cannot EEH reset changes go to a smaller separate patchset
 >(before hotplug)?
 >

 As I explained before, the patchset's order is: PCI generic part,
 PowerNV PCI related, EEH related, device-tree part and hotplug driver.

 The EEH reset change is included in PATCH[37/45]. There is no point
 to reorder the patches.
>>>
>>>I don't understand all of the dependencies but if possible splitting the
>>>series up into a set of smaller self-contained patch series makes things
>>>easier to review and may make it easier for you to get this functionality
>>>reviewed and accepted into upstream.
>>>
>>
>> Thanks, Alistair. I will move those cleanup/refactor related patches
>> to form a separate series which is expected to be merged first. That
>> will helps the reviewers to focus on the patches with complicated
>> changes as you suggested. Alexey, please let me know if that way is
>> you like to see or not.
>
>As I said last cycle, I'll happily take the DT refactoring patches
>separately, but you have to tell me if you want me to apply them and
>it has to be well before the merge window.
>

Thanks, Rob. I hope to post next revision (v9) soon and the device-tree
related cleanup patches should be ready for next merge window in it.

>Rob
>

___
Linuxppc-dev mailing list
Linuxppc-dev@lists.ozlabs.org
https://lists.ozlabs.org/listinfo/linuxppc-dev

Re: [PATCH v8 00/45] powerpc/powernv: PCI hotplug support

2016-04-15 Thread Rob Herring 
On Wed, Apr 13, 2016 at 8:30 PM, Gavin Shan  wrote:
> On Thu, Apr 14, 2016 at 09:57:32AM +1000, Alistair Popple wrote:
>>Hi Gavin,
>>
>>
>>
>>> >Why exactly cannot EEH reset changes go to a smaller separate patchset
>>> >(before hotplug)?
>>> >
>>>
>>> As I explained before, the patchset's order is: PCI generic part,
>>> PowerNV PCI related, EEH related, device-tree part and hotplug driver.
>>>
>>> The EEH reset change is included in PATCH[37/45]. There is no point
>>> to reorder the patches.
>>
>>I don't understand all of the dependencies but if possible splitting the
>>series up into a set of smaller self-contained patch series makes things
>>easier to review and may make it easier for you to get this functionality
>>reviewed and accepted into upstream.
>>
>
> Thanks, Alistair. I will move those cleanup/refactor related patches
> to form a separate series which is expected to be merged first. That
> will helps the reviewers to focus on the patches with complicated
> changes as you suggested. Alexey, please let me know if that way is
> you like to see or not.

As I said last cycle, I'll happily take the DT refactoring patches
separately, but you have to tell me if you want me to apply them and
it has to be well before the merge window.

Rob
___
Linuxppc-dev mailing list
Linuxppc-dev@lists.ozlabs.org
https://lists.ozlabs.org/listinfo/linuxppc-dev

Re: [PATCH v8 00/45] powerpc/powernv: PCI hotplug support

2016-04-13 Thread Gavin Shan 
On Thu, Apr 14, 2016 at 01:26:51PM +1000, Alexey Kardashevskiy wrote:

.../...

>>
>>Do you mean physically pull the adapter out and insert the same
>>adapter back? What's the point for the test case?
>
>
>Because this is what the patchset is for - to replace a physical device on a
>physical machine. Powering on/off the slots via sysfs is just an
>approximation (which is fine when you are debugging), something can go wrong
>and require some work but you do not know it for sure.
>

Yes, It's absolutely worthy to be covered by the test cases though case (2)
covers part of that. Anyway, I'll test it through in next revision. Thanks
for your review.

>
>
>-- 
>Alexey
>

___
Linuxppc-dev mailing list
Linuxppc-dev@lists.ozlabs.org
https://lists.ozlabs.org/listinfo/linuxppc-dev

Re: [PATCH v8 00/45] powerpc/powernv: PCI hotplug support

2016-04-13 Thread Alexey Kardashevskiy 

On 04/14/2016 11:30 AM, Gavin Shan wrote:

On Thu, Apr 14, 2016 at 09:57:32AM +1000, Alistair Popple wrote:

Hi Gavin,




Why exactly cannot EEH reset changes go to a smaller separate patchset
(before hotplug)?



As I explained before, the patchset's order is: PCI generic part,
PowerNV PCI related, EEH related, device-tree part and hotplug driver.

The EEH reset change is included in PATCH[37/45]. There is no point
to reorder the patches.


I don't understand all of the dependencies but if possible splitting the
series up into a set of smaller self-contained patch series makes things
easier to review and may make it easier for you to get this functionality
reviewed and accepted into upstream.



Thanks, Alistair. I will move those cleanup/refactor related patches
to form a separate series which is expected to be merged first. That
will helps the reviewers to focus on the patches with complicated
changes as you suggested. Alexey, please let me know if that way is
you like to see or not.


I do not know yet, I have not finished reviewing this version. May be the 
EEH reset patch depends on 1/45..36/45; or it only makes sense when 45/45 
is applied - this all is unclear.


If 37/45 has no dependencies and good just by itself, you could have posted 
it separately few months ago and it would have reached upstream by now and 
this patchset would be at least one patch shorter and you would not have to 
rebase all 45 patches over and over again on top of the current upstream 
tree...




--
Alexey
___
Linuxppc-dev mailing list
Linuxppc-dev@lists.ozlabs.org
https://lists.ozlabs.org/listinfo/linuxppc-dev

Re: [PATCH v8 00/45] powerpc/powernv: PCI hotplug support

2016-04-13 Thread Alexey Kardashevskiy 

On 04/14/2016 09:42 AM, Gavin Shan wrote:

On Wed, Apr 13, 2016 at 07:14:59PM +1000, Alexey Kardashevskiy wrote:

On 04/13/2016 05:42 PM, Gavin Shan wrote:

On Wed, Apr 13, 2016 at 05:28:15PM +1000, Alexey Kardashevskiy wrote:

On 02/17/2016 02:43 PM, Gavin Shan wrote:

This series of patches rebases on powerpc/next branch, plus below additional
patches:



https://patchwork.ozlabs.org/patch/581315/  (PATCH[1/9] Richard's SRIOV EEH)
https://patchwork.ozlabs.org/patch/582639/  (PATCH[1/1] Gavin's EEH fix)
https://patchwork.ozlabs.org/patch/582093/  (PATCH[1/1] Gavin's EEH fix)
https://patchwork.ozlabs.org/patch/580626/  (PATCH[1/4] Gavin's PCI fix)
https://patchwork.ozlabs.org/patch/580153/  (PATCH[1/1] Andrew's EEH minor 
fix)
https://patchwork.ozlabs.org/patch/566827/  (PATCH[1/1] Russell's P5IOC2 
removal)
https://patchwork.ozlabs.org/patch/534154/  (PATCH[1/7] Richard's SRIOV 
rework)
commit 388f7b1 ("Linux 4.5-rc3")

The series of patches intend to support PCI slot for PowerPC PowerNV platform,
which is running on top of skiboot firmware. The patchset requires corresponding
changes from skiboot firmware, which is sent to skib...@lists.ozlabs.org
for review. The PCI slots are exposed by skiboot with device node properties,
and kernel utilizes those properties to populated PCI slots accordingly.

The original PCI infrastructure on PowerNV platform can't support hotplug
because the PE is assigned during PHB fixup time, which is called for once
during system boot time. For this, the PCI infrastructure on PowerNV platform
has been reworked for a lot. After that, the PE and its corresponding resources
(IODT, M32DT, M64 segments, DMA32 and bypass window) are assigned upon updating
PCI bridge's resources, which might decide PE# assigned to the PE (e.g. M64
resources, on P8 strictly speaking). Each PE will maintain a reference count,
which is (number of child PCI devices + 1). That indicates when last child PCI
device leaves the PE, the PE and its included resources will be relased and put
back into free pool again. With this design, the PE will be released when EEH PE
is released. PATCH[1 - 23] are related to this part.

 From skiboot perspective, PCI slot is providing (hot/fundamental/complete)
resets to EEH. The kernel gets to know if skiboot supports various reset on one
particular PCI slot through device-tree node. If it does, EEH will utilize the
functionality provided by skiboot. Besides, the device-tree nodes have to change
in order to support PCI hotplug. For example, when one PCI adapter inserted to
one slot, its device-tree node should be added to the system dynamically. 
Conversely,
the device-tree node should be removed from the system when the PCI adapter is 
going
to be offline. Since pci_dn and eeh_dev have same life cyle as PCI device nodes,
they should be added/removed accordingly during PCI hotplug. PATCH[24 - 39] are
doing the related work.

The OF driver is changed to support unflattening FDT blob for sub-stree, which
is covered by PATCH[40 - 44].

The last one, PATCH[45], is the standalone PCI hotplug driver for PowerPC 
PowerNV
platform.

===
Testing
===
1. Unplug adapters behind non-empty slot, then plug them.

1.1 Check status
# cat /sys/bus/pci/slots/C10/address
0003:09:00
# cat /sys/bus/pci/slots/C10/adapter
1
# cat /sys/bus/pci/slots/C10/power
1
# lspci
0003:09:00.0 Ethernet controller: \
Broadcom Corporation NetXtreme BCM5719 Gigabit Ethernet PCIe (rev 01)
0003:09:00.1 Ethernet controller: \
Broadcom Corporation NetXtreme BCM5719 Gigabit Ethernet PCIe (rev 01)
0003:09:00.2 Ethernet controller: \
Broadcom Corporation NetXtreme BCM5719 Gigabit Ethernet PCIe (rev 01)
0003:09:00.3 Ethernet controller: \
Broadcom Corporation NetXtreme BCM5719 Gigabit Ethernet PCIe (rev 01)
# lspci -t
# lspci -t
-+-[0003:00]---00.0-[01-13]00.0-[02-13]--+-01.0-[03]00.0
 |   +-08.0-[04-08]--
 |   +-09.0-[09]--+-00.0
 |   |+-00.1
 |   |+-00.2
 |   |\-00.3
 |   +-10.0-[0a-0e]--
 |   \-11.0-[0f-13]--

1.2 Unplug adapter 0003:09.00.x
# echo 0 > /sys/bus/pci/slots/C10/power
# lspci -t
-+-[0003:00]---00.0-[01-13]00.0-[02-13]--+-01.0-[03]00.0
 |   +-08.0-[04-08]--
 |   +-09.0-[09]--
 |   +-10.0-[0a-0e]--
 |   \-11.0-[0f-13]--

1.3 Plug adapter 0003:09.00.x
# echo 1 > /sys/bus/pci/slots/C10/power



Do I understand correctly that the adapter

Re: [PATCH v8 00/45] powerpc/powernv: PCI hotplug support

2016-04-13 Thread Gavin Shan 
On Thu, Apr 14, 2016 at 09:57:32AM +1000, Alistair Popple wrote:
>Hi Gavin,
>
>
>
>> >Why exactly cannot EEH reset changes go to a smaller separate patchset
>> >(before hotplug)?
>> >
>> 
>> As I explained before, the patchset's order is: PCI generic part,
>> PowerNV PCI related, EEH related, device-tree part and hotplug driver.
>> 
>> The EEH reset change is included in PATCH[37/45]. There is no point
>> to reorder the patches.
>
>I don't understand all of the dependencies but if possible splitting the 
>series up into a set of smaller self-contained patch series makes things 
>easier to review and may make it easier for you to get this functionality 
>reviewed and accepted into upstream.
>

Thanks, Alistair. I will move those cleanup/refactor related patches
to form a separate series which is expected to be merged first. That
will helps the reviewers to focus on the patches with complicated
changes as you suggested. Alexey, please let me know if that way is
you like to see or not.

Thanks,
Gavin

>Regards,
>
>Alistair
>

___
Linuxppc-dev mailing list
Linuxppc-dev@lists.ozlabs.org
https://lists.ozlabs.org/listinfo/linuxppc-dev

Re: [PATCH v8 00/45] powerpc/powernv: PCI hotplug support

2016-04-13 Thread Alistair Popple 
Hi Gavin,



> >Why exactly cannot EEH reset changes go to a smaller separate patchset
> >(before hotplug)?
> >
> 
> As I explained before, the patchset's order is: PCI generic part,
> PowerNV PCI related, EEH related, device-tree part and hotplug driver.
> 
> The EEH reset change is included in PATCH[37/45]. There is no point
> to reorder the patches.
 
I don't understand all of the dependencies but if possible splitting the 
series up into a set of smaller self-contained patch series makes things 
easier to review and may make it easier for you to get this functionality 
reviewed and accepted into upstream.

Regards,

Alistair

___
Linuxppc-dev mailing list
Linuxppc-dev@lists.ozlabs.org
https://lists.ozlabs.org/listinfo/linuxppc-dev

Re: [PATCH v8 00/45] powerpc/powernv: PCI hotplug support

2016-04-13 Thread Gavin Shan 
On Wed, Apr 13, 2016 at 07:14:59PM +1000, Alexey Kardashevskiy wrote:
>On 04/13/2016 05:42 PM, Gavin Shan wrote:
>>On Wed, Apr 13, 2016 at 05:28:15PM +1000, Alexey Kardashevskiy wrote:
>>>On 02/17/2016 02:43 PM, Gavin Shan wrote:
This series of patches rebases on powerpc/next branch, plus below additional
patches:



https://patchwork.ozlabs.org/patch/581315/  (PATCH[1/9] Richard's 
 SRIOV EEH)
https://patchwork.ozlabs.org/patch/582639/  (PATCH[1/1] Gavin's EEH 
 fix)
https://patchwork.ozlabs.org/patch/582093/  (PATCH[1/1] Gavin's EEH 
 fix)
https://patchwork.ozlabs.org/patch/580626/  (PATCH[1/4] Gavin's PCI 
 fix)
https://patchwork.ozlabs.org/patch/580153/  (PATCH[1/1] Andrew's 
 EEH minor fix)
https://patchwork.ozlabs.org/patch/566827/  (PATCH[1/1] Russell's 
 P5IOC2 removal)
https://patchwork.ozlabs.org/patch/534154/  (PATCH[1/7] Richard's 
 SRIOV rework)
commit 388f7b1 ("Linux 4.5-rc3")

The series of patches intend to support PCI slot for PowerPC PowerNV 
platform,
which is running on top of skiboot firmware. The patchset requires 
corresponding
changes from skiboot firmware, which is sent to skib...@lists.ozlabs.org
for review. The PCI slots are exposed by skiboot with device node 
properties,
and kernel utilizes those properties to populated PCI slots accordingly.

The original PCI infrastructure on PowerNV platform can't support hotplug
because the PE is assigned during PHB fixup time, which is called for once
during system boot time. For this, the PCI infrastructure on PowerNV 
platform
has been reworked for a lot. After that, the PE and its corresponding 
resources
(IODT, M32DT, M64 segments, DMA32 and bypass window) are assigned upon 
updating
PCI bridge's resources, which might decide PE# assigned to the PE (e.g. M64
resources, on P8 strictly speaking). Each PE will maintain a reference 
count,
which is (number of child PCI devices + 1). That indicates when last child 
PCI
device leaves the PE, the PE and its included resources will be relased and 
put
back into free pool again. With this design, the PE will be released when 
EEH PE
is released. PATCH[1 - 23] are related to this part.

 From skiboot perspective, PCI slot is providing (hot/fundamental/complete)
resets to EEH. The kernel gets to know if skiboot supports various reset on 
one
particular PCI slot through device-tree node. If it does, EEH will utilize 
the
functionality provided by skiboot. Besides, the device-tree nodes have to 
change
in order to support PCI hotplug. For example, when one PCI adapter inserted 
to
one slot, its device-tree node should be added to the system dynamically. 
Conversely,
the device-tree node should be removed from the system when the PCI adapter 
is going
to be offline. Since pci_dn and eeh_dev have same life cyle as PCI device 
nodes,
they should be added/removed accordingly during PCI hotplug. PATCH[24 - 39] 
are
doing the related work.

The OF driver is changed to support unflattening FDT blob for sub-stree, 
which
is covered by PATCH[40 - 44].

The last one, PATCH[45], is the standalone PCI hotplug driver for PowerPC 
PowerNV
platform.

===
Testing
===
1. Unplug adapters behind non-empty slot, then plug them.

1.1 Check status
# cat /sys/bus/pci/slots/C10/address
0003:09:00
# cat /sys/bus/pci/slots/C10/adapter
1
# cat /sys/bus/pci/slots/C10/power
1
# lspci
0003:09:00.0 Ethernet controller: \
Broadcom Corporation NetXtreme BCM5719 Gigabit Ethernet PCIe (rev 01)
0003:09:00.1 Ethernet controller: \
Broadcom Corporation NetXtreme BCM5719 Gigabit Ethernet PCIe (rev 01)
0003:09:00.2 Ethernet controller: \
Broadcom Corporation NetXtreme BCM5719 Gigabit Ethernet PCIe (rev 01)
0003:09:00.3 Ethernet controller: \
Broadcom Corporation NetXtreme BCM5719 Gigabit Ethernet PCIe (rev 01)
# lspci -t
# lspci -t
-+-[0003:00]---00.0-[01-13]00.0-[02-13]--+-01.0-[03]00.0
 |   +-08.0-[04-08]--
 |   +-09.0-[09]--+-00.0
 |   |+-00.1
 |   |+-00.2
 |   |\-00.3
 |   +-10.0-[0a-0e]--
 |   \-11.0-[0f-13]--

1.2 Unplug adapter 0003:09.00.x
# echo 0 > /sys/bus/pci/slots/C10/power
# lspci

Re: [PATCH v8 00/45] powerpc/powernv: PCI hotplug support

2016-04-13 Thread Alexey Kardashevskiy 

On 04/13/2016 05:42 PM, Gavin Shan wrote:

On Wed, Apr 13, 2016 at 05:28:15PM +1000, Alexey Kardashevskiy wrote:

On 02/17/2016 02:43 PM, Gavin Shan wrote:

This series of patches rebases on powerpc/next branch, plus below additional
patches:



https://patchwork.ozlabs.org/patch/581315/  (PATCH[1/9] Richard's SRIOV EEH)
https://patchwork.ozlabs.org/patch/582639/  (PATCH[1/1] Gavin's EEH fix)
https://patchwork.ozlabs.org/patch/582093/  (PATCH[1/1] Gavin's EEH fix)
https://patchwork.ozlabs.org/patch/580626/  (PATCH[1/4] Gavin's PCI fix)
https://patchwork.ozlabs.org/patch/580153/  (PATCH[1/1] Andrew's EEH minor 
fix)
https://patchwork.ozlabs.org/patch/566827/  (PATCH[1/1] Russell's P5IOC2 
removal)
https://patchwork.ozlabs.org/patch/534154/  (PATCH[1/7] Richard's SRIOV 
rework)
commit 388f7b1 ("Linux 4.5-rc3")

The series of patches intend to support PCI slot for PowerPC PowerNV platform,
which is running on top of skiboot firmware. The patchset requires corresponding
changes from skiboot firmware, which is sent to skib...@lists.ozlabs.org
for review. The PCI slots are exposed by skiboot with device node properties,
and kernel utilizes those properties to populated PCI slots accordingly.

The original PCI infrastructure on PowerNV platform can't support hotplug
because the PE is assigned during PHB fixup time, which is called for once
during system boot time. For this, the PCI infrastructure on PowerNV platform
has been reworked for a lot. After that, the PE and its corresponding resources
(IODT, M32DT, M64 segments, DMA32 and bypass window) are assigned upon updating
PCI bridge's resources, which might decide PE# assigned to the PE (e.g. M64
resources, on P8 strictly speaking). Each PE will maintain a reference count,
which is (number of child PCI devices + 1). That indicates when last child PCI
device leaves the PE, the PE and its included resources will be relased and put
back into free pool again. With this design, the PE will be released when EEH PE
is released. PATCH[1 - 23] are related to this part.

 From skiboot perspective, PCI slot is providing (hot/fundamental/complete)
resets to EEH. The kernel gets to know if skiboot supports various reset on one
particular PCI slot through device-tree node. If it does, EEH will utilize the
functionality provided by skiboot. Besides, the device-tree nodes have to change
in order to support PCI hotplug. For example, when one PCI adapter inserted to
one slot, its device-tree node should be added to the system dynamically. 
Conversely,
the device-tree node should be removed from the system when the PCI adapter is 
going
to be offline. Since pci_dn and eeh_dev have same life cyle as PCI device nodes,
they should be added/removed accordingly during PCI hotplug. PATCH[24 - 39] are
doing the related work.

The OF driver is changed to support unflattening FDT blob for sub-stree, which
is covered by PATCH[40 - 44].

The last one, PATCH[45], is the standalone PCI hotplug driver for PowerPC 
PowerNV
platform.

===
Testing
===
1. Unplug adapters behind non-empty slot, then plug them.

1.1 Check status
# cat /sys/bus/pci/slots/C10/address
0003:09:00
# cat /sys/bus/pci/slots/C10/adapter
1
# cat /sys/bus/pci/slots/C10/power
1
# lspci
0003:09:00.0 Ethernet controller: \
Broadcom Corporation NetXtreme BCM5719 Gigabit Ethernet PCIe (rev 01)
0003:09:00.1 Ethernet controller: \
Broadcom Corporation NetXtreme BCM5719 Gigabit Ethernet PCIe (rev 01)
0003:09:00.2 Ethernet controller: \
Broadcom Corporation NetXtreme BCM5719 Gigabit Ethernet PCIe (rev 01)
0003:09:00.3 Ethernet controller: \
Broadcom Corporation NetXtreme BCM5719 Gigabit Ethernet PCIe (rev 01)
# lspci -t
# lspci -t
-+-[0003:00]---00.0-[01-13]00.0-[02-13]--+-01.0-[03]00.0
 |   +-08.0-[04-08]--
 |   +-09.0-[09]--+-00.0
 |   |+-00.1
 |   |+-00.2
 |   |\-00.3
 |   +-10.0-[0a-0e]--
 |   \-11.0-[0f-13]--

1.2 Unplug adapter 0003:09.00.x
# echo 0 > /sys/bus/pci/slots/C10/power
# lspci -t
-+-[0003:00]---00.0-[01-13]00.0-[02-13]--+-01.0-[03]00.0
 |   +-08.0-[04-08]--
 |   +-09.0-[09]--
 |   +-10.0-[0a-0e]--
 |   \-11.0-[0f-13]--

1.3 Plug adapter 0003:09.00.x
# echo 1 > /sys/bus/pci/slots/C10/power



Do I understand correctly that the adapter was not physically moved in/out of
the slot between 1.2 and 1.3?



Correct.



This is not right then... Someone

Re: [PATCH v8 00/45] powerpc/powernv: PCI hotplug support

2016-04-13 Thread Gavin Shan 
On Wed, Apr 13, 2016 at 05:28:15PM +1000, Alexey Kardashevskiy wrote:
>On 02/17/2016 02:43 PM, Gavin Shan wrote:
>>This series of patches rebases on powerpc/next branch, plus below additional
>>patches:
>>
>>
>>
>>https://patchwork.ozlabs.org/patch/581315/(PATCH[1/9] Richard's 
>> SRIOV EEH)
>>https://patchwork.ozlabs.org/patch/582639/(PATCH[1/1] Gavin's EEH 
>> fix)
>>https://patchwork.ozlabs.org/patch/582093/(PATCH[1/1] Gavin's EEH 
>> fix)
>>https://patchwork.ozlabs.org/patch/580626/(PATCH[1/4] Gavin's PCI 
>> fix)
>>https://patchwork.ozlabs.org/patch/580153/(PATCH[1/1] Andrew's 
>> EEH minor fix)
>>https://patchwork.ozlabs.org/patch/566827/(PATCH[1/1] Russell's 
>> P5IOC2 removal)
>>https://patchwork.ozlabs.org/patch/534154/(PATCH[1/7] Richard's 
>> SRIOV rework)
>>commit 388f7b1 ("Linux 4.5-rc3")
>>
>>The series of patches intend to support PCI slot for PowerPC PowerNV platform,
>>which is running on top of skiboot firmware. The patchset requires 
>>corresponding
>>changes from skiboot firmware, which is sent to skib...@lists.ozlabs.org
>>for review. The PCI slots are exposed by skiboot with device node properties,
>>and kernel utilizes those properties to populated PCI slots accordingly.
>>
>>The original PCI infrastructure on PowerNV platform can't support hotplug
>>because the PE is assigned during PHB fixup time, which is called for once
>>during system boot time. For this, the PCI infrastructure on PowerNV platform
>>has been reworked for a lot. After that, the PE and its corresponding 
>>resources
>>(IODT, M32DT, M64 segments, DMA32 and bypass window) are assigned upon 
>>updating
>>PCI bridge's resources, which might decide PE# assigned to the PE (e.g. M64
>>resources, on P8 strictly speaking). Each PE will maintain a reference count,
>>which is (number of child PCI devices + 1). That indicates when last child PCI
>>device leaves the PE, the PE and its included resources will be relased and 
>>put
>>back into free pool again. With this design, the PE will be released when EEH 
>>PE
>>is released. PATCH[1 - 23] are related to this part.
>>
>> From skiboot perspective, PCI slot is providing (hot/fundamental/complete)
>>resets to EEH. The kernel gets to know if skiboot supports various reset on 
>>one
>>particular PCI slot through device-tree node. If it does, EEH will utilize the
>>functionality provided by skiboot. Besides, the device-tree nodes have to 
>>change
>>in order to support PCI hotplug. For example, when one PCI adapter inserted to
>>one slot, its device-tree node should be added to the system dynamically. 
>>Conversely,
>>the device-tree node should be removed from the system when the PCI adapter 
>>is going
>>to be offline. Since pci_dn and eeh_dev have same life cyle as PCI device 
>>nodes,
>>they should be added/removed accordingly during PCI hotplug. PATCH[24 - 39] 
>>are
>>doing the related work.
>>
>>The OF driver is changed to support unflattening FDT blob for sub-stree, which
>>is covered by PATCH[40 - 44].
>>
>>The last one, PATCH[45], is the standalone PCI hotplug driver for PowerPC 
>>PowerNV
>>platform.
>>
>>===
>>Testing
>>===
>>1. Unplug adapters behind non-empty slot, then plug them.
>>
>>1.1 Check status
>># cat /sys/bus/pci/slots/C10/address
>>0003:09:00
>># cat /sys/bus/pci/slots/C10/adapter
>>1
>># cat /sys/bus/pci/slots/C10/power
>>1
>># lspci
>>0003:09:00.0 Ethernet controller: \
>>Broadcom Corporation NetXtreme BCM5719 Gigabit Ethernet PCIe (rev 01)
>>0003:09:00.1 Ethernet controller: \
>>Broadcom Corporation NetXtreme BCM5719 Gigabit Ethernet PCIe (rev 01)
>>0003:09:00.2 Ethernet controller: \
>>Broadcom Corporation NetXtreme BCM5719 Gigabit Ethernet PCIe (rev 01)
>>0003:09:00.3 Ethernet controller: \
>>Broadcom Corporation NetXtreme BCM5719 Gigabit Ethernet PCIe (rev 01)
>># lspci -t
>># lspci -t
>>-+-[0003:00]---00.0-[01-13]00.0-[02-13]--+-01.0-[03]00.0
>> |   +-08.0-[04-08]--
>> |   +-09.0-[09]--+-00.0
>> |   |+-00.1
>> |   |+-00.2
>> |   |\-00.3
>> |   +-10.0-[0a-0e]--
>> |   \-11.0-[0f-13]--
>>
>>1.2 Unplug adapter 0003:09.00.x
>># echo 0 > /sys/bus/pci/slots/C10/power
>># lspci -t
>>-+-[0003:00]---00.0-[01-13]00.0-[02-13]--+-01.0-[03]00.0
>> |   +-08.0-[04-08]--
>> |   +-09.0-[09]--
>> |   +-10.0-[0a-0e]--
>> |   \-11.0-[0f-13]--
>>
>>

Re: [PATCH v8 00/45] powerpc/powernv: PCI hotplug support

2016-04-13 Thread Alexey Kardashevskiy 

On 02/17/2016 02:43 PM, Gavin Shan wrote:

This series of patches rebases on powerpc/next branch, plus below additional
patches:



https://patchwork.ozlabs.org/patch/581315/  (PATCH[1/9] Richard's SRIOV EEH)
https://patchwork.ozlabs.org/patch/582639/  (PATCH[1/1] Gavin's EEH fix)
https://patchwork.ozlabs.org/patch/582093/  (PATCH[1/1] Gavin's EEH fix)
https://patchwork.ozlabs.org/patch/580626/  (PATCH[1/4] Gavin's PCI fix)
https://patchwork.ozlabs.org/patch/580153/  (PATCH[1/1] Andrew's EEH minor 
fix)
https://patchwork.ozlabs.org/patch/566827/  (PATCH[1/1] Russell's P5IOC2 
removal)
https://patchwork.ozlabs.org/patch/534154/  (PATCH[1/7] Richard's SRIOV 
rework)
commit 388f7b1 ("Linux 4.5-rc3")

The series of patches intend to support PCI slot for PowerPC PowerNV platform,
which is running on top of skiboot firmware. The patchset requires corresponding
changes from skiboot firmware, which is sent to skib...@lists.ozlabs.org
for review. The PCI slots are exposed by skiboot with device node properties,
and kernel utilizes those properties to populated PCI slots accordingly.

The original PCI infrastructure on PowerNV platform can't support hotplug
because the PE is assigned during PHB fixup time, which is called for once
during system boot time. For this, the PCI infrastructure on PowerNV platform
has been reworked for a lot. After that, the PE and its corresponding resources
(IODT, M32DT, M64 segments, DMA32 and bypass window) are assigned upon updating
PCI bridge's resources, which might decide PE# assigned to the PE (e.g. M64
resources, on P8 strictly speaking). Each PE will maintain a reference count,
which is (number of child PCI devices + 1). That indicates when last child PCI
device leaves the PE, the PE and its included resources will be relased and put
back into free pool again. With this design, the PE will be released when EEH PE
is released. PATCH[1 - 23] are related to this part.

 From skiboot perspective, PCI slot is providing (hot/fundamental/complete)
resets to EEH. The kernel gets to know if skiboot supports various reset on one
particular PCI slot through device-tree node. If it does, EEH will utilize the
functionality provided by skiboot. Besides, the device-tree nodes have to change
in order to support PCI hotplug. For example, when one PCI adapter inserted to
one slot, its device-tree node should be added to the system dynamically. 
Conversely,
the device-tree node should be removed from the system when the PCI adapter is 
going
to be offline. Since pci_dn and eeh_dev have same life cyle as PCI device nodes,
they should be added/removed accordingly during PCI hotplug. PATCH[24 - 39] are
doing the related work.

The OF driver is changed to support unflattening FDT blob for sub-stree, which
is covered by PATCH[40 - 44].

The last one, PATCH[45], is the standalone PCI hotplug driver for PowerPC 
PowerNV
platform.

===
Testing
===
1. Unplug adapters behind non-empty slot, then plug them.

1.1 Check status
# cat /sys/bus/pci/slots/C10/address
0003:09:00
# cat /sys/bus/pci/slots/C10/adapter
1
# cat /sys/bus/pci/slots/C10/power
1
# lspci
0003:09:00.0 Ethernet controller: \
Broadcom Corporation NetXtreme BCM5719 Gigabit Ethernet PCIe (rev 01)
0003:09:00.1 Ethernet controller: \
Broadcom Corporation NetXtreme BCM5719 Gigabit Ethernet PCIe (rev 01)
0003:09:00.2 Ethernet controller: \
Broadcom Corporation NetXtreme BCM5719 Gigabit Ethernet PCIe (rev 01)
0003:09:00.3 Ethernet controller: \
Broadcom Corporation NetXtreme BCM5719 Gigabit Ethernet PCIe (rev 01)
# lspci -t
# lspci -t
-+-[0003:00]---00.0-[01-13]00.0-[02-13]--+-01.0-[03]00.0
 |   +-08.0-[04-08]--
 |   +-09.0-[09]--+-00.0
 |   |+-00.1
 |   |+-00.2
 |   |\-00.3
 |   +-10.0-[0a-0e]--
 |   \-11.0-[0f-13]--

1.2 Unplug adapter 0003:09.00.x
# echo 0 > /sys/bus/pci/slots/C10/power
# lspci -t
-+-[0003:00]---00.0-[01-13]00.0-[02-13]--+-01.0-[03]00.0
 |   +-08.0-[04-08]--
 |   +-09.0-[09]--
 |   +-10.0-[0a-0e]--
 |   \-11.0-[0f-13]--

1.3 Plug adapter 0003:09.00.x
# echo 1 > /sys/bus/pci/slots/C10/power



Do I understand correctly that the adapter was not physically moved in/out 
of the slot between 1.2 and 1.3?





# lspci -t
-+-[0003:00]---00.0-[01-13]00.0-[02-13]--+-01.0-[03]00.0
 |   +-08.0-[04-08]--
 |

[PATCH v8 00/45] powerpc/powernv: PCI hotplug support

2016-02-16 Thread Gavin Shan 
This series of patches rebases on powerpc/next branch, plus below additional
patches:

   
   
   https://patchwork.ozlabs.org/patch/581315/   (PATCH[1/9] Richard's SRIOV EEH)
   https://patchwork.ozlabs.org/patch/582639/   (PATCH[1/1] Gavin's EEH fix)
   https://patchwork.ozlabs.org/patch/582093/   (PATCH[1/1] Gavin's EEH fix)
   https://patchwork.ozlabs.org/patch/580626/   (PATCH[1/4] Gavin's PCI fix)
   https://patchwork.ozlabs.org/patch/580153/   (PATCH[1/1] Andrew's EEH minor 
fix)
   https://patchwork.ozlabs.org/patch/566827/   (PATCH[1/1] Russell's P5IOC2 
removal)
   https://patchwork.ozlabs.org/patch/534154/   (PATCH[1/7] Richard's SRIOV 
rework)
   commit 388f7b1 ("Linux 4.5-rc3")
   
The series of patches intend to support PCI slot for PowerPC PowerNV platform,
which is running on top of skiboot firmware. The patchset requires corresponding
changes from skiboot firmware, which is sent to skib...@lists.ozlabs.org
for review. The PCI slots are exposed by skiboot with device node properties,
and kernel utilizes those properties to populated PCI slots accordingly.

The original PCI infrastructure on PowerNV platform can't support hotplug
because the PE is assigned during PHB fixup time, which is called for once
during system boot time. For this, the PCI infrastructure on PowerNV platform
has been reworked for a lot. After that, the PE and its corresponding resources
(IODT, M32DT, M64 segments, DMA32 and bypass window) are assigned upon updating
PCI bridge's resources, which might decide PE# assigned to the PE (e.g. M64
resources, on P8 strictly speaking). Each PE will maintain a reference count,
which is (number of child PCI devices + 1). That indicates when last child PCI
device leaves the PE, the PE and its included resources will be relased and put
back into free pool again. With this design, the PE will be released when EEH PE
is released. PATCH[1 - 23] are related to this part.

From skiboot perspective, PCI slot is providing (hot/fundamental/complete)
resets to EEH. The kernel gets to know if skiboot supports various reset on one
particular PCI slot through device-tree node. If it does, EEH will utilize the
functionality provided by skiboot. Besides, the device-tree nodes have to change
in order to support PCI hotplug. For example, when one PCI adapter inserted to
one slot, its device-tree node should be added to the system dynamically. 
Conversely,
the device-tree node should be removed from the system when the PCI adapter is 
going
to be offline. Since pci_dn and eeh_dev have same life cyle as PCI device nodes,
they should be added/removed accordingly during PCI hotplug. PATCH[24 - 39] are
doing the related work.

The OF driver is changed to support unflattening FDT blob for sub-stree, which
is covered by PATCH[40 - 44].

The last one, PATCH[45], is the standalone PCI hotplug driver for PowerPC 
PowerNV
platform.

===
Testing
===
1. Unplug adapters behind non-empty slot, then plug them.

   1.1 Check status
   # cat /sys/bus/pci/slots/C10/address 
   0003:09:00
   # cat /sys/bus/pci/slots/C10/adapter 
   1
   # cat /sys/bus/pci/slots/C10/power 
   1
   # lspci
   0003:09:00.0 Ethernet controller: \
   Broadcom Corporation NetXtreme BCM5719 Gigabit Ethernet PCIe (rev 01)
   0003:09:00.1 Ethernet controller: \
   Broadcom Corporation NetXtreme BCM5719 Gigabit Ethernet PCIe (rev 01)
   0003:09:00.2 Ethernet controller: \
   Broadcom Corporation NetXtreme BCM5719 Gigabit Ethernet PCIe (rev 01)
   0003:09:00.3 Ethernet controller: \
   Broadcom Corporation NetXtreme BCM5719 Gigabit Ethernet PCIe (rev 01)
   # lspci -t
   # lspci -t
   -+-[0003:00]---00.0-[01-13]00.0-[02-13]--+-01.0-[03]00.0
|   +-08.0-[04-08]--
|   +-09.0-[09]--+-00.0
|   |+-00.1
|   |+-00.2
|   |\-00.3
|   +-10.0-[0a-0e]--
|   \-11.0-[0f-13]--

   1.2 Unplug adapter 0003:09.00.x
   # echo 0 > /sys/bus/pci/slots/C10/power 
   # lspci -t
   -+-[0003:00]---00.0-[01-13]00.0-[02-13]--+-01.0-[03]00.0
|   +-08.0-[04-08]--
|   +-09.0-[09]--
|   +-10.0-[0a-0e]--
|   \-11.0-[0f-13]--

   1.3 Plug adapter 0003:09.00.x
   # echo 1 > /sys/bus/pci/slots/C10/power 
   # lspci -t
   -+-[0003:00]---00.0-[01-13]00.0-[02-13]--+-01.0-[03]00.0
|   +-08.0-[04-08]--
|   +-09.0-[09]--+-00.0
|   |+-00.1
|   |+-00.2