Re: [RESEND PATCH 0/3] Allow custom PCI resource alignment on pseries
On Tue, May 28, 2019 at 2:09 PM Shawn Anastasio wrote:
>
>
>
> On 5/27/19 11:01 PM, Oliver wrote:
> > On Tue, May 28, 2019 at 8:56 AM Shawn Anastasio wrote:
> >>
> >> Hello all,
> >>
> >> This patch set implements support for user-specified PCI resource
> >> alignment on the pseries platform for hotplugged PCI devices.
> >> Currently on pseries, PCI resource alignments specified with the
> >> pci=resource_alignment commandline argument are ignored, since
> >> the firmware is in charge of managing the PCI resources. In the
> >> case of hotplugged devices, though, the kernel is in charge of
> >> configuring the resources and should obey alignment requirements.
> >
> > Are you using hotplug to work around SLOF (the OF we use under qemu)
> > not aligning BARs to 64K? It looks like there is a commit in SLOF to
> > fix that
> > (https://git.qemu.org/?p=SLOF.git;a=commit;f=board-qemu/slof/pci-phb.fs;h=1903174472f8800caf50c959b304501b4c01153c).
> >
>
> No, my application actually requires PCI hotplug at run-time.
>
> >> The current behavior of ignoring the alignment for hotplugged devices
> >> results in sub-page BARs landing between page boundaries and
> >> becoming un-mappable from userspace via the VFIO framework.
> >> This issue was observed on a pseries KVM guest with hotplugged
> >> ivshmem devices.
> >
> >> With these changes, users can specify an appropriate
> >> pci=resource_alignment argument on boot for devices they wish to use
> >> with VFIO.
> >>
> >> In the future, this could be extended to provide page-aligned
> >> resources by default for hotplugged devices, similar to what is done
> >> on powernv by commit 382746376993 ("powerpc/powernv: Override
> >> pcibios_default_alignment() to force PCI devices to be page aligned").
> >
> > Can we make aligning the BARs to PAGE_SIZE the default behaviour? The
> > BAR assignment process is complex enough as-is so I'd rather we didn't
> > add another platform hack into the mix.
>
> Absolutely. This will still require the existing changes so that the
> custom alignment isn't flat-out ignored on pseries, but I can set
> it to default to PAGE_SIZE as well, similar to how it's done on PowerNV.
> I've just pushed a v3 to fix a typo and I'll incorporate this change
> in v4.
I was thinking we could get rid of the ppcmd callback and do it in
kernel/pci-common.c. PowerNV is the only platform that implements the
callback and the pseries implementation is going to be identical so I
don't think there's much of point in keeping the callback.
> >> Feedback is appreciated.
> >>
> >> Thanks,
> >> Shawn
> >>
> >> Shawn Anastasio (3):
> >>PCI: Introduce pcibios_ignore_alignment_request
> >>powerpc/64: Enable pcibios_after_init hook on ppc64
> >>powerpc/pseries: Allow user-specified PCI resource alignment after
> >> init
> >>
> >> arch/powerpc/include/asm/machdep.h | 6 --
> >> arch/powerpc/kernel/pci-common.c | 9 +
> >> arch/powerpc/kernel/pci_64.c | 4
> >> arch/powerpc/platforms/pseries/setup.c | 22 ++
> >> drivers/pci/pci.c | 9 +++--
> >> 5 files changed, 46 insertions(+), 4 deletions(-)
> >>
> >> --
> >> 2.20.1
> >>
Re: [RESEND PATCH 0/3] Allow custom PCI resource alignment on pseries
On 5/27/19 11:01 PM, Oliver wrote:
On Tue, May 28, 2019 at 8:56 AM Shawn Anastasio wrote:
Hello all,
This patch set implements support for user-specified PCI resource
alignment on the pseries platform for hotplugged PCI devices.
Currently on pseries, PCI resource alignments specified with the
pci=resource_alignment commandline argument are ignored, since
the firmware is in charge of managing the PCI resources. In the
case of hotplugged devices, though, the kernel is in charge of
configuring the resources and should obey alignment requirements.
Are you using hotplug to work around SLOF (the OF we use under qemu)
not aligning BARs to 64K? It looks like there is a commit in SLOF to
fix that
(https://git.qemu.org/?p=SLOF.git;a=commit;f=board-qemu/slof/pci-phb.fs;h=1903174472f8800caf50c959b304501b4c01153c).
No, my application actually requires PCI hotplug at run-time.
The current behavior of ignoring the alignment for hotplugged devices
results in sub-page BARs landing between page boundaries and
becoming un-mappable from userspace via the VFIO framework.
This issue was observed on a pseries KVM guest with hotplugged
ivshmem devices.
With these changes, users can specify an appropriate
pci=resource_alignment argument on boot for devices they wish to use
with VFIO.
In the future, this could be extended to provide page-aligned
resources by default for hotplugged devices, similar to what is done
on powernv by commit 382746376993 ("powerpc/powernv: Override
pcibios_default_alignment() to force PCI devices to be page aligned").
Can we make aligning the BARs to PAGE_SIZE the default behaviour? The
BAR assignment process is complex enough as-is so I'd rather we didn't
add another platform hack into the mix.
Absolutely. This will still require the existing changes so that the
custom alignment isn't flat-out ignored on pseries, but I can set
it to default to PAGE_SIZE as well, similar to how it's done on PowerNV.
I've just pushed a v3 to fix a typo and I'll incorporate this change
in v4.
Feedback is appreciated.
Thanks,
Shawn
Shawn Anastasio (3):
PCI: Introduce pcibios_ignore_alignment_request
powerpc/64: Enable pcibios_after_init hook on ppc64
powerpc/pseries: Allow user-specified PCI resource alignment after
init
arch/powerpc/include/asm/machdep.h | 6 --
arch/powerpc/kernel/pci-common.c | 9 +
arch/powerpc/kernel/pci_64.c | 4
arch/powerpc/platforms/pseries/setup.c | 22 ++
drivers/pci/pci.c | 9 +++--
5 files changed, 46 insertions(+), 4 deletions(-)
--
2.20.1
Re: [RESEND PATCH 0/3] Allow custom PCI resource alignment on pseries
On Tue, May 28, 2019 at 8:56 AM Shawn Anastasio wrote:
>
> Hello all,
>
> This patch set implements support for user-specified PCI resource
> alignment on the pseries platform for hotplugged PCI devices.
> Currently on pseries, PCI resource alignments specified with the
> pci=resource_alignment commandline argument are ignored, since
> the firmware is in charge of managing the PCI resources. In the
> case of hotplugged devices, though, the kernel is in charge of
> configuring the resources and should obey alignment requirements.
Are you using hotplug to work around SLOF (the OF we use under qemu)
not aligning BARs to 64K? It looks like there is a commit in SLOF to
fix that
(https://git.qemu.org/?p=SLOF.git;a=commit;f=board-qemu/slof/pci-phb.fs;h=1903174472f8800caf50c959b304501b4c01153c).
> The current behavior of ignoring the alignment for hotplugged devices
> results in sub-page BARs landing between page boundaries and
> becoming un-mappable from userspace via the VFIO framework.
> This issue was observed on a pseries KVM guest with hotplugged
> ivshmem devices.
> With these changes, users can specify an appropriate
> pci=resource_alignment argument on boot for devices they wish to use
> with VFIO.
>
> In the future, this could be extended to provide page-aligned
> resources by default for hotplugged devices, similar to what is done
> on powernv by commit 382746376993 ("powerpc/powernv: Override
> pcibios_default_alignment() to force PCI devices to be page aligned").
Can we make aligning the BARs to PAGE_SIZE the default behaviour? The
BAR assignment process is complex enough as-is so I'd rather we didn't
add another platform hack into the mix.
> Feedback is appreciated.
>
> Thanks,
> Shawn
>
> Shawn Anastasio (3):
> PCI: Introduce pcibios_ignore_alignment_request
> powerpc/64: Enable pcibios_after_init hook on ppc64
> powerpc/pseries: Allow user-specified PCI resource alignment after
> init
>
> arch/powerpc/include/asm/machdep.h | 6 --
> arch/powerpc/kernel/pci-common.c | 9 +
> arch/powerpc/kernel/pci_64.c | 4
> arch/powerpc/platforms/pseries/setup.c | 22 ++
> drivers/pci/pci.c | 9 +++--
> 5 files changed, 46 insertions(+), 4 deletions(-)
>
> --
> 2.20.1
>
[RESEND PATCH 0/3] Allow custom PCI resource alignment on pseries
Hello all,
This patch set implements support for user-specified PCI resource
alignment on the pseries platform for hotplugged PCI devices.
Currently on pseries, PCI resource alignments specified with the
pci=resource_alignment commandline argument are ignored, since
the firmware is in charge of managing the PCI resources. In the
case of hotplugged devices, though, the kernel is in charge of
configuring the resources and should obey alignment requirements.
The current behavior of ignoring the alignment for hotplugged devices
results in sub-page BARs landing between page boundaries and
becoming un-mappable from userspace via the VFIO framework.
This issue was observed on a pseries KVM guest with hotplugged
ivshmem devices.
With these changes, users can specify an appropriate
pci=resource_alignment argument on boot for devices they wish to use
with VFIO.
In the future, this could be extended to provide page-aligned
resources by default for hotplugged devices, similar to what is done
on powernv by commit 382746376993 ("powerpc/powernv: Override
pcibios_default_alignment() to force PCI devices to be page aligned").
Feedback is appreciated.
Thanks,
Shawn
Shawn Anastasio (3):
PCI: Introduce pcibios_ignore_alignment_request
powerpc/64: Enable pcibios_after_init hook on ppc64
powerpc/pseries: Allow user-specified PCI resource alignment after
init
arch/powerpc/include/asm/machdep.h | 6 --
arch/powerpc/kernel/pci-common.c | 9 +
arch/powerpc/kernel/pci_64.c | 4
arch/powerpc/platforms/pseries/setup.c | 22 ++
drivers/pci/pci.c | 9 +++--
5 files changed, 46 insertions(+), 4 deletions(-)
--
2.20.1
[PATCH 0/3] Allow custom PCI resource alignment on pseries
(Resent to include relevant mailing lists - sorry about that!)
Hello all,
This patch set implements support for user-specified PCI resource
alignment on the pseries platform for hotplugged PCI devices.
Currently on pseries, PCI resource alignments specified with the
pci=resource_alignment commandline argument are ignored, since
the firmware is in charge of managing the PCI resources. In the
case of hotplugged devices, though, the kernel is in charge of
configuring the resources and should obey alignment requirements.
The current behavior of ignoring the alignment for hotplugged devices
results in sub-page BARs landing between page boundaries and
becoming un-mappable from userspace via the VFIO framework.
This issue was observed on a pseries KVM guest with hotplugged
ivshmem devices.
With these changes, users can specify an appropriate
pci=resource_alignment argument on boot for devices they wish to use
with VFIO.
In the future, this could be extended to provide page-aligned
resources by default for hotplugged devices, similar to what is done
on powernv by commit 382746376993 ("powerpc/powernv: Override
pcibios_default_alignment() to force PCI devices to be page aligned").
Feedback is appreciated.
Thanks,
Shawn
Shawn Anastasio (3):
PCI: Introduce pcibios_ignore_alignment_request
powerpc/64: Enable pcibios_after_init hook on ppc64
powerpc/pseries: Allow user-specified PCI resource alignment after
init
arch/powerpc/include/asm/machdep.h | 6 --
arch/powerpc/kernel/pci-common.c | 9 +
arch/powerpc/kernel/pci_64.c | 4
arch/powerpc/platforms/pseries/setup.c | 22 ++
drivers/pci/pci.c | 9 +++--
5 files changed, 46 insertions(+), 4 deletions(-)
--
2.20.1
