Re: [PATCH] x86: enable swiotlb for > 4GiG ram on 32-bit kernels
* tedheadster wrote: > > But you said without the fix it doesn't work at all? Or is this > > the same box, just with the aic7xxx controller disabled? > > > > In general the patch should only have two effects: > > > > - set a small amount of memory aside for bounce buffering > > - switch the default dma_ops from dma_direct_ops to swiotlb_ops > > > > I can't really see how either could have such a huge effect, even with > > swiotlb having a couple more wired up ops for which we'd enable spectre > > mitigations. > > > > So a strict before and after would be very interesting, if it is really > > just this one change that causes such a huge drop we have hidden dragons > > somewhere.. > > Christoph, > I did a very controlled before-and-after and got very sensible > results. All compiles were close in time with patched and un-patched > kernels. > > I must have screwed something up with my last round of testing. > > Ingo: I am confident this patch should be accepted. Thanks for the update, I've re-applied this to tip:x86/urgent. Ingo ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH] x86: enable swiotlb for > 4GiG ram on 32-bit kernels
> But you said without the fix it doesn't work at all? Or is this > the same box, just with the aic7xxx controller disabled? > > In general the patch should only have two effects: > > - set a small amount of memory aside for bounce buffering > - switch the default dma_ops from dma_direct_ops to swiotlb_ops > > I can't really see how either could have such a huge effect, even with > swiotlb having a couple more wired up ops for which we'd enable spectre > mitigations. > > So a strict before and after would be very interesting, if it is really > just this one change that causes such a huge drop we have hidden dragons > somewhere.. Christoph, I did a very controlled before-and-after and got very sensible results. All compiles were close in time with patched and un-patched kernels. I must have screwed something up with my last round of testing. Ingo: I am confident this patch should be accepted. - Matthew Whitehead ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH] x86: enable swiotlb for > 4GiG ram on 32-bit kernels
On Wed, Oct 17, 2018 at 01:41:49PM -0400, tedheadster wrote: > > What is the comparism? Is this on a non-aic7xxx kernel with and without > > the fix? > > Christoph, > I will try and do some strict before-and-after benchmarking of your > patch. It will take a few days because I am on travel. This is on an > aic7xxx (it does have that controller) kernel with and without the > fix. But you said without the fix it doesn't work at all? Or is this the same box, just with the aic7xxx controller disabled? In general the patch should only have two effects: - set a small amount of memory aside for bounce buffering - switch the default dma_ops from dma_direct_ops to swiotlb_ops I can't really see how either could have such a huge effect, even with swiotlb having a couple more wired up ops for which we'd enable spectre mitigations. So a strict before and after would be very interesting, if it is really just this one change that causes such a huge drop we have hidden dragons somewhere.. ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH] x86: enable swiotlb for > 4GiG ram on 32-bit kernels
* tedheadster wrote:
> On Sun, Oct 14, 2018 at 3:52 AM Christoph Hellwig wrote:
> >
> > We already build the swiotlb code for 32b-t kernels with PAE support,
> > but the code to actually use swiotlb has only been enabled for 64-bit
> > kernel for an unknown reason.
> >
> > Before Linux 4.18 we papers over this fact because the networking code,
> > the scsi layer and some random block drivers implenented their own
> > bounce buffering scheme.
> >
> > Fixes: 21e07dba ("scsi: reduce use of block bounce buffers")
> > Fixes: ab74cfeb ("net: remove the PCI_DMA_BUS_IS_PHYS check in
> > illegal_highdma")
> > Reported-by: tedheadster
> > Tested-by: tedheadster
> > ---
> >
>
> Christoph,
> this fix has causes performance to decrease dramatically. Kernel
> builds that used to take 10-15 minutes are now taking 45-60 minutes on
> the same machine.
Ok, this is way too severe regression, and because the two offending
commits:
Fixes: 21e07dba9fb1 ("scsi: reduce use of block bounce buffers")
Fixes: ab74cfebafa3 ("net: remove the PCI_DMA_BUS_IS_PHYS check in
illegal_highdma")
... are from half a year ago and are in v4.18 already. Fixes should not
cause new regressions in any case.
So I've removed this patch from tip:x86/urgent for now, could you please
re-apply it when you do your testing? I've attached it below.
Thanks,
Ingo
===>
>From 6f3bc8028570e4c326030e8795dbcd57c561b723 Mon Sep 17 00:00:00 2001
From: Christoph Hellwig
Date: Sun, 14 Oct 2018 09:52:08 +0200
Subject: [PATCH] x86/swiotlb: Enable swiotlb for > 4GiG ram on 32-bit kernels
We already build the swiotlb code for 32b-t kernels with PAE support,
but the code to actually use swiotlb has only been enabled for 64-bit
kernel for an unknown reason.
Before Linux 4.18 we paper over this fact because the networking code,
the scsi layer and some random block drivers implemented their own
bounce buffering scheme.
Fixes: 21e07dba9fb1 ("scsi: reduce use of block bounce buffers")
Fixes: ab74cfebafa3 ("net: remove the PCI_DMA_BUS_IS_PHYS check in
illegal_highdma")
Reported-by: Matthew Whitehead
Signed-off-by: Christoph Hellwig
Signed-off-by: Thomas Gleixner
Tested-by: Matthew Whitehead
Cc: konrad.w...@oracle.com
Cc: iommu@lists.linux-foundation.org
Cc: sta...@vger.kernel.org
Link: https://lkml.kernel.org/r/20181014075208.2715-1-...@lst.de
---
arch/x86/kernel/pci-swiotlb.c | 2 --
1 file changed, 2 deletions(-)
diff --git a/arch/x86/kernel/pci-swiotlb.c b/arch/x86/kernel/pci-swiotlb.c
index 661583662430..71c0b01d93b1 100644
--- a/arch/x86/kernel/pci-swiotlb.c
+++ b/arch/x86/kernel/pci-swiotlb.c
@@ -42,10 +42,8 @@ IOMMU_INIT_FINISH(pci_swiotlb_detect_override,
int __init pci_swiotlb_detect_4gb(void)
{
/* don't initialize swiotlb if iommu=off (no_iommu=1) */
-#ifdef CONFIG_X86_64
if (!no_iommu && max_possible_pfn > MAX_DMA32_PFN)
swiotlb = 1;
-#endif
/*
* If SME is active then swiotlb will be set to 1 so that bounce
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH] x86: enable swiotlb for > 4GiG ram on 32-bit kernels
> What is the comparism? Is this on a non-aic7xxx kernel with and without > the fix? Christoph, I will try and do some strict before-and-after benchmarking of your patch. It will take a few days because I am on travel. This is on an aic7xxx (it does have that controller) kernel with and without the fix. - Matthew Whitehead ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH] x86: enable swiotlb for > 4GiG ram on 32-bit kernels
On Tue, Oct 16, 2018 at 09:11:09AM -0400, tedheadster wrote: > Christoph, > this fix has causes performance to decrease dramatically. Kernel > builds that used to take 10-15 minutes are now taking 45-60 minutes on > the same machine. What is the comparism? Is this on a non-aic7xxx kernel with and without the fix? ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH] x86: enable swiotlb for > 4GiG ram on 32-bit kernels
On Tue, 16 Oct 2018, tedheadster wrote:
> On Sun, Oct 14, 2018 at 3:52 AM Christoph Hellwig wrote:
> >
> > We already build the swiotlb code for 32b-t kernels with PAE support,
> > but the code to actually use swiotlb has only been enabled for 64-bit
> > kernel for an unknown reason.
> >
> > Before Linux 4.18 we papers over this fact because the networking code,
> > the scsi layer and some random block drivers implenented their own
> > bounce buffering scheme.
> >
> > Fixes: 21e07dba ("scsi: reduce use of block bounce buffers")
> > Fixes: ab74cfeb ("net: remove the PCI_DMA_BUS_IS_PHYS check in
> > illegal_highdma")
> > Reported-by: tedheadster
> > Tested-by: tedheadster
> > ---
> >
>
> Christoph,
> this fix has causes performance to decrease dramatically. Kernel
> builds that used to take 10-15 minutes are now taking 45-60 minutes on
> the same machine.
Christoph, can you have a look please?
Thanks,
tglx
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH] x86: enable swiotlb for > 4GiG ram on 32-bit kernels
On Sun, Oct 14, 2018 at 3:52 AM Christoph Hellwig wrote:
>
> We already build the swiotlb code for 32b-t kernels with PAE support,
> but the code to actually use swiotlb has only been enabled for 64-bit
> kernel for an unknown reason.
>
> Before Linux 4.18 we papers over this fact because the networking code,
> the scsi layer and some random block drivers implenented their own
> bounce buffering scheme.
>
> Fixes: 21e07dba ("scsi: reduce use of block bounce buffers")
> Fixes: ab74cfeb ("net: remove the PCI_DMA_BUS_IS_PHYS check in
> illegal_highdma")
> Reported-by: tedheadster
> Tested-by: tedheadster
> ---
>
Christoph,
this fix has causes performance to decrease dramatically. Kernel
builds that used to take 10-15 minutes are now taking 45-60 minutes on
the same machine.
- Matthew Whitehead
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH] x86: enable swiotlb for > 4GiG ram on 32-bit kernels
On 10/14/18 3:52 AM, Christoph Hellwig wrote:
We already build the swiotlb code for 32b-t kernels with PAE support,
but the code to actually use swiotlb has only been enabled for 64-bit
kernel for an unknown reason.
Before Linux 4.18 we papers over this fact because the networking code,
the scsi layer and some random block drivers implenented their own
bounce buffering scheme.
Fixes: 21e07dba ("scsi: reduce use of block bounce buffers")
Fixes: ab74cfeb ("net: remove the PCI_DMA_BUS_IS_PHYS check in illegal_highdma")
Reported-by: tedheadster
Tested-by: tedheadster
Reviewed-by: Konrad Rzeszutek Wilk
But since it is already in 'tip' feel free to ignore my tag.
Thanks!
---
arch/x86/kernel/pci-swiotlb.c | 2 --
1 file changed, 2 deletions(-)
diff --git a/arch/x86/kernel/pci-swiotlb.c b/arch/x86/kernel/pci-swiotlb.c
index 661583662430..71c0b01d93b1 100644
--- a/arch/x86/kernel/pci-swiotlb.c
+++ b/arch/x86/kernel/pci-swiotlb.c
@@ -42,10 +42,8 @@ IOMMU_INIT_FINISH(pci_swiotlb_detect_override,
int __init pci_swiotlb_detect_4gb(void)
{
/* don't initialize swiotlb if iommu=off (no_iommu=1) */
-#ifdef CONFIG_X86_64
if (!no_iommu && max_possible_pfn > MAX_DMA32_PFN)
swiotlb = 1;
-#endif
/*
* If SME is active then swiotlb will be set to 1 so that bounce
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH] x86: enable swiotlb for > 4GiG ram on 32-bit kernels
Please change: Reported-by: tedheadster Tested-by: tedheadster to Reported-by: Matthew Whitehead Tested-by: Matthew Whitehead - Matthew ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH] x86: enable swiotlb for > 4GiG ram on 32-bit kernels
* Thomas Gleixner wrote:
> On Sun, 14 Oct 2018, Christoph Hellwig wrote:
>
> > On Sun, Oct 14, 2018 at 10:13:31AM +0200, Thomas Gleixner wrote:
> > > On Sun, 14 Oct 2018, Christoph Hellwig wrote:
> > >
> > > > We already build the swiotlb code for 32b-t kernels with PAE support,
> > > > but the code to actually use swiotlb has only been enabled for 64-bit
> > > > kernel for an unknown reason.
> > > >
> > > > Before Linux 4.18 we papers over this fact because the networking code,
> > > > the scsi layer and some random block drivers implenented their own
> > > > bounce buffering scheme.
> > > >
> > > > Fixes: 21e07dba ("scsi: reduce use of block bounce buffers")
>
> Please use the first 12 characters of the commit SHA for fixes tags in the
> future, as documented. No need to resend, I fixed it up for you and added a
> Cc: stable as well
For those who have their ~/.gitconfig's from ancient Git history, this can be
done via:
git config --global core.abbrev 12
Thanks,
Ingo
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH] x86: enable swiotlb for > 4GiG ram on 32-bit kernels
On Sun, 14 Oct 2018, Christoph Hellwig wrote:
> On Sun, Oct 14, 2018 at 10:13:31AM +0200, Thomas Gleixner wrote:
> > On Sun, 14 Oct 2018, Christoph Hellwig wrote:
> >
> > > We already build the swiotlb code for 32b-t kernels with PAE support,
> > > but the code to actually use swiotlb has only been enabled for 64-bit
> > > kernel for an unknown reason.
> > >
> > > Before Linux 4.18 we papers over this fact because the networking code,
> > > the scsi layer and some random block drivers implenented their own
> > > bounce buffering scheme.
> > >
> > > Fixes: 21e07dba ("scsi: reduce use of block bounce buffers")
Please use the first 12 characters of the commit SHA for fixes tags in the
future, as documented. No need to resend, I fixed it up for you and added a
Cc: stable as well
Thanks,
tglx
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH] x86: enable swiotlb for > 4GiG ram on 32-bit kernels
On Sun, Oct 14, 2018 at 10:13:31AM +0200, Thomas Gleixner wrote:
> On Sun, 14 Oct 2018, Christoph Hellwig wrote:
>
> > We already build the swiotlb code for 32b-t kernels with PAE support,
> > but the code to actually use swiotlb has only been enabled for 64-bit
> > kernel for an unknown reason.
> >
> > Before Linux 4.18 we papers over this fact because the networking code,
> > the scsi layer and some random block drivers implenented their own
> > bounce buffering scheme.
> >
> > Fixes: 21e07dba ("scsi: reduce use of block bounce buffers")
> > Fixes: ab74cfeb ("net: remove the PCI_DMA_BUS_IS_PHYS check in
> > illegal_highdma")
> > Reported-by: tedheadster
> > Tested-by: tedheadster
>
> I'll add your SOB when picking this up :)
Thanks. Here it is in writing:
Signed-off-by: Christoph Hellwig
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH] x86: enable swiotlb for > 4GiG ram on 32-bit kernels
On Sun, 14 Oct 2018, Christoph Hellwig wrote:
> We already build the swiotlb code for 32b-t kernels with PAE support,
> but the code to actually use swiotlb has only been enabled for 64-bit
> kernel for an unknown reason.
>
> Before Linux 4.18 we papers over this fact because the networking code,
> the scsi layer and some random block drivers implenented their own
> bounce buffering scheme.
>
> Fixes: 21e07dba ("scsi: reduce use of block bounce buffers")
> Fixes: ab74cfeb ("net: remove the PCI_DMA_BUS_IS_PHYS check in
> illegal_highdma")
> Reported-by: tedheadster
> Tested-by: tedheadster
I'll add your SOB when picking this up :)
> ---
> arch/x86/kernel/pci-swiotlb.c | 2 --
> 1 file changed, 2 deletions(-)
>
> diff --git a/arch/x86/kernel/pci-swiotlb.c b/arch/x86/kernel/pci-swiotlb.c
> index 661583662430..71c0b01d93b1 100644
> --- a/arch/x86/kernel/pci-swiotlb.c
> +++ b/arch/x86/kernel/pci-swiotlb.c
> @@ -42,10 +42,8 @@ IOMMU_INIT_FINISH(pci_swiotlb_detect_override,
> int __init pci_swiotlb_detect_4gb(void)
> {
> /* don't initialize swiotlb if iommu=off (no_iommu=1) */
> -#ifdef CONFIG_X86_64
> if (!no_iommu && max_possible_pfn > MAX_DMA32_PFN)
> swiotlb = 1;
> -#endif
>
> /*
>* If SME is active then swiotlb will be set to 1 so that bounce
> --
> 2.19.1
>
>
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
