Re: [PATCH] drm/radeon: insist on 32-bit DMA for Cedar on PPC64/PPC64LE
Hi Christian, Michel, Alex, et al., I mentioned the HD 5450 purely as a matter of archeological interest. Back to the FirePro 2270 and Embedded Radeon E6465: I've attached text from both "lspci -tv" and "lspci -v." Actually I'm attaching a couple of different "lspci -v" outputs, one with the FirePro 2270 in place and one with the E6465 in (the same) place. I suspected, but couldn't prove, that the problem might have something to do with the bridge (IBM POWER8 Host Bridge (PHB3)). But I want to reiterate: - Cedar GPU -> problem (before patch, that is); - Caicos GPU in the same slot -> no problem -- Ben P.S. Alex, thanks for applying the patch so expeditiously! On Fri, Feb 23, 2018 at 2:45 AM, Christian König < ckoenig.leichtzumer...@gmail.com> wrote: > Am 22.02.2018 um 18:56 schrieb Michel Dänzer: > >> On 2018-02-22 06:37 PM, Ben Crocker wrote: >> >>> One of my colleagues did discover a "Radeon HG 5450 PCI" from February >>> 2010 which did, apparently, have a Cedar GPU and very definitely had a >>> (plain old) PCI connector. >>> >> There must be a PCIe-to-PCI bridge on that board. The GPU itself is >> always PCIe, and treated accordingly by the driver. >> > > Ben, just an educated guess but is this one the one which is failing to > work correctly? > > Cause the PCIe bus interface is pretty much identical over all generations > of the last decade or so. Only the newest Vega10 generation is a bit > different. > > So I strongly thing that this isn't related to the device being a Cedar at > all, but rather that you have a bridge above it which doesn't correctly > handle 64bit transfers. > > Can you please send and "lspci -t" of both the working and the problematic > devices? > > Thanks, > Christian. > -+-[0045:00]---00.0-[01]00.0 Texas Instruments TUSB73x0 SuperSpeed USB 3.0 xHCI Host Controller +-[0044:00]---00.0-[01]-- +-[0040:00]---00.0-[01]-- +-[0005:00]---00.0-[01-0f]00.0-[02-0f]--+-01.0-[03]00.0 Texas Instruments TUSB73x0 SuperSpeed USB 3.0 xHCI Host Controller | +-08.0-[04]00.0 IBM PCI-E IPR SAS Adapter (ASIC) | +-09.0-[05]--+-00.0 Broadcom Limited NetXtreme BCM5719 Gigabit Ethernet PCIe | |+-00.1 Broadcom Limited NetXtreme BCM5719 Gigabit Ethernet PCIe | |+-00.2 Broadcom Limited NetXtreme BCM5719 Gigabit Ethernet PCIe | |\-00.3 Broadcom Limited NetXtreme BCM5719 Gigabit Ethernet PCIe | +-10.0-[06-0a]-- | \-11.0-[0b-0f]-- +-[0004:00]---00.0-[01]--+-00.0 Advanced Micro Devices, Inc. [AMD/ATI] Cedar GL [FirePro 2270] |\-00.1 Advanced Micro Devices, Inc. [AMD/ATI] Cedar HDMI Audio [Radeon HD 5400/6300/7300 Series] +-[0001:00]---00.0-[01-0d]00.0-[02-0d]--+-01.0-[03-07]-- | +-08.0-[08]00.0 IBM PCI-E IPR SAS Adapter (ASIC) | \-09.0-[09-0d]-- \-[:00]---00.0-[01]-- :00:00.0 PCI bridge: IBM POWER8 Host Bridge (PHB3) (prog-if 00 [Normal decode]) Flags: bus master, fast devsel, latency 0, NUMA node 0 Bus: primary=00, secondary=01, subordinate=01, sec-latency=0 I/O behind bridge: None Memory behind bridge: None Prefetchable memory behind bridge: None Capabilities: [40] Power Management version 3 Capabilities: [48] Express Root Port (Slot-), MSI 00 Capabilities: [100] Advanced Error Reporting Capabilities: [148] #19 Kernel modules: shpchp 0001:00:00.0 PCI bridge: IBM POWER8 Host Bridge (PHB3) (prog-if 00 [Normal decode]) Flags: bus master, fast devsel, latency 0, NUMA node 0 Bus: primary=00, secondary=01, subordinate=0d, sec-latency=0 I/O behind bridge: None Memory behind bridge: 8000-ffef [size=2047M] Prefetchable memory behind bridge: 2100-21fd [size=1016G] Capabilities: [40] Power Management version 3 Capabilities: [48] Express Root Port (Slot-), MSI 00 Capabilities: [100] Advanced Error Reporting Capabilities: [148] #19 Kernel modules: shpchp 0001:01:00.0 PCI bridge: PLX Technology, Inc. PEX 8732 32-lane, 8-Port PCI Express Gen 3 (8.0 GT/s) Switch (rev ca) (prog-if 00 [Normal decode]) Flags: bus master, fast devsel, latency 0, IRQ 493, NUMA node 0 Memory at 3fe08280 (32-bit, non-prefetchable) [size=256K] Bus: primary=01, secondary=02, subordinate=0d, sec-latency=0 I/O behind br
[PATCH] drm/radeon: insist on 32-bit DMA for Cedar on PPC64/PPC64LE
In radeon_device_init, set the need_dma32 flag for Cedar chips (e.g. FirePro 2270). This fixes, or at least works around, a bug on PowerPC exposed by last year's commits 8e3f1b1d8255105f31556aacf8aeb6071b00d469 (Russell Currey) and 253fd51e2f533552ae35a0c661705da6c4842c1b (Alistair Popple) which enabled the 64-bit DMA iommu bypass. This caused the device to freeze, in some cases unrecoverably, and is the subject of several bug reports internal to Red Hat. Signed-off-by: Ben Crocker <bcroc...@redhat.com> --- drivers/gpu/drm/radeon/radeon_device.c | 4 1 file changed, 4 insertions(+) diff --git a/drivers/gpu/drm/radeon/radeon_device.c b/drivers/gpu/drm/radeon/radeon_device.c index ffc10cadcf34..32b577c776b9 100644 --- a/drivers/gpu/drm/radeon/radeon_device.c +++ b/drivers/gpu/drm/radeon/radeon_device.c @@ -1397,6 +1397,10 @@ int radeon_device_init(struct radeon_device *rdev, if ((rdev->flags & RADEON_IS_PCI) && (rdev->family <= CHIP_RS740)) rdev->need_dma32 = true; +#ifdef CONFIG_PPC64 + if (rdev->family == CHIP_CEDAR) + rdev->need_dma32 = true; +#endif dma_bits = rdev->need_dma32 ? 32 : 40; r = pci_set_dma_mask(rdev->pdev, DMA_BIT_MASK(dma_bits)); -- 2.13.6 ___ dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel
Re: [PATCH] drm/radeon: insist on 32-bit DMA for Cedar on PPC64/PPC64LE
One of my colleagues did discover a "Radeon HG 5450 PCI" from February 2010 which did, apparently, have a Cedar GPU and very definitely had a (plain old) PCI connector. But I take your point, and will just drop the PCI/PCIE check. On Thu, Feb 22, 2018 at 12:04 PM, Michel Dänzer <mic...@daenzer.net> wrote: > On 2018-02-22 10:57 PM, Ben Crocker wrote: > > In radeon_device_init, set the need_dma32 flag for Cedar chips > > (e.g. FirePro 2270). This fixes, or at least works around, a bug > > on PowerPC exposed by last year's commits > > > > 8e3f1b1d8255105f31556aacf8aeb6071b00d469 (Russell Currey) > > > > and > > > > 253fd51e2f533552ae35a0c661705da6c4842c1b (Alistair Popple) > > > > which enabled the 64-bit DMA iommu bypass. > > > > This caused the device to freeze, in some cases unrecoverably, and is > > the subject of several bug reports internal to Red Hat. > > > > Signed-off-by: Ben Crocker <bcroc...@redhat.com> > > --- > > drivers/gpu/drm/radeon/radeon_device.c | 5 + > > 1 file changed, 5 insertions(+) > > > > diff --git a/drivers/gpu/drm/radeon/radeon_device.c > b/drivers/gpu/drm/radeon/radeon_device.c > > index ffc10cadcf34..ba7435bcd208 100644 > > --- a/drivers/gpu/drm/radeon/radeon_device.c > > +++ b/drivers/gpu/drm/radeon/radeon_device.c > > @@ -1397,6 +1397,11 @@ int radeon_device_init(struct radeon_device *rdev, > > if ((rdev->flags & RADEON_IS_PCI) && > > (rdev->family <= CHIP_RS740)) > > rdev->need_dma32 = true; > > +#ifdef CONFIG_PPC64 > > + if ((rdev->flags & (RADEON_IS_PCI | RADEON_IS_PCIE)) && > > Cedar is (like all Radeons from the last decade) always PCIe, so this > flags check is redundant FWIW. > > > -- > Earthling Michel Dänzer | http://www.amd.com > Libre software enthusiast | Mesa and X developer > ___ > amd-gfx mailing list > amd-...@lists.freedesktop.org > https://lists.freedesktop.org/mailman/listinfo/amd-gfx > ___ dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel
[PATCH] drm/radeon: insist on 32-bit DMA for Cedar on PPC64/PPC64LE
In radeon_device_init, set the need_dma32 flag for Cedar chips (e.g. FirePro 2270). This fixes, or at least works around, a bug on PowerPC exposed by last year's commits 8e3f1b1d8255105f31556aacf8aeb6071b00d469 (Russell Currey) and 253fd51e2f533552ae35a0c661705da6c4842c1b (Alistair Popple) which enabled the 64-bit DMA iommu bypass. This caused the device to freeze, in some cases unrecoverably, and is the subject of several bug reports internal to Red Hat. Signed-off-by: Ben Crocker <bcroc...@redhat.com> --- drivers/gpu/drm/radeon/radeon_device.c | 5 + 1 file changed, 5 insertions(+) diff --git a/drivers/gpu/drm/radeon/radeon_device.c b/drivers/gpu/drm/radeon/radeon_device.c index ffc10cadcf34..ba7435bcd208 100644 --- a/drivers/gpu/drm/radeon/radeon_device.c +++ b/drivers/gpu/drm/radeon/radeon_device.c @@ -1397,6 +1397,11 @@ int radeon_device_init(struct radeon_device *rdev, if ((rdev->flags & RADEON_IS_PCI) && (rdev->family <= CHIP_RS740)) rdev->need_dma32 = true; +#ifdef CONFIG_PPC64 + if ((rdev->flags & (RADEON_IS_PCI | RADEON_IS_PCIE)) && + (rdev->family == CHIP_CEDAR)) + rdev->need_dma32 = true; +#endif dma_bits = rdev->need_dma32 ? 32 : 40; r = pci_set_dma_mask(rdev->pdev, DMA_BIT_MASK(dma_bits)); -- 2.13.6 ___ dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel
[PATCH] drm/radeon: insist on 32-bit DMA for Cedar on PPC64/PPC64LE
In radeon_device_init, set the need_dma32 flag for Cedar chips (e.g. FirePro 2270). This fixes, or at least works around, a bug on PowerPC exposed by last year's commits 8e3f1b1d8255105f31556aacf8aeb6071b00d469 (Russell Currey) and 253fd51e2f533552ae35a0c661705da6c4842c1b (Alistair Popple) which enabled the 64-bit DMA iommu bypass. This caused the device to freeze, in some cases unrecoverably, and is the subject of several bug reports internal to Red Hat. Signed-off-by: Ben Crocker <bcroc...@redhat.com> --- drivers/gpu/drm/radeon/radeon_device.c | 5 + 1 file changed, 5 insertions(+) diff --git a/drivers/gpu/drm/radeon/radeon_device.c b/drivers/gpu/drm/radeon/radeon_device.c index ffc10cadcf34..ba7435bcd208 100644 --- a/drivers/gpu/drm/radeon/radeon_device.c +++ b/drivers/gpu/drm/radeon/radeon_device.c @@ -1397,6 +1397,11 @@ int radeon_device_init(struct radeon_device *rdev, if ((rdev->flags & RADEON_IS_PCI) && (rdev->family <= CHIP_RS740)) rdev->need_dma32 = true; +#ifdef CONFIG_PPC64 + if ((rdev->flags & (RADEON_IS_PCI | RADEON_IS_PCIE)) && + (rdev->family == CHIP_CEDAR)) + rdev->need_dma32 = true; +#endif dma_bits = rdev->need_dma32 ? 32 : 40; r = pci_set_dma_mask(rdev->pdev, DMA_BIT_MASK(dma_bits)); -- 2.13.6 ___ dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel
Re: [PATCH] drm/radeon: insist on 32-bit DMA for Cedar
I considered that, but I'm only equipped to test on a few cards--the ones IBM is interested in. So this Cedar-specific change fixes the problem on the FirePro 2270; the other cards seem perfectly capable of handling 40+ bit DMA addresses on the Power8 platform. On Thu, Feb 22, 2018 at 11:07 AM, Alex Deucher <alexdeuc...@gmail.com> wrote: > On Thu, Feb 22, 2018 at 10:55 AM, Ben Crocker <bcroc...@redhat.com> wrote: > > Alex, Christian, et al. > > > > Thanks for your rapid response! > > > > I can certainly make the change PPC-only (#ifdef CONFIG_PPC64...) and I > > think > > that's the safest way to go right now. It makes the patch very specific > to > > Cedar > > on PowerPC64, so it doesn't break anything that may have come to depend > on > > Russell's and Alistair's changes, and my timetable for getting a fix in > is > > very short. > > If you want to go this route, you might just set need_dma32 > unconditionally for PPC64. The bus interface hw is pretty much the > same across all asics in this time period. > > Alex > > > > > > -- Ben > > > > > > > > On Thu, Feb 22, 2018 at 2:25 AM, Christian König > > <ckoenig.leichtzumer...@gmail.com> wrote: > >> > >> Am 22.02.2018 um 03:05 schrieb Alex Deucher: > >>> > >>> On Wed, Feb 21, 2018 at 6:41 PM, Ben Crocker <bcroc...@redhat.com> > wrote: > >>>> > >>>> In radeon_device_init, set the need_dma32 flag for Cedar chips > >>>> (e.g. FirePro 2270). This fixes, or at least works around, a bug > >>>> on PowerPC exposed by last year's commits > >>>> > >>>> 8e3f1b1d8255105f31556aacf8aeb6071b00d469 (Russell Currey) > >>>> > >>>> and > >>>> > >>>> 253fd51e2f533552ae35a0c661705da6c4842c1b (Alistair Popple) > >>>> > >>>> which enabled the 64-bit DMA iommu bypass. > >>>> > >>>> This caused the device to freeze, in some cases unrecoverably, and is > >>>> the subject of several bug reports internal to Red Hat. > >>> > >>> Can we make this ppc only? 40 bit DMA works fine on x86. > >> > >> > >> Yeah and at least when the dma_coherent allocator path is used it should > >> work fine on PPC as well. > >> > >> So that is not really a driver bug, but a platform bug and we should > think > >> about reverting or at least disabling those two patches instead. > >> > >> Christian. > >> > >>> > >>> Alex > >>> > >>>> Signed-off-by: Ben Crocker <bcroc...@redhat.com> > >>>> --- > >>>> drivers/gpu/drm/radeon/radeon_device.c | 5 - > >>>> 1 file changed, 4 insertions(+), 1 deletion(-) > >>>> > >>>> diff --git a/drivers/gpu/drm/radeon/radeon_device.c > >>>> b/drivers/gpu/drm/radeon/radeon_device.c > >>>> index ffc10cadcf34..02538903830d 100644 > >>>> --- a/drivers/gpu/drm/radeon/radeon_device.c > >>>> +++ b/drivers/gpu/drm/radeon/radeon_device.c > >>>> @@ -1395,7 +1395,10 @@ int radeon_device_init(struct radeon_device > >>>> *rdev, > >>>> if (rdev->flags & RADEON_IS_AGP) > >>>> rdev->need_dma32 = true; > >>>> if ((rdev->flags & RADEON_IS_PCI) && > >>>> - (rdev->family <= CHIP_RS740)) > >>>> + (rdev->family <= CHIP_RS740 || rdev->family == > CHIP_CEDAR)) > >>>> + rdev->need_dma32 = true; > >>>> + if ((rdev->flags & RADEON_IS_PCIE) && > >>>> + (rdev->family == CHIP_CEDAR)) > >>>> rdev->need_dma32 = true; > >>>> > >>>> dma_bits = rdev->need_dma32 ? 32 : 40; > >>>> -- > >>>> 2.13.6 > >>>> > >>>> ___ > >>>> dri-devel mailing list > >>>> dri-devel@lists.freedesktop.org > >>>> https://lists.freedesktop.org/mailman/listinfo/dri-devel > >>> > >>> ___ > >>> amd-gfx mailing list > >>> amd-...@lists.freedesktop.org > >>> https://lists.freedesktop.org/mailman/listinfo/amd-gfx > >> > >> > > > ___ > amd-gfx mailing list > amd-...@lists.freedesktop.org > https://lists.freedesktop.org/mailman/listinfo/amd-gfx > ___ dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel
Re: [PATCH] drm/radeon: insist on 32-bit DMA for Cedar
Alex, Christian, et al. Thanks for your rapid response! I can certainly make the change PPC-only (#ifdef CONFIG_PPC64...) and I think that's the safest way to go right now. It makes the patch very specific to Cedar on PowerPC64, so it doesn't break anything that may have come to depend on Russell's and Alistair's changes, and my timetable for getting a fix in is very short. -- Ben On Thu, Feb 22, 2018 at 2:25 AM, Christian König < ckoenig.leichtzumer...@gmail.com> wrote: > Am 22.02.2018 um 03:05 schrieb Alex Deucher: > >> On Wed, Feb 21, 2018 at 6:41 PM, Ben Crocker <bcroc...@redhat.com> wrote: >> >>> In radeon_device_init, set the need_dma32 flag for Cedar chips >>> (e.g. FirePro 2270). This fixes, or at least works around, a bug >>> on PowerPC exposed by last year's commits >>> >>> 8e3f1b1d8255105f31556aacf8aeb6071b00d469 (Russell Currey) >>> >>> and >>> >>> 253fd51e2f533552ae35a0c661705da6c4842c1b (Alistair Popple) >>> >>> which enabled the 64-bit DMA iommu bypass. >>> >>> This caused the device to freeze, in some cases unrecoverably, and is >>> the subject of several bug reports internal to Red Hat. >>> >> Can we make this ppc only? 40 bit DMA works fine on x86. >> > > Yeah and at least when the dma_coherent allocator path is used it should > work fine on PPC as well. > > So that is not really a driver bug, but a platform bug and we should think > about reverting or at least disabling those two patches instead. > > Christian. > > >> Alex >> >> Signed-off-by: Ben Crocker <bcroc...@redhat.com> >>> --- >>> drivers/gpu/drm/radeon/radeon_device.c | 5 - >>> 1 file changed, 4 insertions(+), 1 deletion(-) >>> >>> diff --git a/drivers/gpu/drm/radeon/radeon_device.c >>> b/drivers/gpu/drm/radeon/radeon_device.c >>> index ffc10cadcf34..02538903830d 100644 >>> --- a/drivers/gpu/drm/radeon/radeon_device.c >>> +++ b/drivers/gpu/drm/radeon/radeon_device.c >>> @@ -1395,7 +1395,10 @@ int radeon_device_init(struct radeon_device *rdev, >>> if (rdev->flags & RADEON_IS_AGP) >>> rdev->need_dma32 = true; >>> if ((rdev->flags & RADEON_IS_PCI) && >>> - (rdev->family <= CHIP_RS740)) >>> + (rdev->family <= CHIP_RS740 || rdev->family == CHIP_CEDAR)) >>> + rdev->need_dma32 = true; >>> + if ((rdev->flags & RADEON_IS_PCIE) && >>> + (rdev->family == CHIP_CEDAR)) >>> rdev->need_dma32 = true; >>> >>> dma_bits = rdev->need_dma32 ? 32 : 40; >>> -- >>> 2.13.6 >>> >>> ___ >>> dri-devel mailing list >>> dri-devel@lists.freedesktop.org >>> https://lists.freedesktop.org/mailman/listinfo/dri-devel >>> >> ___ >> amd-gfx mailing list >> amd-...@lists.freedesktop.org >> https://lists.freedesktop.org/mailman/listinfo/amd-gfx >> > > ___ dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel
[PATCH] drm/radeon: insist on 32-bit DMA for Cedar
In radeon_device_init, set the need_dma32 flag for Cedar chips (e.g. FirePro 2270). This fixes, or at least works around, a bug on PowerPC exposed by last year's commits 8e3f1b1d8255105f31556aacf8aeb6071b00d469 (Russell Currey) and 253fd51e2f533552ae35a0c661705da6c4842c1b (Alistair Popple) which enabled the 64-bit DMA iommu bypass. This caused the device to freeze, in some cases unrecoverably, and is the subject of several bug reports internal to Red Hat. Signed-off-by: Ben Crocker <bcroc...@redhat.com> --- drivers/gpu/drm/radeon/radeon_device.c | 5 - 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/drivers/gpu/drm/radeon/radeon_device.c b/drivers/gpu/drm/radeon/radeon_device.c index ffc10cadcf34..02538903830d 100644 --- a/drivers/gpu/drm/radeon/radeon_device.c +++ b/drivers/gpu/drm/radeon/radeon_device.c @@ -1395,7 +1395,10 @@ int radeon_device_init(struct radeon_device *rdev, if (rdev->flags & RADEON_IS_AGP) rdev->need_dma32 = true; if ((rdev->flags & RADEON_IS_PCI) && - (rdev->family <= CHIP_RS740)) + (rdev->family <= CHIP_RS740 || rdev->family == CHIP_CEDAR)) + rdev->need_dma32 = true; + if ((rdev->flags & RADEON_IS_PCIE) && + (rdev->family == CHIP_CEDAR)) rdev->need_dma32 = true; dma_bits = rdev->need_dma32 ? 32 : 40; -- 2.13.6 ___ dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel
