Re: [PATCH 09/15] swiotlb: make the swiotlb_init interface more useful
On Wed, Jun 01, 2022 at 08:21:41PM +0200, Christoph Hellwig wrote:
> On Wed, Jun 01, 2022 at 11:11:57AM -0700, Nathan Chancellor wrote:
> > On Wed, Jun 01, 2022 at 07:57:43PM +0200, Christoph Hellwig wrote:
> > > On Wed, Jun 01, 2022 at 10:46:54AM -0700, Nathan Chancellor wrote:
> > > > On Wed, Jun 01, 2022 at 07:34:41PM +0200, Christoph Hellwig wrote:
> > > > > Can you send me the full dmesg and the content of
> > > > > /sys/kernel/debug/swiotlb/io_tlb_nslabs for a good and a bad boot?
> > > >
> > > > Sure thing, they are attached! If there is anything else I can provide
> > > > or test, I am more than happy to do so.
> > >
> > > Nothing interesting. But the performance numbers almost look like
> > > swiotlb=force got ignored before (even if I can't explain why).
> >
> > I was able to get my performance back with this diff but I don't know if
> > this is a hack or a proper fix in the context of the series.
>
> This looks good, but needs a little tweak. I'd go for this variant of
> it:
Tested-by: Nathan Chancellor
Thanks a lot for the quick fix!
> diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c
> index dfa1de89dc944..cb50f8d383606 100644
> --- a/kernel/dma/swiotlb.c
> +++ b/kernel/dma/swiotlb.c
> @@ -192,7 +192,7 @@ void __init swiotlb_update_mem_attributes(void)
> }
>
> static void swiotlb_init_io_tlb_mem(struct io_tlb_mem *mem, phys_addr_t
> start,
> - unsigned long nslabs, bool late_alloc)
> + unsigned long nslabs, unsigned int flags, bool late_alloc)
> {
> void *vaddr = phys_to_virt(start);
> unsigned long bytes = nslabs << IO_TLB_SHIFT, i;
> @@ -203,8 +203,7 @@ static void swiotlb_init_io_tlb_mem(struct io_tlb_mem
> *mem, phys_addr_t start,
> mem->index = 0;
> mem->late_alloc = late_alloc;
>
> - if (swiotlb_force_bounce)
> - mem->force_bounce = true;
> + mem->force_bounce = swiotlb_force_bounce || (flags & SWIOTLB_FORCE);
>
> spin_lock_init(>lock);
> for (i = 0; i < mem->nslabs; i++) {
> @@ -275,8 +274,7 @@ void __init swiotlb_init_remap(bool addressing_limit,
> unsigned int flags,
> panic("%s: Failed to allocate %zu bytes align=0x%lx\n",
> __func__, alloc_size, PAGE_SIZE);
>
> - swiotlb_init_io_tlb_mem(mem, __pa(tlb), nslabs, false);
> - mem->force_bounce = flags & SWIOTLB_FORCE;
> + swiotlb_init_io_tlb_mem(mem, __pa(tlb), nslabs, flags, false);
>
> if (flags & SWIOTLB_VERBOSE)
> swiotlb_print_info();
> @@ -348,7 +346,7 @@ int swiotlb_init_late(size_t size, gfp_t gfp_mask,
>
> set_memory_decrypted((unsigned long)vstart,
>(nslabs << IO_TLB_SHIFT) >> PAGE_SHIFT);
> - swiotlb_init_io_tlb_mem(mem, virt_to_phys(vstart), nslabs, true);
> + swiotlb_init_io_tlb_mem(mem, virt_to_phys(vstart), nslabs, 0, true);
>
> swiotlb_print_info();
> return 0;
> @@ -835,8 +833,8 @@ static int rmem_swiotlb_device_init(struct reserved_mem
> *rmem,
>
> set_memory_decrypted((unsigned long)phys_to_virt(rmem->base),
>rmem->size >> PAGE_SHIFT);
> - swiotlb_init_io_tlb_mem(mem, rmem->base, nslabs, false);
> - mem->force_bounce = true;
> + swiotlb_init_io_tlb_mem(mem, rmem->base, nslabs, SWIOTLB_FORCE,
> + false);
> mem->for_alloc = true;
>
> rmem->priv = mem;
>
Cheers,
Nathan
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH 09/15] swiotlb: make the swiotlb_init interface more useful
On Wed, Jun 01, 2022 at 07:57:43PM +0200, Christoph Hellwig wrote: > On Wed, Jun 01, 2022 at 10:46:54AM -0700, Nathan Chancellor wrote: > > On Wed, Jun 01, 2022 at 07:34:41PM +0200, Christoph Hellwig wrote: > > > Can you send me the full dmesg and the content of > > > /sys/kernel/debug/swiotlb/io_tlb_nslabs for a good and a bad boot? > > > > Sure thing, they are attached! If there is anything else I can provide > > or test, I am more than happy to do so. > > Nothing interesting. But the performance numbers almost look like > swiotlb=force got ignored before (even if I can't explain why). I was able to get my performance back with this diff but I don't know if this is a hack or a proper fix in the context of the series. diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c index dfa1de89dc94..0bfb2fe3d8c5 100644 --- a/kernel/dma/swiotlb.c +++ b/kernel/dma/swiotlb.c @@ -276,7 +276,7 @@ void __init swiotlb_init_remap(bool addressing_limit, unsigned int flags, __func__, alloc_size, PAGE_SIZE); swiotlb_init_io_tlb_mem(mem, __pa(tlb), nslabs, false); - mem->force_bounce = flags & SWIOTLB_FORCE; + mem->force_bounce = swiotlb_force_bounce || (flags & SWIOTLB_FORCE); if (flags & SWIOTLB_VERBOSE) swiotlb_print_info(); > Do you get a similar performance with the new kernel without > swiotlb=force as the old one with that argument by any chance? I'll see if I can test that, as I am not sure I have control over those cmdline arguments. Cheers, Nathan ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH 09/15] swiotlb: make the swiotlb_init interface more useful
On Wed, Jun 01, 2022 at 07:34:41PM +0200, Christoph Hellwig wrote: > Can you send me the full dmesg and the content of > /sys/kernel/debug/swiotlb/io_tlb_nslabs for a good and a bad boot? Sure thing, they are attached! If there is anything else I can provide or test, I am more than happy to do so. Cheers, Nathan # cat /sys/kernel/debug/swiotlb/io_tlb_nslabs 32768 # dmesg [0.00] Linux version 5.18.0-rc3-microsoft-standard-WSL2-8-ga3e230926708 (nathan@dev-arch.thelio-3990X) (gcc (GCC) 12.1.0, GNU ld (GNU Binutils) 2.38) #1 SMP PREEMPT_DYNAMIC Wed Jun 1 10:38:34 MST 2022 [0.00] Command line: initrd=\initrd.img panic=-1 nr_cpus=8 swiotlb=force earlycon=uart8250,io,0x3f8,115200 console=hvc0 debug pty.legacy_count=0 [0.00] KERNEL supported cpus: [0.00] Intel GenuineIntel [0.00] AMD AuthenticAMD [0.00] Centaur CentaurHauls [0.00] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [0.00] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [0.00] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [0.00] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [0.00] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'compacted' format. [0.00] signal: max sigframe size: 1776 [0.00] BIOS-provided physical RAM map: [0.00] BIOS-e820: [mem 0x-0x0009] usable [0.00] BIOS-e820: [mem 0x000e-0x000e0fff] reserved [0.00] BIOS-e820: [mem 0x0010-0x001f] ACPI data [0.00] BIOS-e820: [mem 0x0020-0xf7ff] usable [0.00] BIOS-e820: [mem 0x0001-0x000407ff] usable [0.00] earlycon: uart8250 at I/O port 0x3f8 (options '115200') [0.00] printk: bootconsole [uart8250] enabled [0.00] NX (Execute Disable) protection: active [0.00] DMI not present or invalid. [0.00] Hypervisor detected: Microsoft Hyper-V [0.00] Hyper-V: privilege flags low 0xae7f, high 0x3b8030, hints 0xc2c, misc 0xe0bed7b6 [0.00] Hyper-V: Host Build 10.0.22000.708-0-0 [0.00] Hyper-V: Nested features: 0x4a [0.00] Hyper-V: LAPIC Timer Frequency: 0x1e8480 [0.00] Hyper-V: Using hypercall for remote TLB flush [0.00] clocksource: hyperv_clocksource_tsc_page: mask: 0x max_cycles: 0x24e6a1710, max_idle_ns: 440795202120 ns [0.05] tsc: Detected 3800.008 MHz processor [0.001901] e820: update [mem 0x-0x0fff] usable ==> reserved [0.004593] e820: remove [mem 0x000a-0x000f] usable [0.006806] last_pfn = 0x408000 max_arch_pfn = 0x4 [0.009042] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [0.011760] last_pfn = 0xf8000 max_arch_pfn = 0x4 [0.013959] Using GB pages for direct mapping [0.015749] RAMDISK: [mem 0x0371f000-0x03779fff] [0.017616] ACPI: Early table checksum verification disabled [0.019854] ACPI: RSDP 0x000E 24 (v02 VRTUAL) [0.022162] ACPI: XSDT 0x0010 44 (v01 VRTUAL MICROSFT 0001 MSFT 0001) [0.025624] ACPI: FACP 0x00101000 000114 (v06 VRTUAL MICROSFT 0001 MSFT 0001) [0.029022] ACPI: DSDT 0x001011B8 01E184 (v02 MSFTVM DSDT01 0001 MSFT 0500) [0.032413] ACPI: FACS 0x00101114 40 [0.034280] ACPI: OEM0 0x00101154 64 (v01 VRTUAL MICROSFT 0001 MSFT 0001) [0.037699] ACPI: SRAT 0x0011F33C 000330 (v02 VRTUAL MICROSFT 0001 MSFT 0001) [0.041089] ACPI: APIC 0x0011F66C 88 (v04 VRTUAL MICROSFT 0001 MSFT 0001) [0.044475] ACPI: Reserving FACP table memory at [mem 0x101000-0x101113] [0.047159] ACPI: Reserving DSDT table memory at [mem 0x1011b8-0x11f33b] [0.049905] ACPI: Reserving FACS table memory at [mem 0x101114-0x101153] [0.052693] ACPI: Reserving OEM0 table memory at [mem 0x101154-0x1011b7] [0.055404] ACPI: Reserving SRAT table memory at [mem 0x11f33c-0x11f66b] [0.058040] ACPI: Reserving APIC table memory at [mem 0x11f66c-0x11f6f3] [0.061078] Zone ranges: [0.062074] DMA [mem 0x1000-0x00ff] [0.066106] DMA32[mem 0x0100-0x] [0.068763] Normal [mem 0x0001-0x000407ff] [0.071235] Device empty [0.072384] Movable zone start for each node [0.074058] Early memory node ranges [0.075515] node 0: [mem 0x1000-0x0009] [0.077979] node 0: [mem 0x0020-0xf7ff] [0.080483] node 0: [mem 0x0001-0x000407ff] [0.082980] Initmem setup node 0 [mem 0x1000-0x000407ff] [0.085954] On node 0, zone DMA: 1 pages in unavailable ranges [0.085972] On node 0, zone DMA: 352
Re: [PATCH 09/15] swiotlb: make the swiotlb_init interface more useful
Hi Christoph,
On Mon, Apr 04, 2022 at 07:05:53AM +0200, Christoph Hellwig wrote:
> Pass a bool to pass if swiotlb needs to be enabled based on the
> addressing needs and replace the verbose argument with a set of
> flags, including one to force enable bounce buffering.
>
> Note that this patch removes the possibility to force xen-swiotlb
> use using swiotlb=force on the command line on x86 (arm and arm64
> never supported that), but this interface will be restored shortly.
>
> Signed-off-by: Christoph Hellwig
I bisected a performance regression in WSL2 to this change as commit
c6af2aa9ffc9 ("swiotlb: make the swiotlb_init interface more useful") in
mainline (bisect log below). I initially noticed it because accessing the
Windows filesystem through the /mnt/c mount is about 40x slower if I am doing
my math right based on the benchmarks below.
Before:
$ uname -r; and hyperfine "ls -l /mnt/c/Users/natec/Downloads"
5.18.0-rc3-microsoft-standard-WSL2-8-ga3e230926708
Benchmark 1: ls -l /mnt/c/Users/natec/Downloads
Time (mean ± σ): 564.5 ms ± 24.1 ms[User: 2.5 ms, System: 130.3 ms]
Range (min … max): 510.2 ms … 588.0 ms10 runs
After
$ uname -r; and hyperfine "ls -l /mnt/c/Users/natec/Downloads"
5.18.0-rc3-microsoft-standard-WSL2-9-gc6af2aa9ffc9
Benchmark 1: ls -l /mnt/c/Users/natec/Downloads
Time (mean ± σ): 23.282 s ± 1.220 s[User: 0.013 s, System: 0.101 s]
Range (min … max): 21.793 s … 25.317 s10 runs
I do see 'swiotlb=force' on the cmdline:
$ cat /proc/cmdline
initrd=\initrd.img panic=-1 nr_cpus=8 swiotlb=force
earlycon=uart8250,io,0x3f8,115200 console=hvc0 debug pty.legacy_count=0
/mnt/c appears to be a 9p mount, not sure if that is relevant here:
$ mount &| grep /mnt/c
drvfs on /mnt/c type 9p
(rw,noatime,dirsync,aname=drvfs;path=C:\;uid=1000;gid=1000;symlinkroot=/mnt/,mmap,access=client,msize=262144,trans=virtio)
If there is any other information I can provide, please let me know.
Cheers,
Nathan
# bad: [700170bf6b4d773e328fa54ebb70ba444007c702] Merge tag 'nfs-for-5.19-1' of
git://git.linux-nfs.org/projects/anna/linux-nfs
# good: [4b0986a3613c92f4ec1bdc7f60ec66fea135991f] Linux 5.18
git bisect start '700170bf6b4d773e328fa54ebb70ba444007c702' 'v5.18'
# good: [86c87bea6b42100c67418af690919c44de6ede6e] Merge tag
'devicetree-for-5.19' of
git://git.kernel.org/pub/scm/linux/kernel/git/robh/linux
git bisect good 86c87bea6b42100c67418af690919c44de6ede6e
# bad: [ae862183285cbb2ef9032770d98ffa9becffe9d5] Merge tag 'arm-dt-5.19' of
git://git.kernel.org/pub/scm/linux/kernel/git/soc/soc
git bisect bad ae862183285cbb2ef9032770d98ffa9becffe9d5
# good: [2518f226c60d8e04d18ba4295500a5b0b8ac7659] Merge tag
'drm-next-2022-05-25' of git://anongit.freedesktop.org/drm/drm
git bisect good 2518f226c60d8e04d18ba4295500a5b0b8ac7659
# bad: [babf0bb978e3c9fce6c4eba6b744c8754fd43d8e] Merge tag
'xfs-5.19-for-linus' of git://git.kernel.org/pub/scm/fs/xfs/xfs-linux
git bisect bad babf0bb978e3c9fce6c4eba6b744c8754fd43d8e
# good: [beed983621fbdfd291e6e3a0cdc4d10517e60af8] ASoC: Intel: avs: Machine
board registration
git bisect good beed983621fbdfd291e6e3a0cdc4d10517e60af8
# good: [fbe86daca0ba878b04fa241b85e26e54d17d4229] Merge tag 'scsi-misc' of
git://git.kernel.org/pub/scm/linux/kernel/git/jejb/scsi
git bisect good fbe86daca0ba878b04fa241b85e26e54d17d4229
# good: [166afc45ed5523298541fd0297f9ad585cc2708c] Merge tag
'reflink-speedups-5.19_2022-04-28' of
git://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux into
xfs-5.19-for-next
git bisect good 166afc45ed5523298541fd0297f9ad585cc2708c
# bad: [e375780b631a5fc2a61a3b4fa12429255361a31e] Merge tag
'fsnotify_for_v5.19-rc1' of
git://git.kernel.org/pub/scm/linux/kernel/git/jack/linux-fs
git bisect bad e375780b631a5fc2a61a3b4fa12429255361a31e
# bad: [4a37f3dd9a83186cb88d44808ab35b78375082c9] dma-direct: don't
over-decrypt memory
git bisect bad 4a37f3dd9a83186cb88d44808ab35b78375082c9
# bad: [742519538e6b07250c8085bbff4bd358bc03bf16] swiotlb: pass a gfp_mask
argument to swiotlb_init_late
git bisect bad 742519538e6b07250c8085bbff4bd358bc03bf16
# good: [9bbe7a7fc126e3d14fefa4b035854aba080926d9] arm/xen: don't check for
xen_initial_domain() in xen_create_contiguous_region
git bisect good 9bbe7a7fc126e3d14fefa4b035854aba080926d9
# good: [a3e230926708125205ffd06d3dc2175a8263ae7e] x86: centralize setting
SWIOTLB_FORCE when guest memory encryption is enabled
git bisect good a3e230926708125205ffd06d3dc2175a8263ae7e
# bad: [8ba2ed1be90fc210126f68186564707478552c95] swiotlb: add a SWIOTLB_ANY
flag to lift the low memory restriction
git bisect bad 8ba2ed1be90fc210126f68186564707478552c95
# bad: [c6af2aa9ffc9763826607bc2664ef3ea4475ed18] swiotlb: make the
swiotlb_init interface more useful
git bisect bad c6af2aa9ffc9763826607bc2664ef3ea4475ed18
# first bad commit: [c6af2aa9ffc9763826607bc2664ef3ea4475ed18] swiotlb: make
the swiotlb_init interface more useful
___
iommu mailing
Re: [PATCH v1] driver core: Extend deferred probe timeout on driver registration
On Mon, May 23, 2022 at 01:04:03PM -0700, Saravana Kannan wrote:
> On Mon, May 23, 2022 at 8:17 AM Nathan Chancellor wrote:
> >
> > On Fri, May 20, 2022 at 05:15:55PM -0700, Saravana Kannan wrote:
> > > On Fri, May 20, 2022 at 5:04 PM Nathan Chancellor
> > > wrote:
> > > >
> > > > On Fri, May 20, 2022 at 04:49:48PM -0700, Saravana Kannan wrote:
> > > > > On Fri, May 20, 2022 at 4:30 PM Nathan Chancellor
> > > > > wrote:
> > > > > >
> > > > > > Hi Saravana,
> > > > > >
> > > > > > On Fri, Apr 29, 2022 at 03:09:32PM -0700, Saravana Kannan wrote:
> > > > > > > The deferred probe timer that's used for this currently starts at
> > > > > > > late_initcall and runs for driver_deferred_probe_timeout seconds.
> > > > > > > The
> > > > > > > assumption being that all available drivers would be loaded and
> > > > > > > registered before the timer expires. This means, the
> > > > > > > driver_deferred_probe_timeout has to be pretty large for it to
> > > > > > > cover the
> > > > > > > worst case. But if we set the default value for it to cover the
> > > > > > > worst
> > > > > > > case, it would significantly slow down the average case. For this
> > > > > > > reason, the default value is set to 0.
> > > > > > >
> > > > > > > Also, with CONFIG_MODULES=y and the current default values of
> > > > > > > driver_deferred_probe_timeout=0 and fw_devlink=on, devices with
> > > > > > > missing
> > > > > > > drivers will cause their consumer devices to always defer their
> > > > > > > probes.
> > > > > > > This is because device links created by fw_devlink defer the
> > > > > > > probe even
> > > > > > > before the consumer driver's probe() is called.
> > > > > > >
> > > > > > > Instead of a fixed timeout, if we extend an unexpired deferred
> > > > > > > probe
> > > > > > > timer on every successful driver registration, with the
> > > > > > > expectation more
> > > > > > > modules would be loaded in the near future, then the default
> > > > > > > value of
> > > > > > > driver_deferred_probe_timeout only needs to be as long as the
> > > > > > > worst case
> > > > > > > time difference between two consecutive module loads.
> > > > > > >
> > > > > > > So let's implement that and set the default value to 10 seconds
> > > > > > > when
> > > > > > > CONFIG_MODULES=y.
> > > > > > >
> > > > > > > Cc: Greg Kroah-Hartman
> > > > > > > Cc: "Rafael J. Wysocki"
> > > > > > > Cc: Rob Herring
> > > > > > > Cc: Linus Walleij
> > > > > > > Cc: Will Deacon
> > > > > > > Cc: Ulf Hansson
> > > > > > > Cc: Kevin Hilman
> > > > > > > Cc: Thierry Reding
> > > > > > > Cc: Mark Brown
> > > > > > > Cc: Pavel Machek
> > > > > > > Cc: Geert Uytterhoeven
> > > > > > > Cc: Yoshihiro Shimoda
> > > > > > > Cc: Paul Kocialkowski
> > > > > > > Cc: linux-g...@vger.kernel.org
> > > > > > > Cc: linux...@vger.kernel.org
> > > > > > > Cc: iommu@lists.linux-foundation.org
> > > > > > > Signed-off-by: Saravana Kannan
> > > > > >
> > > > > > I bisected a boot hang with ARCH=s390 defconfig in QEMU down to this
> > > > > > change as commit 2b28a1a84a0e ("driver core: Extend deferred probe
> > > > > > timeout on driver registration") in next-20220520 (bisect log
> > > > > > below).
> > > > > >
> > > > > > $ make -skj"$(nproc)" ARCH=s390 CROSS_COMPILE=s390x-linux-gnu-
> > > > > > defconfig bzImage
> > > > > >
> > > > > > $ timeout --foreground 15m stdbuf -oL -eL \
> > > > > > qemu-system-s390x \
> > > > > > -initrd ... \
> > > > > > -M s390-ccw-
Re: [PATCH v1] driver core: Extend deferred probe timeout on driver registration
On Fri, May 20, 2022 at 05:15:55PM -0700, Saravana Kannan wrote:
> On Fri, May 20, 2022 at 5:04 PM Nathan Chancellor wrote:
> >
> > On Fri, May 20, 2022 at 04:49:48PM -0700, Saravana Kannan wrote:
> > > On Fri, May 20, 2022 at 4:30 PM Nathan Chancellor
> > > wrote:
> > > >
> > > > Hi Saravana,
> > > >
> > > > On Fri, Apr 29, 2022 at 03:09:32PM -0700, Saravana Kannan wrote:
> > > > > The deferred probe timer that's used for this currently starts at
> > > > > late_initcall and runs for driver_deferred_probe_timeout seconds. The
> > > > > assumption being that all available drivers would be loaded and
> > > > > registered before the timer expires. This means, the
> > > > > driver_deferred_probe_timeout has to be pretty large for it to cover
> > > > > the
> > > > > worst case. But if we set the default value for it to cover the worst
> > > > > case, it would significantly slow down the average case. For this
> > > > > reason, the default value is set to 0.
> > > > >
> > > > > Also, with CONFIG_MODULES=y and the current default values of
> > > > > driver_deferred_probe_timeout=0 and fw_devlink=on, devices with
> > > > > missing
> > > > > drivers will cause their consumer devices to always defer their
> > > > > probes.
> > > > > This is because device links created by fw_devlink defer the probe
> > > > > even
> > > > > before the consumer driver's probe() is called.
> > > > >
> > > > > Instead of a fixed timeout, if we extend an unexpired deferred probe
> > > > > timer on every successful driver registration, with the expectation
> > > > > more
> > > > > modules would be loaded in the near future, then the default value of
> > > > > driver_deferred_probe_timeout only needs to be as long as the worst
> > > > > case
> > > > > time difference between two consecutive module loads.
> > > > >
> > > > > So let's implement that and set the default value to 10 seconds when
> > > > > CONFIG_MODULES=y.
> > > > >
> > > > > Cc: Greg Kroah-Hartman
> > > > > Cc: "Rafael J. Wysocki"
> > > > > Cc: Rob Herring
> > > > > Cc: Linus Walleij
> > > > > Cc: Will Deacon
> > > > > Cc: Ulf Hansson
> > > > > Cc: Kevin Hilman
> > > > > Cc: Thierry Reding
> > > > > Cc: Mark Brown
> > > > > Cc: Pavel Machek
> > > > > Cc: Geert Uytterhoeven
> > > > > Cc: Yoshihiro Shimoda
> > > > > Cc: Paul Kocialkowski
> > > > > Cc: linux-g...@vger.kernel.org
> > > > > Cc: linux...@vger.kernel.org
> > > > > Cc: iommu@lists.linux-foundation.org
> > > > > Signed-off-by: Saravana Kannan
> > > >
> > > > I bisected a boot hang with ARCH=s390 defconfig in QEMU down to this
> > > > change as commit 2b28a1a84a0e ("driver core: Extend deferred probe
> > > > timeout on driver registration") in next-20220520 (bisect log below).
> > > >
> > > > $ make -skj"$(nproc)" ARCH=s390 CROSS_COMPILE=s390x-linux-gnu-
> > > > defconfig bzImage
> > > >
> > > > $ timeout --foreground 15m stdbuf -oL -eL \
> > > > qemu-system-s390x \
> > > > -initrd ... \
> > > > -M s390-ccw-virtio \
> > > > -display none \
> > > > -kernel arch/s390/boot/bzImage \
> > > > -m 512m \
> > > > -nodefaults \
> > > > -serial mon:stdio
> > > > ...
> > > > [2.077303] In-situ OAM (IOAM) with IPv6
> > > > [2.077639] NET: Registered PF_PACKET protocol family
> > > > [2.078063] bridge: filtering via arp/ip/ip6tables is no longer
> > > > available by default. Update your scripts to load br_netfilter if you
> > > > need this.
> > > > [2.078795] Key type dns_resolver registered
> > > > [2.079317] cio: Channel measurement facility initialized using
> > > > format extended (mode autodetected)
> > > > [2.081494] Discipline DIAG cannot be used without z/VM
> > > > [ 260.626363] random: crng init done
> > > > qemu-system-s390x: terminating on signal 15 from pid 3815762 (timeout)
> > > >
> > > > We have a
Re: [PATCH v1] driver core: Extend deferred probe timeout on driver registration
On Fri, May 20, 2022 at 04:49:48PM -0700, Saravana Kannan wrote:
> On Fri, May 20, 2022 at 4:30 PM Nathan Chancellor wrote:
> >
> > Hi Saravana,
> >
> > On Fri, Apr 29, 2022 at 03:09:32PM -0700, Saravana Kannan wrote:
> > > The deferred probe timer that's used for this currently starts at
> > > late_initcall and runs for driver_deferred_probe_timeout seconds. The
> > > assumption being that all available drivers would be loaded and
> > > registered before the timer expires. This means, the
> > > driver_deferred_probe_timeout has to be pretty large for it to cover the
> > > worst case. But if we set the default value for it to cover the worst
> > > case, it would significantly slow down the average case. For this
> > > reason, the default value is set to 0.
> > >
> > > Also, with CONFIG_MODULES=y and the current default values of
> > > driver_deferred_probe_timeout=0 and fw_devlink=on, devices with missing
> > > drivers will cause their consumer devices to always defer their probes.
> > > This is because device links created by fw_devlink defer the probe even
> > > before the consumer driver's probe() is called.
> > >
> > > Instead of a fixed timeout, if we extend an unexpired deferred probe
> > > timer on every successful driver registration, with the expectation more
> > > modules would be loaded in the near future, then the default value of
> > > driver_deferred_probe_timeout only needs to be as long as the worst case
> > > time difference between two consecutive module loads.
> > >
> > > So let's implement that and set the default value to 10 seconds when
> > > CONFIG_MODULES=y.
> > >
> > > Cc: Greg Kroah-Hartman
> > > Cc: "Rafael J. Wysocki"
> > > Cc: Rob Herring
> > > Cc: Linus Walleij
> > > Cc: Will Deacon
> > > Cc: Ulf Hansson
> > > Cc: Kevin Hilman
> > > Cc: Thierry Reding
> > > Cc: Mark Brown
> > > Cc: Pavel Machek
> > > Cc: Geert Uytterhoeven
> > > Cc: Yoshihiro Shimoda
> > > Cc: Paul Kocialkowski
> > > Cc: linux-g...@vger.kernel.org
> > > Cc: linux...@vger.kernel.org
> > > Cc: iommu@lists.linux-foundation.org
> > > Signed-off-by: Saravana Kannan
> >
> > I bisected a boot hang with ARCH=s390 defconfig in QEMU down to this
> > change as commit 2b28a1a84a0e ("driver core: Extend deferred probe
> > timeout on driver registration") in next-20220520 (bisect log below).
> >
> > $ make -skj"$(nproc)" ARCH=s390 CROSS_COMPILE=s390x-linux-gnu- defconfig
> > bzImage
> >
> > $ timeout --foreground 15m stdbuf -oL -eL \
> > qemu-system-s390x \
> > -initrd ... \
> > -M s390-ccw-virtio \
> > -display none \
> > -kernel arch/s390/boot/bzImage \
> > -m 512m \
> > -nodefaults \
> > -serial mon:stdio
> > ...
> > [2.077303] In-situ OAM (IOAM) with IPv6
> > [2.077639] NET: Registered PF_PACKET protocol family
> > [2.078063] bridge: filtering via arp/ip/ip6tables is no longer
> > available by default. Update your scripts to load br_netfilter if you need
> > this.
> > [2.078795] Key type dns_resolver registered
> > [2.079317] cio: Channel measurement facility initialized using format
> > extended (mode autodetected)
> > [2.081494] Discipline DIAG cannot be used without z/VM
> > [ 260.626363] random: crng init done
> > qemu-system-s390x: terminating on signal 15 from pid 3815762 (timeout)
> >
> > We have a simple rootfs available if necessary:
> >
> > https://github.com/ClangBuiltLinux/boot-utils/raw/bc0d17785eb67f1edd0ee0a134970a807895f741/images/s390/rootfs.cpio.zst
> >
> > If there is any other information I can provide, please let me know!
>
> Hmm... strange. Can you please try the following command line options
> and tell me which of these has the issue and which don't?
Sure thing!
> 1) deferred_probe_timeout=0
No issue.
> 2) deferred_probe_timeout=1
> 3) deferred_probe_timeout=300
Both of these appear to hang in the same way, I let each sit for five
minutes.
Cheers,
Nathan
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH v1] driver core: Extend deferred probe timeout on driver registration
Hi Saravana,
On Fri, Apr 29, 2022 at 03:09:32PM -0700, Saravana Kannan wrote:
> The deferred probe timer that's used for this currently starts at
> late_initcall and runs for driver_deferred_probe_timeout seconds. The
> assumption being that all available drivers would be loaded and
> registered before the timer expires. This means, the
> driver_deferred_probe_timeout has to be pretty large for it to cover the
> worst case. But if we set the default value for it to cover the worst
> case, it would significantly slow down the average case. For this
> reason, the default value is set to 0.
>
> Also, with CONFIG_MODULES=y and the current default values of
> driver_deferred_probe_timeout=0 and fw_devlink=on, devices with missing
> drivers will cause their consumer devices to always defer their probes.
> This is because device links created by fw_devlink defer the probe even
> before the consumer driver's probe() is called.
>
> Instead of a fixed timeout, if we extend an unexpired deferred probe
> timer on every successful driver registration, with the expectation more
> modules would be loaded in the near future, then the default value of
> driver_deferred_probe_timeout only needs to be as long as the worst case
> time difference between two consecutive module loads.
>
> So let's implement that and set the default value to 10 seconds when
> CONFIG_MODULES=y.
>
> Cc: Greg Kroah-Hartman
> Cc: "Rafael J. Wysocki"
> Cc: Rob Herring
> Cc: Linus Walleij
> Cc: Will Deacon
> Cc: Ulf Hansson
> Cc: Kevin Hilman
> Cc: Thierry Reding
> Cc: Mark Brown
> Cc: Pavel Machek
> Cc: Geert Uytterhoeven
> Cc: Yoshihiro Shimoda
> Cc: Paul Kocialkowski
> Cc: linux-g...@vger.kernel.org
> Cc: linux...@vger.kernel.org
> Cc: iommu@lists.linux-foundation.org
> Signed-off-by: Saravana Kannan
I bisected a boot hang with ARCH=s390 defconfig in QEMU down to this
change as commit 2b28a1a84a0e ("driver core: Extend deferred probe
timeout on driver registration") in next-20220520 (bisect log below).
$ make -skj"$(nproc)" ARCH=s390 CROSS_COMPILE=s390x-linux-gnu- defconfig bzImage
$ timeout --foreground 15m stdbuf -oL -eL \
qemu-system-s390x \
-initrd ... \
-M s390-ccw-virtio \
-display none \
-kernel arch/s390/boot/bzImage \
-m 512m \
-nodefaults \
-serial mon:stdio
...
[2.077303] In-situ OAM (IOAM) with IPv6
[2.077639] NET: Registered PF_PACKET protocol family
[2.078063] bridge: filtering via arp/ip/ip6tables is no longer available by
default. Update your scripts to load br_netfilter if you need this.
[2.078795] Key type dns_resolver registered
[2.079317] cio: Channel measurement facility initialized using format
extended (mode autodetected)
[2.081494] Discipline DIAG cannot be used without z/VM
[ 260.626363] random: crng init done
qemu-system-s390x: terminating on signal 15 from pid 3815762 (timeout)
We have a simple rootfs available if necessary:
https://github.com/ClangBuiltLinux/boot-utils/raw/bc0d17785eb67f1edd0ee0a134970a807895f741/images/s390/rootfs.cpio.zst
If there is any other information I can provide, please let me know!
Cheers,
Nathan
# bad: [18ecd30af1a8402c162cca1bd58771c0e5be7815] Add linux-next specific files
for 20220520
# good: [b015dcd62b86d298829990f8261d5d154b8d7af5] Merge tag
'for-5.18/parisc-4' of
git://git.kernel.org/pub/scm/linux/kernel/git/deller/parisc-linux
git bisect start '18ecd30af1a8402c162cca1bd58771c0e5be7815'
'b015dcd62b86d298829990f8261d5d154b8d7af5'
# good: [f9b63740b666dd9887eb0282d21b5f65bb0cadd0] Merge branch 'master' of
git://git.kernel.org/pub/scm/linux/kernel/git/herbert/cryptodev-2.6.git
git bisect good f9b63740b666dd9887eb0282d21b5f65bb0cadd0
# good: [1f5eb3e76303572f0318e8c50da51c516580aa03] Merge branch 'master' of
git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git
git bisect good 1f5eb3e76303572f0318e8c50da51c516580aa03
# bad: [4c1d9cc0363691893ef94fa0d798faca013e27d3] Merge branch 'staging-next'
of git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/staging.git
git bisect bad 4c1d9cc0363691893ef94fa0d798faca013e27d3
# bad: [dcb68304485c0ba5f84f1a54687c751b68263d93] Merge branch 'usb-next' of
git://git.kernel.org/pub/scm/linux/kernel/git/johan/usb-serial.git
git bisect bad dcb68304485c0ba5f84f1a54687c751b68263d93
# good: [61271996dc46aecb40fd26f89a4ec0a6bd8f3a8f] Merge branch 'next' of
git://git.kernel.org/pub/scm/virt/kvm/kvm.git
git bisect good 61271996dc46aecb40fd26f89a4ec0a6bd8f3a8f
# good: [d4db45a71f56032b552e161968bb0e5fdd2767f8] Merge branch 'for-next' of
git://git.kernel.org/pub/scm/linux/kernel/git/pavel/linux-leds.git
git bisect good d4db45a71f56032b552e161968bb0e5fdd2767f8
# good: [d090c7a2ab84663185e4abda21d7d83880937c8a] USB / dwc3: Fix a checkpatch
warning in core.c
git bisect good d090c7a2ab84663185e4abda21d7d83880937c8a
# bad: [b232b02bf3c205b13a26dcec08e53baddd8e59ed] driver core: fix deadlock in
__device_attach
git bisect bad b232b02bf3c205b13a26dcec08e53baddd8e59ed
# good:
Re: [patch V3 28/35] PCI/MSI: Simplify pci_irq_get_affinity()
On Sat, Dec 18, 2021 at 11:25:14AM +0100, Thomas Gleixner wrote:
> On Fri, Dec 17 2021 at 15:30, Nathan Chancellor wrote:
> > On Fri, Dec 10, 2021 at 11:19:26PM +0100, Thomas Gleixner wrote:
> > I just bisected a boot failure on my AMD test desktop to this patch as
> > commit f48235900182 ("PCI/MSI: Simplify pci_irq_get_affinity()") in
> > -next. It looks like there is a problem with the NVMe drive after this
> > change according to the logs. Given that the hard drive is not getting
> > mounted for journald to write logs to, I am not really sure how to get
> > them from the machine so I have at least taken a picture of what I see
> > on my screen; open to ideas on that front!
>
> Bah. Fix below.
Tested-by: Nathan Chancellor
> Thanks,
>
> tglx
> ---
> diff --git a/drivers/pci/msi/msi.c b/drivers/pci/msi/msi.c
> index 71802410e2ab..9b4910befeda 100644
> --- a/drivers/pci/msi/msi.c
> +++ b/drivers/pci/msi/msi.c
> @@ -1100,7 +1100,7 @@ EXPORT_SYMBOL(pci_irq_vector);
> */
> const struct cpumask *pci_irq_get_affinity(struct pci_dev *dev, int nr)
> {
> - int irq = pci_irq_vector(dev, nr);
> + int idx, irq = pci_irq_vector(dev, nr);
> struct msi_desc *desc;
>
> if (WARN_ON_ONCE(irq <= 0))
> @@ -1113,7 +1113,10 @@ const struct cpumask *pci_irq_get_affinity(struct
> pci_dev *dev, int nr)
>
> if (WARN_ON_ONCE(!desc->affinity))
> return NULL;
> - return >affinity[nr].mask;
> +
> + /* MSI has a mask array in the descriptor. */
> + idx = dev->msi_enabled ? nr : 0;
> + return >affinity[idx].mask;
> }
> EXPORT_SYMBOL(pci_irq_get_affinity);
>
>
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [patch V3 28/35] PCI/MSI: Simplify pci_irq_get_affinity()
Hi Thomas,
On Fri, Dec 10, 2021 at 11:19:26PM +0100, Thomas Gleixner wrote:
> From: Thomas Gleixner
>
> Replace open coded MSI descriptor chasing and use the proper accessor
> functions instead.
>
> Signed-off-by: Thomas Gleixner
> Reviewed-by: Greg Kroah-Hartman
> Reviewed-by: Jason Gunthorpe
Apologies if this has already been reported somewhere else or already
fixed, I did a search of all of lore and did not see anything similar to
it and I did not see any new commits in -tip around this.
I just bisected a boot failure on my AMD test desktop to this patch as
commit f48235900182 ("PCI/MSI: Simplify pci_irq_get_affinity()") in
-next. It looks like there is a problem with the NVMe drive after this
change according to the logs. Given that the hard drive is not getting
mounted for journald to write logs to, I am not really sure how to get
them from the machine so I have at least taken a picture of what I see
on my screen; open to ideas on that front!
https://github.com/nathanchance/bug-files/blob/0d25d78b5bc1d5e9c15192b3bc80676364de8287/f48235900182/crash.jpg
Please let me know what information I can provide to make debugging this
easier and I am more than happy to apply and test patches as needed.
Cheers,
Nathan
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [powerpc][next-20210727] Boot failure - kernel BUG at arch/powerpc/kernel/interrupt.c:98!
On 7/29/2021 9:35 AM, Konrad Rzeszutek Wilk wrote:
On Thu, Jul 29, 2021 at 05:13:36PM +0100, Will Deacon wrote:
On Wed, Jul 28, 2021 at 10:35:34AM -0700, Nathan Chancellor wrote:
On Wed, Jul 28, 2021 at 01:31:06PM +0530, Sachin Sant wrote:
next-20210723 was good. The boot failure seems to have been introduced with
next-20210726.
I have attached the boot log.
I noticed this with OpenSUSE's ppc64le config [1] and my bisect landed on
commit ad6c00283163 ("swiotlb: Free tbl memory in swiotlb_exit()"). That
series just keeps on giving...
Low-level across platform do that. And thank you for testing it and
finding this bug. Please let me know if the patch works so I can add it
in in the patch series.
That was not meant to sound as sarcastic as it did so my apologies for
that :(
Will's patch looks good to me in QEMU, I do not have a bare metal POWER
system to test it on.
Tested-by: Nathan Chancellor
Yes, but look how handy our new print is!
:)
[0.010799] software IO TLB: tearing down default memory pool
[0.010805] [ cut here ]
[0.010808] kernel BUG at arch/powerpc/kernel/interrupt.c:98!
Following Nick's suggestion, the diff below should help? I don't have a
relevant box on which I can test it though.
Will
--->8
diff --git a/arch/powerpc/platforms/pseries/svm.c
b/arch/powerpc/platforms/pseries/svm.c
index 1d829e257996..87f001b4c4e4 100644
--- a/arch/powerpc/platforms/pseries/svm.c
+++ b/arch/powerpc/platforms/pseries/svm.c
@@ -63,6 +63,9 @@ void __init svm_swiotlb_init(void)
int set_memory_encrypted(unsigned long addr, int numpages)
{
+ if (!mem_encrypt_active())
+ return 0;
+
if (!PAGE_ALIGNED(addr))
return -EINVAL;
@@ -73,6 +76,9 @@ int set_memory_encrypted(unsigned long addr, int numpages)
int set_memory_decrypted(unsigned long addr, int numpages)
{
+ if (!mem_encrypt_active())
+ return 0;
+
if (!PAGE_ALIGNED(addr))
return -EINVAL;
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [powerpc][next-20210727] Boot failure - kernel BUG at arch/powerpc/kernel/interrupt.c:98!
On Wed, Jul 28, 2021 at 01:31:06PM +0530, Sachin Sant wrote:
> linux-next fails to boot on Power server (POWER8/POWER9). Following traces
> are seen during boot
>
> [0.010799] software IO TLB: tearing down default memory pool
> [0.010805] [ cut here ]
> [0.010808] kernel BUG at arch/powerpc/kernel/interrupt.c:98!
> [0.010812] Oops: Exception in kernel mode, sig: 5 [#1]
> [0.010816] LE PAGE_SIZE=64K MMU=Hash SMP NR_CPUS=2048 NUMA pSeries
> [0.010820] Modules linked in:
> [0.010824] CPU: 1 PID: 1 Comm: swapper/0 Not tainted
> 5.14.0-rc3-next-20210727 #1
> [0.010830] NIP: c0032cfc LR: c000c764 CTR:
> c000c670
> [0.010834] REGS: c3603b10 TRAP: 0700 Not tainted
> (5.14.0-rc3-next-20210727)
> [0.010838] MSR: 80029033 CR: 28000222
> XER: 0002
> [0.010848] CFAR: c000c760 IRQMASK: 3
> [0.010848] GPR00: c000c764 c3603db0 c29bd000
> 0001
> [0.010848] GPR04: 0a68 0400 c3603868
>
> [0.010848] GPR08:
> 0003
> [0.010848] GPR12: c0001ec9ee80 c0012a28
>
> [0.010848] GPR16:
>
> [0.010848] GPR20:
>
> [0.010848] GPR24: f134
> c3603868
> [0.010848] GPR28: 0400 0a68 c202e9c0
> c3603e80
> [0.010896] NIP [c0032cfc] system_call_exception+0x8c/0x2e0
> [0.010901] LR [c000c764] system_call_common+0xf4/0x258
> [0.010907] Call Trace:
> [0.010909] [c3603db0] [c016a6dc]
> calculate_sigpending+0x4c/0xe0 (unreliable)
> [0.010915] [c3603e10] [c000c764]
> system_call_common+0xf4/0x258
> [0.010921] --- interrupt: c00 at kvm_template_end+0x4/0x8
> [0.010926] NIP: c0092dec LR: c0114fc8 CTR:
>
> [0.010930] REGS: c3603e80 TRAP: 0c00 Not tainted
> (5.14.0-rc3-next-20210727)
> [0.010934] MSR: 80009033 CR: 28000222
> XER:
> [0.010943] IRQMASK: 0
> [0.010943] GPR00: c202e9c0 c3603b00 c29bd000
> f134
> [0.010943] GPR04: 0a68 0400 c3603868
>
> [0.010943] GPR08:
>
> [0.010943] GPR12: c0001ec9ee80 c0012a28
>
> [0.010943] GPR16:
>
> [0.010943] GPR20:
>
> [0.010943] GPR24: c20033c4 c110afc0 c2081950
> c3277d40
> [0.010943] GPR28: ca68 0400
> 000d
> [0.010989] NIP [c0092dec] kvm_template_end+0x4/0x8
> [0.010993] LR [c0114fc8] set_memory_encrypted+0x38/0x60
> [0.010999] --- interrupt: c00
> [0.011001] [c3603b00] [c000c764]
> system_call_common+0xf4/0x258 (unreliable)
> [0.011008] Instruction dump:
> [0.011011] 694a0003 312a 7d495110 0b0a 6000 6000 e87f0108
> 68690002
> [0.011019] 7929ffe2 0b09 68634000 786397e2 <0b03> e93f0138
> 792907e0 0b09
> [0.011029] ---[ end trace a20ad55589efcb10 ]---
> [0.012297]
> [1.012304] Kernel panic - not syncing: Fatal exception
>
> next-20210723 was good. The boot failure seems to have been introduced with
> next-20210726.
>
> I have attached the boot log.
I noticed this with OpenSUSE's ppc64le config [1] and my bisect landed on
commit ad6c00283163 ("swiotlb: Free tbl memory in swiotlb_exit()"). That
series just keeps on giving... Adding some people from that thread to
this one. Original thread:
https://lore.kernel.org/r/1905cd70-7656-42ae-99e2-a31fc3812...@linux.vnet.ibm.com/
[1]: https://github.com/openSUSE/kernel-source/raw/master/config/ppc64le/default
Cheers,
Nathan
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH v15 06/12] swiotlb: Use is_swiotlb_force_bounce for swiotlb data bouncing
Hi Will and Robin, On 7/6/2021 10:06 AM, Will Deacon wrote: On Tue, Jul 06, 2021 at 04:39:11PM +0100, Robin Murphy wrote: On 2021-07-06 15:05, Christoph Hellwig wrote: On Tue, Jul 06, 2021 at 03:01:04PM +0100, Robin Murphy wrote: FWIW I was pondering the question of whether to do something along those lines or just scrap the default assignment entirely, so since I hadn't got round to saying that I've gone ahead and hacked up the alternative (similarly untested) for comparison :) TBH I'm still not sure which one I prefer... Claire did implement something like your suggestion originally, but I don't really like it as it doesn't scale for adding multiple global pools, e.g. for the 64-bit addressable one for the various encrypted secure guest schemes. Ah yes, that had slipped my mind, and it's a fair point indeed. Since we're not concerned with a minimal fix for backports anyway I'm more than happy to focus on Will's approach. Another thing is that that looks to take us a quiet step closer to the possibility of dynamically resizing a SWIOTLB pool, which is something that some of the hypervisor protection schemes looking to build on top of this series may want to explore at some point. Ok, I'll split that nasty diff I posted up into a reviewable series and we can take it from there. For what it's worth, I attempted to boot Will's diff on top of Konrad's devel/for-linus-5.14 and it did not work; in fact, I got no output on my monitor period, even with earlyprintk=, and I do not think this machine has a serial console. Robin's fix does work, it survived ten reboots with no issues getting to X and I do not see the KASAN and slub debug messages anymore but I understand that this is not the preferred solution it seems (although Konrad did want to know if it works). I am happy to test any further patches or follow ups as needed, just keep me on CC. Cheers, Nathan ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH v15 06/12] swiotlb: Use is_swiotlb_force_bounce for swiotlb data bouncing
Hi Will and Robin, On Fri, Jul 02, 2021 at 04:13:50PM +0100, Robin Murphy wrote: > On 2021-07-02 14:58, Will Deacon wrote: > > Hi Nathan, > > > > On Thu, Jul 01, 2021 at 12:52:20AM -0700, Nathan Chancellor wrote: > > > On 7/1/2021 12:40 AM, Will Deacon wrote: > > > > On Wed, Jun 30, 2021 at 08:56:51AM -0700, Nathan Chancellor wrote: > > > > > On Wed, Jun 30, 2021 at 12:43:48PM +0100, Will Deacon wrote: > > > > > > On Wed, Jun 30, 2021 at 05:17:27PM +0800, Claire Chang wrote: > > > > > > > `BUG: unable to handle page fault for address: 003a8290` > > > > > > > and > > > > > > > the fact it crashed at `_raw_spin_lock_irqsave` look like the > > > > > > > memory > > > > > > > (maybe dev->dma_io_tlb_mem) was corrupted? > > > > > > > The dev->dma_io_tlb_mem should be set here > > > > > > > (https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/tree/drivers/pci/probe.c#n2528) > > > > > > > through device_initialize. > > > > > > > > > > > > I'm less sure about this. 'dma_io_tlb_mem' should be pointing at > > > > > > 'io_tlb_default_mem', which is a page-aligned allocation from > > > > > > memblock. > > > > > > The spinlock is at offset 0x24 in that structure, and looking at the > > > > > > register dump from the crash: > > > > > > > > > > > > Jun 29 18:28:42 hp-4300G kernel: RSP: 0018:adb4013db9e8 EFLAGS: > > > > > > 00010006 > > > > > > Jun 29 18:28:42 hp-4300G kernel: RAX: 003a8290 RBX: > > > > > > RCX: 8900572ad580 > > > > > > Jun 29 18:28:42 hp-4300G kernel: RDX: 89005653f024 RSI: > > > > > > 000c RDI: 1d17 > > > > > > Jun 29 18:28:42 hp-4300G kernel: RBP: 0a20d000 R08: > > > > > > 000c R09: > > > > > > Jun 29 18:28:42 hp-4300G kernel: R10: 0a20d000 R11: > > > > > > 89005653f000 R12: 0212 > > > > > > Jun 29 18:28:42 hp-4300G kernel: R13: 1000 R14: > > > > > > 0002 R15: 0020 > > > > > > Jun 29 18:28:42 hp-4300G kernel: FS: 7f1f8898ea40() > > > > > > GS:89005728() knlGS: > > > > > > Jun 29 18:28:42 hp-4300G kernel: CS: 0010 DS: ES: CR0: > > > > > > 80050033 > > > > > > Jun 29 18:28:42 hp-4300G kernel: CR2: 003a8290 CR3: > > > > > > 0001020d CR4: 00350ee0 > > > > > > Jun 29 18:28:42 hp-4300G kernel: Call Trace: > > > > > > Jun 29 18:28:42 hp-4300G kernel: _raw_spin_lock_irqsave+0x39/0x50 > > > > > > Jun 29 18:28:42 hp-4300G kernel: swiotlb_tbl_map_single+0x12b/0x4c0 > > > > > > > > > > > > Then that correlates with R11 holding the 'dma_io_tlb_mem' pointer > > > > > > and > > > > > > RDX pointing at the spinlock. Yet RAX is holding junk :/ > > > > > > > > > > > > I agree that enabling KASAN would be a good idea, but I also think > > > > > > we > > > > > > probably need to get some more information out of > > > > > > swiotlb_tbl_map_single() > > > > > > to see see what exactly is going wrong in there. > > > > > > > > > > I can certainly enable KASAN and if there is any debug print I can add > > > > > or dump anything, let me know! > > > > > > > > I bit the bullet and took v5.13 with swiotlb/for-linus-5.14 merged in, > > > > built > > > > x86 defconfig and ran it on my laptop. However, it seems to work fine! > > > > > > > > Please can you share your .config? > > > > > > Sure thing, it is attached. It is just Arch Linux's config run through > > > olddefconfig. The original is below in case you need to diff it. > > > > > > https://raw.githubusercontent.com/archlinux/svntogit-packages/9045405dc835527164f3034b3ceb9a67c7a53cd4/trunk/config > > > > > > If there is anything more that I can provide, please let me know. > > > > I eventually got this booting (for some reason it was causing LD to SEGV > > trying to link it for a
Re: [PATCH v15 06/12] swiotlb: Use is_swiotlb_force_bounce for swiotlb data bouncing
Hi Will and Claire,
On Wed, Jun 30, 2021 at 12:43:48PM +0100, Will Deacon wrote:
> On Wed, Jun 30, 2021 at 05:17:27PM +0800, Claire Chang wrote:
> > On Wed, Jun 30, 2021 at 9:43 AM Nathan Chancellor wrote:
> > >
> > > On Thu, Jun 24, 2021 at 11:55:20PM +0800, Claire Chang wrote:
> > > > Propagate the swiotlb_force into io_tlb_default_mem->force_bounce and
> > > > use it to determine whether to bounce the data or not. This will be
> > > > useful later to allow for different pools.
> > > >
> > > > Signed-off-by: Claire Chang
> > > > Reviewed-by: Christoph Hellwig
> > > > Tested-by: Stefano Stabellini
> > > > Tested-by: Will Deacon
> > > > Acked-by: Stefano Stabellini
> > >
> > > This patch as commit af452ec1b1a3 ("swiotlb: Use is_swiotlb_force_bounce
> > > for swiotlb data bouncing") causes my Ryzen 3 4300G system to fail to
> > > get to an X session consistently (although not every single time),
> > > presumably due to a crash in the AMDGPU driver that I see in dmesg.
> > >
> > > I have attached logs at af452ec1b1a3 and f127c9556a8e and I am happy
> > > to provide any further information, debug, or test patches as necessary.
> >
> > Are you using swiotlb=force? or the swiotlb_map is called because of
> > !dma_capable?
> > (https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/tree/kernel/dma/direct.h#n93)
>
> The command line is in the dmesg:
>
> | Kernel command line: initrd=\amd-ucode.img
> initrd=\initramfs-linux-next-llvm.img
> root=PARTUUID=8680aa0c-cf09-4a69-8cf3-970478040ee7 rw intel_pstate=no_hwp
> irqpoll
>
> but I worry that this looks _very_ similar to the issue reported by Qian
> Cai which we thought we had fixed. Nathan -- is the failure deterministic?
Yes, for the most part. It does not happen every single boot so when I
was bisecting, I did a series of seven boots and only considered the
revision good when all seven of them made it to LightDM's greeter. My
results that I notated show most bad revisions failed anywhere from four
to six times.
> > `BUG: unable to handle page fault for address: 003a8290` and
> > the fact it crashed at `_raw_spin_lock_irqsave` look like the memory
> > (maybe dev->dma_io_tlb_mem) was corrupted?
> > The dev->dma_io_tlb_mem should be set here
> > (https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/tree/drivers/pci/probe.c#n2528)
> > through device_initialize.
>
> I'm less sure about this. 'dma_io_tlb_mem' should be pointing at
> 'io_tlb_default_mem', which is a page-aligned allocation from memblock.
> The spinlock is at offset 0x24 in that structure, and looking at the
> register dump from the crash:
>
> Jun 29 18:28:42 hp-4300G kernel: RSP: 0018:adb4013db9e8 EFLAGS: 00010006
> Jun 29 18:28:42 hp-4300G kernel: RAX: 003a8290 RBX:
> RCX: 8900572ad580
> Jun 29 18:28:42 hp-4300G kernel: RDX: 89005653f024 RSI: 000c
> RDI: 1d17
> Jun 29 18:28:42 hp-4300G kernel: RBP: 0a20d000 R08: 000c
> R09:
> Jun 29 18:28:42 hp-4300G kernel: R10: 0a20d000 R11: 89005653f000
> R12: 0212
> Jun 29 18:28:42 hp-4300G kernel: R13: 1000 R14: 0002
> R15: 0020
> Jun 29 18:28:42 hp-4300G kernel: FS: 7f1f8898ea40()
> GS:89005728() knlGS:
> Jun 29 18:28:42 hp-4300G kernel: CS: 0010 DS: ES: CR0:
> 80050033
> Jun 29 18:28:42 hp-4300G kernel: CR2: 003a8290 CR3: 0001020d
> CR4: 00350ee0
> Jun 29 18:28:42 hp-4300G kernel: Call Trace:
> Jun 29 18:28:42 hp-4300G kernel: _raw_spin_lock_irqsave+0x39/0x50
> Jun 29 18:28:42 hp-4300G kernel: swiotlb_tbl_map_single+0x12b/0x4c0
>
> Then that correlates with R11 holding the 'dma_io_tlb_mem' pointer and
> RDX pointing at the spinlock. Yet RAX is holding junk :/
>
> I agree that enabling KASAN would be a good idea, but I also think we
> probably need to get some more information out of swiotlb_tbl_map_single()
> to see see what exactly is going wrong in there.
I can certainly enable KASAN and if there is any debug print I can add
or dump anything, let me know!
Cheers,
Nathan
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH] iommu/vt-d: Fix W=1 clang warning in intel/perf.c
On 6/17/2021 1:30 PM, Joerg Roedel wrote: On Thu, Jun 17, 2021 at 10:16:50AM -0700, Nick Desaulniers wrote: On Thu, Jun 17, 2021 at 7:54 AM Joerg Roedel wrote: From: Joerg Roedel Fix this warning when compiled with clang and W=1: drivers/iommu/intel/perf.c:16: warning: Function parameter or member 'latency_lock' not described in 'DEFINE_SPINLOCK' drivers/iommu/intel/perf.c:16: warning: expecting prototype for perf.c(). Prototype was for DEFINE_SPINLOCK() instead I think these warnings are actually produced by kernel-doc? (not clang) Will kernel-doc check automatically when COMPILER=clang is set and W=1? Because I did not explicitly enable any kernel-doc checks. Regards, Joerg kernel-doc is run automatically with W=1, regardless of gcc versus clang. Cheers, Nathan ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH v11 07/11] device-mapping: Introduce DMA range map, supplanting dma_pfn_offset
On Tue, Sep 08, 2020 at 11:43:45AM +0200, Christoph Hellwig wrote: > And because I like replying to myself so much, here is a link to the > version with the arm cleanup patch applied. Unlike the previous two > attempts this has at least survived very basic sanity testing: > > http://git.infradead.org/users/hch/misc.git/shortlog/refs/heads/dma-ranges.2 Tested-by: Nathan Chancellor Thank you for fixing this up quickly! ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH v11 07/11] device-mapping: Introduce DMA range map, supplanting dma_pfn_offset
On Wed, Sep 02, 2020 at 05:36:29PM -0700, Florian Fainelli wrote: > > > On 9/2/2020 3:38 PM, Nathan Chancellor wrote: > [snip] > > > Hello Nathan, > > > > > > Can you tell me how much memory your RPI has and if all of it is > > > > This is the 4GB version. > > > > > accessible by the PCIe device? Could you also please include the DTS > > > of the PCIe node? IIRC, the RPI firmware does some mangling of the > > > PCIe DT before Linux boots -- could you describe what is going on > > > there? > > > > Unfortunately, I am not familiar with how to get this information. If > > you could provide some instructions for how to do so, I am more than > > happy to. I am not very knowleagable about the inner working of the Pi, > > I mainly use it as a test platform for making sure that LLVM does not > > cause problems on real devices. > > Can you bring the dtc application to your Pi root filesystem, and if so, can > you run the following: > > dtc -I fs -O dtb /proc/device-tree -f > /tmp/device.dtb Sure, the result is attached. > or cat /sys/firmware/fdt > device.dtb > > and attach the resulting file? > > > > > > Finally, can you attach the text of the full boot log? > > > > I have attached a working and broken boot log. Thank you for the quick > > response! > > Is it possible for you to rebuild your kernel with CONFIG_MMC_DEBUG by any > chance? Of course. A new log is attached with the debug output from that config. > I have a suspicion that this part of the DTS for the bcm2711.dtsi platform > is at fault: > > https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/arch/arm/boot/dts/bcm2711.dtsi#n264 > > and the resulting dma-ranges parsing is just not working for reasons to be > determined. > -- > Florian Let me know if you need anything else out of me. Cheers, Nathan device.dtb Description: Binary data [0.00] Booting Linux on physical CPU 0x00 [0x410fd083] [0.00] Linux version 5.9.0-rc3-next-20200902-dirty (nathan@ubuntu-n2-xlarge-x86) (ClangBuiltLinux clang version 12.0.0 (https://github.com/llvm/llvm-project.git b21ddded8f04fee925bbf9e6458347104b5b99eb), LLD 12.0.0 (https://github.com/llvm/llvm-project.git b21ddded8f04fee925bbf9e6458347104b5b99eb)) #1 SMP PREEMPT Wed Sep 2 17:41:42 MST 2020 [0.00] Machine model: Raspberry Pi 4 Model B Rev 1.2 [0.00] efi: UEFI not found. [0.00] Reserved memory: created CMA memory pool at 0x3740, size 64 MiB [0.00] OF: reserved mem: initialized node linux,cma, compatible id shared-dma-pool [0.00] NUMA: No NUMA configuration found [0.00] NUMA: Faking a node at [mem 0x-0xfbff] [0.00] NUMA: NODE_DATA [mem 0xfb81f100-0xfb820fff] [0.00] Zone ranges: [0.00] DMA [mem 0x-0x3fff] [0.00] DMA32[mem 0x4000-0xfbff] [0.00] Normal empty [0.00] Movable zone start for each node [0.00] Early memory node ranges [0.00] node 0: [mem 0x-0x3b3f] [0.00] node 0: [mem 0x4000-0xfbff] [0.00] Initmem setup node 0 [mem 0x-0xfbff] [0.00] percpu: Embedded 23 pages/cpu s54168 r8192 d31848 u94208 [0.00] Detected PIPT I-cache on CPU0 [0.00] CPU features: detected: EL2 vector hardening [0.00] CPU features: kernel page table isolation forced ON by KASLR [0.00] CPU features: detected: Kernel page table isolation (KPTI) [0.00] ARM_SMCCC_ARCH_WORKAROUND_1 missing from firmware [0.00] CPU features: detected: ARM errata 1165522, 1319367, or 1530923 [0.00] Built 1 zonelists, mobility grouping on. Total pages: 996912 [0.00] Policy zone: DMA32 [0.00] Kernel command line: dma.dmachans=0x71f5 bcm2709.boardrev=0xc03112 bcm2709.serial=0xb78d398 bcm2709.uart_clock=4800 bcm2709.disk_led_gpio=42 bcm2709.disk_led_active_low=0 smsc95xx.macaddr=DC:A6:32:60:6C:87 vc_mem.mem_base=0x3ec0 vc_mem.mem_size=0x4000 console=ttyS1,115200 console=tty1 root=PARTUUID=45a8dd8a-02 rootfstype=ext4 elevator=deadline fsck.repair=yes rootwait plymouth.ignore-serial-consoles [0.00] Kernel parameter elevator= does not have any effect anymore. [0.00] Please use sysfs to set IO scheduler for individual devices. [0.00] Dentry cache hash table entries: 524288 (order: 10, 4194304 bytes, linear) [0.00] Inode-cache hash table entries: 262144 (order: 9, 2097152 bytes, linear) [0.00] mem auto-init: stack:off, heap alloc:off, heap free:off [0.00] software IO TLB: mapped [mem 0x3340-0x3740] (64
Re: [PATCH v11 07/11] device-mapping: Introduce DMA range map, supplanting dma_pfn_offset
On Wed, Sep 02, 2020 at 06:11:08PM -0400, Jim Quinlan wrote: > On Wed, Sep 2, 2020 at 5:53 PM Nathan Chancellor > wrote: > > > > On Mon, Aug 24, 2020 at 03:30:20PM -0400, Jim Quinlan wrote: > > > The new field 'dma_range_map' in struct device is used to facilitate the > > > use of single or multiple offsets between mapping regions of cpu addrs and > > > dma addrs. It subsumes the role of "dev->dma_pfn_offset" which was only > > > capable of holding a single uniform offset and had no region bounds > > > checking. > > > > > > The function of_dma_get_range() has been modified so that it takes a > > > single > > > argument -- the device node -- and returns a map, NULL, or an error code. > > > The map is an array that holds the information regarding the DMA regions. > > > Each range entry contains the address offset, the cpu_start address, the > > > dma_start address, and the size of the region. > > > > > > of_dma_configure() is the typical manner to set range offsets but there > > > are > > > a number of ad hoc assignments to "dev->dma_pfn_offset" in the kernel > > > driver code. These cases now invoke the function > > > dma_attach_offset_range(dev, cpu_addr, dma_addr, size). > > > > > > Signed-off-by: Jim Quinlan > > > --- > > > arch/arm/include/asm/dma-mapping.h| 10 +-- > > > arch/arm/mach-keystone/keystone.c | 17 +++-- > > > arch/sh/drivers/pci/pcie-sh7786.c | 9 +-- > > > arch/x86/pci/sta2x11-fixup.c | 7 +- > > > drivers/acpi/arm64/iort.c | 5 +- > > > drivers/base/core.c | 2 + > > > drivers/gpu/drm/sun4i/sun4i_backend.c | 5 +- > > > drivers/iommu/io-pgtable-arm.c| 2 +- > > > .../platform/sunxi/sun4i-csi/sun4i_csi.c | 5 +- > > > .../platform/sunxi/sun6i-csi/sun6i_csi.c | 4 +- > > > drivers/of/address.c | 72 +-- > > > drivers/of/device.c | 43 ++- > > > drivers/of/of_private.h | 10 +-- > > > drivers/of/unittest.c | 34 ++--- > > > drivers/remoteproc/remoteproc_core.c | 8 ++- > > > .../staging/media/sunxi/cedrus/cedrus_hw.c| 7 +- > > > drivers/usb/core/message.c| 9 ++- > > > drivers/usb/core/usb.c| 7 +- > > > include/linux/device.h| 4 +- > > > include/linux/dma-direct.h| 8 +-- > > > include/linux/dma-mapping.h | 36 ++ > > > kernel/dma/coherent.c | 10 +-- > > > kernel/dma/mapping.c | 66 + > > > 23 files changed, 265 insertions(+), 115 deletions(-) > > > > Apologies if this has already been reported or is known but this commit > > is now in next-20200902 and it causes my Raspberry Pi 4 to no longer > > make it to userspace, instead spewing mmc errors: > > > > That commit causes my Raspberry Pi 4 to no longer make it to userspace, > > instead spewing mmc errors: > > > > [0.00] Booting Linux on physical CPU 0x00 [0x410fd083] > > [0.00] Linux version 5.9.0-rc3-4-geef520b232c6-dirty > > (nathan@ubuntu-n2-xlarge-x86) (ClangBuiltLinux clang version 12.0.0 > > (https://github.com/llvm/llvm-project.git > > b21ddded8f04fee925bbf9e6458347104b5b99eb), LLD 12.0.0 > > (https://github.com/llvm/llvm-project.git > > b21ddded8f04fee925bbf9e6458347104b5b99eb)) #1 SMP PREEMPT Wed Sep 2 > > 13:48:49 MST 2020 > > [0.00] Machine model: Raspberry Pi 4 Model B Rev 1.2 > > ... > > [1.459752] raspberrypi-firmware soc:firmware: Attached to firmware from > > 2020-08-24T18:50:56 > > [1.57] dwc2 fe98.usb: supply vusb_d not found, using dummy > > regulator > > [1.507454] dwc2 fe98.usb: supply vusb_a not found, using dummy > > regulator > > [1.615547] dwc2 fe98.usb: EPs: 8, dedicated fifos, 4080 entries in > > SPRAM > > [1.627537] sdhci-iproc fe30.sdhci: allocated mmc-pwrseq > > [1.665497] mmc0: SDHCI controller on fe30.sdhci [fe30.sdhci] > > using PIO > > [1.690601] mmc0: queuing unknown CIS tuple 0x80 (2 bytes) > > [1.697892] mmc0: queuing unknown CIS tuple 0x80 (3 bytes) > > [1.705173] mmc0: queuing unknown CIS tuple 0
Re: [PATCH v11 07/11] device-mapping: Introduce DMA range map, supplanting dma_pfn_offset
On Mon, Aug 24, 2020 at 03:30:20PM -0400, Jim Quinlan wrote: > The new field 'dma_range_map' in struct device is used to facilitate the > use of single or multiple offsets between mapping regions of cpu addrs and > dma addrs. It subsumes the role of "dev->dma_pfn_offset" which was only > capable of holding a single uniform offset and had no region bounds > checking. > > The function of_dma_get_range() has been modified so that it takes a single > argument -- the device node -- and returns a map, NULL, or an error code. > The map is an array that holds the information regarding the DMA regions. > Each range entry contains the address offset, the cpu_start address, the > dma_start address, and the size of the region. > > of_dma_configure() is the typical manner to set range offsets but there are > a number of ad hoc assignments to "dev->dma_pfn_offset" in the kernel > driver code. These cases now invoke the function > dma_attach_offset_range(dev, cpu_addr, dma_addr, size). > > Signed-off-by: Jim Quinlan > --- > arch/arm/include/asm/dma-mapping.h| 10 +-- > arch/arm/mach-keystone/keystone.c | 17 +++-- > arch/sh/drivers/pci/pcie-sh7786.c | 9 +-- > arch/x86/pci/sta2x11-fixup.c | 7 +- > drivers/acpi/arm64/iort.c | 5 +- > drivers/base/core.c | 2 + > drivers/gpu/drm/sun4i/sun4i_backend.c | 5 +- > drivers/iommu/io-pgtable-arm.c| 2 +- > .../platform/sunxi/sun4i-csi/sun4i_csi.c | 5 +- > .../platform/sunxi/sun6i-csi/sun6i_csi.c | 4 +- > drivers/of/address.c | 72 +-- > drivers/of/device.c | 43 ++- > drivers/of/of_private.h | 10 +-- > drivers/of/unittest.c | 34 ++--- > drivers/remoteproc/remoteproc_core.c | 8 ++- > .../staging/media/sunxi/cedrus/cedrus_hw.c| 7 +- > drivers/usb/core/message.c| 9 ++- > drivers/usb/core/usb.c| 7 +- > include/linux/device.h| 4 +- > include/linux/dma-direct.h| 8 +-- > include/linux/dma-mapping.h | 36 ++ > kernel/dma/coherent.c | 10 +-- > kernel/dma/mapping.c | 66 + > 23 files changed, 265 insertions(+), 115 deletions(-) Apologies if this has already been reported or is known but this commit is now in next-20200902 and it causes my Raspberry Pi 4 to no longer make it to userspace, instead spewing mmc errors: That commit causes my Raspberry Pi 4 to no longer make it to userspace, instead spewing mmc errors: [0.00] Booting Linux on physical CPU 0x00 [0x410fd083] [0.00] Linux version 5.9.0-rc3-4-geef520b232c6-dirty (nathan@ubuntu-n2-xlarge-x86) (ClangBuiltLinux clang version 12.0.0 (https://github.com/llvm/llvm-project.git b21ddded8f04fee925bbf9e6458347104b5b99eb), LLD 12.0.0 (https://github.com/llvm/llvm-project.git b21ddded8f04fee925bbf9e6458347104b5b99eb)) #1 SMP PREEMPT Wed Sep 2 13:48:49 MST 2020 [0.00] Machine model: Raspberry Pi 4 Model B Rev 1.2 ... [1.459752] raspberrypi-firmware soc:firmware: Attached to firmware from 2020-08-24T18:50:56 [1.57] dwc2 fe98.usb: supply vusb_d not found, using dummy regulator [1.507454] dwc2 fe98.usb: supply vusb_a not found, using dummy regulator [1.615547] dwc2 fe98.usb: EPs: 8, dedicated fifos, 4080 entries in SPRAM [1.627537] sdhci-iproc fe30.sdhci: allocated mmc-pwrseq [1.665497] mmc0: SDHCI controller on fe30.sdhci [fe30.sdhci] using PIO [1.690601] mmc0: queuing unknown CIS tuple 0x80 (2 bytes) [1.697892] mmc0: queuing unknown CIS tuple 0x80 (3 bytes) [1.705173] mmc0: queuing unknown CIS tuple 0x80 (3 bytes) [1.713788] mmc0: queuing unknown CIS tuple 0x80 (7 bytes) [1.721228] mmc0: queuing unknown CIS tuple 0x80 (3 bytes) [1.732062] mmc1: SDHCI controller on fe34.emmc2 [fe34.emmc2] using ADMA [1.741828] ALSA device list: [1.744885] No soundcards found. [1.748540] Waiting for root device PARTUUID=45a8dd8a-02... [1.788865] random: fast init done [1.793489] mmc1: unrecognised SCR structure version 4 [1.798814] mmc1: error -22 whilst initialising SD card [1.813969] mmc0: new high speed SDIO card at address 0001 [1.883178] mmc1: unrecognised SCR structure version 2 [1.888423] mmc1: error -22 whilst initialising SD card [1.964069] mmc1: unrecognised SCR structure version 4 [1.969314] mmc1: error -22 whilst initialising SD card [2.061225] mmc1: unrecognised SCR structure version 4 [2.066470] mmc1: error -22 whilst initialising SD card [3.160476] mmc1: unrecognised SCR structure version 4 [3.165718] mmc1: error -22 whilst initialising SD card This is what it looks like before that
Re: [PATCH] dma-pool: Do not allocate pool memory from CMA
On Tue, Jul 28, 2020 at 12:09:18PM +0200, Christoph Hellwig wrote:
> Ok, I found a slight bug that wasn't intended. I wanted to make sure
> we can always fall back to a lower pool, but got that wrong. Should be
> fixed in the next version.
Hi Christoph and Nicolas,
Did a version of that series ever get send out? I am coming into the
conversation late but I am running into an issue with the Raspberry Pi 4
not booting on linux-next, which appears to be due to this patch now in
mainline as commit d9765e41d8e9 ("dma-pool: do not allocate pool memory
from CMA") combined with
https://lore.kernel.org/lkml/20200725014529.1143208-2-jiaxun.y...@flygoat.com/
in -next:
[1.423163] raspberrypi-firmware soc:firmware: Request 0x0001 returned
status 0x
[1.431883] raspberrypi-firmware soc:firmware: Request 0x00030046 returned
status 0x
[1.443888] raspberrypi-firmware soc:firmware: Request 0x00030043 returned
status 0x
[1.452527] raspberrypi-exp-gpio soc:firmware:gpio: Failed to get GPIO 0
config (-22 80)
[1.460836] raspberrypi-firmware soc:firmware: Request 0x00030043 returned
status 0x
[1.469445] raspberrypi-exp-gpio soc:firmware:gpio: Failed to get GPIO 1
config (-22 81)
[1.477735] raspberrypi-firmware soc:firmware: Request 0x00030043 returned
status 0x
[1.486350] raspberrypi-exp-gpio soc:firmware:gpio: Failed to get GPIO 2
config (-22 82)
[1.494639] raspberrypi-firmware soc:firmware: Request 0x00030043 returned
status 0x
[1.503246] raspberrypi-exp-gpio soc:firmware:gpio: Failed to get GPIO 3
config (-22 83)
[1.511529] raspberrypi-firmware soc:firmware: Request 0x00030043 returned
status 0x
[1.520131] raspberrypi-exp-gpio soc:firmware:gpio: Failed to get GPIO 4
config (-22 84)
[1.528414] raspberrypi-firmware soc:firmware: Request 0x00030043 returned
status 0x
[1.537017] raspberrypi-exp-gpio soc:firmware:gpio: Failed to get GPIO 5
config (-22 85)
[1.545299] raspberrypi-firmware soc:firmware: Request 0x00030043 returned
status 0x
[1.553903] raspberrypi-exp-gpio soc:firmware:gpio: Failed to get GPIO 6
config (-22 86)
[1.562184] raspberrypi-firmware soc:firmware: Request 0x00030043 returned
status 0x
[1.570787] raspberrypi-exp-gpio soc:firmware:gpio: Failed to get GPIO 7
config (-22 87)
[1.579897] raspberrypi-firmware soc:firmware: Request 0x00030030 returned
status 0x
[1.589419] raspberrypi-firmware soc:firmware: Request 0x00028001 returned
status 0x
[1.599391] raspberrypi-firmware soc:firmware: Request 0x00030043 returned
status 0x
[1.608018] raspberrypi-exp-gpio soc:firmware:gpio: Failed to get GPIO 1
config (-22 81)
[1.616313] raspberrypi-firmware soc:firmware: Request 0x00030043 returned
status 0x
[1.624932] raspberrypi-exp-gpio soc:firmware:gpio: Failed to get GPIO 1
config (-22 81)
[1.633195] pwrseq_simple: probe of wifi-pwrseq failed with error -22
[1.643904] raspberrypi-firmware soc:firmware: Request 0x00030043 returned
status 0x
[1.652544] raspberrypi-exp-gpio soc:firmware:gpio: Failed to get GPIO 2
config (-22 82)
[1.660839] raspberrypi-firmware soc:firmware: Request 0x00030041 returned
status 0x
[1.669446] raspberrypi-exp-gpio soc:firmware:gpio: Failed to get GPIO 2
state (-22 82)
[1.677727] leds-gpio: probe of leds failed with error -22
[1.683735] raspberrypi-firmware soc:firmware: Request 0x00030043 returned
status 0x
[1.692346] raspberrypi-exp-gpio soc:firmware:gpio: Failed to get GPIO 6
config (-22 86)
[1.700636] raspberrypi-firmware soc:firmware: Request 0x00030043 returned
status 0x
[1.709240] raspberrypi-exp-gpio soc:firmware:gpio: Failed to get GPIO 6
config (-22 86)
[1.717496] reg-fixed-voltage: probe of sd_vcc_reg failed with error -22
[1.725546] raspberrypi-firmware soc:firmware: Request 0x00030043 returned
status 0x
[1.734176] raspberrypi-exp-gpio soc:firmware:gpio: Failed to get GPIO 4
config (-22 84)
[1.742465] raspberrypi-firmware soc:firmware: Request 0x00030043 returned
status 0x
[1.751072] raspberrypi-exp-gpio soc:firmware:gpio: Failed to get GPIO 4
config (-22 84)
[1.759332] gpio-regulator: probe of sd_io_1v8_reg failed with error -22
[1.768042] raspberrypi-firmware soc:firmware: Request 0x00028001 returned
status 0x
[1.780871] ALSA device list:
[1.783960] No soundcards found.
[1.787633] Waiting for root device PARTUUID=45a8dd8a-02...
I am unsure if it is related to the issue that Amit is having or
if that makes sense at all but I can reliably reproduce it.
v5.8-rc1: OK
v5.8-rc1 + d9765e41d8e9e: OK
v5.8-rc1 + "of_address: Add bus type match for pci ranges parser": OK
v5.8-rc1 + both: BROKEN
I wanted to test the series to see if this fixes anything. If you would
prefer a different thread for this or further
Re: [PATCH v2] iommu/dma: Rationalise types for DMA masks
On Wed, Dec 11, 2019 at 06:33:26PM +, Robin Murphy wrote: > Since iommu_dma_alloc_iova() combines incoming masks with the u64 bus > limit, it makes more sense to pass them around in their native u64 > rather than converting to dma_addr_t early. Do that, and resolve the > remaining type discrepancy against the domain geometry with a cheeky > cast to keep things simple. > > Signed-off-by: Robin Murphy Tested-by: Nathan Chancellor # build ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH v2] dma-mapping: treat dev->bus_dma_mask as a DMA limit
On Thu, Nov 21, 2019 at 10:26:44AM +0100, Nicolas Saenz Julienne wrote: > Using a mask to represent bus DMA constraints has a set of limitations. > The biggest one being it can only hold a power of two (minus one). The > DMA mapping code is already aware of this and treats dev->bus_dma_mask > as a limit. This quirk is already used by some architectures although > still rare. > > With the introduction of the Raspberry Pi 4 we've found a new contender > for the use of bus DMA limits, as its PCIe bus can only address the > lower 3GB of memory (of a total of 4GB). This is impossible to represent > with a mask. To make things worse the device-tree code rounds non power > of two bus DMA limits to the next power of two, which is unacceptable in > this case. > > In the light of this, rename dev->bus_dma_mask to dev->bus_dma_limit all > over the tree and treat it as such. Note that dev->bus_dma_limit should > contain the higher accesible DMA address. > > Signed-off-by: Nicolas Saenz Julienne > diff --git a/drivers/iommu/dma-iommu.c b/drivers/iommu/dma-iommu.c > index 041066f3ec99..0cc702a70a96 100644 > --- a/drivers/iommu/dma-iommu.c > +++ b/drivers/iommu/dma-iommu.c > @@ -421,8 +421,7 @@ static dma_addr_t iommu_dma_alloc_iova(struct > iommu_domain *domain, > if (iova_len < (1 << (IOVA_RANGE_CACHE_MAX_SIZE - 1))) > iova_len = roundup_pow_of_two(iova_len); > > - if (dev->bus_dma_mask) > - dma_limit &= dev->bus_dma_mask; > + dma_limit = min_not_zero(dma_limit, dev->bus_dma_limit); > > if (domain->geometry.force_aperture) > dma_limit = min(dma_limit, domain->geometry.aperture_end); Just as an FYI, this introduces a warning on arm32 allyesconfig for me: In file included from ../include/linux/list.h:9, from ../include/linux/kobject.h:19, from ../include/linux/of.h:17, from ../include/linux/irqdomain.h:35, from ../include/linux/acpi.h:13, from ../include/linux/acpi_iort.h:10, from ../drivers/iommu/dma-iommu.c:11: ../drivers/iommu/dma-iommu.c: In function 'iommu_dma_alloc_iova': ../include/linux/kernel.h:851:29: warning: comparison of distinct pointer types lacks a cast 851 | (!!(sizeof((typeof(x) *)1 == (typeof(y) *)1))) | ^~ ../include/linux/kernel.h:865:4: note: in expansion of macro '__typecheck' 865 | (__typecheck(x, y) && __no_side_effects(x, y)) |^~~ ../include/linux/kernel.h:875:24: note: in expansion of macro '__safe_cmp' 875 | __builtin_choose_expr(__safe_cmp(x, y), \ |^~ ../include/linux/kernel.h:884:19: note: in expansion of macro '__careful_cmp' 884 | #define min(x, y) __careful_cmp(x, y, <) | ^ ../include/linux/kernel.h:917:39: note: in expansion of macro 'min' 917 | __x == 0 ? __y : ((__y == 0) ? __x : min(__x, __y)); }) | ^~~ ../drivers/iommu/dma-iommu.c:424:14: note: in expansion of macro 'min_not_zero' 424 | dma_limit = min_not_zero(dma_limit, dev->bus_dma_limit); | ^~~~ Cheers, Nathan ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH] kernel: dma: Make CMA boot parameters __ro_after_init
On Sat, Oct 12, 2019 at 05:59:18PM +0530, Shyam Saini wrote: > This parameters are not changed after early boot. > By making them __ro_after_init will reduce any attack surface in the > kernel. > > Link: https://lwn.net/Articles/676145/ > Cc: Christoph Hellwig > Cc: Marek Szyprowski > Cc: Robin Murphy > Cc: Matthew Wilcox > Cc: Christopher Lameter > Cc: Kees Cook > Signed-off-by: Shyam Saini > --- > kernel/dma/contiguous.c | 8 > 1 file changed, 4 insertions(+), 4 deletions(-) > > diff --git a/kernel/dma/contiguous.c b/kernel/dma/contiguous.c > index 69cfb4345388..1b689b1303cd 100644 > --- a/kernel/dma/contiguous.c > +++ b/kernel/dma/contiguous.c > @@ -42,10 +42,10 @@ struct cma *dma_contiguous_default_area; > * Users, who want to set the size of global CMA area for their system > * should use cma= kernel parameter. > */ > -static const phys_addr_t size_bytes = (phys_addr_t)CMA_SIZE_MBYTES * SZ_1M; > -static phys_addr_t size_cmdline = -1; > -static phys_addr_t base_cmdline; > -static phys_addr_t limit_cmdline; > +static const phys_addr_t __ro_after_init size_bytes = > (phys_addr_t)CMA_SIZE_MBYTES * SZ_1M; The 0day bot reported an issue with this change with clang: https://groups.google.com/d/msgid/clang-built-linux/201910140334.nhultlt8%25lkp%40intel.com kernel/dma/contiguous.c:46:36: error: 'size_cmdline' causes a section type conflict with 'size_bytes' static phys_addr_t __ro_after_init size_cmdline = -1; ^ kernel/dma/contiguous.c:45:42: note: declared here static const phys_addr_t __ro_after_init size_bytes = (phys_addr_t)CMA_SIZE_MBYTES * SZ_1M; ^ kernel/dma/contiguous.c:47:36: error: 'base_cmdline' causes a section type conflict with 'size_bytes' static phys_addr_t __ro_after_init base_cmdline; ^ kernel/dma/contiguous.c:45:42: note: declared here static const phys_addr_t __ro_after_init size_bytes = (phys_addr_t)CMA_SIZE_MBYTES * SZ_1M; ^ kernel/dma/contiguous.c:48:36: error: 'limit_cmdline' causes a section type conflict with 'size_bytes' static phys_addr_t __ro_after_init limit_cmdline; ^ kernel/dma/contiguous.c:45:42: note: declared here static const phys_addr_t __ro_after_init size_bytes = (phys_addr_t)CMA_SIZE_MBYTES * SZ_1M; ^ 3 errors generated. The errors seem kind of cryptic at first but something that is const should automatically be in the read only section, this part of the commit seems unnecessary. Removing that part of the change fixes the error. Cheers, Nathan ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH v3 1/2] dma-contiguous: Abstract dma_{alloc,free}_contiguous()
Hi Nicolin,
On Thu, May 23, 2019 at 09:06:32PM -0700, Nicolin Chen wrote:
> Both dma_alloc_from_contiguous() and dma_release_from_contiguous()
> are very simply implemented, but requiring callers to pass certain
> parameters like count and align, and taking a boolean parameter to
> check __GFP_NOWARN in the allocation flags. So every function call
> duplicates similar work:
> /* A piece of example */
> unsigned long order = get_order(size);
> size_t count = size >> PAGE_SHIFT;
> page = dma_alloc_from_contiguous(dev, count, order, gfp & __GFP_NOWARN);
> [...]
> dma_release_from_contiguous(dev, page, size >> PAGE_SHIFT);
>
> Additionally, as CMA can be used only in the context which permits
> sleeping, most of callers do a gfpflags_allow_blocking() check and
> a corresponding fallback allocation of normal pages upon any false
> result:
> /* A piece of example */
> if (gfpflags_allow_blocking(flag))
> page = dma_alloc_from_contiguous();
> if (!page)
> page = alloc_pages();
> [...]
> if (!dma_release_from_contiguous(dev, page, count))
> __free_pages(page, get_order(size));
>
> So this patch simplifies those function calls by abstracting these
> operations into the two new functions: dma_{alloc,free}_contiguous.
>
> As some callers of dma_{alloc,release}_from_contiguous() might be
> complicated, this patch just implements these two new functions to
> kernel/dma/direct.c only as an initial step.
>
> Suggested-by: Christoph Hellwig
> Signed-off-by: Nicolin Chen
> ---
This commit is causing boot failures in QEMU on x86_64 defconfig:
https://travis-ci.com/ClangBuiltLinux/continuous-integration/jobs/203825363
Attached is a bisect log and a boot log with GCC (just to show it is not
a compiler thing).
My QEMU command line is:
qemu-system-x86_64 -m 512m \
-drive file=images/x86_64/rootfs.ext4,format=raw,if=ide \
-append 'console=ttyS0 root=/dev/sda' \
-nographic \
-kernel arch/x86_64/boot/bzImage
and the rootfs is available here:
https://github.com/ClangBuiltLinux/continuous-integration/raw/master/images/x86_64/rootfs.ext4
I haven't seen a report on this yet so apologize if there is already a
fix in the works. Let me know if you need anythnig from me.
Cheers,
Nathan
git bisect start
# good: [9fb67d643f6f1892a08ee3a04ea54022d1060bb0] Merge tag 'pinctrl-v5.2-2'
of git://git.kernel.org/pub/scm/linux/kernel/git/linusw/linux-pinctrl
git bisect good 9fb67d643f6f1892a08ee3a04ea54022d1060bb0
# bad: [9a15d2e3fd03e38a6ee7d7bc34d28bb7340f05f2] Add linux-next specific files
for 20190529
git bisect bad 9a15d2e3fd03e38a6ee7d7bc34d28bb7340f05f2
# bad: [eb756b9cce06b6c30b478a7ead67ddc0aa52b421] Merge remote-tracking branch
'crypto/master'
git bisect bad eb756b9cce06b6c30b478a7ead67ddc0aa52b421
# bad: [ffd5fc17ee0cd5f90258a8d4e0b87af913943f72] Merge remote-tracking branch
'xtensa/xtensa-for-next'
git bisect bad ffd5fc17ee0cd5f90258a8d4e0b87af913943f72
# bad: [df846fa0031f0e11dbfd5dd7c959cb45b3c0b3e2] Merge remote-tracking branch
'actions/for-next'
git bisect bad df846fa0031f0e11dbfd5dd7c959cb45b3c0b3e2
# good: [673e28ea2b579adcec369cd7f7295142a6b7e017] Merge remote-tracking branch
'usb.current/usb-linus'
git bisect good 673e28ea2b579adcec369cd7f7295142a6b7e017
# good: [d4f1f6efe84b20ff5f3a4874c580c2bee6cba68a] Merge remote-tracking branch
'kvms390-fixes/master'
git bisect good d4f1f6efe84b20ff5f3a4874c580c2bee6cba68a
# bad: [d0756fb0025937a5ed866d27f4452c66b066c089] Merge remote-tracking branch
'dma-mapping/for-next'
git bisect bad d0756fb0025937a5ed866d27f4452c66b066c089
# good: [49af21d0007414838a27eb4ade21277350e8ef1d] Merge remote-tracking branch
'drm-misc-fixes/for-linux-next-fixes'
git bisect good 49af21d0007414838a27eb4ade21277350e8ef1d
# good: [8680aa5a58abfe6087a3d8248c02232d3e05dc80] iommu/dma: Don't remap CMA
unnecessarily
git bisect good 8680aa5a58abfe6087a3d8248c02232d3e05dc80
# good: [efd9f10b70689fdeacadc95b6e0ea6dc311fa64f] iommu/dma: Refactor
iommu_dma_mmap
git bisect good efd9f10b70689fdeacadc95b6e0ea6dc311fa64f
# good: [b5f75a3639ff3b547e4eee7671e4321a429747a6] arm64: switch copyright
boilerplace to SPDX in dma-mapping.c
git bisect good b5f75a3639ff3b547e4eee7671e4321a429747a6
# bad: [fdaeec198ada8c48bff03c85fab542e5b241f5bc] dma-contiguous: add
dma_{alloc,free}_contiguous() helpers
git bisect bad fdaeec198ada8c48bff03c85fab542e5b241f5bc
# good: [a84cc69eb53715d37242a21ada398b0d8cd316fc] arm64: trim includes in
dma-mapping.c
git bisect good a84cc69eb53715d37242a21ada398b0d8cd316fc
# first bad commit: [fdaeec198ada8c48bff03c85fab542e5b241f5bc] dma-contiguous:
add dma_{alloc,free}_contiguous() helpers
c[?7l[2J[0mSeaBIOS (version 1.12.0-20181126_142135-anatol)
iPXE (http://ipxe.org) 00:03.0 C980 PCI2.10 PnP PMM+1FF92110+1FEF2110 C980
Press Ctrl-B to configure iPXE (PCI 00:03.0)...
[PATCH] iommu/dma: Fix condition check in iommu_dma_unmap_sg
Clang warns:
drivers/iommu/dma-iommu.c:897:6: warning: logical not is only applied to
the left hand side of this comparison [-Wlogical-not-parentheses]
if (!(attrs & DMA_ATTR_SKIP_CPU_SYNC) == 0)
^ ~~
drivers/iommu/dma-iommu.c:897:6: note: add parentheses after the '!' to
evaluate the comparison first
if (!(attrs & DMA_ATTR_SKIP_CPU_SYNC) == 0)
^
()
drivers/iommu/dma-iommu.c:897:6: note: add parentheses around left hand
side expression to silence this warning
if (!(attrs & DMA_ATTR_SKIP_CPU_SYNC) == 0)
^
()
1 warning generated.
Judging from the rest of the commit and the conditional in
iommu_dma_map_sg, either
if (!(attrs & DMA_ATTR_SKIP_CPU_SYNC))
or
if ((attrs & DMA_ATTR_SKIP_CPU_SYNC) == 0)
was intended, not a combination of the two.
I personally think that the former is easier to understand so use that.
Fixes: 06d60728ff5c ("iommu/dma: move the arm64 wrappers to common code")
Link: https://github.com/ClangBuiltLinux/linux/issues/497
Signed-off-by: Nathan Chancellor
---
drivers/iommu/dma-iommu.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/drivers/iommu/dma-iommu.c b/drivers/iommu/dma-iommu.c
index 0cd49c2d3770..0dee374fc64a 100644
--- a/drivers/iommu/dma-iommu.c
+++ b/drivers/iommu/dma-iommu.c
@@ -894,7 +894,7 @@ static void iommu_dma_unmap_sg(struct device *dev, struct
scatterlist *sg,
struct scatterlist *tmp;
int i;
- if (!(attrs & DMA_ATTR_SKIP_CPU_SYNC) == 0)
+ if (!(attrs & DMA_ATTR_SKIP_CPU_SYNC))
iommu_dma_sync_sg_for_cpu(dev, sg, nents, dir);
/*
--
2.22.0.rc1
