On 18.12.2025 09:29, Uwe Kleine-König wrote: > Control: forwarded -1 > https://lore.kernel.org/linux-iommu/hnwgn7glub4klmkzabzcm76eldaa3xm5ua72eldwzesblveo2a@4xq2bxhmbsyq > Hello, > > I extended the audience to people involved in the creation of > 2c223f7239f376a90d71903ec474ba887cf21d94. > > On Wed, Dec 17, 2025 at 11:07:00PM +0100, Benjamin Drung wrote: >> On Wed, 2025-12-17 at 22:20 +0100, Benjamin Drung wrote: >>> On Sun, 2025-11-30 at 10:27 +0100, Salvatore Bonaccorso wrote: >>>> On Sat, Nov 29, 2025 at 12:02:00AM +0100, Salvatore Bonaccorso wrote: >>>>> On Fri, Nov 28, 2025 at 10:53:17PM +0100, Benjamin Drung wrote: >>>>>> On Wed, 2025-11-26 at 08:52 +0100, Salvatore Bonaccorso wrote: >>>>>>> On Sat, Nov 22, 2025 at 04:27:53PM +0100, Benjamin Drung wrote: >>>>>>>> On Fri, 2025-11-21 at 09:52 +0100, Uwe Kleine-König wrote: >>>>>>>>> On Thu, Nov 20, 2025 at 11:11:07AM +0100, Benjamin Drung wrote: >>>>>>>>>> On Thu, 2025-11-20 at 08:58 +0100, Uwe Kleine-König wrote: >>>>>>>>>>> On Mon, Nov 17, 2025 at 02:05:41AM +0100, Benjamin Drung wrote: >>>>>>>>>>>> On Wed, 2025-11-05 at 20:14 +0100, Uwe Kleine-König wrote: >>>>>>>>>>>>> Can you try adding >>>>>>>>>>>>> >>>>>>>>>>>>> earlyprintk=serial,0x7e215040 >>>>>>>>>>>>> >>>>>>>>>>>>> to the kernel commandline. Not sure this activates the right >>>>>>>>>>>>> procedures, >>>>>>>>>>>>> maybe try it with the working kernel first. >>>>>>>>>>>> I tried with the working kernel 6.12.43+deb13-rpi but these were >>>>>>>>>>>> the >>>>>>>>>>>> earliest lines: >>>>>>>>>>>> >>>>>>>>>>>> [ 4.116565] printk: legacy console [ttyS1] enabled >>>>>>>>>>>> [ 4.129635] printk: legacy bootconsole [earlycon0] disabled >>>>>>>>>>>> [ 4.147050] bcm2835-power bcm2835-power: Broadcom BCM2835 power >>>>>>>>>>>> domains driver >>>>>>>>>>>> [ 4.160972] mousedev: PS/2 mouse device common for all mice >>>>>>>>>>>> [ 4.172622] i2c-bcm2835 20805000.i2c: Could not read >>>>>>>>>>>> clock-frequency property >>>>>>>>>>>> [ 4.186585] bcm2835-wdt bcm2835-wdt: Broadcom BCM2835 watchdog >>>>>>>>>>>> timer >>>>>>>>>>>> >>>>>>>>>>>> I tried with 6.12.57+deb13-rpi but that failed to boot and the >>>>>>>>>>>> earlyprintk line did not give any additional logs. >>>>>>>>>>> Can you please provide a full kernel log of the working kernel? >>>>>>>>>> dmesg log is attached. >>>>>>>>>> >>>>>>>>>>> Also please provide the output of: >>>>>>>>>>> >>>>>>>>>>> for f in /proc/tty/driver/*; do echo $f; sed 's/^/ /' < $f; >>>>>>>>>>> done >>>>>>>>>> /proc/tty/driver/serial >>>>>>>>>> serinfo:1.0 driver revision: >>>>>>>>>> 0: uart:unknown port:00000000 irq:0 >>>>>>>>>> 1: uart:16550 mmio:0x20215040 irq:53 tx:15629 rx:0 RTS|CTS|DTR >>>>>>>>>> 2: uart:unknown port:00000000 irq:0 >>>>>>>>>> 3: uart:unknown port:00000000 irq:0 >>>>>>>>>> /proc/tty/driver/ttyAMA >>>>>>>>>> serinfo:1.0 driver revision: >>>>>>>>>> 0: uart:PL011 rev2 mmio:0x20201000 irq:81 tx:452 rx:980 >>>>>>>>>> RTS|CTS|DTR >>>>>>>>> OK, is >>>>>>>>> >>>>>>>>> earlyprintk=serial,0x20215040 >>>>>>>>> >>>>>>>>> more communicative then? >>>>>>>> Nope, but I found >>>>>>>> https://www.raspberrypi.com/documentation/computers/configuration.html#enabling-early-console-for-linux >>>>>>>> and setting earlycon=uart8250,mmio32,0x20215040 worked. The kernel >>>>>>>> 6.12.43+deb13-rpi booted with printing the early lines as well (see >>>>>>>> dmesg of previous mail). >>>>>>>> >>>>>>>> With all other experiments cleaned, I tried the latest kernel: >>>>>>>> >>>>>>>> ``` >>>>>>>> $ cat /etc/default/raspi-extra-cmdline >>>>>>>> earlycon=uart8250,mmio32,0x20215040 >>>>>>>> $ update-initramfs -u >>>>>>>> ``` >>>>>>>> >>>>>>>> I got following logs before it got stuck: >>>>>>>> >>>>>>>> ``` >>>>>>>> [ 0.000000] Booting Linux on physical CPU 0x0 >>>>>>>> [ 0.000000] Linux version 6.12.57+deb13-rpi >>>>>>>> ([email protected]) (arm-linux-gnueabi-gcc-14 (Debian >>>>>>>> 14.2.0-19) 14.2.0, GNU ld (GNU Binutils for Debian) 2.44) #1 Debian >>>>>>>> 6.12.57-1 (2025-11-05) >>>>>>>> [ 0.000000] CPU: ARMv6-compatible processor [410fb767] revision 7 >>>>>>>> (ARMv7), cr=00c5387d >>>>>>>> [ 0.000000] CPU: PIPT / VIPT nonaliasing data cache, VIPT >>>>>>>> nonaliasing instruction cache >>>>>>>> [ 0.000000] OF: fdt: Machine model: Raspberry Pi Zero W Rev 1.1 >>>>>>>> [ 0.000000] random: crng init done >>>>>>>> [ 0.000000] earlycon: uart8250 at MMIO32 0x20215040 (options '') >>>>>>>> [ 0.000000] printk: legacy bootconsole [uart8250] enabled >>>>>>>> [ 0.000000] Memory policy: Data cache writeback >>>>>>>> [ 0.000000] Reserved memory: bypass linux,cma node, using cmdline >>>>>>>> CMA params instead >>>>>>>> [ 0.000000] OF: reserved mem: node linux,cma compatible matching >>>>>>>> fail >>>>>>>> [ 0.000000] cma: Reserved 64 MiB at 0x17000000 on node -1 >>>>>>>> ``` >>>>>>>> >>>>>>>> It looks like being cma related and I found a cma boot option in >>>>>>>> /etc/default/raspi-firmware. So I modified /boot/firmware/cmdline.txt >>>>>>>> and changed cma=64M to cma=0. Then the system booted without problems: >>>>>>>> >>>>>>>> ``` >>>>>>>> [...] >>>>>>>> [ 0.000000] Reserved memory: bypass linux,cma node, using cmdline >>>>>>>> CMA params instead >>>>>>>> [ 0.000000] OF: reserved mem: node linux,cma compatible matching >>>>>>>> fail >>>>>>>> [ 0.000000] Zone ranges: >>>>>>>> [ 0.000000] Normal [mem 0x0000000000000000-0x000000001bffffff] >>>>>>>> [ 0.000000] Movable zone start for each node >>>>>>>> [ 0.000000] Early memory node ranges >>>>>>>> [ 0.000000] node 0: [mem 0x0000000000000000-0x000000001bffffff] >>>>>>>> [ 0.000000] Initmem setup node 0 [mem >>>>>>>> 0x0000000000000000-0x000000001bffffff] >>>>>>>> [ 0.000000] Kernel command line: dma.dmachans=0x7ff5 >>>>>>>> bcm2708.boardrev=0x9000c1 bcm2708.serial=0x8bb1238 >>>>>>>> bcm2708.uart_clock=48000000 bcm2708.disk_led_gpio=47 >>>>>>>> smsc95xx.macaddr=B8:27:EB:BB:12:38 vc_mem.mem_base=0x1ec00000 >>>>>>>> vc_mem.mem_size=0x20000000 console=tty0 console=ttyS1,115200 >>>>>>>> root=/dev/mmcblk0p2 rw fsck.repair=yes net.ifnames=0 cma=0 rootwait >>>>>>>> earlycon=uart8250,mmio32,0x20215040 >>>>>>>> [...] >>>>>>>> ``` >>>>>>>> >>>>>>>> Then I persisted this change by setting CMA=0 in >>>>>>>> /etc/default/raspi-firmware: >>>>>>>> >>>>>>>> ``` >>>>>>>> $ grep -Ev '^(#|$)' /etc/default/raspi-firmware >>>>>>>> CMA=0 >>>>>>>> $ update-initramfs -u >>>>>>>> ``` >>>>>>>> >>>>>>>> This config removes the cma cmdline option completely. The system still >>>>>>>> boots: >>>>>>>> >>>>>>>> ``` >>>>>>>> [...] >>>>>>>> [ 0.000000] Memory policy: Data cache writeback >>>>>>>> [ 0.000000] Reserved memory: created CMA memory pool at 0x17000000, >>>>>>>> size 64 MiB >>>>>>>> [ 0.000000] OF: reserved mem: initialized node linux,cma, >>>>>>>> compatible id shared-dma-pool >>>>>>>> [ 0.000000] OF: reserved mem: 0x17000000..0x1affffff (65536 KiB) >>>>>>>> map reusable linux,cma >>>>>>>> [ 0.000000] Zone ranges: >>>>>>>> [ 0.000000] Normal [mem 0x0000000000000000-0x000000001bffffff] >>>>>>>> [ 0.000000] Movable zone start for each node >>>>>>>> [ 0.000000] Early memory node ranges >>>>>>>> [ 0.000000] node 0: [mem 0x0000000000000000-0x000000001bffffff] >>>>>>>> [ 0.000000] Initmem setup node 0 [mem >>>>>>>> 0x0000000000000000-0x000000001bffffff] >>>>>>>> [ 0.000000] Kernel command line: dma.dmachans=0x7ff5 >>>>>>>> bcm2708.boardrev=0x9000c1 bcm2708.serial=0x8bb1238 >>>>>>>> bcm2708.uart_clock=48000000 bcm2708.disk_led_gpio=47 >>>>>>>> smsc95xx.macaddr=B8:27:EB:BB:12:38 vc_mem.mem_base=0x1ec00000 >>>>>>>> vc_mem.mem_size=0x20000000 console=tty0 console=ttyS1,115200 >>>>>>>> root=/dev/mmcblk0p2 rw fsck.repair=yes net.ifnames=0 rootwait >>>>>>>> earlycon=uart8250,mmio32,0x20215040 >>>>>>>> [...] >>>>>>>> ``` >>>>>>>> >>>>>>>> That should hopefully narrow down the issue space. >>>>>>> If I'm not wrong the only cma related change between 6.12.43 and >>>>>>> 6.12.48 was the backport of 2c223f7239f3 ("of: reserved_mem: >>>>>>> Restructure call site for dma_contiguous_early_fixup()"): >>>>>>> >>>>>>> commit 46efab01648a04082266115a8e917c3b26b97fa8 >>>>>>> Author: Oreoluwa Babatunde <[email protected]> >>>>>>> Date: Wed Aug 6 10:24:21 2025 -0700 >>>>>>> >>>>>>> of: reserved_mem: Restructure call site for >>>>>>> dma_contiguous_early_fixup() >>>>>>> >>>>>>> [ Upstream commit 2c223f7239f376a90d71903ec474ba887cf21d94 ] >>>>>>> >>>>>>> Restructure the call site for dma_contiguous_early_fixup() >>>>>>> to >>>>>>> where the reserved_mem nodes are being parsed from the DT >>>>>>> so that >>>>>>> dma_mmu_remap[] is populated before dma_contiguous_remap() >>>>>>> is called. >>>>>>> >>>>>>> Fixes: 8a6e02d0c00e ("of: reserved_mem: Restructure how the >>>>>>> reserved memory regions are processed") >>>>>>> Signed-off-by: Oreoluwa Babatunde >>>>>>> <[email protected]> >>>>>>> Tested-by: William Zhang <[email protected]> >>>>>>> Signed-off-by: Marek Szyprowski <[email protected]> >>>>>>> Link: >>>>>>> https://lore.kernel.org/r/[email protected] >>>>>>> Signed-off-by: Sasha Levin <[email protected]> >>>>>>> >>>>>>> Uwe, would that make sense? Benjamin can you try to make a build with >>>>>>> that commit reverted, does it fix the problem? >>>>>> Building the kernel on the Pi Zero would probably take ages. I have a >>>>>> Raspberry Pi 5 running arm64. That should be able to build the kernel in >>>>>> a armel schroot, shouldn't it? Is there documentation for building the >>>>>> kernel? >>>>> I hope it is fine for Uwe that i respond here, he might have another >>>>> more efficient way to reach our goal. >>>>> >>>>> But we have the simple-patching guideline here: >>>>> https://protect2.fireeye.com/v1/url?k=6d29844b-0ca29171-6d280f04-74fe4860008a-0bd2c61b0efaf6bb&q=1&e=9d70d889-a8df-4ca7-962f-11b3a6da1b85&u=https%3A%2F%2Fkernel-team.pages.debian.net%2Fkernel-handbook%2Fch-common-tasks.html%23id-1.6.6.4 >>>>> >>>>> That should work within an armel schroot work well, e.g. on a >>>>> porterbox amdahl.d.o i would first setup a schroot via: >>>>> >>>>> schroot -b -c chroot:trixie_armel-dchroot -n linux >>>>> dd-schroot-cmd -c linux apt-get update >>>>> dd-schroot-cmd -c linux apt-get dist-upgrade >>>>> dd-schroot-cmd -c linux apt-get build-dep linux >>>>> schroot -r -c linux >>>>> (see https://dsa.debian.org/doc/schroot/) >>>>> >>>>> Then download the linux/6.12.57-1 source and make the revert patch >>>>> available. >>>>> >>>>> apt-get source linux >>>>> cd linux-* >>>>> ./debian/bin/test-patches -f rpi >>>>> ../0001-Revert-of-reserved_mem-Restructure-call-site-for-dma.patch >>>> In case you can/want trust unsigned packages (but I have put a >>>> sha256sum along signed with my key in the DD keyring), you can test >>>> the patch revert via the packages on: >>>> https://people.debian.org/~carnil/tmp/linux/1116251/ >>> Thanks for providing the kernel. I downloaded linux-image- >>> 6.12+unreleased-rpi_6.12.57-1a~test_armel.deb and installed it. This >>> kernel hangs on boot (see attached boot.log). It hangs there >>> reproducible. >> The "Linux version 4.14.98+" in the log puzzled me and >> /etc/kernel/postinst.d/z50-raspi-firmware did not copy kernel >> 6.12.57-1a~test to /boot/firmware. >> >> I found the old kernel.img and kernel7.img that I deleted. I copied the >> 6.12.57-1a~test kernel manually: >> >> ``` >> update-initramfs -u -k 6.12+unreleased-rpi >> cd /boot/firmware/ >> cp ../vmlinuz-6.12+unreleased-rpi . >> cp ../initrd.img-6.12+unreleased-rpi . >> vim config.txt >> reboot >> ``` >> >> Then the Pi booted the correct kernel (6.12+unreleased-rpi) successfully >> with following cmdline configurations: >> >> * no cma set >> * cma=0 >> * cma=64M > If I understood right that means that > 46efab01648a04082266115a8e917c3b26b97fa8 aka > 2c223f7239f376a90d71903ec474ba887cf21d94 broke booting on the Raspberry > Pi Zero W when CMA is used. > > #regzbot introduced: 2c223f7239f376a90d71903ec474ba887cf21d94 > > Does this ring a bell for someone, or do you have some debugging > instructions? Complete context is available at > https://bugs.debian.org/1116251.
There is a fix posted for this case, but it needs a bit more discussion: https://lore.kernel.org/all/[email protected]/ Best regards -- Marek Szyprowski, PhD Samsung R&D Institute Poland
