Re: dtbs_install recursing on subdirs-y and dtbs-subdir leading to race?
On Wed, Mar 16, 2016 at 08:54:34AM +, Ian Campbell wrote: > Where it appears that multiple instance of __dtbs_install_prep have > been running in parallel at least the apm and arm subdirectories of > arch/arm64/boot/dts, with both of them then racing in the > $(Q)if [ -d $(INSTALL_DTBS_PATH) ]; then mv $(INSTALL_DTBS_PATH) > $(INSTALL_DTBS_PATH).old; fi > rule since apparently $(INSTALL_DTBS_PATH) existed during the "-d" > check but had gone by the time of the move. I've already sent a patch several times to remove this line, I believe it's finally queued for this merge window. -- RMK's Patch system: http://www.arm.linux.org.uk/developer/patches/ FTTC broadband for 0.8mile line: currently at 9.6Mbps down 400kbps up according to speedtest.net.
Re: dtbs_install recursing on subdirs-y and dtbs-subdir leading to race?
On Wed, 2016-03-16 at 09:04 +, Russell King - ARM Linux wrote: > On Wed, Mar 16, 2016 at 08:54:34AM +, Ian Campbell wrote: > > Where it appears that multiple instance of __dtbs_install_prep have > > been running in parallel at least the apm and arm subdirectories of > > arch/arm64/boot/dts, with both of them then racing in the > > $(Q)if [ -d $(INSTALL_DTBS_PATH) ]; then mv > $(INSTALL_DTBS_PATH) $(INSTALL_DTBS_PATH).old; fi > > rule since apparently $(INSTALL_DTBS_PATH) existed during the "-d" > > check but had gone by the time of the move. > > I've already sent a patch several times to remove this line, I believe > it's finally queued for this merge window. Yes, as I said further down in my mail: I understand that the mv bit of the rule in question is likely to be removed quite soon[1] but I think the underlying race / extra recursion still exits and might have other implications. (where [1] was a link to your patch). I still think it is unexpected (or at least unintended) that this rune is run in all subdirectories. Ian.
Bug#813881: linux-image-4.3.0-1-armmp install wrong dtb on Wandboard Quad Rev B1
On Mon, 2016-03-14 at 10:48 -0700, Vagrant Cascadian wrote: > On 2016-03-14, Ian Campbell wrote: > > On Sun, 2016-02-07 at 19:50 -0800, Vagrant Cascadian wrote: > >> On 2016-02-06, Heinrich Schuchardt wrote: > >> > Booting with u-boot-imx requires imx6q-wandboard-revb1.dtb. > >> > linux-image-4.3.0-1-armmp installs imx6q-wandboard.dtb > >> > leaving me with a system that will not boot. > >> > > >> > With imx6q-wandboard-revb1.dtb the system boots. > ... > >> When the revc was added, backwards compatibility was broken by renaming > >> the revb .dtb file instead of keeping it and introducing the revc in a > >> new .dtb... kind of hard to fix correctly now... > > >> Adding support for flash-kernel to copy multiple, or even optionally all > >> .dtb files could at least work around the issue. > > > flash-kernel's DTB entry can reference a script to run which prints the > > DTB filename to use, so if you can distinguish the variants by poking > > at /sys etc (e.g the current sole user is kirkwood-qnap which looks at > > properties of the PCI host bridge etc) then that might be an option? > > I still think it would be better to copy multiple .dtb files, to make > sure all variants are available. This also makes it possible to use the > same SD card image on multiple wandboards. That would be fine too. > > Were any of these boards supported in Jessie? > > In Jessie, they both work using the same .dtb provided by linux 3.16.x, > although installing 4.x from jessie-backports on a wandboard rev B might > cause issues. OK, so we do need to worry about the upgrade path then. > > If so then making upgrade work smoothly would be nice, but if not then > > this might just be a case of Testing/Unstable users having > > occasionally to manually fix things, but once this is done and the > > correct DTB is in use flash-kernel should form then on DTRT and > > Stretch will just work for fresh installs. > > Upgrading u-boot is the tricky part, as we don't currently automatically > upgrade u-boot(and it's a bit tricky to do so). Depending on which > u-boot version is installed, u-boot will set fdtfile to a value that may > not be correct depending on which combination of linux + flash-kernel + > board variant is being booted. That would imply that the "wrong" u-boot was running on the board, wouldn't it? Does u-boot actually need updating or is "setenv fdtname ...; saveenv" sufficient? > I think this can partially be worked around by updating the wandboard > bootscript to have fallbacks to /boot/dtb-$ver (like the u-boot-generic > bootscript). Then the user can set the appropriate .dtb in > /etc/flash-kernel/db. With support for installing multiple dtb files we would perhaps want to still have a notion of a "primary" DTB, i.e. the one linked to /boot/dtb-$ver and if that is the case then using the script callout to try and pick the most appropriate fallback would make sense to me. Ian.
dtbs_install recursing on subdirs-y and dtbs-subdir leading to race?
Hello, As part of an automated build of the Debian Linux kernel packages I think we have observed a race (or at least some unexpected extra recursion) in the handling of dtb-subdirs vs subdirs-y in arch/arm64/boot/dts when using make -j. Looking at the log at [0] and removing the unrelated stuff happening due to other parallelism we see: make[8]: Nothing to be done for '__dtbs_install'. mv: cannot stat '/«PKGBUILDDIR»/debian/linux-image-4.5.0-rc7-arm64/usr/lib/linux-image-4.5.0-rc7-arm64': No such file or directory mv: cannot stat '/«PKGBUILDDIR»/debian/linux-image-4.5.0-rc7-arm64/usr/lib/linux-image-4.5.0-rc7-arm64': No such file or directory /«PKGBUILDDIR»/scripts/Makefile.dtbinst:26: recipe for target '__dtbs_install_prep' failed make[8]: *** [__dtbs_install_prep] Error 1 /«PKGBUILDDIR»/scripts/Makefile.dtbinst:26: recipe for target '__dtbs_install_prep' failed make[8]: *** [__dtbs_install_prep] Error 1 INSTALL net/bridge/netfilter/ebtable_nat.ko /«PKGBUILDDIR»/scripts/Makefile.dtbinst:46: recipe for target 'apm' failed make[7]: *** [apm] Error 2 make[7]: *** Waiting for unfinished jobs /«PKGBUILDDIR»/scripts/Makefile.dtbinst:46: recipe for target 'arm' failed make[7]: *** [arm] Error 2 make[8]: Nothing to be done for '__dtbs_install'. arch/arm64/Makefile:103: recipe for target 'dtbs_install' failed make[6]: *** [dtbs_install] Error 2 Makefile:146: recipe for target 'sub-make' failed make[5]: *** [sub-make] Error 2 Makefile:24: recipe for target '__sub-make' failed make[4]: *** [__sub-make] Error 2 make[4]: Leaving directory '/«PKGBUILDDIR»/debian/build/build_arm64_none_arm64' debian/rules.real:394: recipe for target 'install-image_arm64_none_arm64_dt' failed make[3]: *** [install-image_arm64_none_arm64_dt] Error 2 make[3]: Leaving directory '/«PKGBUILDDIR»' debian/rules.real:362: recipe for target 'install-image_arm64_none_arm64' failed make[2]: *** [install-image_arm64_none_arm64] Error 2 make[2]: *** Waiting for unfinished jobs Where it appears that multiple instance of __dtbs_install_prep have been running in parallel at least the apm and arm subdirectories of arch/arm64/boot/dts, with both of them then racing in the $(Q)if [ -d $(INSTALL_DTBS_PATH) ]; then mv $(INSTALL_DTBS_PATH) $(INSTALL_DTBS_PATH).old; fi rule since apparently $(INSTALL_DTBS_PATH) existed during the "-d" check but had gone by the time of the move. The build is in an automated buildd pristine environment with INSTALL_DTBS_PATH pointing to a brand new directory, so for $(INSTALL_DTBS_PATH) to exist at all there must have been a third instance of __dtbs_install_prep earlier which created it. I understand that the mv bit of the rule in question is likely to be removed quite soon[1] but I think the underlying race / extra recursion still exits and might have other implications. Ben and I have hypothesised that this is because arch/arm64/boot/dts/Makefile has: subdir-y:= $(dts-dirs) which means that dtbs_install recurses twice, once due to the dts-dirs handling in scripts/Makefile.dtbinst rules (via $(dtbsinst-dirs)) and once again for the (generic) subdir-y handling. BTW many of the subdir Makefiles have the same construct, I'm not sure why since they have no sub-sub-dirs, although it seems more harmless in that context. By my reading the __dtbs_install_prep rule is supposed to run once as part of arch/*/boot/dts/Makefile and not as part of any each of the subdirectories. I've experimented with removing the subdir-y assignment, but that seems to break the dtbs and clean rules. I'm not sure how else this can/should be fixed with Kbuild. Any ideas? Thanks, Ian. [0] https://buildd.debian.org/status/fetch.php?pkg=linux=arm64=4.5~rc7-1~exp1=1457444794 [1] https://git.kernel.org/cgit/linux/kernel/git/mmarek/kbuild.git/commit/?id=5399eb9b39081d6d2fc1a13d4ea85f1ceb2c8b44
