Bug#608538: btrfs root installation results in initramfs busybox prompt
On Mon, 2011-01-03 at 19:31 -0700, Tim Gardner wrote: > On 01/03/2011 03:32 PM, Ben Hutchings wrote: > > > > Try this. I've made it reasonably generic so you can easily add other > > hidden dependencies like tg3 -> broadcom. > > > > Ben. > > > > I wrote a couple of macros for Karmic to solve a race problem that might > be applicable here. [...] There are (at least) three different though similar problems: 1. The GART/GPU initialisation ordering problem. The artificial symbol dependency works for i915, but the problem remains for other GPU drivers which can't depend on a specific GART driver. Our solution was to build-in all the GART drivers, as they are quite small. 2. Network driver dependency on a PHY driver. There's no ordering problem because we now generate and use module aliases based on MDIO IDs. However, the PHY module dependency is discovered dynamically, so it is hidden from initramfs-tools. (This is not a problem when MODULES=most, because mkinitramfs then includes all modules under drivers/net/phy.) The artifical symbol dependency can be used as a workaround, at the cost of loading modules that aren't needed. 3. For crypto users and drivers, the dependency is discovered dynamically, so again it is hidden from initramfs-tools. The artifical symbol dependency can be used as a workaround, but optimised drivers will not be loaded automatically. (Though for the specific case of crc32c this is broken anyway.) Ben. -- Ben Hutchings Once a job is fouled up, anything done to improve it makes it worse. signature.asc Description: This is a digitally signed message part
Bug#608538: btrfs root installation results in initramfs busybox prompt
On 01/03/2011 03:32 PM, Ben Hutchings wrote:
Try this. I've made it reasonably generic so you can easily add other
hidden dependencies like tg3 -> broadcom.
Ben.
I wrote a couple of macros for Karmic to solve a race problem that might
be applicable here. I think there are several areas of the kernel that
suffer from this type of indirect runtime registration issue.
Unfortunately, this patch requires that you hack the macro values into
the offending modules.
rtg
--
Tim Gardner tim.gard...@canonical.com
>From 3a9048b7ebc1aefd539290238cfd4d2dfafbfa0a Mon Sep 17 00:00:00 2001
From: Tim Gardner
Date: Tue, 6 Oct 2009 17:28:34 -0600
Subject: [PATCH] UBUNTU: SAUCE: Created MODULE_EXPORT/MODULE_IMPORT macros
BugLink: http://bugs.launchpad.net/bugs/430694
Create a macro pair that can be used to symbolicly link two modules
if they are not already directly linked.
This relationship uses the facilities of depmod and modprobe to
get soft dependencies loaded in the correct order.
Examples of soft dependencies are:
1) i915 and intel_agp (agp _must_ be loaded before i915)
2) any net driver that uses phylib modules.
Signed-off-by: Tim Gardner
---
include/linux/module.h |9 +
1 files changed, 9 insertions(+), 0 deletions(-)
diff --git a/include/linux/module.h b/include/linux/module.h
index 8b17fd8..9733e7f 100644
--- a/include/linux/module.h
+++ b/include/linux/module.h
@@ -712,4 +712,13 @@ static inline void module_bug_finalize(const Elf_Ehdr *hdr,
static inline void module_bug_cleanup(struct module *mod) {}
#endif /* CONFIG_GENERIC_BUG */
+/*
+ * Establish a symbolic link between 2 modules so that depmod
+ * and modprobe do the heavy lifting of loading the modules in the
+ * correct dependency order.
+ */
+#define MODULE_EXPORT(mod_name) int sym_link_##mod_name; EXPORT_SYMBOL(sym_link_##mod_name);
+#define MODULE_IMPORT(mod_name) extern int sym_link_##mod_name; int func_sym_link_##mod_name(void) {sym_link_##mod_name=1;}; EXPORT_SYMBOL(func_sym_link_##mod_name);
+
+
#endif /* _LINUX_MODULE_H */
--
1.7.0.4
Bug#608538: btrfs root installation results in initramfs busybox prompt
On Mon, 2011-01-03 at 22:57 +0100, Michael Prokop wrote:
> * Ben Hutchings [Mon Jan 03, 2011 at 09:50:46PM +]:
> > On Mon, 2011-01-03 at 22:16 +0100, Michael Prokop wrote:
> > > * Ben Hutchings [Mon Jan 03, 2011 at 09:03:37PM
> > > +]:
> > > > On Mon, 2011-01-03 at 16:37 -0400, Joey Hess wrote:
>
> > > > > So, at least a workaround would be for the initramfs to have crc32c
> > > > > added to
> > > > > it whenever libcrc32c is. Attached patch just adds it to the base
> > > > > modules
> > > > > list; since btrfs is already there that seems like an ok quick fix.
>
> > > > This is stupid. Without a declared module dependency, a MODULES=dep
> > > > configuration will remain broken. I think this needs to be fixed in the
> > > > kernel instead.
>
> > > > > Note that it would probably be better to try first loading hardware
> > > > > optimised
> > > > > versions like crc32c-intel, and only load crc32c if they fail to load.
>
> > > > I believe that happens automatically, as the hardware-optimised modules
> > > > provide an alias of 'crc32c'.
>
> > > > > (BTW, this bug probably also breaks netbooting with certian ethernet
> > > > > cards
> > > > > whose drivers also use libcrc32c.)
>
> > > > Right.
>
> > > Thanks for showing up, would be nice to see this fixed.
>
> > > I just want to make sure you're aware of my bugreport #602254
> > > (which Joey seemed to have noticed already according to its
> > > BTS history).
>
> > > Ben, do you think this could be fixed at the kernel side?
>
> > Eventually, yes, but unfortunately it turns out that we can't fix it
> > immediately. 'depmod' only looks at symbol dependencies; there is no
> > way for modules to declare explicit dependencies through module
> > information. We can bodge it by exporting a specific symbol from crc32c
> > and referring to that from libcrc32c, but since the optimised version
> > doesn't satisfy that dependency it will not be loaded.
>
> Ok, that's what I expected.
Actually, 'modprobe' will completely ignore aliases if there is a module
whose real name matches the requested name. So currently crc32c-intel
will never be automatically loaded!
> > So I'm afraid this will have to be worked around in initramfs-tools for
> > now: whenever you add libcrc32c, add crc32c as well (no matter how
> > libcrc32c was selected).
>
> So we should just add crc32c-intel, libcrc32c and crc32c by default
> via initramfs-tools for now, ACK?
Try this. I've made it reasonably generic so you can easily add other
hidden dependencies like tg3 -> broadcom.
Ben.
From: Ben Hutchings
Date: Mon, 3 Jan 2011 21:58:24 +
Subject: [PATCH] Handle hidden dependency of libcrc32c on crc32c
'depmod' only looks at symbol dependencies; there is no way for
modules to declare explicit dependencies through module information,
so dependencies on e.g. crypto providers are hidden. Until this is
fixed, we need to handle those hidden dependencies.
Signed-off-by: Ben Hutchings
---
hook-functions | 14 ++
mkinitramfs|3 +++
2 files changed, 17 insertions(+), 0 deletions(-)
diff --git a/hook-functions b/hook-functions
index c559651..d77c295 100644
--- a/hook-functions
+++ b/hook-functions
@@ -484,6 +484,20 @@ auto_add_modules()
esac
}
+# 'depmod' only looks at symbol dependencies; there is no way for
+# modules to declare explicit dependencies through module information,
+# so dependencies on e.g. crypto providers are hidden. Until this is
+# fixed, we need to handle those hidden dependencies.
+hidden_dep_add_modules()
+{
+ for dep in "lib/libcrc32c crc32c"; do
+ set -- $dep
+ if [ -f "${DESTDIR}/lib/modules/${version}/kernel/$1.ko" ]; then
+ manual_add_modules "$2"
+ fi
+ done
+}
+
# mkinitramfs help message
usage()
{
diff --git a/mkinitramfs b/mkinitramfs
index 0bb7806..40eb35b 100755
--- a/mkinitramfs
+++ b/mkinitramfs
@@ -232,6 +232,9 @@ list)
;;
esac
+# Resolve hidden dependencies
+hidden_dep_add_modules
+
# Have to do each file, because cpio --dereference doesn't recurse down
# symlinks.
--
1.7.2.3
--
Ben Hutchings
Once a job is fouled up, anything done to improve it makes it worse.
signature.asc
Description: This is a digitally signed message part
Bug#608538: btrfs root installation results in initramfs busybox prompt
* Ben Hutchings [Mon Jan 03, 2011 at 09:50:46PM +]: > On Mon, 2011-01-03 at 22:16 +0100, Michael Prokop wrote: > > * Ben Hutchings [Mon Jan 03, 2011 at 09:03:37PM > > +]: > > > On Mon, 2011-01-03 at 16:37 -0400, Joey Hess wrote: > > > > So, at least a workaround would be for the initramfs to have crc32c > > > > added to > > > > it whenever libcrc32c is. Attached patch just adds it to the base > > > > modules > > > > list; since btrfs is already there that seems like an ok quick fix. > > > This is stupid. Without a declared module dependency, a MODULES=dep > > > configuration will remain broken. I think this needs to be fixed in the > > > kernel instead. > > > > Note that it would probably be better to try first loading hardware > > > > optimised > > > > versions like crc32c-intel, and only load crc32c if they fail to load. > > > I believe that happens automatically, as the hardware-optimised modules > > > provide an alias of 'crc32c'. > > > > (BTW, this bug probably also breaks netbooting with certian ethernet > > > > cards > > > > whose drivers also use libcrc32c.) > > > Right. > > Thanks for showing up, would be nice to see this fixed. > > I just want to make sure you're aware of my bugreport #602254 > > (which Joey seemed to have noticed already according to its > > BTS history). > > Ben, do you think this could be fixed at the kernel side? > Eventually, yes, but unfortunately it turns out that we can't fix it > immediately. 'depmod' only looks at symbol dependencies; there is no > way for modules to declare explicit dependencies through module > information. We can bodge it by exporting a specific symbol from crc32c > and referring to that from libcrc32c, but since the optimised version > doesn't satisfy that dependency it will not be loaded. Ok, that's what I expected. > So I'm afraid this will have to be worked around in initramfs-tools for > now: whenever you add libcrc32c, add crc32c as well (no matter how > libcrc32c was selected). So we should just add crc32c-intel, libcrc32c and crc32c by default via initramfs-tools for now, ACK? Thanks. regards, -mika- signature.asc Description: Digital signature
Bug#608538: btrfs root installation results in initramfs busybox prompt
On Mon, 2011-01-03 at 22:16 +0100, Michael Prokop wrote: > * Ben Hutchings [Mon Jan 03, 2011 at 09:03:37PM +]: > > On Mon, 2011-01-03 at 16:37 -0400, Joey Hess wrote: > > > I hope this can be dealt with, it seems to be the only remaining > > > issue in getting Debian to support btrfs root filesystems. > > > > This is easily reproducible, I installed from a recent daily build > > > netinst, put /boot on ext3 and / on btrfs and same problem on boot. > > > > The problem is that btrfs depends on libcrc32c, which demand loads > > > any of several crc32c implementations, depending on hardware. Those > > > modules are not declared as dependencies, so initramfs-tools does not > > > include them. > > > > So, another way to see the same problem: > > > > r...@gnu:/lib/modules/2.6.32-5-686/kernel/crypto>mv crc32c.ko ~ > > > r...@gnu:/lib/modules/2.6.32-5-686/kernel/crypto>insmod > > > ../lib/libcrc32c.ko > > > insmod: error inserting '../lib/libcrc32c.ko': -1 Unknown symbol in module > > > r...@gnu:/lib/modules/2.6.32-5-686/kernel/crypto>mv ~/crc32c.ko . > > > r...@gnu:/lib/modules/2.6.32-5-686/kernel/crypto>insmod > > > ../lib/libcrc32c.ko > > > r...@gnu:/lib/modules/2.6.32-5-686/kernel/crypto> > > > > So, at least a workaround would be for the initramfs to have crc32c added > > > to > > > it whenever libcrc32c is. Attached patch just adds it to the base modules > > > list; since btrfs is already there that seems like an ok quick fix. > > > This is stupid. Without a declared module dependency, a MODULES=dep > > configuration will remain broken. I think this needs to be fixed in the > > kernel instead. > > > > Note that it would probably be better to try first loading hardware > > > optimised > > > versions like crc32c-intel, and only load crc32c if they fail to load. > > > I believe that happens automatically, as the hardware-optimised modules > > provide an alias of 'crc32c'. > > > > (BTW, this bug probably also breaks netbooting with certian ethernet cards > > > whose drivers also use libcrc32c.) > > > Right. > > Thanks for showing up, would be nice to see this fixed. > > I just want to make sure you're aware of my bugreport #602254 > (which Joey seemed to have noticed already according to its > BTS history). > > Ben, do you think this could be fixed at the kernel side? Eventually, yes, but unfortunately it turns out that we can't fix it immediately. 'depmod' only looks at symbol dependencies; there is no way for modules to declare explicit dependencies through module information. We can bodge it by exporting a specific symbol from crc32c and referring to that from libcrc32c, but since the optimised version doesn't satisfy that dependency it will not be loaded. So I'm afraid this will have to be worked around in initramfs-tools for now: whenever you add libcrc32c, add crc32c as well (no matter how libcrc32c was selected). Ben. -- Ben Hutchings Once a job is fouled upbv, anything done to improve it makes it worse. signature.asc Description: This is a digitally signed message part
Bug#608538: btrfs root installation results in initramfs busybox prompt
* Ben Hutchings [Mon Jan 03, 2011 at 09:03:37PM +]: > On Mon, 2011-01-03 at 16:37 -0400, Joey Hess wrote: > > I hope this can be dealt with, it seems to be the only remaining > > issue in getting Debian to support btrfs root filesystems. > > This is easily reproducible, I installed from a recent daily build > > netinst, put /boot on ext3 and / on btrfs and same problem on boot. > > The problem is that btrfs depends on libcrc32c, which demand loads > > any of several crc32c implementations, depending on hardware. Those > > modules are not declared as dependencies, so initramfs-tools does not > > include them. > > So, another way to see the same problem: > > r...@gnu:/lib/modules/2.6.32-5-686/kernel/crypto>mv crc32c.ko ~ > > r...@gnu:/lib/modules/2.6.32-5-686/kernel/crypto>insmod ../lib/libcrc32c.ko > > insmod: error inserting '../lib/libcrc32c.ko': -1 Unknown symbol in module > > r...@gnu:/lib/modules/2.6.32-5-686/kernel/crypto>mv ~/crc32c.ko . > > r...@gnu:/lib/modules/2.6.32-5-686/kernel/crypto>insmod ../lib/libcrc32c.ko > > r...@gnu:/lib/modules/2.6.32-5-686/kernel/crypto> > > So, at least a workaround would be for the initramfs to have crc32c added to > > it whenever libcrc32c is. Attached patch just adds it to the base modules > > list; since btrfs is already there that seems like an ok quick fix. > This is stupid. Without a declared module dependency, a MODULES=dep > configuration will remain broken. I think this needs to be fixed in the > kernel instead. > > Note that it would probably be better to try first loading hardware > > optimised > > versions like crc32c-intel, and only load crc32c if they fail to load. > I believe that happens automatically, as the hardware-optimised modules > provide an alias of 'crc32c'. > > (BTW, this bug probably also breaks netbooting with certian ethernet cards > > whose drivers also use libcrc32c.) > Right. Thanks for showing up, would be nice to see this fixed. I just want to make sure you're aware of my bugreport #602254 (which Joey seemed to have noticed already according to its BTS history). Ben, do you think this could be fixed at the kernel side? regards, -mika- signature.asc Description: Digital signature
Bug#608538: btrfs root installation results in initramfs busybox prompt
On Mon, 2011-01-03 at 16:37 -0400, Joey Hess wrote: > I hope this can be dealt with, it seems to be the only remaining > issue in getting Debian to support btrfs root filesystems. > > This is easily reproducible, I installed from a recent daily build > netinst, put /boot on ext3 and / on btrfs and same problem on boot. > > The problem is that btrfs depends on libcrc32c, which demand loads > any of several crc32c implementations, depending on hardware. Those > modules are not declared as dependencies, so initramfs-tools does not > include them. > > So, another way to see the same problem: > > r...@gnu:/lib/modules/2.6.32-5-686/kernel/crypto>mv crc32c.ko ~ > r...@gnu:/lib/modules/2.6.32-5-686/kernel/crypto>insmod ../lib/libcrc32c.ko > insmod: error inserting '../lib/libcrc32c.ko': -1 Unknown symbol in module > r...@gnu:/lib/modules/2.6.32-5-686/kernel/crypto>mv ~/crc32c.ko . > r...@gnu:/lib/modules/2.6.32-5-686/kernel/crypto>insmod ../lib/libcrc32c.ko > r...@gnu:/lib/modules/2.6.32-5-686/kernel/crypto> > > So, at least a workaround would be for the initramfs to have crc32c added to > it whenever libcrc32c is. Attached patch just adds it to the base modules > list; since btrfs is already there that seems like an ok quick fix. This is stupid. Without a declared module dependency, a MODULES=dep configuration will remain broken. I think this needs to be fixed in the kernel instead. > Note that it would probably be better to try first loading hardware optimised > versions like crc32c-intel, and only load crc32c if they fail to load. I believe that happens automatically, as the hardware-optimised modules provide an alias of 'crc32c'. > (BTW, this bug probably also breaks netbooting with certian ethernet cards > whose drivers also use libcrc32c.) Right. Ben. -- Ben Hutchings Once a job is fouled up, anything done to improve it makes it worse. signature.asc Description: This is a digitally signed message part
Bug#608538: btrfs root installation results in initramfs busybox prompt
I hope this can be dealt with, it seems to be the only remaining
issue in getting Debian to support btrfs root filesystems.
This is easily reproducible, I installed from a recent daily build
netinst, put /boot on ext3 and / on btrfs and same problem on boot.
The problem is that btrfs depends on libcrc32c, which demand loads
any of several crc32c implementations, depending on hardware. Those
modules are not declared as dependencies, so initramfs-tools does not
include them.
So, another way to see the same problem:
r...@gnu:/lib/modules/2.6.32-5-686/kernel/crypto>mv crc32c.ko ~
r...@gnu:/lib/modules/2.6.32-5-686/kernel/crypto>insmod ../lib/libcrc32c.ko
insmod: error inserting '../lib/libcrc32c.ko': -1 Unknown symbol in module
r...@gnu:/lib/modules/2.6.32-5-686/kernel/crypto>mv ~/crc32c.ko .
r...@gnu:/lib/modules/2.6.32-5-686/kernel/crypto>insmod ../lib/libcrc32c.ko
r...@gnu:/lib/modules/2.6.32-5-686/kernel/crypto>
So, at least a workaround would be for the initramfs to have crc32c added to
it whenever libcrc32c is. Attached patch just adds it to the base modules
list; since btrfs is already there that seems like an ok quick fix.
Note that it would probably be better to try first loading hardware optimised
versions like crc32c-intel, and only load crc32c if they fail to load.
(BTW, this bug probably also breaks netbooting with certian ethernet cards
whose drivers also use libcrc32c.)
--
see shy jo
--- hook-functions.orig 2011-01-03 16:20:26.705813570 -0400
+++ hook-functions 2011-01-03 16:20:30.278780384 -0400
@@ -412,7 +412,7 @@
for x in ehci-hcd ohci-hcd uhci-hcd usbhid xhci hid-apple \
hid-cherry hid-logitech hid-microsoft \
btrfs ext2 ext3 ext4 ext4dev isofs jfs nfs reiserfs udf xfs \
- af_packet atkbd i8042 virtio_pci; do
+ af_packet atkbd i8042 virtio_pci crc32c; do
manual_add_modules "${x}"
done
;;
signature.asc
Description: Digital signature
Processed: Re: Bug#608538: btrfs root installation results in initramfs busybox prompt
Processing commands for cont...@bugs.debian.org: > reassign 608538 btrfs-tools Bug #608538 [initramfs-tools] btrfs root installation results in initramfs busybox prompt Bug #602254 [initramfs-tools] initramfs-tools: [lib]crc32c* modules handling Bug reassigned from package 'initramfs-tools' to 'btrfs-tools'. Bug reassigned from package 'initramfs-tools' to 'btrfs-tools'. Bug No longer marked as found in versions initramfs-tools/0.98.5. Bug No longer marked as found in versions initramfs-tools/0.98.5. > retitle 608538 Didn't include required module in initramfs Bug #608538 [btrfs-tools] btrfs root installation results in initramfs busybox prompt Bug #602254 [btrfs-tools] initramfs-tools: [lib]crc32c* modules handling Changed Bug title to 'Didn't include required module in initramfs' from 'btrfs root installation results in initramfs busybox prompt' Changed Bug title to 'Didn't include required module in initramfs' from 'initramfs-tools: [lib]crc32c* modules handling' > thanks Stopping processing here. Please contact me if you need assistance. -- 608538: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=608538 Debian Bug Tracking System Contact ow...@bugs.debian.org with problems -- To UNSUBSCRIBE, email to debian-kernel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/handler.s.c.129399199114685.transcr...@bugs.debian.org
Bug#608538: btrfs root installation results in initramfs busybox prompt
reassign 608538 btrfs-tools retitle 608538 Didn't include required module in initramfs thanks On Sat, Jan 1, 2011 at 09:06, Aron Xu wrote: > No, nothing. But I truly only get an initramfs prompt. It looks to be a btrfs-tools hook issue. Reassigning it. -- Otavio Salvador O.S. Systems E-mail: ota...@ossystems.com.br http://www.ossystems.com.br Mobile: +55 53 9981-7854 http://projetos.ossystems.com.br -- To UNSUBSCRIBE, email to debian-kernel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/aanlktimtu9xi7f5yq9puwkbqvcuv2ozbmk0pdrnxh...@mail.gmail.com
