Bug#518412: linux-2.6 install errors due to incompatible depmod files [was Re: Bug#518412: initramfs-tools: must support relative paths in modules.dep ]
On Friday 20 March 2009, Michal Marek wrote: Frans Pop napsal(a): The use case here, which I suspect is not all that uncommon, is that I built a kernel from upstream source on a (Debian unstable) system with the new version of depmod and then installed that kernel on a (Debian stable) system that has an older version of modprobe [1]. Backward vs. forward compatibility discussion aside, there is a reason why most (all?) distro kernel packages run depmod at *install time* on the target machine. There might be modules installed by other packages, the format might change, newer depmod has a configuration file and finally the files are not exactly small and can be recreated rather easily. It might be a good idea to do the same when compiling kernels manually and copying them around. After some more reflection I've come to agree with this. Cheers, FJP -- To UNSUBSCRIBE, email to debian-kernel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#518412: linux-2.6 install errors due to incompatible depmod files [was Re: Bug#518412: initramfs-tools: must support relative paths in modules.dep ]
Frans Pop napsal(a): The use case here, which I suspect is not all that uncommon, is that I built a kernel from upstream source on a (Debian unstable) system with the new version of depmod and then installed that kernel on a (Debian stable) system that has an older version of modprobe [1]. Backward vs. forward compatibility discussion aside, there is a reason why most (all?) distro kernel packages run depmod at *install time* on the target machine. There might be modules installed by other packages, the format might change, newer depmod has a configuration file and finally the files are not exactly small and can be recreated rather easily. It might be a good idea to do the same when compiling kernels manually and copying them around. Michal -- To UNSUBSCRIBE, email to debian-kernel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#518412: linux-2.6 install errors due to incompatible depmod files [was Re: Bug#518412: initramfs-tools: must support relative paths in modules.dep ]
On Fri, 2009-03-20 at 00:40 +0100, Frans Pop wrote: I think my 2 cents are played out by now, so I'll drop things here. Maybe someone else will be willing to take up the batton. At least the issue is somewhat documented now. I'll inform others in Debian that the issue exists and fix things locally for my own use case. Doesn't Debian run depmod in the postinst of the kernel package - and iirc, again on boot anyway? In which case, you'd always have module files that match the version of depmod in the host environment not the build environment. Scott -- Scott James Remnant sc...@canonical.com signature.asc Description: This is a digitally signed message part
Bug#518412: linux-2.6 install errors due to incompatible depmod files [was Re: Bug#518412: initramfs-tools: must support relative paths in modules.dep ]
On Mar 20, Scott James Remnant sc...@canonical.com wrote: Doesn't Debian run depmod in the postinst of the kernel package - and iirc, again on boot anyway? Not anymore on boot, but I can't see why depmod should NOT being run when the kernel is installed. -- ciao, Marco signature.asc Description: Digital signature
Bug#518412: linux-2.6 install errors due to incompatible depmod files [was Re: Bug#518412: initramfs-tools: must support relative paths in modules.dep ]
On Fri, 2009-03-20 at 12:05 +0100, Marco d'Itri wrote: On Mar 20, Scott James Remnant sc...@canonical.com wrote: Doesn't Debian run depmod in the postinst of the kernel package - and iirc, again on boot anyway? Not anymore on boot, but I can't see why depmod should NOT being run when the kernel is installed. Me neither. The only file in /lib/modules/2.x.y-z (other than the modules) we include in the kernel packages is modules.order. The rest of the modules.* come by running depmod when the package is installed; we also update it when any further packages containing modules for that kernel are installed. Scott -- Scott James Remnant sc...@canonical.com signature.asc Description: This is a digitally signed message part
Bug#518412: linux-2.6 install errors due to incompatible depmod files [was Re: Bug#518412: initramfs-tools: must support relative paths in modules.dep ]
On Fri, 2009-03-20 at 12:05 +0100, Marco d'Itri wrote: On Mar 20, Scott James Remnant sc...@canonical.com wrote: Doesn't Debian run depmod in the postinst of the kernel package - and iirc, again on boot anyway? Not anymore on boot, but I can't see why depmod should NOT being run when the kernel is installed. This is the thing I'm trying to understand. That, and what I can really do in the absence of a time machine to change a former release :) Jon. -- To UNSUBSCRIBE, email to debian-kernel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#518412: linux-2.6 install errors due to incompatible depmod files [was Re: Bug#518412: initramfs-tools: must support relative paths in modules.dep ]
[ adding jcm and lkml to Cc: ]
On Thu, Mar 19, 2009 at 07:40:46PM +0100, Frans Pop wrote:
Hmmm. I wonder if it is the old m-i-t's modprobe that is the problem when
you do:
modprobe --set-version=2.6.26.3 --ignore-install --show-depends module
Looks like that's it:
# modprobe -V
module-init-tools version 3.4
# modprobe --set-version=2.6.26.3 --ignore-install --show-depends nfs
WARNING: Could not open 'kernel/net/sunrpc/sunrpc.ko': No such file or
directory
WARNING: Could not open 'kernel/fs/nfs_common/nfs_acl.ko': No such file or
directory
WARNING: Could not open 'kernel/fs/lockd/lockd.ko': No such file or
directory
FATAL: Could not open 'kernel/fs/nfs/nfs.ko': No such file or directory
# modprobe -V
module-init-tools version 3.7-pre9
# modprobe --set-version=2.6.26.3 --ignore-install --show-depends nfs
WARNING: All config files need .conf: /etc/modprobe.d/pnp-hotplug, it will
be ignored in a future release.
WARNING: All config files need .conf: /etc/modprobe.d/display_class, it
will be ignored in a future release.
WARNING: All config files need .conf: /etc/modprobe.d/blacklist, it will
be ignored in a future release.
insmod /lib/modules/2.6.26.3/kernel/net/sunrpc/sunrpc.ko
insmod /lib/modules/2.6.26.3/kernel/fs/nfs_common/nfs_acl.ko
insmod /lib/modules/2.6.26.3/kernel/fs/lockd/lockd.ko
insmod /lib/modules/2.6.26.3/kernel/fs/nfs/nfs.ko
That would mean that m-i-t has created a backwards incompatibility problem
_with itself_ and that the problem actually is installing a kernel, that
was built on a system with new m-i-t, on a system with old m-i-t.
Or, installing a kernel, built on a system running unstable or testing, on
a system running oldstable or stable. That sucks.
argh indeed.
arguably it is also a bug by initramfs-tools to suppress the error of above
modprobe calls see manual_add_modules() in hook-functions:
for mam_x in $(modprobe --set-version=${version} --ignore-install \
--show-depends ${1} 2/dev/null | awk '/^insmod/ { print $2 }'); do
git history tells that jbailey added it from the start on,
so it must have been because modprobe was/is too chatty.
suse mkinitrd seems also only to have lost that suppression lately.
thanks for finding out fjp.
kind regards
--
maks
--
To UNSUBSCRIBE, email to debian-kernel-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#518412: linux-2.6 install errors due to incompatible depmod files [was Re: Bug#518412: initramfs-tools: must support relative paths in modules.dep ]
On Thu, 2009-03-19 at 21:13 +0100, maximilian attems wrote: [ adding jcm and lkml to Cc: ] [ You want linux-modu...@vger.kernel.org rather than LKML. I've added the former to the CC list, we can kill LKML off the CC shortly. ] That would mean that m-i-t has created a backwards incompatibility problem _with itself_ and that the problem actually is installing a kernel, that was built on a system with new m-i-t, on a system with old m-i-t. Looks like the problem is actually that depmod was run under the newer version and then you tried to use the generated files with an older modprobe. I'm not sure that's actually an error - it was noted that the slight format change was unideal for such unlikely cases and in fact we won't do that again in the future. But if you were just moving forwards from one release to the next you would have been fine - you're talking lack of forward compatibility actually. Unless I'm missing something, in which case please clarify. I would like it also if you'd send me the generated files, distro info, etc. Jon. -- To UNSUBSCRIBE, email to debian-kernel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#518412: linux-2.6 install errors due to incompatible depmod files [was Re: Bug#518412: initramfs-tools: must support relative paths in modules.dep ]
(lkml dropped) On Thursday 19 March 2009, Jon Masters wrote: On Thu, 2009-03-19 at 21:13 +0100, maximilian attems wrote: That would mean that m-i-t has created a backwards incompatibility problem _with itself_ and that the problem actually is installing a kernel, that was built on a system with new m-i-t, on a system with old m-i-t. Looks like the problem is actually that depmod was run under the newer version and then you tried to use the generated files with an older modprobe. I'm not sure that's actually an error - it was noted that the slight format change was unideal for such unlikely cases and in fact we won't do that again in the future. But if you were just moving forwards from one release to the next you would have been fine - you're talking lack of forward compatibility actually. The use case here, which I suspect is not all that uncommon, is that I built a kernel from upstream source on a (Debian unstable) system with the new version of depmod and then installed that kernel on a (Debian stable) system that has an older version of modprobe [1]. The kernel Makefile of course does: /sbin/depmod -ae -F System.map -b $(INSTALL_MOD_PATH) $(KERNELRELEASE) Because the old modprobe does not understand the new relative (or rather rootless) paths, aggravated by the fact that initramfs-tools does not error out or display errors from modprobe (probably for good historic reasons), I suddenly had an initramfs that contained no modules and thus a system that failed to reboot with the new kernel. It took me quite a lot of time to trace it back to the upgrade of module-init-tools. Needless to say that this had always worked without any problems before. -- To UNSUBSCRIBE, email to debian-kernel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#518412: linux-2.6 install errors due to incompatible depmod files [was Re: Bug#518412: initramfs-tools: must support relative paths in modules.dep ]
On Thu, 2009-03-19 at 21:57 +0100, Frans Pop wrote: Because the old modprobe does not understand the new relative (or rather rootless) paths, aggravated by the fact that initramfs-tools does not error out or display errors from modprobe (probably for good historic reasons), I suddenly had an initramfs that contained no modules and thus a system that failed to reboot with the new kernel. Well, that lack of understanding the difference between relative/explict paths was fixed but of course we can't go back in time and fix what's already out there and in use. Yes, it was a bad idea of mine (perhaps) to change the existing file format and I've learned something, but it should only have affected for example that 3.4 release you're using. Jon. -- To UNSUBSCRIBE, email to debian-kernel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#518412: linux-2.6 install errors due to incompatible depmod files [was Re: Bug#518412: initramfs-tools: must support relative paths in modules.dep ]
On Thursday 19 March 2009, Jon Masters wrote: Yes, it was a bad idea of mine (perhaps) to change the existing file format and I've learned something, but it should only have affected for example that 3.4 release you're using. Do you mean that earlier versions are not affected? Hasn't depmod generated full paths basically any version up to 3.6? But even if it is only 3.4, that still makes it every Debian stable user (and unknown other distros) who runs the risk of ending up with an unbootable system for hard to trace reasons... Potentially painful for example for NAS devices where the kernel and initrd get installed in flash, replacing the previous version. As the kernel Makefile does run depmod during a build, I don't think it's strange to assume users rely on that modules.dep being valid, even for older versions of modprobe. What exactly are the resons behind the change in file format that're so strong that depmod cannot continue to generate the old format for the next 5 years or so as a transition period, until the risk is much lower that users run into problems because their current version of modprobe does not understand the new format? Are they really worth the potential consequences? -- To UNSUBSCRIBE, email to debian-kernel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#518412: linux-2.6 install errors due to incompatible depmod files [was Re: Bug#518412: initramfs-tools: must support relative paths in modules.dep ]
On Thu, 2009-03-19 at 23:09 +0100, Frans Pop wrote: On Thursday 19 March 2009, Jon Masters wrote: Yes, it was a bad idea of mine (perhaps) to change the existing file format and I've learned something, but it should only have affected for example that 3.4 release you're using. Do you mean that earlier versions are not affected? Hasn't depmod generated full paths basically any version up to 3.6? Yes, it was 3.6 where it changed, not 3.4. However, there was a backward compatibility issue during development that I was referring to. That isn't the problem here anyway. The problem is that you want to use a newly generated .dep file on an old install. I know that would be nice, and I'm sorry that it bothers you, but at the same time you don't expect newly built binaries to run against an older version of glibc they weren't built against, etc. But even if it is only 3.4, that still makes it every Debian stable user (and unknown other distros) who runs the risk of ending up with an unbootable system for hard to trace reasons... No, it really doesn't. If you upgrade to a new module-init-tools and run depmod, then everything works. If you use the old .dep files then it still works. If you force downgrade and don't rebuild your .dep files then there certainly is the risk of being out of sync - but there are other times where such things can happen too. I'm sure other utilities have similar. Potentially painful for example for NAS devices where the kernel and initrd get installed in flash, replacing the previous version. Yes, and in that case there's no problem. If you do a forward upgrade everything works just fine. It's only if you try to mix versions and go backwards that you get a problem. I can see this in a cross-build setup being a slight annoyance, but even there most people would use the same version of the tools or expect to have problems building on a newer box for an older release. As the kernel Makefile does run depmod during a build, I don't think it's strange to assume users rely on that modules.dep being valid, even for older versions of modprobe. It works fine as long as your box is self-consistent. What is the actual problem other than that mixing versions and trying to go backward without quickly re-running depmod might cause module load problems? What exactly are the resons behind the change in file format that're so strong that depmod cannot continue to generate the old format for the next 5 years or so as a transition period I guess it's called progress ;) Sarcasm aside, if you can give me an example of an actual real life set of users who are adversely affected then I'll try to do something to help out. But if you're asking for old versions of software to be compatible with newer releases in every case I think you're not being terribly realistic. The kernel changes to make progress, and so do other utilities. Jon. -- To UNSUBSCRIBE, email to debian-kernel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#518412: linux-2.6 install errors due to incompatible depmod files [was Re: Bug#518412: initramfs-tools: must support relative paths in modules.dep ]
On Thursday 19 March 2009, Jon Masters wrote: [...] I understand how and why and when it works now. I can also easily avoid the problem now that I know about it. The question here is if the breakage is really necessary. I ran into the problem within days of installing the new m-i-t. I don't think I'm very special, so my guess is it's going to affect others too. I guess it's called progress ;) Sarcasm aside, if you can give me an example of an actual real life set of users who are adversely affected then I'll try to do something to help out. But if you're asking for old versions of software to be compatible with newer releases in every case I think you're not being terribly realistic. The kernel changes to make progress, and so do other utilities. No, sorry. That isn't, at least AIUI, how kernel (related) development is supposed to be done: http://lkml.org/lkml/2005/12/29/173 and similar discussions later. Sure, if there are very strong reasons to break things, fine. But whenever possible the kernel has ensured backwards compatibility, mostly only _after_ someone complained. Think of the i386 and x86_64 symlinks after the x86 integration, think of the COMPAT_SYSFS flags, think of optional support for old /proc files, think feature-removal-schedule.txt. So far you seem to be avoiding to give the reasons for the change. What would be so wrong with ensuring the compatibility for some transition period and avoiding the problem? P.S. Thanks a lot for your prompt replies. I do appreciate the discussion. -- To UNSUBSCRIBE, email to debian-kernel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#518412: linux-2.6 install errors due to incompatible depmod files [was Re: Bug#518412: initramfs-tools: must support relative paths in modules.dep ]
On Thu, 2009-03-19 at 23:58 +0100, Frans Pop wrote: I guess it's called progress ;) Sarcasm aside, if you can give me an example of an actual real life set of users who are adversely affected then I'll try to do something to help out. But if you're asking for old versions of software to be compatible with newer releases in every case I think you're not being terribly realistic. The kernel changes to make progress, and so do other utilities. Sure, if there are very strong reasons to break things, fine. But whenever possible the kernel has ensured backwards compatibility, mostly only _after_ someone complained. Think of the i386 and x86_64 symlinks after the x86 integration, think of the COMPAT_SYSFS flags, think of optional support for old /proc files, think feature-removal-schedule.txt. All of these examples refer to the future. The *future*. Things that will change, preserving backward compatibility, keeping symlinks around for those who are used to the old location. *Backward* compatibility. But the issue you raised was about *Forward* compatibility. This is nice but isn't the same. That would be like guaranteeing that all future kernel features will work with a kernel from 6 months ago, or that modules will, or other similar stuff. So far you seem to be avoiding to give the reasons for the change. What would be so wrong with ensuring the compatibility for some transition period and avoiding the problem? There is compatibility. From the past, to the present, into the future. But you want to use *future* generated files with a *past* version. That is not backward compatibility and it is not the kind of thing where you can preserve for 5 years, etc. How does the old version know something is going to change? It doesn't and it can't. v3.4 will always be the same, and it won't change. The files changed slightly in v3.6, but v3.4 can never know about that. P.S. Thanks a lot for your prompt replies. I do appreciate the discussion. Sure. Hopefully you see that this is not a regular compatibility issue. Jon. -- To UNSUBSCRIBE, email to debian-kernel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#518412: linux-2.6 install errors due to incompatible depmod files [was Re: Bug#518412: initramfs-tools: must support relative paths in modules.dep ]
On Friday 20 March 2009, Jon Masters wrote: Sure, if there are very strong reasons to break things, fine. But whenever possible the kernel has ensured backwards compatibility, mostly only _after_ someone complained. Think of the i386 and x86_64 symlinks after the x86 integration, think of the COMPAT_SYSFS flags, think of optional support for old /proc files, think feature-removal-schedule.txt. All of these examples refer to the future. The *future*. Things that will change, preserving backward compatibility, keeping symlinks around for those who are used to the old location. *Backward* compatibility. Exactly: making sure that tools in older userspace environments (old version of modprobe) continue to work with new kernels (or built in an environment that happens to have the latest and greatest depmod). But the issue you raised was about *Forward* compatibility. This is nice but isn't the same. That would be like guaranteeing that all future kernel features will work with a kernel from 6 months ago, or that modules will, or other similar stuff. I really don't see the huge difference. IMO you are comparing apples with oranges. I'm really don't think I'm trying to use modules from a 2.4 kernel with a 2.6 kernel here. m-i-t is _not_ an integral part of the kernel. It's just another tool, and one that's used in two different environments: a kernel build environment and a kernel user environment. And the format of the files generated by one and read by the other is the glue that keeps things together. Changing that format should IMO be done with due consideration for relevant use cases, and only if there are very strong arguments to do so. So far you seem to be avoiding to give the reasons for the change. What would be so wrong with ensuring the compatibility for some transition period and avoiding the problem? [...] Hopefully you see that this is not a regular compatibility issue. What I mainly see is that you seem to be avoiding answering this question and are apparently unwilling to consider to repair the situation. I think my 2 cents are played out by now, so I'll drop things here. Maybe someone else will be willing to take up the batton. At least the issue is somewhat documented now. I'll inform others in Debian that the issue exists and fix things locally for my own use case. Thanks again for your replies. Cheers, FJP -- To UNSUBSCRIBE, email to debian-kernel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
