Re: Large initrd [Was: Re: booting problem (udev related?)]
On Thu, Aug 02, 2007 at 08:34:00PM -0400, Douglas Allan Tutty wrote: However, don't all those modules in the initrd end up staying in the kernel anyway, or do they get unloaded during boot? If they stay, and 'most' modules get added, how is that different than having a huge monolithic kernel? It may not matter on a box with huge memory, but I have mostly small-memory boxes. I may be wrong, but I think that only the needed modules are actually loaded. As for xorg-video-foo, that's why I don't install the xorg metapackage. I choose from its dependencies what I need. Same here /rant There's a growing kitchen-sink approach in Debian (perhaps all of Linux, I don't know). There's the kernel/initrd size, there's the variable device name problems, to name two. It suggests to me that there's a missing piece of infrastructure. Perhaps the installer system should create a hardware inventory file that initrdtools (or whatever the nom de jure) can access to generate a tailord initrd, that apt can consult for what drivers to download, etc. The installer rescue mode could offer a tool to regenerate the inventory file for times when one changes hardware. /end rant True, but you have to consider the competition. If you plug a new device into a Windows machine the driver gets installed automatically or you get prompted for the drivers if Windows doesn't have them. You have to admit that this is pretty convenient functionality which has been there at least since Windows 2000 (how this is cluttering the registry and the fact that it isn't always working is a totally different topic). The big advantage on linux (and especially Debian) is that power users still have the possibility to customize the setup (like using a different mkinitrd, different options, purge unneeded packages, ...) that a Windows user doesn't have. Regards, Andrei -- If you can't explain it simply, you don't understand it well enough. (Albert Einstein) signature.asc Description: Digital signature
design focus [was Large initrd, was booting problem (udev related?)]
On Fri, Aug 03, 2007 at 05:54:57PM +0300, Andrei Popescu wrote: On Thu, Aug 02, 2007 at 08:34:00PM -0400, Douglas Allan Tutty wrote: However, don't all those modules in the initrd end up staying in the kernel anyway, or do they get unloaded during boot? If they stay, and 'most' modules get added, how is that different than having a huge monolithic kernel? It may not matter on a box with huge memory, but I have mostly small-memory boxes. I may be wrong, but I think that only the needed modules are actually loaded. As for xorg-video-foo, that's why I don't install the xorg metapackage. I choose from its dependencies what I need. Same here All these extra packages together take a lot of disk space, a lot of download bandwidth to install and maintain. /rant There's a growing kitchen-sink approach in Debian (perhaps all of Linux, I don't know). There's the kernel/initrd size, there's the variable device name problems, to name two. It suggests to me that there's a missing piece of infrastructure. Perhaps the installer system should create a hardware inventory file that initrdtools (or whatever the nom de jure) can access to generate a tailord initrd, that apt can consult for what drivers to download, etc. The installer rescue mode could offer a tool to regenerate the inventory file for times when one changes hardware. /end rant True, but you have to consider the competition. I guess the problem is related to this notion of trying to compete with MS. If people 'buy' brand A because they like features x,y, and z, and brand B has the goal of gaining market share, it will tend to morph into a clone (feature-wise) of brand A. However, it will tend to take on some of the compromises of brand B that go with features x, y, and z. I stick with debian on my big box because of inertia, the debian policy, the debian security support for all packages in debian/main, and the absolute ease of applying bug fixes with aptitude. Debian also supports my trackball mouse's scroll wheel (IMPS/2) whereas OpenBSD does not. However, my older computers are transitioning away from Debian to BSD because of the newer debian (perhaps all linuxes) being so much slower on them than either older debians or new BSDs. If you plug a new device into a Windows machine the driver gets installed automatically or you get prompted for the drivers if Windows doesn't have them. You have to admit that this is pretty convenient functionality which has been there at least since Windows 2000 (how this is cluttering the registry and the fact that it isn't always working is a totally different topic). That convenience comes at a huge price in terms of system resource utilization on boxes with few resources. Compare it to OpenBSD, for example, where there is no such thing as eth0, but network interfaces based on driver name (eg. ne) and configuration; my 486 has one NIC as ne1. Its not convenient to have to look up in a file for the supported configurations of different hardware to ensure that your NIC is set up to match one of them then configure networking based on ne1. However, its only done once. The big advantage on linux (and especially Debian) is that power users still have the possibility to customize the setup (like using a different mkinitrd, different options, purge unneeded packages, ...) that a Windows user doesn't have. True, but rather than hotplugging, I would prefer a program that can be run as needed each time a new piece of hardware is attached for the first time, which would create the device node and load the appropriate module and parameters. Once done, it would get out of the way. On subsequent attachment of a device, everything would be pre-existing. It all comes down to the notion of competition and market share. If Debian is going to focus on market share and competing with MS it will have to target MS's target market. Since I'm not in that market, Debian will be shifting its focus on the market I'm in. It won't be that I'm drifting away from Debian but that Debian is drifting away from me. Doug. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: design focus [was Large initrd, was booting problem (udev related?)]
On Aug 3, 2007, at 9:25 AM, Douglas Allan Tutty wrote: I guess the problem is related to this notion of trying to compete with MS. If people 'buy' brand A because they like features x,y, and z, and brand B has the goal of gaining market share, it will tend to morph into a clone (feature-wise) of brand A. However, it will tend to take on some of the compromises of brand B that go with features x, y, and z. I stick with debian on my big box because of inertia, the debian policy, the debian security support for all packages in debian/main, and the absolute ease of applying bug fixes with aptitude. Debian also supports my trackball mouse's scroll wheel (IMPS/2) whereas OpenBSD does not. However, my older computers are transitioning away from Debian to BSD because of the newer debian (perhaps all linuxes) being so much slower on them than either older debians or new BSDs. I don't think it's so much Microsoft's influence as it is a difference in philosophy. Linux distributions put a lot of effort into being convenient desktop OSs. BSD tends to be aimed more at servers, where things like hotplugging aren't as important. If you have to check dmesg for the right device node and then run 'mount' to access a USB flash drive on a server, it doesn't matter much because you aren't going to be doing that often. If you have to do that on your desktop machine every time you plug in your digital camera, it gets old in a hurry. For that matter, ten years ago Linux distributions were already doing fully automated installers while NetBSD and OpenBSD still required you to get out a calculator to figure out the cylinder boundaries for the slices on your hard disk. The two OSs just occupy different points on the easy of use vs. compactness scale. You see this in hardware support, too. Linux tries to support the newest stuff, because that's what's in desktop machines (and sometimes suffers instability because of it), while BSD tends to take a more conservative approach. Hardware that's seen in desktops but rarely in servers often isn't supported or maintained well in BSD, because it's just not a priority. (The 3c509 ethernet driver, for example, was buggy for *years* in FreeBSD. It never really got fixed, the cards just became obsolete. ;) Another example: The Marvell Yukon gigabit ethernet chipset, common in desktops but rare in servers, is much slower under FreeBSD than under Linux.) It could be for your particular application, BSD is just the right tool for the job. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: design focus [was Large initrd, was booting problem (udev related?)]
On Fri, Aug 03, 2007 at 12:25:15PM -0400, Douglas Allan Tutty wrote: On Fri, Aug 03, 2007 at 05:54:57PM +0300, Andrei Popescu wrote: On Thu, Aug 02, 2007 at 08:34:00PM -0400, Douglas Allan Tutty wrote: However, don't all those modules in the initrd end up staying in the kernel anyway, or do they get unloaded during boot? If they stay, and 'most' modules get added, how is that different than having a huge monolithic kernel? It may not matter on a box with huge memory, but I have mostly small-memory boxes. I may be wrong, but I think that only the needed modules are actually loaded. I think this is correct, only the needed modules are actually loaded into the kernel. The initrd makes the *available* for loading. And when / pivots, I think the initrd memory gets freed. So its really only an issue during the initial bootstrap. A really large initrd on a memory-bound machine could get in the way. A really large initrd on an I/O bound machine can take a long time to load in. But, IMO, for general purpose machines, its not a big deal. As for xorg-video-foo, that's why I don't install the xorg metapackage. I choose from its dependencies what I need. Same here All these extra packages together take a lot of disk space, a lot of download bandwidth to install and maintain. yeah, the extra packages definitely are an issue. I'm not so sure tht the extra kernel modules are all that big a deal in the long run. but that's just a gut feeling. /rant There's a growing kitchen-sink approach in Debian (perhaps all of Linux, I don't know). There's the kernel/initrd size, there's the variable device name problems, to name two. It suggests to me that there's a missing piece of infrastructure. Perhaps the installer system should create a hardware inventory file that initrdtools (or whatever the nom de jure) can access to generate a tailord initrd, that apt can consult for what drivers to download, etc. The installer rescue mode could offer a tool to regenerate the inventory file for times when one changes hardware. /end rant True, but you have to consider the competition. I guess the problem is related to this notion of trying to compete with MS. If people 'buy' brand A because they like features x,y, and z, and brand B has the goal of gaining market share, it will tend to morph into a clone (feature-wise) of brand A. However, it will tend to take on some of the compromises of brand B that go with features x, y, and z. I think that on the whole, debian strikes a decent balance. You get the kitchen sink, but have the option to switch over to a bare pipe sticking out of the wall for no charge other than your own labor. :) A signature.asc Description: Digital signature
Large initrd [Was: Re: booting problem (udev related?)]
On Wed, Aug 01, 2007 at 11:10:25PM -0600, Bob Proulx wrote: Miles Bader wrote: Hmm, I didn't realize it analyzed the system when building the ramfs contents. Maybe I could just reinstall the kernel while the new kernel is running (or is there an official hint mechanism I could use)? Yes. Please try that. [I thought it just included _every_ possible module on the ramfs -- judging from the enormous size of the installed kernel package, it seems like it!] :-) Yes, I know what you mean. I was using yaird to make my initrd, but it gave some errors on the latest upgrade (and Steve Langasek, Debian kernel maintainer suggested it is no longer maintained). So now I'm exploring the initramfs-tools package. The first initrd was about *5 (five)* times bigger! I changed the config for including modules from 'most' to 'dep' and I got a much smaller (but still a bit bigger then yaird) initrd. Haven't tried to boot with it yet, though ;) Regards, Andrei -- If you can't explain it simply, you don't understand it well enough. (Albert Einstein) signature.asc Description: Digital signature
Re: Large initrd [Was: Re: booting problem (udev related?)]
On Thu, Aug 02, 2007 at 18:23:04 +0300, Andrei Popescu wrote: [...] Yes, I know what you mean. I was using yaird to make my initrd, but it gave some errors on the latest upgrade (and Steve Langasek, Debian kernel maintainer suggested it is no longer maintained). Do you mean this problem? yaird error: unrecognised line in /proc/bus/input/devices: U: Uniq= (fatal) That can be fixed relatively easily, see bug #431534, followup 1. (/usr/lib/yaird/perl/InputTab.pm is patched to simply ignore these new lines in /proc/bus/input/devices.) -- Regards,| http://users.icfo.es/Florian.Kulzer Florian | -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Large initrd [Was: Re: booting problem (udev related?)]
On Thu, Aug 02, 2007 at 06:23:04PM +0300, Andrei Popescu wrote: On Wed, Aug 01, 2007 at 11:10:25PM -0600, Bob Proulx wrote: Miles Bader wrote: Hmm, I didn't realize it analyzed the system when building the ramfs contents. Maybe I could just reinstall the kernel while the new kernel is running (or is there an official hint mechanism I could use)? Yes. Please try that. [I thought it just included _every_ possible module on the ramfs -- judging from the enormous size of the installed kernel package, it seems like it!] :-) Yes, I know what you mean. I was using yaird to make my initrd, but it gave some errors on the latest upgrade (and Steve Langasek, Debian kernel maintainer suggested it is no longer maintained). So now I'm exploring the initramfs-tools package. The first initrd was about *5 (five)* times bigger! wow! I never noticed that. And in fact I probably wouldn't have as this system doesn't have any initrd's left from yaird. My server which was an etch/testing box for a while has a couple older initrd's that are 1.4 megs or so versus the newer ones at 5-6megs. yikes. I changed the config for including modules from 'most' to 'dep' and I got a much smaller (but still a bit bigger then yaird) initrd. Haven't tried to boot with it yet, though ;) same here. interesting. I'll have to play with that. You could probably tighten it up even more by using the 'list' option and putting a minimum-necessary list in /etc/initramfs-tools/modules. At least that's how I read it. So what is the significance of initrd size? (other than the obvious filling up /boot issue). Is it really a problem to have most modules in there? I can think of some situations where it might be nice to have most of them -- mobo fails catastrophically and you want to be able to just boot, for example. Finally, I have on this (sid) system both initrd-tools and initramfs-tools installed. The latter is brought in by the kernel dependencies, and the former is manually installed. Who knows why or when I did that, but is one preferred over the other? A signature.asc Description: Digital signature
Re: Large initrd [Was: Re: booting problem (udev related?)]
On Thu, Aug 02, 2007 at 10:35:01AM -0700, Andrew Sackville-West wrote: same here. interesting. I'll have to play with that. You could probably tighten it up even more by using the 'list' option and putting a minimum-necessary list in /etc/initramfs-tools/modules. At least that's how I read it. That's too much hacking for my taste. So what is the significance of initrd size? (other than the obvious filling up /boot issue). Is it really a problem to have most modules in there? I can think of some situations where it might be nice to have most of them -- mobo fails catastrophically and you want to be able to just boot, for example. This is about it. Debian wants to provide an initrd that works even ehn changing hardware. Same reason for installing all -xorg-video-foo packages. Finally, I have on this (sid) system both initrd-tools and initramfs-tools installed. The latter is brought in by the kernel dependencies, and the former is manually installed. Who knows why or when I did that, but is one preferred over the other? AFAIU initrd-tools are deprecated and should not be used: http://wiki.debian.org/InitrdReplacementOptions There is also a nice comparison of initramfs-tools vs. yaird, though I'm not sure how recent this is. Regards, Andrei -- If you can't explain it simply, you don't understand it well enough. (Albert Einstein) signature.asc Description: Digital signature
Re: Large initrd [Was: Re: booting problem (udev related?)]
On Thu, Aug 02, 2007 at 06:40:52PM +0200, Florian Kulzer wrote: On Thu, Aug 02, 2007 at 18:23:04 +0300, Andrei Popescu wrote: [...] Yes, I know what you mean. I was using yaird to make my initrd, but it gave some errors on the latest upgrade (and Steve Langasek, Debian kernel maintainer suggested it is no longer maintained). Do you mean this problem? yaird error: unrecognised line in /proc/bus/input/devices: U: Uniq= (fatal) That can be fixed relatively easily, see bug #431534, followup 1. (/usr/lib/yaird/perl/InputTab.pm is patched to simply ignore these new lines in /proc/bus/input/devices.) Sure I could do this (actually I found another workaround, see #435560), but that's not the point. And I'm (by far) not knowledgeable enough to take over. Regards, Andrei -- If you can't explain it simply, you don't understand it well enough. (Albert Einstein) signature.asc Description: Digital signature
Re: Large initrd [Was: Re: booting problem (udev related?)]
On Fri, Aug 03, 2007 at 12:19:36AM +0300, Andrei Popescu wrote: On Thu, Aug 02, 2007 at 10:35:01AM -0700, Andrew Sackville-West wrote: So what is the significance of initrd size? (other than the obvious filling up /boot issue). Is it really a problem to have most modules in there? I can think of some situations where it might be nice to have most of them -- mobo fails catastrophically and you want to be able to just boot, for example. This is about it. Debian wants to provide an initrd that works even ehn changing hardware. Same reason for installing all -xorg-video-foo packages. However, don't all those modules in the initrd end up staying in the kernel anyway, or do they get unloaded during boot? If they stay, and 'most' modules get added, how is that different than having a huge monolithic kernel? It may not matter on a box with huge memory, but I have mostly small-memory boxes. As for xorg-video-foo, that's why I don't install the xorg metapackage. I choose from its dependencies what I need. /rant There's a growing kitchen-sink approach in Debian (perhaps all of Linux, I don't know). There's the kernel/initrd size, there's the variable device name problems, to name two. It suggests to me that there's a missing piece of infrastructure. Perhaps the installer system should create a hardware inventory file that initrdtools (or whatever the nom de jure) can access to generate a tailord initrd, that apt can consult for what drivers to download, etc. The installer rescue mode could offer a tool to regenerate the inventory file for times when one changes hardware. /end rant Doug. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
booting problem (udev related?)
For a long time, I used self-compiled kernels, with no problems. Recently I installed a debian kernel package, linux-image-2.6.22-1-686 (version 2.6.22-3). [It was a tight fit -- my root partition only has 130MB on it, and the debian kernel package used up 60MB -- but it did fit with about 4MB to spare!] The problem is that with the new kernel, the system won't boot all the way. It fails when it tries to mount the root partition, and dumps me into the ramfs emergency shell. The error message is something generic like File not found (sorry for the vagueness, those boot messages don't get saved anywhere and I didn't write them down). I seems like it may be related to udev because if I look in /dev, the disk device nodes which should be there _aren't there_, even though the disk hardware is recognized fine by the kernel. Indeed, I can fix things enough in the emergency shell to get the boot to succeed; I just use the following commands: mknod /dev/sda1 b 8 1 mount -text2 /dev/sda1 /root Then I hit ^D to the shell prompt to exit the shell, and the boot continues sucessfully (I'm typing in that running system now)! So it appears that for some reason, udev didn't create the appropriate /dev/sda1 node for the root to be mounted?!? Oddly, once booting continues, everything works fine, including mounting of my /usr partition from /dev/sda3 (notice that I didn't create /dev/sda3 above, and it certainly wasn't there initially). [I know my raw /dev directory has an entry for /dev/sda3 from before udev existed; does udev notice that and somehow copy it?] With my old self-compiled kernels, I have no problems, using exactly the same system. Those kernels though, have compiled-in drivers for all my devices, and don't use initramfs (or initrd) at all, so perhaps it's a module-loading issue? The last self-compiled kernel I used was version 2.6.19.7 btw. Does anybody have any idea what's going on, and how I might try to fix it? BTW, my system uses a SCSI disk with an old Adaptec SCSI card, if that's relevant... here's some related msgs from dmsg: SCSI subsystem initialized ... scsi0 : Adaptec AIC7XXX EISA/VLB/PCI SCSI HBA DRIVER, Rev 7.0 Adaptec 2940 Ultra SCSI adapter aic7880: Ultra Single Channel A, SCSI Id=7, 16/253 SCBs ... scsi 0:0:5:0: Direct-Access SEAGATE ST34555N 0930 PQ: 0 ANSI: 2 target0:0:5: Beginning Domain Validation target0:0:5: FAST-10 SCSI 10.0 MB/s ST (100 ns, offset 15) target0:0:5: Domain Validation skipping write tests target0:0:5: Ending Domain Validation sd 0:0:5:0: [sda] 924 512-byte hardware sectors (4551 MB) sd 0:0:5:0: [sda] Write Protect is off sd 0:0:5:0: [sda] Mode Sense: 93 00 10 08 sd 0:0:5:0: [sda] Write cache: enabled, read cache: enabled, supports DPO and FUA sd 0:0:5:0: [sda] 924 512-byte hardware sectors (4551 MB) sd 0:0:5:0: [sda] Write Protect is off sd 0:0:5:0: [sda] Mode Sense: 93 00 10 08 sd 0:0:5:0: [sda] Write cache: enabled, read cache: enabled, supports DPO and FUA sda: sda1 sda2 sda3 sd 0:0:5:0: [sda] Attached SCSI disk Thanks greatly, -Miles -- If you can't beat them, arrange to have them beaten. [George Carlin] -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: booting problem (udev related?)
Miles Bader wrote: The problem is that with the new kernel, the system won't boot all the way. It fails when it tries to mount the root partition, and dumps me into the ramfs emergency shell. The error message is something generic like File not found (sorry for the vagueness, those boot messages don't get saved anywhere and I didn't write them down). This sounds to me as if the initrd did not load the device drivers, as you suggested later in your message. Look in the /boot/grub/menu.lst file and verify that the initrd is being loaded. Check that you have current versions of 'module-init-tools' and 'initramfs-tools' packages. The 'mkinitramfs' command is what builds the initrd.img file. I think the problem is your clue that previously you had compiled into your kernel your required modules. This may be making it difficult for mkinitramfs to determine which modules are required. If it fails to detect this it would build an incorrect initrd.img and have similar results to what you are reporting. You may need to hint to it a module to help bootstrap the system along. BTW, my system uses a SCSI disk with an old Adaptec SCSI card scsi0 : Adaptec AIC7XXX EISA/VLB/PCI SCSI HBA DRIVER, Rev 7.0 Adaptec 2940 Ultra SCSI adapter That was a great card. I have one too but it is no longer in use. I miss SCSI. Bob -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: booting problem (udev related?)
[EMAIL PROTECTED] (Bob Proulx) writes: I think the problem is your clue that previously you had compiled into your kernel your required modules. This may be making it difficult for mkinitramfs to determine which modules are required. If it fails to detect this it would build an incorrect initrd.img and have similar results to what you are reporting. You may need to hint to it a module to help bootstrap the system along. Hmm, I didn't realize it analyzed the system when building the ramfs contents. Maybe I could just reinstall the kernel while the new kernel is running (or is there an official hint mechanism I could use)? [I thought it just included _every_ possible module on the ramfs -- judging from the enormous size of the installed kernel package, it seems like it!] BTW, my system uses a SCSI disk with an old Adaptec SCSI card scsi0 : Adaptec AIC7XXX EISA/VLB/PCI SCSI HBA DRIVER, Rev 7.0 Adaptec 2940 Ultra SCSI adapter That was a great card. I have one too but it is no longer in use. I miss SCSI. Yeah this system gives me the warm fuzzies, despite the very small disk size; I guess next system will be SATA though, it's just too hard to justify anything else... :-/ Thanks, -Miles -- My spirit felt washed. With blood. [Eli Shin, on The Passion of the Christ] -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: booting problem (udev related?)
On Thu, Aug 02, 2007 at 08:50:08AM +0900, Miles Bader wrote: I seems like it may be related to udev because if I look in /dev, the disk device nodes which should be there _aren't there_, even though the disk hardware is recognized fine by the kernel. Udev isn't running yet. The boot devices/modules are loaded in the initramfs. I've never compiled a kernel so I haven't had to fitz with initramfs. Doug. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: booting problem (udev related?)
Miles Bader wrote: Hmm, I didn't realize it analyzed the system when building the ramfs contents. Maybe I could just reinstall the kernel while the new kernel is running (or is there an official hint mechanism I could use)? Yes. Please try that. [I thought it just included _every_ possible module on the ramfs -- judging from the enormous size of the installed kernel package, it seems like it!] :-) Yeah this system gives me the warm fuzzies, despite the very small disk size; I guess next system will be SATA though, it's just too hard to justify anything else... :-/ Yep. I have given into the dark side too. They have cookies. Bob -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
