Re: Customizing the kernel installation process
On Sun, 4 Apr 2010 12:00:58 -0400 (EDT), Hugo Vanwoerkom wrote: Stephen Powell wrote: ... I have a document on my web page which addresses the subject of creating a custom kernel for Debian. ... I welcome all comments. What about this: http://wiki.debian.org/HowToRebuildAnOfficialDebianKernelPackage Thanks for the link, Hugo. -- .''`. Stephen Powellzlinux...@wowway.com : :' : `. `'` `- -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/890068129.1169721270480633270.javamail.r...@md01.wow.synacor.com
Re: Customizing the kernel installation process
Stephen Powell wrote: I have seen a number of help requests on this list recently that indicate that there are a number of people who initially installed grub, had problems, and reverted to lilo. Then they experience subsequent problems when a kernel is updated, a new kernel is installed, or some kind of upgrade requires that the initial RAM filesystem be rebuilt. It does not appear to me that they have lilo fully integrated into the Debian package management system. I have a document on my web page which addresses the subject of creating a custom kernel for Debian. But one of the steps, Step 10: Customize the Kernel Installation Process, is of interest even to people who only run stock kernels, especially if they run lilo. In response to recent problem reports, I have totally re-written that Step and updated the document on the Internet just today. (If you have looked at the page before, it may be in your browser cache. Manually reload the page if necessary to make sure that you have the current version. The maintenance log should show an update on April 3, 2010.) I particularly wanted to make sure that users who change boot loaders from grub to lilo do all the necessary steps to fully integrate lilo into the Debian package management system. Some of the steps are not at all obvious. I welcome all comments. Am I full of excrement? Is there something missing? What can be improved? I am especially interested in hearing from lilo users. Here is the link: http://www.wowway.com/~zlinuxman/Kernel.htm. Scroll down to Step 10 and have a good read. (Unfortunately, it's fairly long. But I wanted to treat the topic thoroughly.) Commence firing. Fire at will. What about this: http://wiki.debian.org/HowToRebuildAnOfficialDebianKernelPackage Hugo -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/hpad3q$k3...@dough.gmane.org
Re: Customizing the kernel installation process
On Sun, 04 Apr 2010 11:00:58 -0500 Hugo Vanwoerkom hvw59...@care2.com wrote: Stephen Powell wrote: I welcome all comments. Am I full of excrement? Is there something missing? What can be improved? I am especially interested in hearing from lilo users. lilo user here. works great. The _only_ gotcha that I ran into was forgetting to modify the postinst scripts to point to lilo instead of that other loader. But you clearly talk about this, so my bad. Here is the link: http://www.wowway.com/~zlinuxman/Kernel.htm. Scroll down to Step 10 and have a good read. (Unfortunately, it's fairly long. But I wanted to treat the topic thoroughly.) Commence firing. Fire at will. What about this: http://wiki.debian.org/HowToRebuildAnOfficialDebianKernelPackage that's the link that I tried to use when first trying to build a linux kernel the debian way and for some reason I just couldn't figure it out. Stephen's explanation is much more detailed. Also the postinst and postrm scripts are invaluable. and for whatever reason, even with the extensive explanation I found it _simpler_ to follow. for the longest time I have downloaded the kernel tarball and built outside of debian. however I'd like to use the debian nvidia packages, so I'm trying to build the kernel in the debian framework. however, rebuilding nvidia for the latest kernel has now become a problem (in the debian framework), so I'm thinking about dropping back to the tarball method. Maybe I need to be a little more persistent and then write something about rebuilding nvidia to complement Stephen's work. It is very handy having things installed as packages. Brian -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20100404100434.58e18...@windy.deldotd.com
Re: Customizing the kernel installation process
On Sun, Apr 04 2010, bri...@aracnet.com wrote:
On Sun, 04 Apr 2010 11:00:58 -0500
Hugo Vanwoerkom hvw59...@care2.com wrote:
for the longest time I have downloaded the kernel tarball and built
outside of debian. however I'd like to use the debian nvidia packages,
so I'm trying to build the kernel in the debian framework. however,
rebuilding nvidia for the latest kernel has now become a problem (in
the debian framework), so I'm thinking about dropping back to the
tarball method.
Maybe I need to be a little more persistent and then write something
about rebuilding nvidia to complement Stephen's work.
It is very handy having things installed as packages.
For what it is worth, I also have an nvidia card. I nowadays
just use the kernel git tree. My usual sequence of action is to
,
| % cd /usr/local/src/kernel/linus-tree.git
| % git fetch stable
| % git co -b my-machine-v2.6.33.2 v2.6.33.2
| % make oldconfig
| % ./.compile_command
| % sudo dpkg -i ../*.deb
`
Where .compile command looks like:
,
| #!/bin/sh
|
| export MODULE_LOC=/usr/local/src/kernel/modules
|
| # Optionally, refresh the nvidia module
| # rm -rf ${MODULE_LOC}/nvidia-kernel
| # (cd /usr/local/src/kernel; tar jfx /usr/src/nvidia-kernel.tar.bz2
|
| # make sure we get a machine specific name for the image, even if
| # I forgot toe specify one on the command line
| ev=$(uname -n)
|
| # Use the version extension given on the command line, if any
| if [ -n $1 ]; then
| ev=$1
| fi
|
| make-kpkg --rootcmd=fakeroot --append-to-version=-$ev kernel_image
| fakeroot make-kpkg --append-to-version=-$ev modules_image
`
Once you have your variant of .compile_command, building kernels
and nvidia packages is painless :-)
manoj
--
War is an equal opportunity destroyer.
Manoj Srivastava sriva...@acm.org http://www.golden-gryphon.com/
4096R/C5779A1C E37E 5EC5 2A01 DA25 AD20 05B6 CF48 9438 C577 9A1C
--
To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/87vdc7xeqr@anzu.internal.golden-gryphon.com
Re: Customizing the kernel installation process
On Sat, 3 Apr 2010 20:34:13 -0400 (EDT), Paul E Condon wrote: One of the reasons for boot problems in general is the lack of stablity in the names assigned by the kernel to SCSI devices. Lilo seems to require that you know at lilo-update time what the kernel will think the name of a device will be at boot time. Grub was supposed to offer a solution. Or am I just totally misunderstanding ... ? Lilo has two distict phases: (1) The process of writing out a new boot block, which occurs as the result of issuing the lilo command at a shell prompt, and (2) The process of booting the kernel, which is done by the code written to the boot block during phase 1. The examination of /etc/lilo.conf and the interpretation of Linux device names specified in /etc/lilo.conf, such as /dev/hda1, occur only during phase 1, the process of writing out a new boot block. During phase 1, the Linux device names are converted into the equivalent parameters needed to do I/O to the device via the BIOS. For example, /dev/hda1 would be converted into something like primary IDE controller, master device, partition number 1. The files themselves (the kernel image file and the initial RAM disk image file) are converted into a sequential list of LBA addresses to read. Thus, during phase 2, lilo doesn't need to know anything about the file system (except for file system control structures which may exist inside the data blocks themselves) in order to read the file. If something causes the device names to change, such as a change to udev, that will not cause a boot failure, since the phase 2 boot code doesn't use the device names at all. It does all its I/O via the BIOS and uses parameters that the BIOS understands. However, /etc/lilo.conf will need to be edited to specify the new device names before the lilo command is issued again to write out a new boot block. The method used by grub does bypass the Linux device name issue. However, grub version 1 and grub version 2 use incompatible methods for specifying the partition numbers. Grub version 1 numbers partitions starting with 0: grub version 2 numbers partitions starting with 1. Furthermore, the boot loader configuration files are not the only things that may need to be changed if Linux device names change. In particular /etc/fstab comes to mind, not to mention various home-grown scripts that administrators may have created. I hope that answers your question. From my perspective, the primary advantage of grub over lilo is that grub does not read the kernel image file and the intial RAM disk image file via a list of LBA addresses created during the writing of the boot block: it understands the file system and can dynamically determine at boot time which blocks it needs to read to load the kernel and the initial RAM filesystem into memory. This means that the boot block needs to be re-written less often. With lilo, simply copying the kernel image file somewhere else, erasing the original, then copying it back will require that the boot block be re-written. Why? Because even though the kernel image file exists in the /boot directory, just like it did before, chances are that it no longer occupies the exact same list of blocks in the exact same order that it did before. Therefore, the lilo command must be re-run to generate a fresh (and correct) list of LBA addresses to read in order to read the kernel image. Similarly, anything that causes or may cause the list of LBA addresses which comprise the initial RAM filesytem file to change will also necessitate that lilo be re-run. This includes, but is not limited to, regenerating the initial RAM file system by running update-initramfs -u. Since grub understands the file sytem, it can determine which blocks need to be read dynamically. That, in my humble opionion, is the primary advantage of grub. With grub version 1, running update-grub does not re-write the boot block. It only regenerates /boot/grub/menu.lst based on which kernels are installed. With grub version 2, this list is effectively generated on the fly, and even this requirement is eliminated. These advantages come at a price, however. The extra code needed to understand the many file systems makes grub bigger (and more vulnerable to bugs). And that is the primary reason I use lilo. lilo occupies only the master boot record (cylinder 0, head 0, record 1). grub (both versions) occupies additional blocks (record 2, record 3, etc.). These blocks are outside of a partition and are not backed up by the backup software that we use. The backup software knows to back up the master boot record, and it also backs up all the partitions and their boot blocks, but the extra blocks occupied by grub are not backed up. Thus, if we have a hard drive failure, we replace the hard drive; we run a restore; and we're good to go, as long as the machine runs lilo. But if the machine runs grub, and we do a restore, we have an unbootable machine. There are some ways around
Re: Customizing the kernel installation process
On Sun, 4 Apr 2010 13:04:34 -0400 (EDT), Brian D. wrote: On Sun, 04 Apr 2010 11:00:58 -0500 Hugo Vanwoerkom wrote: Stephen Powell wrote: I welcome all comments. Am I full of excrement? Is there something missing? What can be improved? I am especially interested in hearing from lilo users. Here is the link: http://www.wowway.com/~zlinuxman/Kernel.htm. Scroll down to Step 10 and have a good read. (Unfortunately, it's fairly long. But I wanted to treat the topic thoroughly.) Commence firing. Fire at will. What about this: http://wiki.debian.org/HowToRebuildAnOfficialDebianKernelPackage lilo user here. works great. The _only_ gotcha that I ran into was forgetting to modify the postinst scripts to point to lilo instead of that other loader. But you clearly talk about this, so my bad. that's the link that I tried to use when first trying to build a linux kernel the debian way and for some reason I just couldn't figure it out. Stephen's explanation is much more detailed. Also the postinst and postrm scripts are invaluable. and for whatever reason, even with the extensive explanation I found it _simpler_ to follow. for the longest time I have downloaded the kernel tarball and built outside of debian. however I'd like to use the debian nvidia packages, so I'm trying to build the kernel in the debian framework. however, rebuilding nvidia for the latest kernel has now become a problem (in the debian framework), so I'm thinking about dropping back to the tarball method. Maybe I need to be a little more persistent and then write something about rebuilding nvidia to complement Stephen's work. It is very handy having things installed as packages. I didn't realize you were going to read the *whole thing*. I was mainly interested in people's comments on Step 10, which is useful information even to those using stock kernels. I didn't realize you were a user of a custom kernel. I do cover external module packages in my treatise, but it assumes that you have the source code. The proprietary nvidia driver is partially closed source. I've never tried to do anything in that environment. Maybe after April 15 (tax season is over), I'll play around with this, since this is of interest to a number of people on the list. But right now I'm too busy trying to get my taxes done. -- .''`. Stephen Powellzlinux...@wowway.com : :' : `. `'` `- -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/1491629930.1051041270415868026.javamail.r...@md01.wow.synacor.com
Re: Customizing the kernel installation process
On Sun, 4 Apr 2010 14:19:08 -0400 (EDT), Manoj Srivastava wrote:
On Sun, Apr 04 2010, bri...@aracnet.com wrote:
for the longest time I have downloaded the kernel tarball and built
outside of debian. however I'd like to use the debian nvidia packages,
so I'm trying to build the kernel in the debian framework. however,
rebuilding nvidia for the latest kernel has now become a problem (in
the debian framework), so I'm thinking about dropping back to the
tarball method.
Maybe I need to be a little more persistent and then write something
about rebuilding nvidia to complement Stephen's work.
It is very handy having things installed as packages.
For what it is worth, I also have an nvidia card. I nowadays
just use the kernel git tree. My usual sequence of action is to
,
| % cd /usr/local/src/kernel/linus-tree.git
| % git fetch stable
| % git co -b my-machine-v2.6.33.2 v2.6.33.2
| % make oldconfig
| % ./.compile_command
| % sudo dpkg -i ../*.deb
`
Where .compile command looks like:
,
| #!/bin/sh
|
| export MODULE_LOC=/usr/local/src/kernel/modules
|
| # Optionally, refresh the nvidia module
| # rm -rf ${MODULE_LOC}/nvidia-kernel
| # (cd /usr/local/src/kernel; tar jfx /usr/src/nvidia-kernel.tar.bz2
|
| # make sure we get a machine specific name for the image, even if
| # I forgot to specify one on the command line
| ev=$(uname -n)
|
| # Use the version extension given on the command line, if any
| if [ -n $1 ]; then
| ev=$1
| fi
|
| make-kpkg --rootcmd=fakeroot --append-to-version=-$ev kernel_image
| fakeroot make-kpkg --append-to-version=-$ev modules_image
`
Once you have your variant of .compile_command, building kernels
and nvidia packages is painless :-)
Thanks for participating, Manoj. If you needed to compile a custom
kernel for some reason, and you were going to use an official Debian
kernel source package (linux-source-2.6.32, for example), and you
were going to use make-kpkg to create your custom kernel image, and
you also wanted to use the proprietary Nvidia drivers, how would you
do it? That's a specific example that I would like to integrate into
my HOWTO. If I were smarter, or more experienced, I would probably
be able to derive the steps from the above. But unfortunately, I'm not.
Use official Debian packages wherever possible. But if the user has
a custom kernel, I seem to remember that a kernel module that is
customized for that kernel has to be built somehow. And I'd like
make-kpkg do do as much of the work as possible. A single invocation
of make-kpkg that has both the kernel_image and modules_image targets
would be ideal. I haven't built a separate modules_image package since
the days when ALSA was not part of the kernel. That's been a while!
--
.''`. Stephen Powellzlinux...@wowway.com
: :' :
`. `'`
`-
--
To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive:
http://lists.debian.org/1741854818.1053401270417891931.javamail.r...@md01.wow.synacor.com
Re: Customizing the kernel installation process
On Sun, Apr 04 2010, Stephen Powell wrote:
On Sun, 4 Apr 2010 14:19:08 -0400 (EDT), Manoj Srivastava wrote:
On Sun, Apr 04 2010, bri...@aracnet.com wrote:
for the longest time I have downloaded the kernel tarball and built
outside of debian. however I'd like to use the debian nvidia packages,
so I'm trying to build the kernel in the debian framework. however,
rebuilding nvidia for the latest kernel has now become a problem (in
the debian framework), so I'm thinking about dropping back to the
tarball method.
Maybe I need to be a little more persistent and then write something
about rebuilding nvidia to complement Stephen's work.
It is very handy having things installed as packages.
For what it is worth, I also have an nvidia card. I nowadays
just use the kernel git tree. My usual sequence of action is to
,
| % cd /usr/local/src/kernel/linus-tree.git
| % git fetch stable
| % git co -b my-machine-v2.6.33.2 v2.6.33.2
| % make oldconfig
| % ./.compile_command
| % sudo dpkg -i ../*.deb
`
Where .compile command looks like:
,
| #!/bin/sh
|
| export MODULE_LOC=/usr/local/src/kernel/modules
|
| # Optionally, refresh the nvidia module
| # rm -rf ${MODULE_LOC}/nvidia-kernel
| # (cd /usr/local/src/kernel; tar jfx /usr/src/nvidia-kernel.tar.bz2
|
| # make sure we get a machine specific name for the image, even if
| # I forgot to specify one on the command line
| ev=$(uname -n)
|
| # Use the version extension given on the command line, if any
| if [ -n $1 ]; then
| ev=$1
| fi
|
| make-kpkg --rootcmd=fakeroot --append-to-version=-$ev kernel_image
| fakeroot make-kpkg --append-to-version=-$ev modules_image
`
Once you have your variant of .compile_command, building kernels
and nvidia packages is painless :-)
Thanks for participating, Manoj. If you needed to compile a custom
kernel for some reason, and you were going to use an official Debian
kernel source package (linux-source-2.6.32, for example), and you
were going to use make-kpkg to create your custom kernel image, and
you also wanted to use the proprietary Nvidia drivers, how would you
do it? That's a specific example that I would like to integrate into
my HOWTO. If I were smarter, or more experienced, I would probably
be able to derive the steps from the above. But unfortunately, I'm not.
Oh, if it were me, I would extract the kernel sources, throw
away the ./debian directory, and use exactly the same steps as
above. That way, I'd get whatever patches are applied to Debian, and
yet not have to deal with -tree. -build, or what have you. (Indeed,
just call the script, the first thing it does is remove ./debian, so
you probably have to do nothing).
See, the above approach does not care how you arrived at the
kernel source tree -- I can jump from the bleeding edge rcX tree, to
stable trees, to any other tree I want to add a remote repo for, and
the script works. It would make no difference if you tree was somehow
derived from the official kernel tree sources. make-kpkg and the little
wrapper just don't _care_.
I ... have reservations about the ./debian directory structure
that the official kernel uses, so I can't actually advocate using that.
Use official Debian packages wherever possible. But if the user has
a custom kernel, I seem to remember that a kernel module that is
customized for that kernel has to be built somehow. And I'd like
make-kpkg do do as much of the work as possible. A single invocation
of make-kpkg that has both the kernel_image and modules_image targets
would be ideal. I haven't built a separate modules_image package since
the days when ALSA was not part of the kernel. That's been a while!
Well, if you don't mind running the whole thing under fakeroot,
you could do:
fakeroot make-kpkg --append-to-version=-$ev kernel_image modules_image
and you are done. You run some stuff nuder fakeroot, but usually thsat
does not matter. Me, since it is a script, I just run make-kpkg
twice. Usually I am not even looking at the screen. So if ever there
is a conerner case that bites fakeroot (comething like that happened a
few years ago), I am covered somewhat better. But then, fakeroot bugs
are now far and few in between, so this is mostly historical reasons.
manoj
--
The sendmail configuration file is one of those files that looks like someone
beat their head on the keyboard. After working with it... I can see why! --
Harry Skelton
Manoj Srivastava sriva...@acm.org http://www.golden-gryphon.com/
4096R/C5779A1C E37E 5EC5 2A01 DA25 AD20 05B6 CF48 9438 C577 9A1C
--
To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/87r5muyi5f@anzu.internal.golden-gryphon.com
Customizing the kernel installation process
I have seen a number of help requests on this list recently that indicate that there are a number of people who initially installed grub, had problems, and reverted to lilo. Then they experience subsequent problems when a kernel is updated, a new kernel is installed, or some kind of upgrade requires that the initial RAM filesystem be rebuilt. It does not appear to me that they have lilo fully integrated into the Debian package management system. I have a document on my web page which addresses the subject of creating a custom kernel for Debian. But one of the steps, Step 10: Customize the Kernel Installation Process, is of interest even to people who only run stock kernels, especially if they run lilo. In response to recent problem reports, I have totally re-written that Step and updated the document on the Internet just today. (If you have looked at the page before, it may be in your browser cache. Manually reload the page if necessary to make sure that you have the current version. The maintenance log should show an update on April 3, 2010.) I particularly wanted to make sure that users who change boot loaders from grub to lilo do all the necessary steps to fully integrate lilo into the Debian package management system. Some of the steps are not at all obvious. I welcome all comments. Am I full of excrement? Is there something missing? What can be improved? I am especially interested in hearing from lilo users. Here is the link: http://www.wowway.com/~zlinuxman/Kernel.htm. Scroll down to Step 10 and have a good read. (Unfortunately, it's fairly long. But I wanted to treat the topic thoroughly.) Commence firing. Fire at will. -- .''`. Stephen Powellzlinux...@wowway.com : :' : `. `'` `- -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/180943899.887521270303721562.javamail.r...@md01.wow.synacor.com
Re: Customizing the kernel installation process
On 20100403_100841, Stephen Powell wrote: I have seen a number of help requests on this list recently that indicate that there are a number of people who initially installed grub, had problems, and reverted to lilo. Then they experience subsequent problems when a kernel is updated, a new kernel is installed, or some kind of upgrade requires that the initial RAM filesystem be rebuilt. It does not appear to me that they have lilo fully integrated into the Debian package management system. I have a document on my web page which addresses the subject of creating a custom kernel for Debian. But one of the steps, Step 10: Customize the Kernel Installation Process, is of interest even to people who only run stock kernels, especially if they run lilo. In response to recent problem reports, I have totally re-written that Step and updated the document on the Internet just today. (If you have looked at the page before, it may be in your browser cache. Manually reload the page if necessary to make sure that you have the current version. The maintenance log should show an update on April 3, 2010.) I particularly wanted to make sure that users who change boot loaders from grub to lilo do all the necessary steps to fully integrate lilo into the Debian package management system. Some of the steps are not at all obvious. I welcome all comments. Am I full of excrement? Is there something missing? What can be improved? I am especially interested in hearing from lilo users. Here is the link: http://www.wowway.com/~zlinuxman/Kernel.htm. Scroll down to Step 10 and have a good read. (Unfortunately, it's fairly long. But I wanted to treat the topic thoroughly.) Commence firing. Fire at will. Thanks Steve. I looked into reverting to lilo several weeks ago and decided that I was sufficiently confused that I couldn't even ask well formed questions. Now I have a feeling that maybe I could do that, but I haven't absolutely given up on grub --- yet. I worry about lilo as time goes on. One of the reasons for boot problems in general is the lack of stablity in the names assigned by the kernel to SCSI devices. Lilo seems to require that you know at lilo-update time what the kernel will think the name of a device will be at boot time. Grub was supposed to offer a solution. Or am I just totally misunderstanding ... ? -- Paul E Condon pecon...@mesanetworks.net -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20100404003413.gb7...@big.lan.gnu
