Re: [Pkg-zfsonlinux-devel] Kernel image and OOT auto builder environment (Was: Using out of tree modules in d-i)?
On Sat, May 25, 2013 at 3:08 AM, Turbo Fredriksson tu...@bayour.com wrote: On May 24, 2013, at 8:54 PM, Aron Xu wrote: What I can think of is to do the trick in d-i, since it already has the ability to retrieve and load udeb on the fly, and even prompt users for missing firmware. Maybe even build them using dkms? I saw that you can make d-i build all packages... So adding a extra hook or something that sees that this is a OOT module, then get it's dkms package, build (to a .deb/.udeb which I have patches sent upstream for) it and use that... It could be possible, but I don't think building it is acceptable, because it means you must pull in everything of build-essential to the d-i image, which is useless for most other people when they run d-i. Ah, ok then I think I know what you mean. I thought you meant when the d-i image is built... But how does this help keeping 'the current kernel' and the OOT's up-to-date? From what I think I understand of you proposal, the user. Which means we can't offer 'whatever-support-the-OOT-provides' globally in our install images. It would be up to the user to make sure that this is available. I was kind'a hoping we could do this as an organization and provide this for our users, instead of putting it to them to make it available for themselves. Now I understand what you mean, just use DKMS to build OOT modules at image generation time? Seems to be a good idea. -- Regards, Aron Xu -- To UNSUBSCRIBE, email to debian-kernel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/CAMr=8w6v=4b1+azgn2a4zub9znfsxmg30ntxdrlbaceucou...@mail.gmail.com
Re: [Pkg-zfsonlinux-devel] Kernel image and OOT auto builder environment (Was: Using out of tree modules in d-i)?
On May 25, 2013, at 2:23 PM, Aron Xu wrote: just use DKMS to build OOT modules at image generation time? Seems to be a good idea. If setting up a kernel-build-system on one of the Debian GNU/Linux servers isn't an option, then this would be a second best.. Then it will be up to the user building the image to make sure (or skip) OOT's they don't want/need. But if an official image is build, the OOT module will still need to be up to date with the current kernel, so this is not a perfect solution anyway. -- Michael Jackson is not going to buried or cremated but recycled into shopping bags so he can remain white, plastic and dangerous for kids to play with. -- To UNSUBSCRIBE, email to debian-kernel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/e73af69d-a04f-4848-b79b-8107f4a11...@bayour.com
Re: [Pkg-zfsonlinux-devel] Kernel image and OOT auto builder environment (Was: Using out of tree modules in d-i)?
On Fri, May 24, 2013 at 11:53 PM, Turbo Fredriksson tu...@bayour.com wrote: On May 22, 2013, at 3:52 PM, Ben Hutchings wrote: Quoting from the report of our 2009 meeting, 20091015123106.ga16...@kyllikki.org: out of tree modules --- After a somewhat involved discussion taking into account the FTP masters extreme irritation about trying to match binaries to source by hand for the lenny release it was resolved to remove linux-modules-extra and -nonfree as they are an impossible to support approach. The Built-Using header should cover FTP masters' concerns. However it is still the case that omnibus source packages are unsustainable as many OOT modules are not kept up to date with the kernel API. I haven't been keeping up with the inner workings of Debian GNU/Linux the last couple of years, so please take this as-is. Is it possible to setup an automatic build system for kernel and modules? I know that we have (had?) an auto builder to build everything automatically (think it was mostly (?) used for the ports), but I was thinking about a special build system here. Very possibly just a modification of what we already have... That is, a special place to upload the kernel source to, and once the auto builder have successfully built a linux-image* package(s), then it will look in a special list for source packages to build as well. And once all those have succeeded to, THEN it will upload the kernel (and the OOT modules) to the ftp archive... To clarify: the kernel maintainer does not build and upload the package to the archives, but only uploads the source package to this special build system and it will do the rest... That way OOT modules should be able to keep up with the kernel releases and uploads without much (hopefully) extra work. And the added benefit is that once a new kernel version is available in the archives, the OOT modules will as well, without any extra lagg... This should be reasonably easy to do, and I volunteer to do the initial setup and/or prof-of-consept for all the versions we currently support (oldstable, stable and unstable). Having something like this may (or may not) generally help, but I'm not sure this is what we want to solve the problem. If OOT modules are added into d-i images directly, then we must set up something like this proposal to make sure the modules are available whenever a new kernel ABI is uploaded, and help d-i people to handle the brokenness if a change in kernel makes the OOT module does not build, or even function badly. This is hard to do, and I think there could be easier way for this specific issue. What I can think of is to do the trick in d-i, since it already has the ability to retrieve and load udeb on the fly, and even prompt users for missing firmware. Such functionality may be reused so that related udebs can be fetched and loaded when selected, and if all effort failed just generate an error message. By doing this a small check is performed to make sure the d-i kernel and the OOT modules are matched, so that d-i may get the ability to include OOT modules in the image, and the worst case is wasting several megabytes in the resulting images. debian-cd may also be improved to check OOT modules included in the image has correct version info to work together with the target d-i kernel to avoid such a waste. Finally, OOT modules may be listed in a separate list so that users are aware what they are doing, and d-i team can coordinate with maintainers of modules they want to include whenever an official image is generated (e.g. release CDs). As for how the small check would be implemented, the question can be discussed later in more detail if we agree that it's the preferred way to go with. -- Regards, Aron Xu -- To UNSUBSCRIBE, email to debian-kernel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/CAMr=8w7TsXC9tOYkws48sV0r+mv=mfzlb8l6z_3-xrqexfy...@mail.gmail.com
Re: [Pkg-zfsonlinux-devel] Kernel image and OOT auto builder environment (Was: Using out of tree modules in d-i)?
On Sat, May 25, 2013 at 2:45 AM, Turbo Fredriksson tu...@bayour.com wrote: On May 24, 2013, at 7:04 PM, Aron Xu wrote: and help d-i people to handle the brokenness if a change in kernel makes the OOT module does not build This should only happen when a new major version of the kernel comes out, which means it should only happen in unstable.. And we could make it so that it is possible to make that particular OOT skipped (manually or automatically after a specified time), and hence won't be available for that run of d-i. But within a oldstable or stable kernel, only the Debian version changes, right, so... But if a kernel changes a lot and often in unstable, support for that specific OOT will be intermittent. Might not be a huge problem in unstable. Unwanted, but not a huge deal... What I can think of is to do the trick in d-i, since it already has the ability to retrieve and load udeb on the fly, and even prompt users for missing firmware. Maybe even build them using dkms? I saw that you can make d-i build all packages... So adding a extra hook or something that sees that this is a OOT module, then get it's dkms package, build (to a .deb/.udeb which I have patches sent upstream for) it and use that... It could be possible, but I don't think building it is acceptable, because it means you must pull in everything of build-essential to the d-i image, which is useless for most other people when they run d-i. But either way would mean that the d-i builder have to do the fixing, instead of a group of people (should be the responsibility of the OOT module maintainer(s)). Such functionality may be reused so that related udebs can be fetched and loaded when selected, and if all effort failed just generate an error message. What if an OOT is a network module? Might be needed before the network is up to fetch it... This is what I've said that d-i can ask users for firmware, and the case is usually network drivers: http://wiki.debian.org/Firmware And if it's included in the monolithic image, then it will be up to the d-i builder/uploader to make sure that the module is available, not a group of people... -- Regards, Aron Xu -- To UNSUBSCRIBE, email to debian-kernel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/CAMr=8w7burLHc=0rStLTDSrge=se67jghvpajmu3rh_-p3q...@mail.gmail.com
Re: [Pkg-zfsonlinux-devel] Kernel image and OOT auto builder environment (Was: Using out of tree modules in d-i)?
On May 24, 2013, at 7:04 PM, Aron Xu wrote: and help d-i people to handle the brokenness if a change in kernel makes the OOT module does not build This should only happen when a new major version of the kernel comes out, which means it should only happen in unstable.. And we could make it so that it is possible to make that particular OOT skipped (manually or automatically after a specified time), and hence won't be available for that run of d-i. But within a oldstable or stable kernel, only the Debian version changes, right, so... But if a kernel changes a lot and often in unstable, support for that specific OOT will be intermittent. Might not be a huge problem in unstable. Unwanted, but not a huge deal... What I can think of is to do the trick in d-i, since it already has the ability to retrieve and load udeb on the fly, and even prompt users for missing firmware. Maybe even build them using dkms? I saw that you can make d-i build all packages... So adding a extra hook or something that sees that this is a OOT module, then get it's dkms package, build (to a .deb/.udeb which I have patches sent upstream for) it and use that... But either way would mean that the d-i builder have to do the fixing, instead of a group of people (should be the responsibility of the OOT module maintainer(s)). Such functionality may be reused so that related udebs can be fetched and loaded when selected, and if all effort failed just generate an error message. What if an OOT is a network module? Might be needed before the network is up to fetch it... And if it's included in the monolithic image, then it will be up to the d-i builder/uploader to make sure that the module is available, not a group of people... -- To UNSUBSCRIBE, email to debian-kernel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/e46ce809-7a99-4572-9af2-64ee1f645...@bayour.com
Re: [Pkg-zfsonlinux-devel] Kernel image and OOT auto builder environment (Was: Using out of tree modules in d-i)?
On May 24, 2013, at 8:54 PM, Aron Xu wrote: What I can think of is to do the trick in d-i, since it already has the ability to retrieve and load udeb on the fly, and even prompt users for missing firmware. Maybe even build them using dkms? I saw that you can make d-i build all packages... So adding a extra hook or something that sees that this is a OOT module, then get it's dkms package, build (to a .deb/.udeb which I have patches sent upstream for) it and use that... It could be possible, but I don't think building it is acceptable, because it means you must pull in everything of build-essential to the d-i image, which is useless for most other people when they run d-i. Ah, ok then I think I know what you mean. I thought you meant when the d-i image is built... But how does this help keeping 'the current kernel' and the OOT's up-to-date? From what I think I understand of you proposal, the user. Which means we can't offer 'whatever-support-the-OOT-provides' globally in our install images. It would be up to the user to make sure that this is available. I was kind'a hoping we could do this as an organization and provide this for our users, instead of putting it to them to make it available for themselves. -- To UNSUBSCRIBE, email to debian-kernel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/cd9b3b40-3a48-473d-8cd0-643aa1440...@bayour.com
