Re: Out-of-tree kernel module udeb
On May 17, 2015, at 1:25 PM, Cyril Brulebois wrote:
Aron Xu a...@debian.org (2015-05-17):
At the moment only spl is available in the archive, using dkms, and
for zfs it's similar in the way of packaging though not uploaded yet.
What we have (code ready to go) is a mechanism that detects/gets
definition of a current KVERS and generate a source package with
dependencies and binary packages with names corresponding to it.
My personal stance on kernel related things would be “upstream first”.
I'm not exactly sure what you mean by that, but if you mean that upstream
(i.e. ZFS On Linux - ZoL in this case) should have the support to build
binary modules package(s), then they do.
BUT, it's somewhat of a hack. The debs upstream build is from alien the
rpm which it natively have.
And if we're talking about udebs, there is no need for that in upstream.
If we're talking about upstream as the Linux kernel sources, because of
the CDDL license of ZoL, we're _NEVER_ (ever, ever!!) going to be able
to put it in the Linux kernel source tree.
Unless someone that's never seen the ZoL code decides to … reverse engineer
one of the most advanced filesystems ever created (so far). I doubt (do
get I a understatement of the year award for that?? :) that will ever
happen either.
If it ain't going to be merged into mainline, or at least accepted as a
patchset (like e.g. aufs3 or rt in wheezy) for src:linux, I'm not sure
we want to support that.
Currently (just to catch everyone up for those that haven't been paying
attention :), I've been doing debs, udebs etc for both SPL (Solaris Porting
Layer) and ZFS for ZoL for years.
I have also made a modified version of D-I (previous patches have been
sent to relevant parties of Debian GNU/Linux although I've been, over the
year, improving on them) that I supply to the ZoL community. But since
Debian GNU/Linux was in release mode, only a few of them was accepted.
So when the packages are built, it also build the udebs for the kernel
it's running on. Because we need ZoL support in the installer, with its
limited space requirements, we can't obviously (!!) ship a build environment
there as well to build the spl and zfs modules (from the existing dkms
packages).
We need the udebs.
_Inside_ the installed system which is created, the dkms packages are used
as normal. It's only in the partition phase of the install we need udebs.
Building this for the current running system is, I agree, somewhat of a
pain. BUT, since the kernel doesn't really change that much/often, I think
_we_ (the pkg-zfsonlinux-devel) can handle it, to make sure that there's udebs
for the kernel D-I is using.
PS1. Because I personally _hate_, _loath_, _really don't like_ dkms, I created
the option to create a {spl,zfs}-modules-source, much like I'm used to
from the OpenAFS code (and others). But it still needs a build environment
(just not necessarily on the host using ZoL - which I prefer not to have).
PS2. Because I got the impression that binary modules wouldn't be allowed
(still haven't heard anything definitive about that from the DPL etc),
I started experimenting with using zfs-fuse for the installer part, and
real ZoL modules (from dkms) for the 'finished product'. After a couple
of weeks, I had a proof-of-concept that seemed to work.
BUT, it's The hack of all hacks in my opinion, and I just can't recommend
that course with a straight face.
We need the udebs to get this all the way (to allow users to install
a fully enabled ZFS system).
--
Choose a job you love, and you will never have
to work a day in your life.
signature.asc
Description: Message signed with OpenPGP using GPGMail
Re: Kernel image and OOT auto builder environment (Was: Using out of tree modules in d-i)?
On May 26, 2013, at 3:31 AM, Ben Hutchings wrote: Absolutely not acceptable. Out-of-tree modules must not hold up fixes to the kernel. It was bad enough when their build failures were blocking each other in linux-modules-extra-2.6. So how much delay would be acceptable? One day? Twelve hours? Six? One? Do note that a build fail for a OOT should only happen at a major version bump. That is, if an OOT build for 'linux-image-3.2.0-4-amd64', version 3.2.41-2 (current wheezy kernel), it should build quite well on whatever would be the next stable dist version (3.2.41-3 or maybe even '3.2.42-1'?). But if not even a second delay would be acceptable, then just fail the OOT, notify it's maintainer but still upload the kernel. My point being, that with a auto build system for kernels and OOT's, at least we get a fighting chance of keeping the OOTs up to date. Without it, either the kernel maintainer(s) need to build the OOT's as well (and either manually try to fix any problems or hand it over to the OOT maintainer), or the OOT maintainer needs to monitor for any kernel update and then do their build and upload.. Both of these introduces a lot of delays and manual work. I really don't think that ALL OOTs will fail in such a case, ZoL I'm quite sure wouldn't If you jump from 3.9 (which I have on my production machines) to whatever comes next, say 4.0, then yes most likely it will fail. _IF_ the Debian GNU/Linux kernel maintainer creates a 4.0 kernel packages almost instantaneous when Linus releases it. But if the kernel maintainer have a little delay here, say 'a few days', it is most likely fixed in ZoL and a new source package is uploaded, so when the kernel maintainer DO make a 4.0 kernel, ZoL (and hopefully all OOTs) is ready for this.. And a jump like this (or 3.8 in wheezy to 4.0 in Jessie for example) would _NEVER_ happen within a release we support, only in 'sid', and 'a few hours' (maybe even a day?) would be acceptable there... ? Of course a fail will introduce manual labor, but that can never be avoided. But with a auto-kernel-build-system (tm), there won't ALWAYS be need for manual labor. -- Geologists recently discovered that earthquakes are nothing more than Bruce Schneier and Chuck Norris communicating via a roundhouse kick-based cryptosystem. -- To UNSUBSCRIBE, email to debian-kernel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/2111b0cc-0683-477b-837b-b0ab06e52...@bayour.com
Re: [Pkg-zfsonlinux-devel] Kernel image and OOT auto builder environment (Was: Using out of tree modules in d-i)?
On May 25, 2013, at 2:23 PM, Aron Xu wrote: just use DKMS to build OOT modules at image generation time? Seems to be a good idea. If setting up a kernel-build-system on one of the Debian GNU/Linux servers isn't an option, then this would be a second best.. Then it will be up to the user building the image to make sure (or skip) OOT's they don't want/need. But if an official image is build, the OOT module will still need to be up to date with the current kernel, so this is not a perfect solution anyway. -- Michael Jackson is not going to buried or cremated but recycled into shopping bags so he can remain white, plastic and dangerous for kids to play with. -- To UNSUBSCRIBE, email to debian-kernel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/e73af69d-a04f-4848-b79b-8107f4a11...@bayour.com
Re: Using out of tree modules in d-i?
On May 25, 2013, at 10:12 PM, Bastian Blank wrote: Please show that CDDL does not impose additional restrictions over GPL-2. Since you seem to have already settled the case once and for all and are so sure about this, how about you prove your point? It really doesn't matter who proves there point, they are mutually exclusive... -- Em - The battle cry of the cronical masturbater. - Charlie Harper -- To UNSUBSCRIBE, email to debian-kernel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/0452f919-d6e1-45be-9bb5-710a8cde9...@bayour.com
Kernel image and OOT auto builder environment (Was: Using out of tree modules in d-i)?
On May 22, 2013, at 3:52 PM, Ben Hutchings wrote: Quoting from the report of our 2009 meeting, 20091015123106.ga16...@kyllikki.org: out of tree modules --- After a somewhat involved discussion taking into account the FTP masters extreme irritation about trying to match binaries to source by hand for the lenny release it was resolved to remove linux-modules-extra and -nonfree as they are an impossible to support approach. The Built-Using header should cover FTP masters' concerns. However it is still the case that omnibus source packages are unsustainable as many OOT modules are not kept up to date with the kernel API. I haven't been keeping up with the inner workings of Debian GNU/Linux the last couple of years, so please take this as-is. Is it possible to setup an automatic build system for kernel and modules? I know that we have (had?) an auto builder to build everything automatically (think it was mostly (?) used for the ports), but I was thinking about a special build system here. Very possibly just a modification of what we already have... That is, a special place to upload the kernel source to, and once the auto builder have successfully built a linux-image* package(s), then it will look in a special list for source packages to build as well. And once all those have succeeded to, THEN it will upload the kernel (and the OOT modules) to the ftp archive... To clarify: the kernel maintainer does not build and upload the package to the archives, but only uploads the source package to this special build system and it will do the rest... That way OOT modules should be able to keep up with the kernel releases and uploads without much (hopefully) extra work. And the added benefit is that once a new kernel version is available in the archives, the OOT modules will as well, without any extra lagg... This should be reasonably easy to do, and I volunteer to do the initial setup and/or prof-of-consept for all the versions we currently support (oldstable, stable and unstable). -- To UNSUBSCRIBE, email to debian-kernel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/7556b503-9234-4767-b6f1-b86d1280d...@bayour.com
Re: [Pkg-zfsonlinux-devel] Kernel image and OOT auto builder environment (Was: Using out of tree modules in d-i)?
On May 24, 2013, at 7:04 PM, Aron Xu wrote: and help d-i people to handle the brokenness if a change in kernel makes the OOT module does not build This should only happen when a new major version of the kernel comes out, which means it should only happen in unstable.. And we could make it so that it is possible to make that particular OOT skipped (manually or automatically after a specified time), and hence won't be available for that run of d-i. But within a oldstable or stable kernel, only the Debian version changes, right, so... But if a kernel changes a lot and often in unstable, support for that specific OOT will be intermittent. Might not be a huge problem in unstable. Unwanted, but not a huge deal... What I can think of is to do the trick in d-i, since it already has the ability to retrieve and load udeb on the fly, and even prompt users for missing firmware. Maybe even build them using dkms? I saw that you can make d-i build all packages... So adding a extra hook or something that sees that this is a OOT module, then get it's dkms package, build (to a .deb/.udeb which I have patches sent upstream for) it and use that... But either way would mean that the d-i builder have to do the fixing, instead of a group of people (should be the responsibility of the OOT module maintainer(s)). Such functionality may be reused so that related udebs can be fetched and loaded when selected, and if all effort failed just generate an error message. What if an OOT is a network module? Might be needed before the network is up to fetch it... And if it's included in the monolithic image, then it will be up to the d-i builder/uploader to make sure that the module is available, not a group of people... -- To UNSUBSCRIBE, email to debian-kernel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/e46ce809-7a99-4572-9af2-64ee1f645...@bayour.com
Re: [Pkg-zfsonlinux-devel] Kernel image and OOT auto builder environment (Was: Using out of tree modules in d-i)?
On May 24, 2013, at 8:54 PM, Aron Xu wrote: What I can think of is to do the trick in d-i, since it already has the ability to retrieve and load udeb on the fly, and even prompt users for missing firmware. Maybe even build them using dkms? I saw that you can make d-i build all packages... So adding a extra hook or something that sees that this is a OOT module, then get it's dkms package, build (to a .deb/.udeb which I have patches sent upstream for) it and use that... It could be possible, but I don't think building it is acceptable, because it means you must pull in everything of build-essential to the d-i image, which is useless for most other people when they run d-i. Ah, ok then I think I know what you mean. I thought you meant when the d-i image is built... But how does this help keeping 'the current kernel' and the OOT's up-to-date? From what I think I understand of you proposal, the user. Which means we can't offer 'whatever-support-the-OOT-provides' globally in our install images. It would be up to the user to make sure that this is available. I was kind'a hoping we could do this as an organization and provide this for our users, instead of putting it to them to make it available for themselves. -- To UNSUBSCRIBE, email to debian-kernel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/cd9b3b40-3a48-473d-8cd0-643aa1440...@bayour.com
Re: Using out of tree modules in d-i?
On May 22, 2013, at 3:52 PM, Ben Hutchings wrote: Linux has plenty of fine filesystems to choose from already, so this is not a must-have. Ohhh, ouch! But I'm not going to bite... :) Also, there are questions as to whether it would be legal. Legal as in CDDL clashes with GPL you mean? I've heard that to. It would be nice to hear, once and for all if this is the case or not... The kernel team would endorse the use of dkms as a way for out-of-tree module maintainers to get their modules auto-built. Doesn't work for d-i, of course. So what you're saying, in short, is that we can forget about it? -- To UNSUBSCRIBE, email to debian-kernel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/f856aa6a-0cb7-41b8-b86c-303170d12...@bayour.com
