Re: [gentoo-dev] Re: [gentoo-kernel] Proper distribution integration of kernel *-sources, patches and configuration.
On 07/01/2013 11:29 PM, Richard Yao wrote: > On 07/01/2013 09:56 PM, Greg KH wrote:> On Mon, Jul 01, 2013 at > 09:36:21PM -0400, Richard Yao wrote: >>> That is because fixes for other filesystems are either held back by a >>> lack of system kernel updates or held hostage by regressions in newer >>> kernels on certain hardware. >> >> I have never heard this being a reason for keeping code upstream, what >> do you mean by it? > > This is an issue that end users tend to encounter where changes in a > newer kernel break stuff that they need. One example is nouveau kexec > support. Another is that the nouveau in the first two RCs of Linux 3.7 > (if I recall) broke my display completely. I probably should clarify that this issue keeps some users from obtaining bug fixes in key components (e.g. their filesystem). That is why I prefer the situation where code lives out-of-tree and works on a variety of kernels over the situation where it is bundled with the kernel for in-tree builds. signature.asc Description: OpenPGP digital signature
Re: [gentoo-dev] Re: [gentoo-kernel] Proper distribution integration of kernel *-sources, patches and configuration.
>> Furthermore, should the kernel kernel choose to engage that out-of-tree >> code, my expectation is that its quality will improve as they do testing >> and write patches. > > What do you mean by this? I probably should have clarified that there was a typo in that. I meant "the kernel team", not "the kernel kernel". I was referring to the Gentoo kernel team. I sent an email to the list with the correction, but I neglected to include you on the CC list by mistake. signature.asc Description: OpenPGP digital signature
Re: [gentoo-dev] Re: [gentoo-kernel] Proper distribution integration of kernel *-sources, patches and configuration.
On 07/01/2013 09:56 PM, Greg KH wrote:> On Mon, Jul 01, 2013 at 09:36:21PM -0400, Richard Yao wrote: >> On 07/01/2013 03:23 PM, Greg KH wrote: >>> On Mon, Jul 01, 2013 at 08:45:16PM +0200, Tom Wijsman wrote: >> Q: What about my stable server? I really don't want to run this >> stuff! >> >> A: These options would depend on !CONFIG_VANILLA or >> CONFIG_EXPERIMENTAL > > What is CONFIG_VANILLA? I don't see that in the upstream kernel tree > at all. > > CONFIG_EXPERIMENTAL is now gone from upstream, so you are going to > have a problem with this. Earlier I mentioned "2) These feature should depend on a non-vanilla / experimental option." which is an option we would introduce under the Gentoo distribution menu section. >>> >>> Distro-specific config options, great :( >>> >>which would be disabled by default, therefore if you keep this >> option the way it is on your stable server; it won't affect you. > > Not always true. Look at aufs as an example. It patches the core > kernel code in ways that are _not_ accepted upstream yet. Now you all > are running that modified code, even if you don't want aufs. Earlier I mentioned "3) The patch should not affect the build by default."; if it does, we have to adjust it to not do that, this is something that can be easily scripted. It's just a matter of embedding each + block in the diff with a config check and updating the counts. >>> >>> Look at aufs as a specific example of why you can't do that, otherwise, >>> don't you think that the aufs developer(s) wouldn't have done so? >> >> I am accquainted with the developer of a stackable filesystem developer. >> According to what he has told me in person offline, the developers on >> the LKML cannot decide on how a stackable filesystem should be >> implemented. I was told three different variations on the design that >> some people liked and others didn't, which ultimately kept the upstream >> kernel from adopting anything. I specifically recall two variations, >> which were doing it as part of the VFS and doing it as part of ext4. If >> you want to criticize stackable filesystems, would you lay out a >> groundwork for getting one implemented upon which people will agree? > > I was not criticising stackable filesystems at all, nor the aufs code, > which I personally run on some of my own systems. > > I agree that the upstream kernel development of stackable filesystems > has been all over the place, with seemingly not much guidance on how to > get things merged properly. Being that I am not part of the subset of > the kernel development community, I am in no position to lay any > groundwork out at all. > > And note, this is totally off-topic from this thread. > > My main point is that if Gentoo wants to start maintaining out-of-tree > kernel patches, and modifying them, the maintenance nightmare is going > to be huge. Been there, done that, have way too many t-shirts > commemorating it, and never want to do it again. > >>> The goal of "don't touch any other kernel code" is a very good one, but >>> not always true for these huge out-of-tree kernel patches. Usually that >>> is the main reason why these patches aren't merged upstream, because >>> those changes are not acceptable. >> >> I was under the impression that there were several reasons for patches >> not being merged upstream: >> >> 1. Lack of signed-off > > No distro will accept code that does not have a signed-off-by line, > otherwise they might get into trouble, as that implies the patch is not > properly licensed, right? That is wrong. Signed-off is an affirmation that the code is under the license, but the absence of signed-off does not imply that the code is not under the license. For example, I doubt you will receive signed-off for PaX/GrSecurity, but is there any doubt that code is under the GPL? To give another example, I doubt that you will receive signed off for code from BSD UNIX, but is there any doubt that code is under the 3-clause BSD license? >> 2. Code drop that no one will maintain > > Agreed. > >> 3. Subsystem maintainers saying no simply because they do not like >> . > > That is very rare and usually the subsystem maintainer can be poked to > resolve this. Past conversations with you have made me reluctant to try, but the next time I see something like this, I will send you an email. >> 4. Risk of patent trolls > > I know of no kernel patches outside of the tree because of this, do you? There is compatibility code for DOS long filenames in fat that is no longer in-tree because of this. >> 5. Actual technical reasons > > That's the majority of the reasons patches aren't accepted. Not all patches that aren't accepted are submitted to be rejected. >> Only some of the patches were rejected. Others were never submitted. The >> PaX/GrSecurity developers prefer their code to stay out-of-tree. As one >> of the people hacking on
Re: [gentoo-dev] Re: [gentoo-kernel] Proper distribution integration of kernel *-sources, patches and configuration.
On Mon, Jul 01, 2013 at 09:36:21PM -0400, Richard Yao wrote: > On 07/01/2013 03:23 PM, Greg KH wrote: > > On Mon, Jul 01, 2013 at 08:45:16PM +0200, Tom Wijsman wrote: > Q: What about my stable server? I really don't want to run this > stuff! > > A: These options would depend on !CONFIG_VANILLA or > CONFIG_EXPERIMENTAL > >>> > >>> What is CONFIG_VANILLA? I don't see that in the upstream kernel tree > >>> at all. > >>> > >>> CONFIG_EXPERIMENTAL is now gone from upstream, so you are going to > >>> have a problem with this. > >> > >> Earlier I mentioned "2) These feature should depend on a non-vanilla / > >> experimental option." which is an option we would introduce under the > >> Gentoo distribution menu section. > > > > Distro-specific config options, great :( > > > which would be disabled by default, therefore if you keep this > option the way it is on your stable server; it won't affect you. > >>> > >>> Not always true. Look at aufs as an example. It patches the core > >>> kernel code in ways that are _not_ accepted upstream yet. Now you all > >>> are running that modified code, even if you don't want aufs. > >> > >> Earlier I mentioned "3) The patch should not affect the build by > >> default."; if it does, we have to adjust it to not do that, this is > >> something that can be easily scripted. It's just a matter of embedding > >> each + block in the diff with a config check and updating the counts. > > > > Look at aufs as a specific example of why you can't do that, otherwise, > > don't you think that the aufs developer(s) wouldn't have done so? > > I am accquainted with the developer of a stackable filesystem developer. > According to what he has told me in person offline, the developers on > the LKML cannot decide on how a stackable filesystem should be > implemented. I was told three different variations on the design that > some people liked and others didn't, which ultimately kept the upstream > kernel from adopting anything. I specifically recall two variations, > which were doing it as part of the VFS and doing it as part of ext4. If > you want to criticize stackable filesystems, would you lay out a > groundwork for getting one implemented upon which people will agree? I was not criticising stackable filesystems at all, nor the aufs code, which I personally run on some of my own systems. I agree that the upstream kernel development of stackable filesystems has been all over the place, with seemingly not much guidance on how to get things merged properly. Being that I am not part of the subset of the kernel development community, I am in no position to lay any groundwork out at all. And note, this is totally off-topic from this thread. My main point is that if Gentoo wants to start maintaining out-of-tree kernel patches, and modifying them, the maintenance nightmare is going to be huge. Been there, done that, have way too many t-shirts commemorating it, and never want to do it again. > > The goal of "don't touch any other kernel code" is a very good one, but > > not always true for these huge out-of-tree kernel patches. Usually that > > is the main reason why these patches aren't merged upstream, because > > those changes are not acceptable. > > I was under the impression that there were several reasons for patches > not being merged upstream: > > 1. Lack of signed-off No distro will accept code that does not have a signed-off-by line, otherwise they might get into trouble, as that implies the patch is not properly licensed, right? > 2. Code drop that no one will maintain Agreed. > 3. Subsystem maintainers saying no simply because they do not like > . That is very rare and usually the subsystem maintainer can be poked to resolve this. > 4. Risk of patent trolls I know of no kernel patches outside of the tree because of this, do you? > 5. Actual technical reasons That's the majority of the reasons patches aren't accepted. > Only some of the patches were rejected. Others were never submitted. The > PaX/GrSecurity developers prefer their code to stay out-of-tree. As one > of the people hacking on ZFSOnLinux, I prefer that the code be > out-of-tree. There's also a small legal issue with the zfs code that might prevent it from being merged :) > That is because fixes for other filesystems are either held back by a > lack of system kernel updates or held hostage by regressions in newer > kernels on certain hardware. I have never heard this being a reason for keeping code upstream, what do you mean by it? > With that said, being in Linus' tree does not make code fall under some > golden standard for quality. There are many significant issues in code > committed to Linus' the kernel, some of which have been problems for > years. Just to name a few: The fact that bugs happen in 16 million lines of kernel code, moving at a rate that is orders of magnitude faster than any other software development project, is not news to anyone, it's a
Re: [gentoo-dev] Re: [gentoo-kernel] Proper distribution integration of kernel *-sources, patches and configuration.
On 07/01/2013 09:36 PM, Richard Yao wrote: > On 07/01/2013 03:23 PM, Greg KH wrote: >> On Mon, Jul 01, 2013 at 08:45:16PM +0200, Tom Wijsman wrote: > Q: What about my stable server? I really don't want to run this > stuff! > > A: These options would depend on !CONFIG_VANILLA or > CONFIG_EXPERIMENTAL What is CONFIG_VANILLA? I don't see that in the upstream kernel tree at all. CONFIG_EXPERIMENTAL is now gone from upstream, so you are going to have a problem with this. >>> >>> Earlier I mentioned "2) These feature should depend on a non-vanilla / >>> experimental option." which is an option we would introduce under the >>> Gentoo distribution menu section. >> >> Distro-specific config options, great :( >> >which would be disabled by default, therefore if you keep this > option the way it is on your stable server; it won't affect you. Not always true. Look at aufs as an example. It patches the core kernel code in ways that are _not_ accepted upstream yet. Now you all are running that modified code, even if you don't want aufs. >>> >>> Earlier I mentioned "3) The patch should not affect the build by >>> default."; if it does, we have to adjust it to not do that, this is >>> something that can be easily scripted. It's just a matter of embedding >>> each + block in the diff with a config check and updating the counts. >> >> Look at aufs as a specific example of why you can't do that, otherwise, >> don't you think that the aufs developer(s) wouldn't have done so? > > I am accquainted with the developer of a stackable filesystem developer. I should probably proofread multiple times before I send emails. Anyway, that should have been: > I am acquainted with the developer of a stackable filesystem. > According to what he has told me in person offline, the developers on > the LKML cannot decide on how a stackable filesystem should be > implemented. I was told three different variations on the design that > some people liked and others didn't, which ultimately kept the upstream > kernel from adopting anything. I specifically recall two variations, > which were doing it as part of the VFS and doing it as part of ext4. If > you want to criticize stackable filesystems, would you lay out a > groundwork for getting one implemented upon which people will agree? > >> The goal of "don't touch any other kernel code" is a very good one, but >> not always true for these huge out-of-tree kernel patches. Usually that >> is the main reason why these patches aren't merged upstream, because >> those changes are not acceptable. > > I was under the impression that there were several reasons for patches > not being merged upstream: > > 1. Lack of signed-off > 2. Code drop that no one will maintain > 3. Subsystem maintainers saying no simply because they do not like > . > 4. Risk of patent trolls > 5. Actual technical reasons > >> So be very careful here, you are messing with things that are rejected >> by upstream. >> >> greg k-h >> > > Only some of the patches were rejected. Others were never submitted. The > PaX/GrSecurity developers prefer their code to stay out-of-tree. As one > of the people hacking on ZFSOnLinux, I prefer that the code be > out-of-tree. That is because fixes for other filesystems are either held > back by a lack of system kernel updates or held hostage by regressions > in newer kernels on certain hardware. > > With that said, being in Linus' tree does not make code fall under some > golden standard for quality. There are many significant issues in code > committed to Linus' the kernel, some of which have been problems for > years. Just to name a few: > > 1. Doing `rm -r /dir` on a directory tree containing millions of inodes > (e.g. ccache) on an ext4 filesystem mounted with discard with the CFQ IO > elevator will cause a system to hang for hours on pre-SATA 3.1 hardware. > This is because TRIM is a non-queued command and is being interleaved > with writes for "fairness". Incidentally, using noop turns a multiple > hour hang into a laggy experience of a few minutes. > > 2. aio_sync() is unimplemented, which means that there is no sane way > for userland software like QEMU and TGT to be both fast and guarantee > data integrity. A single crash and your guest is corrupted. It would > have been better had AIO never been implemented. > > 3. dm-crypt will reorder write requests across flushes. That is because > upon seeing a write, it sends it to a work queue to be processed > asynchronously and upon seeing a flush, it immediately processes it. A > single kernel panic or sudden power loss can damage filesystems stored > on it. > > 4. Under low memory conditions with hundreds of concurrent threads (e.g. > package builds), every thread will enter direct reclaim and there will > be a remarkable drop in system throughput, assuming that the system does > not lockup. There is a fairly substantial amount of time wasted after > o
Re: [gentoo-dev] Re: [gentoo-kernel] Proper distribution integration of kernel *-sources, patches and configuration.
On 07/01/2013 03:23 PM, Greg KH wrote: > On Mon, Jul 01, 2013 at 08:45:16PM +0200, Tom Wijsman wrote: Q: What about my stable server? I really don't want to run this stuff! A: These options would depend on !CONFIG_VANILLA or CONFIG_EXPERIMENTAL >>> >>> What is CONFIG_VANILLA? I don't see that in the upstream kernel tree >>> at all. >>> >>> CONFIG_EXPERIMENTAL is now gone from upstream, so you are going to >>> have a problem with this. >> >> Earlier I mentioned "2) These feature should depend on a non-vanilla / >> experimental option." which is an option we would introduce under the >> Gentoo distribution menu section. > > Distro-specific config options, great :( > which would be disabled by default, therefore if you keep this option the way it is on your stable server; it won't affect you. >>> >>> Not always true. Look at aufs as an example. It patches the core >>> kernel code in ways that are _not_ accepted upstream yet. Now you all >>> are running that modified code, even if you don't want aufs. >> >> Earlier I mentioned "3) The patch should not affect the build by >> default."; if it does, we have to adjust it to not do that, this is >> something that can be easily scripted. It's just a matter of embedding >> each + block in the diff with a config check and updating the counts. > > Look at aufs as a specific example of why you can't do that, otherwise, > don't you think that the aufs developer(s) wouldn't have done so? I am accquainted with the developer of a stackable filesystem developer. According to what he has told me in person offline, the developers on the LKML cannot decide on how a stackable filesystem should be implemented. I was told three different variations on the design that some people liked and others didn't, which ultimately kept the upstream kernel from adopting anything. I specifically recall two variations, which were doing it as part of the VFS and doing it as part of ext4. If you want to criticize stackable filesystems, would you lay out a groundwork for getting one implemented upon which people will agree? > The goal of "don't touch any other kernel code" is a very good one, but > not always true for these huge out-of-tree kernel patches. Usually that > is the main reason why these patches aren't merged upstream, because > those changes are not acceptable. I was under the impression that there were several reasons for patches not being merged upstream: 1. Lack of signed-off 2. Code drop that no one will maintain 3. Subsystem maintainers saying no simply because they do not like . 4. Risk of patent trolls 5. Actual technical reasons > So be very careful here, you are messing with things that are rejected > by upstream. > > greg k-h > Only some of the patches were rejected. Others were never submitted. The PaX/GrSecurity developers prefer their code to stay out-of-tree. As one of the people hacking on ZFSOnLinux, I prefer that the code be out-of-tree. That is because fixes for other filesystems are either held back by a lack of system kernel updates or held hostage by regressions in newer kernels on certain hardware. With that said, being in Linus' tree does not make code fall under some golden standard for quality. There are many significant issues in code committed to Linus' the kernel, some of which have been problems for years. Just to name a few: 1. Doing `rm -r /dir` on a directory tree containing millions of inodes (e.g. ccache) on an ext4 filesystem mounted with discard with the CFQ IO elevator will cause a system to hang for hours on pre-SATA 3.1 hardware. This is because TRIM is a non-queued command and is being interleaved with writes for "fairness". Incidentally, using noop turns a multiple hour hang into a laggy experience of a few minutes. 2. aio_sync() is unimplemented, which means that there is no sane way for userland software like QEMU and TGT to be both fast and guarantee data integrity. A single crash and your guest is corrupted. It would have been better had AIO never been implemented. 3. dm-crypt will reorder write requests across flushes. That is because upon seeing a write, it sends it to a work queue to be processed asynchronously and upon seeing a flush, it immediately processes it. A single kernel panic or sudden power loss can damage filesystems stored on it. 4. Under low memory conditions with hundreds of concurrent threads (e.g. package builds), every thread will enter direct reclaim and there will be a remarkable drop in system throughput, assuming that the system does not lockup. There is a fairly substantial amount of time wasted after one thread finishes direct reclaim in other threads because they will still be performing direct reclaim afterward. 5. The Linux 3.7 nouveau rewrite broke kexec support. The graphics hardware will not reinitialize properly. 6. A throttle mechanism introduced for memory cgroups can cause the system to deadlock whenever it is holding a lock needed for swap and
Re: [gentoo-dev] Re: [gentoo-kernel] Proper distribution integration of kernel *-sources, patches and configuration.
On Mon, 1 Jul 2013 14:24:54 -0700 Greg KH wrote: > > but suppose people > > want BFQ? Why can't we have it in gentoo-sources. It is totally > > disabled by not selecting CONFIG_BFQ. Selecting it is no different > > than emerging pf-sources with the same other options ported over. > > Until you run into a patch that modifies code outside of it's CONFIG_ > option, like the aufs example I pointed out. It would be policy to not add such patches, unless wrapped with config checks by a script; further more, I discussed USE=-experimental with mpagano and he found this separation a good idea, we can split this into a third experimental tarball to not surprise non-Gentoo users as well. mpagano as well as I stand completely behind that gentoo-sources must remain usable for production servers; which this USE flag fulfills, as well as separate from all of this to use live ebuilds in our testing to avoid surprises that even our non-experimental genpatches could bring. (For those in #gentoo-kernel, that conversation happened there) -- With kind regards, Tom Wijsman (TomWij) Gentoo Developer E-mail address : tom...@gentoo.org GPG Public Key : 6D34E57D GPG Fingerprint : C165 AF18 AB4C 400B C3D2 ABF0 95B2 1FCD 6D34 E57D signature.asc Description: PGP signature
Re: [gentoo-dev] Re: [gentoo-kernel] Proper distribution integration of kernel *-sources, patches and configuration.
On 07/01/2013 05:30 PM, Fabio Erculiani wrote: On Mon, Jul 1, 2013 at 11:26 PM, Anthony G. Basile wrote: I'm pretty sure I hit a genuine deadlock with it. I've been trying to reproduce with debugging on but nothing yet. But, having said that: BFQ [Experimtental] This introduced an experimental io scheduler. Have fun with it, but don't dare use it in production else we will laugh. Don't trust outdated documentation. http://algo.ing.unimo.it/people/paolo/disk_sched/sources.php There is nothing about it in the latest BFQ patchset. No, I hit it. Not read about it. Hit it. I can't be 100% sure, but next time it happens, I will be ready with the smoking gun. -- Anthony G. Basile, Ph.D. Gentoo Linux Developer [Hardened] E-Mail: bluen...@gentoo.org GnuPG FP : 1FED FAD9 D82C 52A5 3BAB DC79 9384 FA6E F52D 4BBA GnuPG ID : F52D4BBA
Re: [gentoo-dev] Re: [gentoo-kernel] Proper distribution integration of kernel *-sources, patches and configuration.
On 07/01/2013 05:24 PM, Greg KH wrote: On Mon, Jul 01, 2013 at 05:17:07PM -0400, Anthony G. Basile wrote: On 07/01/2013 03:23 PM, Greg KH wrote: On Mon, Jul 01, 2013 at 08:45:16PM +0200, Tom Wijsman wrote: Q: What about my stable server? I really don't want to run this stuff! A: These options would depend on !CONFIG_VANILLA or CONFIG_EXPERIMENTAL What is CONFIG_VANILLA? I don't see that in the upstream kernel tree at all. CONFIG_EXPERIMENTAL is now gone from upstream, so you are going to have a problem with this. Earlier I mentioned "2) These feature should depend on a non-vanilla / experimental option." which is an option we would introduce under the Gentoo distribution menu section. Distro-specific config options, great :( I'm not sure what you mean by "distro-specific", See later mention of CONFIG_GENTOO_EXPERIMENTAL, that is what I was referring to. but suppose people want BFQ? Why can't we have it in gentoo-sources. It is totally disabled by not selecting CONFIG_BFQ. Selecting it is no different than emerging pf-sources with the same other options ported over. Until you run into a patch that modifies code outside of it's CONFIG_ option, like the aufs example I pointed out. Yeah, that's the situation with hardened-sources and then we are in agreement. If its orthogonal to the rest of the kernel, I maintain that it can safely be included with the appropriate warnings. By your logic, we should not distribut pf-sources either. The truth of the matter is, there are forks of the vanilla kernel out there. Are you suggesting we distribute none of them? That's a total false argument, the discussion here is about our "main" gentoo-kernel tree, not one of our many domain-specific kernel versions that are maintained separately. Now I'm confused because gentoo-sources is gentoo specific. It contains stuff that we need in gentoo but other distros do not need, like our end-to-end support for certain xattr namespaces. If you remove these then we must either 1) maintain a userland which is not in line with other distros or 2) give up on critical features we want in gentoo, like markings on elf object in user.pax.flags and certain caps, as well as in the future preserving selinux labels through emerge. Upstream will not accept them because of "who needs that crap" and we can't give them up without loosing core functionality. Feel free to review those patches but don't ask us to drop them from gentoo-sources because their not in upstream. Only vanilla-sources should be exactly that. upstream vanilla with nothing else. period. NOTE: hardened-sources is its own world. There is not level of turning on/off options that get you back to a vanilla kernel. Agreed, which keeps that from being merged into this tree, hopefully :) Yeah I think everyone is in agreement with that. But it also fits my point about orthogonality above. -- Anthony G. Basile, Ph.D. Gentoo Linux Developer [Hardened] E-Mail: bluen...@gentoo.org GnuPG FP : 1FED FAD9 D82C 52A5 3BAB DC79 9384 FA6E F52D 4BBA GnuPG ID : F52D4BBA
Re: [gentoo-dev] Re: [gentoo-kernel] Proper distribution integration of kernel *-sources, patches and configuration.
On Mon, Jul 1, 2013 at 11:26 PM, Anthony G. Basile wrote: > > > I'm pretty sure I hit a genuine deadlock with it. I've been trying to > reproduce with debugging on but nothing yet. > > But, having said that: > > BFQ [Experimtental] > > This introduced an experimental io scheduler. Have fun with it, but don't > dare use it in production else we will laugh. Don't trust outdated documentation. http://algo.ing.unimo.it/people/paolo/disk_sched/sources.php There is nothing about it in the latest BFQ patchset. > > >> >>> -- >>> Regards, >>> Markos Chandras - Gentoo Linux Developer >>> http://dev.gentoo.org/~hwoarang >>> >> >> > > > -- > Anthony G. Basile, Ph.D. > Gentoo Linux Developer [Hardened] > E-Mail: bluen...@gentoo.org > GnuPG FP : 1FED FAD9 D82C 52A5 3BAB DC79 9384 FA6E F52D 4BBA > GnuPG ID : F52D4BBA > > -- Fabio Erculiani
Re: [gentoo-dev] Re: [gentoo-kernel] Proper distribution integration of kernel *-sources, patches and configuration.
On Mon, Jul 01, 2013 at 05:17:07PM -0400, Anthony G. Basile wrote: > On 07/01/2013 03:23 PM, Greg KH wrote: > >On Mon, Jul 01, 2013 at 08:45:16PM +0200, Tom Wijsman wrote: > Q: What about my stable server? I really don't want to run this > stuff! > > A: These options would depend on !CONFIG_VANILLA or > CONFIG_EXPERIMENTAL > >>>What is CONFIG_VANILLA? I don't see that in the upstream kernel tree > >>>at all. > >>> > >>>CONFIG_EXPERIMENTAL is now gone from upstream, so you are going to > >>>have a problem with this. > >>Earlier I mentioned "2) These feature should depend on a non-vanilla / > >>experimental option." which is an option we would introduce under the > >>Gentoo distribution menu section. > >Distro-specific config options, great :( > > I'm not sure what you mean by "distro-specific", See later mention of CONFIG_GENTOO_EXPERIMENTAL, that is what I was referring to. > but suppose people > want BFQ? Why can't we have it in gentoo-sources. It is totally > disabled by not selecting CONFIG_BFQ. Selecting it is no different > than emerging pf-sources with the same other options ported over. Until you run into a patch that modifies code outside of it's CONFIG_ option, like the aufs example I pointed out. > By your logic, we should not distribut pf-sources either. The truth > of the matter is, there are forks of the vanilla kernel out there. Are > you suggesting we distribute none of them? That's a total false argument, the discussion here is about our "main" gentoo-kernel tree, not one of our many domain-specific kernel versions that are maintained separately. > NOTE: hardened-sources is its own world. There is not level of > turning on/off options that get you back to a vanilla kernel. Agreed, which keeps that from being merged into this tree, hopefully :) thanks, greg k-h
Re: [gentoo-dev] Re: [gentoo-kernel] Proper distribution integration of kernel *-sources, patches and configuration.
On 07/01/2013 04:25 PM, Fabio Erculiani wrote: On Mon, Jul 1, 2013 at 10:14 PM, Markos Chandras wrote: [...] It's really scary to have the BFQ in a stable gentoo-sources ebuild. BFQ is not that scary, it's "just" an iosched (and it's quite easy to write an iosched), what could possibly go wrong? Jokes apart, I've been using it in production for almost 2 years now (can't remember exactly). I'm pretty sure I hit a genuine deadlock with it. I've been trying to reproduce with debugging on but nothing yet. But, having said that: BFQ [Experimtental] This introduced an experimental io scheduler. Have fun with it, but don't dare use it in production else we will laugh. -- Regards, Markos Chandras - Gentoo Linux Developer http://dev.gentoo.org/~hwoarang -- Anthony G. Basile, Ph.D. Gentoo Linux Developer [Hardened] E-Mail: bluen...@gentoo.org GnuPG FP : 1FED FAD9 D82C 52A5 3BAB DC79 9384 FA6E F52D 4BBA GnuPG ID : F52D4BBA
Re: [gentoo-dev] Re: [gentoo-kernel] Proper distribution integration of kernel *-sources, patches and configuration.
On 07/01/2013 03:24 PM, Greg KH wrote: On Mon, Jul 01, 2013 at 02:30:51PM -0400, Anthony G. Basile wrote: Tom, you already know my opinion because we discussed it. I'm all for it. Just a reminder: there's always problems somewhere in the kernel which can be triggered by various options. The kernel is not one big take it or leave it chunk of code, but many chunks selectable by Kconfig with the exception of course of the core. The best we can do wrt to BFQ and other "risky" patches is mark these options as EXPERIMENTAL. I was going to say depend on CONFIG_EXPERIMENTAL in Kconfig, but this is deprecated. See scripts/checkpatch.pl "Use of CONFIG_EXPERIMENTAL is deprecated. For alternatives, see https://lkml.org/lkml/2012/10/23/580"; It's flat out gone now in the 3.10 kernel release, so if you use it, your code just will never be enabled. greg k-h Yeah i noticed that right after sending it. -- Anthony G. Basile, Ph.D. Gentoo Linux Developer [Hardened] E-Mail: bluen...@gentoo.org GnuPG FP : 1FED FAD9 D82C 52A5 3BAB DC79 9384 FA6E F52D 4BBA GnuPG ID : F52D4BBA
Re: [gentoo-dev] Re: [gentoo-kernel] Proper distribution integration of kernel *-sources, patches and configuration.
On 07/01/2013 03:23 PM, Greg KH wrote: On Mon, Jul 01, 2013 at 08:45:16PM +0200, Tom Wijsman wrote: Q: What about my stable server? I really don't want to run this stuff! A: These options would depend on !CONFIG_VANILLA or CONFIG_EXPERIMENTAL What is CONFIG_VANILLA? I don't see that in the upstream kernel tree at all. CONFIG_EXPERIMENTAL is now gone from upstream, so you are going to have a problem with this. Earlier I mentioned "2) These feature should depend on a non-vanilla / experimental option." which is an option we would introduce under the Gentoo distribution menu section. Distro-specific config options, great :( I'm not sure what you mean by "distro-specific", but suppose people want BFQ? Why can't we have it in gentoo-sources. It is totally disabled by not selecting CONFIG_BFQ. Selecting it is no different than emerging pf-sources with the same other options ported over. By your logic, we should not distribut pf-sources either. The truth of the matter is, there are forks of the vanilla kernel out there. Are you suggesting we distribute none of them? NOTE: hardened-sources is its own world. There is not level of turning on/off options that get you back to a vanilla kernel. which would be disabled by default, therefore if you keep this option the way it is on your stable server; it won't affect you. Not always true. Look at aufs as an example. It patches the core kernel code in ways that are _not_ accepted upstream yet. Now you all are running that modified code, even if you don't want aufs. Earlier I mentioned "3) The patch should not affect the build by default."; if it does, we have to adjust it to not do that, this is something that can be easily scripted. It's just a matter of embedding each + block in the diff with a config check and updating the counts. Look at aufs as a specific example of why you can't do that, otherwise, don't you think that the aufs developer(s) wouldn't have done so? The goal of "don't touch any other kernel code" is a very good one, but not always true for these huge out-of-tree kernel patches. Usually that is the main reason why these patches aren't merged upstream, because those changes are not acceptable. So be very careful here, you are messing with things that are rejected by upstream. greg k-h -- Anthony G. Basile, Ph.D. Gentoo Linux Developer [Hardened] E-Mail: bluen...@gentoo.org GnuPG FP : 1FED FAD9 D82C 52A5 3BAB DC79 9384 FA6E F52D 4BBA GnuPG ID : F52D4BBA
Re: [gentoo-dev] Re: [gentoo-kernel] Proper distribution integration of kernel *-sources, patches and configuration.
On Mon, 1 Jul 2013 21:14:45 +0100 Markos Chandras wrote: > And besides that, I am sure that 98% of our users out there do not > know they run a (heavily?) modified upstream kernel when they emerge > the official/supported gentoo-sources. This point I do understand. > The transition between the minimal genpatches to the > "new-shiny-feature-full" was made behind the scenes. This should have > been communicated earlier in time. Apart from the BFQ test in ~3 versions, there did not really happen a transition yet; unless you consider fbcondecor to be part of that, ... > If you ask me, I would prefer if you apply all the 3rd-party patches > conditionally (use flag?, maybe a new gentoo-sources-ng ebuild?) > It's really scary to have the BFQ in a stable gentoo-sources ebuild. An USE flag sounds like something that would make sense; similarly, someone suggested in a mail to cover this as well under a new set in genpatches; currently we do "base" (Linux patches + fixes) and "extra", we could extend this with "experimental" or something like that. An "experimental" USE flag definitely makes sense; and documents where users can find the introduced CONFIG_GENTOO_EXPERIMENTAL kernel option in its USE flag description, as well as a page documenting the patches present in the kernel as well as where they can be found in menuconfig. In the long end, this documentation could be generated from the patches. So, we introduce USE=-experimental in the eclass which users would have to enable to get these patches to even apply; that way users can still see and use a completely untampered kernel, in case they want to want to apply patches that fail against all the experimental patches. -- With kind regards, Tom Wijsman (TomWij) Gentoo Developer E-mail address : tom...@gentoo.org GPG Public Key : 6D34E57D GPG Fingerprint : C165 AF18 AB4C 400B C3D2 ABF0 95B2 1FCD 6D34 E57D signature.asc Description: PGP signature
Re: [gentoo-dev] Re: [gentoo-kernel] Proper distribution integration of kernel *-sources, patches and configuration.
On Mon, Jul 1, 2013 at 10:24 PM, Christoph Junghans wrote: > 2013/7/1 Fabio Erculiani : >> I believe that end users would benefit more from kernel binary ebuilds >> (ebuilds building an actual kernel with an official config), rather >> than all this. > +1 > > The "binary" use flag for sys-kernel/*-sources in Funtoo implements > exactly that. [OT] And why should you throw kernel sources at people as well? Separate pkgs is much better. I don't want to have kernel sources on my servers (for the same reason I don't want to have a compiler, and I've been able to sort this out as well). [/OT] Sorry for the OT. > >> >> -- >> Fabio Erculiani >> > > > > -- > Christoph Junghans > http://dev.gentoo.org/~ottxor/ > -- Fabio Erculiani
Re: [gentoo-dev] Re: [gentoo-kernel] Proper distribution integration of kernel *-sources, patches and configuration.
On Mon, Jul 1, 2013 at 10:14 PM, Markos Chandras wrote: > [...] > It's really scary to have the BFQ in a stable gentoo-sources ebuild. BFQ is not that scary, it's "just" an iosched (and it's quite easy to write an iosched), what could possibly go wrong? Jokes apart, I've been using it in production for almost 2 years now (can't remember exactly). > > -- > Regards, > Markos Chandras - Gentoo Linux Developer > http://dev.gentoo.org/~hwoarang > -- Fabio Erculiani
Re: [gentoo-dev] Re: [gentoo-kernel] Proper distribution integration of kernel *-sources, patches and configuration.
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 07/01/2013 03:55 PM, Fabio Erculiani wrote: > I believe that end users would benefit more from kernel binary ebuilds > (ebuilds building an actual kernel with an official config), rather > than all this. > ++ - -ZC -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.19 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQIcBAEBAgAGBQJR0eW1AAoJEKXdFCfdEflKK4EP/jHyQsECDwVc27eiCglQQ5vV i+FP/cLYkzmNkH8AQwHFq37TVD00XWRiW4vTLu8FrEyu2EIUOze3IwDS9z988rhm TAOcCkoZPF0rA3NoRmapzccSwXMC6W8wOChXSXvRZx+FqYc2kOJ+Yjc7w552ISg3 DQbJlyS/HvspMfAa/zHQYUb2xDX/1HzH1qIdWCBVHo1RpfkmRoblkQ41e7lKnyfA 8oot1XGNc/8jTL5s1rRtpwt0R8Xw2gIEMjSK93B3QLSAzHVkpxmAQ4uELi5m5R5O l+07jit+vqw/cS3/2mSP5nJg8SfOIXNSHLexErUGYg92JsebrlqwmhZjYIHF+ldC h5CbRO30CrcF7kjAjP77gfp0r9tw0AAzOxBqB8cP7/YB62Ee7yAJjXOST7ip7KA5 Bvua4PkgUtyL2fs0orm/Vrkg5ckOVdP92E5oYKFh4gM7UeuDDN2ZJ7fuEKNL9pRC KyCIf3MkQKw18RxO+ecgJlxVN3aj8Qkt4sdJUCZWpiSpry77gVrnciORpHLguKVh yVqhDVkd9b3u2jsnlWs6SeqBQtsChx+xI3OK1lA8kEzNArF6yQxhFxrMeyLUu2Iq LWgBWTuilVIxPAiwDpiWOrXSMjWtjvYGH0vxmfX8wOlvx/L7PA+KEYv+jQeHo59m B1M0btSM6Ut5RJzeS7K8 =cRF9 -END PGP SIGNATURE-
Re: [gentoo-dev] Re: [gentoo-kernel] Proper distribution integration of kernel *-sources, patches and configuration.
2013/7/1 Fabio Erculiani : > I believe that end users would benefit more from kernel binary ebuilds > (ebuilds building an actual kernel with an official config), rather > than all this. +1 The "binary" use flag for sys-kernel/*-sources in Funtoo implements exactly that. > > -- > Fabio Erculiani > -- Christoph Junghans http://dev.gentoo.org/~ottxor/
Re: [gentoo-dev] Re: [gentoo-kernel] Proper distribution integration of kernel *-sources, patches and configuration.
On 1 July 2013 20:09, Matthew Summers wrote: > On Mon, Jul 1, 2013 at 1:56 PM, Tom Wijsman wrote: >> On Mon, 1 Jul 2013 19:38:48 +0100 >> Markos Chandras wrote: >> >>> I certainly don't feel safe anymore running non-upstream code in >>> production boxes. >> >> You don't run it unless you explicitly tick on that you want >> experimental functionality _as well as_ the optional features in >> question; as I said earlier on chat, I don't understand your point here. >> >> If you don't enable them, genpatches is just like it is before; I'm >> not sure why the recommendations should change here, especially with >> vanilla-sources taking a further step away from Gentoo Security and QA. >> > > Tom, > > I think the point was well-made by grehkh. If the patchset patches the > kernel's core, it doesn't matter what CONFIG_* option is set the core > kernel code _has_now_been_changed_. This is the crux of the argument, > I believe. AUFS simply being one example of this. I'm sure there are > others. > > -- > Matthew W. Summers > Gentoo Foundation Inc. > GPG: 111B C438 35FA EDB5 B5D3 736F 45EE 5DC0 0878 9D46 > And besides that, I am sure that 98% of our users out there do not know they run a (heavily?) modified upstream kernel when they emerge the official/supported gentoo-sources. The transition between the minimal genpatches to the "new-shiny-feature-full" was made behind the scenes. This should have been communicated earlier in time. If you ask me, I would prefer if you apply all the 3rd-party patches conditionally (use flag?, maybe a new gentoo-sources-ng ebuild?) It's really scary to have the BFQ in a stable gentoo-sources ebuild. -- Regards, Markos Chandras - Gentoo Linux Developer http://dev.gentoo.org/~hwoarang
Re: [gentoo-dev] Re: [gentoo-kernel] Proper distribution integration of kernel *-sources, patches and configuration.
On Mon, 1 Jul 2013 21:55:23 +0200 Fabio Erculiani wrote: > I believe that end users would benefit more from kernel binary ebuilds > (ebuilds building an actual kernel with an official config), rather > than all this. > There's been a start on this, but we're looking at you now; feel free to communicate with Zero_Chaos about this to clarify why and how you do certain matters, make sure (just ping me on IRC after a conversation with him or so) I can follow along so I can more easily bring this into the Portage tree; as at least someone on the kernel herd needs to understand what is going on with all of that There is a bug filed for this with some work from him. https://bugs.gentoo.org/show_bug.cgi?id=472906 Separate from this whole thread; we definitely have some thoughts about this in mind, and I'll give you one cool thought: Having the kernel build the sources can allow the kernel to automatically rebuild out of kernel modules; eg. `emerge sys-kernel/gentoo-kernel` and it will rebuild nvidia-drivers, virtualbox, ... Of course, for this to work we would need to propagate sub slot rebuilds through virtuals. We thinked through some possible problems and solutions that appear; but I would like to keep them out of this ML thread; as to not mix a thought out idea with one that does some early circles in our minds. On a side note; I'm currently testing genpatches using an ebuild I hacked together myself, allows me to test the way eclass does the patches as well as include some neat local QA checks and testing magic. -- With kind regards, Tom Wijsman (TomWij) Gentoo Developer E-mail address : tom...@gentoo.org GPG Public Key : 6D34E57D GPG Fingerprint : C165 AF18 AB4C 400B C3D2 ABF0 95B2 1FCD 6D34 E57D signature.asc Description: PGP signature
Re: [gentoo-dev] Re: [gentoo-kernel] Proper distribution integration of kernel *-sources, patches and configuration.
On Mon, Jul 1, 2013 at 9:59 PM, Pacho Ramos wrote: > El lun, 01-07-2013 a las 21:55 +0200, Fabio Erculiani escribió: >> I believe that end users would benefit more from kernel binary ebuilds >> (ebuilds building an actual kernel with an official config), rather >> than all this. >> > > I don't see them exclusionary, look different issues to me :/ (with > completely different solutions I think) Of course I'm not saying that they're not orthogonal. OTOH I believe that the complexity of supporting something like this (the original proposal) is not linear. However, I don't see any problem with people implementing it btw... it's their time. > > -- Fabio Erculiani
Re: [gentoo-dev] Re: [gentoo-kernel] Proper distribution integration of kernel *-sources, patches and configuration.
El lun, 01-07-2013 a las 21:55 +0200, Fabio Erculiani escribió: > I believe that end users would benefit more from kernel binary ebuilds > (ebuilds building an actual kernel with an official config), rather > than all this. > I don't see them exclusionary, look different issues to me :/ (with completely different solutions I think)
Re: [gentoo-dev] Re: [gentoo-kernel] Proper distribution integration of kernel *-sources, patches and configuration.
I believe that end users would benefit more from kernel binary ebuilds (ebuilds building an actual kernel with an official config), rather than all this. -- Fabio Erculiani
Re: [gentoo-dev] Re: [gentoo-kernel] Proper distribution integration of kernel *-sources, patches and configuration.
On Mon, 1 Jul 2013 12:33:30 -0700 Greg KH wrote: > On Mon, Jul 01, 2013 at 09:25:42PM +0200, Tom Wijsman wrote: > > On Mon, 1 Jul 2013 14:09:57 -0500 > > Matthew Summers wrote: > > > If the patchset patches the kernel's core, it doesn't matter what > > > CONFIG_* option is set the core kernel code > > > _has_now_been_changed_. This is the crux of the argument, I > > > believe. AUFS simply being one example of this. I'm sure there > > > are others. > > > > As per my response to that point, this statement is no longer true. > > > > Let me re-iterate it here: > > > > Earlier I mentioned "3) The patch should not affect the build by > > default."; if it does, we have to adjust it to not do that, this is > > something that can be easily scripted. It's just a matter of > > embedding each + block in the diff with a config check and updating > > the counts. > > Wonderful, now you are maintaining a patch that looks nothing like the > one created by the original developers, nor tested by anyone else > other than gentoo developers. I can convert the original developer's patch whenever he updates it; or on top of that, write a script to generate the original patch back. > Playing fast-and-loose with kernel patches is a fun thing to do, but > really, why? Users love doing this type of thing, but the > interactions of different kernel patches with core subsystems is > almost always a non-trivial thing. Our main intent is to provide them and deduplicate work; if users have a problem with one of the patches and it isn't clear to us, we can forward them to the authors. Nothing in this proposal inherits that we must support the patches; especially not, since they are "experimental". > I'm not saying not to do this, but consider this a friendly warning > that this is going to be a MAJOR pain to maintain and debug over the > long-run. Maybe, maybe not; plan is to do a slow start, congestion avoidance. :) > But hey, what do I know? It's not like I've ever done this before and > had the experience of the resulting fall-out that took years to > recover from on user's production systems, causing a number of > enterprise Linux companies to swear that they would never do this > type of thing again... I covered this in the conditions as well as the F.A.Q.; feel free to read about it again, this does not affect them unless they explicitly want them to enable CONFIG_GENTOO_EXPERIMENTAL which help includes a warning; thinking it through, we might even suffix -exp to version. > Personally, I wish you luck, it will push the sane users to the > vanilla-sources tree, which I strongly encourage :) I will push them to keep CONFIG_GENTOO_EXPERIMENTAL disabled. :) > greg "kids, get off my lawn!" k-h Gentoo (Penguin) and Larry do not necessarily like grass; they might like ice, fire, water, earth, ... whatever their appetite craves. -- With kind regards, Tom Wijsman (TomWij) Gentoo Developer E-mail address : tom...@gentoo.org GPG Public Key : 6D34E57D GPG Fingerprint : C165 AF18 AB4C 400B C3D2 ABF0 95B2 1FCD 6D34 E57D signature.asc Description: PGP signature
Re: [gentoo-dev] Re: [gentoo-kernel] Proper distribution integration of kernel *-sources, patches and configuration.
On Mon, 1 Jul 2013 12:24:36 -0700 Greg KH wrote: > On Mon, Jul 01, 2013 at 02:30:51PM -0400, Anthony G. Basile wrote: > > Tom, you already know my opinion because we discussed it. I'm all > > for it. Just a reminder: there's always problems somewhere in the > > kernel which can be triggered by various options. The kernel is not > > one big take it or leave it chunk of code, but many chunks > > selectable by Kconfig with the exception of course of the core. The > > best we can do wrt to BFQ and other "risky" patches is mark these > > options as EXPERIMENTAL. I was going to say depend on > > CONFIG_EXPERIMENTAL in Kconfig, but this is deprecated. See > > scripts/checkpatch.pl > > > > "Use of CONFIG_EXPERIMENTAL is deprecated. For alternatives, see > > https://lkml.org/lkml/2012/10/23/580"; > > It's flat out gone now in the 3.10 kernel release, so if you use it, > your code just will never be enabled. As I replied earlier and tried to make clear in my first post, this would be our own config variable; naming it CONFIG_GENTOO_EXPERIMENTAL for that matter as to not confuse people. -- With kind regards, Tom Wijsman (TomWij) Gentoo Developer E-mail address : tom...@gentoo.org GPG Public Key : 6D34E57D GPG Fingerprint : C165 AF18 AB4C 400B C3D2 ABF0 95B2 1FCD 6D34 E57D signature.asc Description: PGP signature
Re: [gentoo-dev] Re: [gentoo-kernel] Proper distribution integration of kernel *-sources, patches and configuration.
On Mon, Jul 01, 2013 at 09:25:42PM +0200, Tom Wijsman wrote: > On Mon, 1 Jul 2013 14:09:57 -0500 > Matthew Summers wrote: > > If the patchset patches the kernel's core, it doesn't matter what > > CONFIG_* option is set the core kernel code _has_now_been_changed_. > > This is the crux of the argument, I believe. AUFS simply being one > > example of this. I'm sure there are others. > > As per my response to that point, this statement is no longer true. > > Let me re-iterate it here: > > Earlier I mentioned "3) The patch should not affect the build by > default."; if it does, we have to adjust it to not do that, this is > something that can be easily scripted. It's just a matter of embedding > each + block in the diff with a config check and updating the counts. Wonderful, now you are maintaining a patch that looks nothing like the one created by the original developers, nor tested by anyone else other than gentoo developers. There's a reason that no other distro does this. Playing fast-and-loose with kernel patches is a fun thing to do, but really, why? Users love doing this type of thing, but the interactions of different kernel patches with core subsystems is almost always a non-trivial thing. I'm not saying not to do this, but consider this a friendly warning that this is going to be a MAJOR pain to maintain and debug over the long-run. But hey, what do I know? It's not like I've ever done this before and had the experience of the resulting fall-out that took years to recover from on user's production systems, causing a number of enterprise Linux companies to swear that they would never do this type of thing again... Personally, I wish you luck, it will push the sane users to the vanilla-sources tree, which I strongly encourage :) greg "kids, get off my lawn!" k-h
Re: [gentoo-dev] Re: [gentoo-kernel] Proper distribution integration of kernel *-sources, patches and configuration.
On Mon, 1 Jul 2013 12:23:24 -0700 Greg KH wrote: > > Earlier I mentioned "3) The patch should not affect the build by > > default."; if it does, we have to adjust it to not do that, this is > > something that can be easily scripted. It's just a matter of > > embedding each + block in the diff with a config check and updating > > the counts. > > Look at aufs as a specific example of why you can't do that, > otherwise, don't you think that the aufs developer(s) wouldn't have > done so? > > The goal of "don't touch any other kernel code" is a very good one, > but not always true for these huge out-of-tree kernel patches. > Usually that is the main reason why these patches aren't merged > upstream, because those changes are not acceptable. > > So be very careful here, you are messing with things that are rejected > by upstream. It sounds to me like some of those developers don't care about providing an option yet; because they would introduce it as an afterthought, based on its need and not just because they can. I don't see why this would be problematic, embedding it in config checks as well as checking whether the patch introduces non-conditional code are trivial to implement; why do you think this is not always true? Thank you for making us aware, feedback on this is nice. -- With kind regards, Tom Wijsman (TomWij) Gentoo Developer E-mail address : tom...@gentoo.org GPG Public Key : 6D34E57D GPG Fingerprint : C165 AF18 AB4C 400B C3D2 ABF0 95B2 1FCD 6D34 E57D signature.asc Description: PGP signature
Re: [gentoo-dev] Re: [gentoo-kernel] Proper distribution integration of kernel *-sources, patches and configuration.
On Mon, 1 Jul 2013 14:09:57 -0500 Matthew Summers wrote: > I think the point was well-made by grehkh. You missed my response to that point. > If the patchset patches the kernel's core, it doesn't matter what > CONFIG_* option is set the core kernel code _has_now_been_changed_. > This is the crux of the argument, I believe. AUFS simply being one > example of this. I'm sure there are others. As per my response to that point, this statement is no longer true. Let me re-iterate it here: Earlier I mentioned "3) The patch should not affect the build by default."; if it does, we have to adjust it to not do that, this is something that can be easily scripted. It's just a matter of embedding each + block in the diff with a config check and updating the counts. -- With kind regards, Tom Wijsman (TomWij) Gentoo Developer E-mail address : tom...@gentoo.org GPG Public Key : 6D34E57D GPG Fingerprint : C165 AF18 AB4C 400B C3D2 ABF0 95B2 1FCD 6D34 E57D signature.asc Description: PGP signature
Re: [gentoo-dev] Re: [gentoo-kernel] Proper distribution integration of kernel *-sources, patches and configuration.
On Mon, Jul 01, 2013 at 02:30:51PM -0400, Anthony G. Basile wrote: > Tom, you already know my opinion because we discussed it. I'm all > for it. Just a reminder: there's always problems somewhere in the > kernel which can be triggered by various options. The kernel is not > one big take it or leave it chunk of code, but many chunks > selectable by Kconfig with the exception of course of the core. The > best we can do wrt to BFQ and other "risky" patches is mark these > options as EXPERIMENTAL. I was going to say depend on > CONFIG_EXPERIMENTAL in Kconfig, but this is deprecated. See > scripts/checkpatch.pl > > "Use of CONFIG_EXPERIMENTAL is deprecated. For alternatives, see > https://lkml.org/lkml/2012/10/23/580"; It's flat out gone now in the 3.10 kernel release, so if you use it, your code just will never be enabled. greg k-h
Re: [gentoo-dev] Re: [gentoo-kernel] Proper distribution integration of kernel *-sources, patches and configuration.
On Mon, Jul 01, 2013 at 08:45:16PM +0200, Tom Wijsman wrote: > > > Q: What about my stable server? I really don't want to run this > > > stuff! > > > > > > A: These options would depend on !CONFIG_VANILLA or > > > CONFIG_EXPERIMENTAL > > > > What is CONFIG_VANILLA? I don't see that in the upstream kernel tree > > at all. > > > > CONFIG_EXPERIMENTAL is now gone from upstream, so you are going to > > have a problem with this. > > Earlier I mentioned "2) These feature should depend on a non-vanilla / > experimental option." which is an option we would introduce under the > Gentoo distribution menu section. Distro-specific config options, great :( > > >which would be disabled by default, therefore if you keep this > > > option the way it is on your stable server; it won't affect you. > > > > Not always true. Look at aufs as an example. It patches the core > > kernel code in ways that are _not_ accepted upstream yet. Now you all > > are running that modified code, even if you don't want aufs. > > Earlier I mentioned "3) The patch should not affect the build by > default."; if it does, we have to adjust it to not do that, this is > something that can be easily scripted. It's just a matter of embedding > each + block in the diff with a config check and updating the counts. Look at aufs as a specific example of why you can't do that, otherwise, don't you think that the aufs developer(s) wouldn't have done so? The goal of "don't touch any other kernel code" is a very good one, but not always true for these huge out-of-tree kernel patches. Usually that is the main reason why these patches aren't merged upstream, because those changes are not acceptable. So be very careful here, you are messing with things that are rejected by upstream. greg k-h
Re: [gentoo-dev] Re: [gentoo-kernel] Proper distribution integration of kernel *-sources, patches and configuration.
On Mon, Jul 1, 2013 at 1:56 PM, Tom Wijsman wrote: > On Mon, 1 Jul 2013 19:38:48 +0100 > Markos Chandras wrote: > >> I certainly don't feel safe anymore running non-upstream code in >> production boxes. > > You don't run it unless you explicitly tick on that you want > experimental functionality _as well as_ the optional features in > question; as I said earlier on chat, I don't understand your point here. > > If you don't enable them, genpatches is just like it is before; I'm > not sure why the recommendations should change here, especially with > vanilla-sources taking a further step away from Gentoo Security and QA. > Tom, I think the point was well-made by grehkh. If the patchset patches the kernel's core, it doesn't matter what CONFIG_* option is set the core kernel code _has_now_been_changed_. This is the crux of the argument, I believe. AUFS simply being one example of this. I'm sure there are others. -- Matthew W. Summers Gentoo Foundation Inc. GPG: 111B C438 35FA EDB5 B5D3 736F 45EE 5DC0 0878 9D46
Re: [gentoo-dev] Re: [gentoo-kernel] Proper distribution integration of kernel *-sources, patches and configuration.
On Mon, 01 Jul 2013 14:30:51 -0400 "Anthony G. Basile" wrote: > I was going to say depend on CONFIG_EXPERIMENTAL in > Kconfig, but this is deprecated. See scripts/checkpatch.pl Yes, I think it wasn't clear from my first post; but the intention was to introduce such option under the Gentoo distribution menu section. I think I forgot to mention that near the end. I'm well aware, which is why we should proceed slowly; since I will have an away period soon, I don't plan to implement this this month and set up the organization before starting anything to ensure we do this properly. -- With kind regards, Tom Wijsman (TomWij) Gentoo Developer E-mail address : tom...@gentoo.org GPG Public Key : 6D34E57D GPG Fingerprint : C165 AF18 AB4C 400B C3D2 ABF0 95B2 1FCD 6D34 E57D signature.asc Description: PGP signature
Re: [gentoo-dev] Re: [gentoo-kernel] Proper distribution integration of kernel *-sources, patches and configuration.
On Mon, 1 Jul 2013 19:38:48 +0100 Markos Chandras wrote: > I certainly don't feel safe anymore running non-upstream code in > production boxes. You don't run it unless you explicitly tick on that you want experimental functionality _as well as_ the optional features in question; as I said earlier on chat, I don't understand your point here. If you don't enable them, genpatches is just like it is before; I'm not sure why the recommendations should change here, especially with vanilla-sources taking a further step away from Gentoo Security and QA. -- With kind regards, Tom Wijsman (TomWij) Gentoo Developer E-mail address : tom...@gentoo.org GPG Public Key : 6D34E57D GPG Fingerprint : C165 AF18 AB4C 400B C3D2 ABF0 95B2 1FCD 6D34 E57D signature.asc Description: PGP signature
Re: [gentoo-dev] Re: [gentoo-kernel] Proper distribution integration of kernel *-sources, patches and configuration.
On Mon, 1 Jul 2013 11:17:49 -0700 Greg KH wrote: > > Meet CONFIG_DEVTMPFS; ... > > This is not the only build option that users must enable to get a > booting system by far. So why single this one out? Because it is an example. Later on I explicitly mention "There are a small set of other variables in this nature, ..."; I'm talking about configuration options not present in the defconfig, specifically those that everyone who runs a Gentoo based system (@system set + stage3) wants to have enabled. > > Q: What about my stable server? I really don't want to run this > > stuff! > > > > A: These options would depend on !CONFIG_VANILLA or > > CONFIG_EXPERIMENTAL > > What is CONFIG_VANILLA? I don't see that in the upstream kernel tree > at all. > > CONFIG_EXPERIMENTAL is now gone from upstream, so you are going to > have a problem with this. Earlier I mentioned "2) These feature should depend on a non-vanilla / experimental option." which is an option we would introduce under the Gentoo distribution menu section. > >which would be disabled by default, therefore if you keep this > > option the way it is on your stable server; it won't affect you. > > Not always true. Look at aufs as an example. It patches the core > kernel code in ways that are _not_ accepted upstream yet. Now you all > are running that modified code, even if you don't want aufs. Earlier I mentioned "3) The patch should not affect the build by default."; if it does, we have to adjust it to not do that, this is something that can be easily scripted. It's just a matter of embedding each + block in the diff with a config check and updating the counts. > >In other words, genpatches stay as stable as before; unless you > >explicitly toggle options that intentionally make it unstable. > > As pointed out above, this is not always true. Under the above condition (3), it is always true. > Also, as others stated, the "hardened" patches also don't always only > touch areas covered by non-config-selected portions of the kernel. Yes, it seems wise to keep hardened-sources separated. > Mix and match your external kernel patches at your own risk, what you > are suggesting does make it "easy" for users, but I bet it will be a > huge support issue for the already-overworked gentoo kernel > developers, the combinations just are too big to test and guarantee > working. I'm waiting for you to push new kernels to kernel.org. Joking aside; users are doing this anyway, it is better to know about it. Who knows some of the bugs we have is the result of our unawareness. Agreed, we have to watch out how far we push this though; which is why we should start with just those patches that appear the most in other sources, carefully meeting our goal over time. Not mimic geek-sources all at once, I believe it took him some time to reach that point... -- With kind regards, Tom Wijsman (TomWij) Gentoo Developer E-mail address : tom...@gentoo.org GPG Public Key : 6D34E57D GPG Fingerprint : C165 AF18 AB4C 400B C3D2 ABF0 95B2 1FCD 6D34 E57D signature.asc Description: PGP signature
Re: [gentoo-dev] Re: [gentoo-kernel] Proper distribution integration of kernel *-sources, patches and configuration.
On 1 July 2013 19:17, Greg KH wrote: > > greg "stick to the vanilla-sources" k-h > Before these changes were introduced, the gentoo-sources and vanilla-sources were quite similar in the sense that genpatches used to contain patches already in Linus' tree. However, given that gentoo-sources will become heavily patched by external 3rd-party patches, it might makes more sense to recommend vanilla-sources to our users (handbook, etc). I certainly don't feel safe anymore running non-upstream code in production boxes. -- Regards, Markos Chandras - Gentoo Linux Developer http://dev.gentoo.org/~hwoarang
Re: [gentoo-dev] Re: [gentoo-kernel] Proper distribution integration of kernel *-sources, patches and configuration.
On 07/01/2013 11:20 AM, Jeff Horelick wrote: On 1 July 2013 10:41, Tom Wijsman wrote: Hello Please reply to gentoo-dev in case ML daemon changes Reply-To. ### TL; DR ### By introducing feature patches which menu options are disabled by default to genpatches, we can deduplicate *-sources maintainers as well as large groups of users work. By introducing a distribution section in the menuconfig, we can provide options that enable minimal Gentoo requirements by default (DEVTMPFS, making Gentoo systems bootable since an udev release a long time ago) and other distribution stuff. I really like this idea. geek-sources has appealed to me massively the past few months, but i didn't want to risk stability and go with a kernel from a unofficial overlay. I can not wait to see this in gentoo-sources and I fully support this idea. Tom, you already know my opinion because we discussed it. I'm all for it. Just a reminder: there's always problems somewhere in the kernel which can be triggered by various options. The kernel is not one big take it or leave it chunk of code, but many chunks selectable by Kconfig with the exception of course of the core. The best we can do wrt to BFQ and other "risky" patches is mark these options as EXPERIMENTAL. I was going to say depend on CONFIG_EXPERIMENTAL in Kconfig, but this is deprecated. See scripts/checkpatch.pl "Use of CONFIG_EXPERIMENTAL is deprecated. For alternatives, see https://lkml.org/lkml/2012/10/23/580"; --Tony -- Anthony G. Basile, Ph.D. Gentoo Linux Developer [Hardened] E-Mail: bluen...@gentoo.org GnuPG FP : 1FED FAD9 D82C 52A5 3BAB DC79 9384 FA6E F52D 4BBA GnuPG ID : F52D4BBA
[gentoo-dev] Re: [gentoo-kernel] Proper distribution integration of kernel *-sources, patches and configuration.
On Mon, Jul 01, 2013 at 04:41:49PM +0200, Tom Wijsman wrote: > This problem is not only visible for patches, but also in the config. > > Meet CONFIG_DEVTMPFS; forget to enable it, greet a failing boot. We're > telling users to enable it in some places, in the handbook it's a single > line you must read, on the Wiki it's kind of missing unless you are > luckily on the right page, on the Quick Install book it is missing too. This is not the only build option that users must enable to get a booting system by far. So why single this one out? > Q: I don't want feature X? Please don't add the patch! > > A: It's optional, don't enable it in your menu config. > > Q: What about my stable server? I really don't want to run this stuff! > > A: These options would depend on !CONFIG_VANILLA or CONFIG_EXPERIMENTAL What is CONFIG_VANILLA? I don't see that in the upstream kernel tree at all. CONFIG_EXPERIMENTAL is now gone from upstream, so you are going to have a problem with this. >which would be disabled by default, therefore if you keep this option >the way it is on your stable server; it won't affect you. Not always true. Look at aufs as an example. It patches the core kernel code in ways that are _not_ accepted upstream yet. Now you all are running that modified code, even if you don't want aufs. Note, I'm just using aufs as an example here, I'm not commenting on the quality of the code, or why it is modifying the core kernel at all, I happen to run it on some of my own servers, but your feelings might differ. >In other words, genpatches stay as stable as before; unless you >explicitly toggle options that intentionally make it unstable. As pointed out above, this is not always true. Also, as others stated, the "hardened" patches also don't always only touch areas covered by non-config-selected portions of the kernel. Mix and match your external kernel patches at your own risk, what you are suggesting does make it "easy" for users, but I bet it will be a huge support issue for the already-overworked gentoo kernel developers, the combinations just are too big to test and guarantee working. good luck, greg "stick to the vanilla-sources" k-h
Re: [gentoo-dev] Proper distribution integration of kernel *-sources, patches and configuration.
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 07/01/2013 01:35 PM, Tom Wijsman wrote: > On Mon, 01 Jul 2013 12:20:09 -0400 > "Rick \"Zero_Chaos\" Farina" wrote: > >> Some patches are reasonably easy to combine, such as genpatches and >> aufs. Some patches are difficult to combine, such as hardened and *. >> When you combine hardened patches and aufs (for example) you need >> extra patches. I would be THRILLED to see the number of sources cut >> down, but hardened-sources must be it's own thing (that said, I'll >> personally maintain the aufs patches for hardened if they wanted to >> add a USE=aufs flag). > > Yes, gave it as an quick example but I indeed remember from going > through the sources ebuilds that hardened ebuilds do quite some things. > I think the downside from extending genpatches is that hardened-sources > can no longer rely on it, but we'll have to see that as we go forward. > > I don't think that apart from hardened the optional patches on their own > are hard to combine; they each have their own separate goal, I don't > see them conflict on anything. If it happens once in a while, we can > still maintain them to work together. Hardened has K_WANT_GENPATCHES="base" which means it already doesn't take the extra patches. We could either introduce a new flag for your patches like K_WANT_GENPATCHES="base extra geek" or more likely make each one with their own name so that hardened et al can take what they like and leave the rest. > > Also note that I do not plan to introduce any USE flags, since that > would duplicate the options to be listed in the kernel menuconfig. Good point. Thanks, Zero > >> If users want a vanilla kernel, they want vanilla-sources. Nothing >> about that should change. I don't feel that it would be honest to >> add a vanilla use flag to gentoo-sources as in no reality are those >> vanilla. > > Apart from the changes discussed on the gentoo-kernel ML, nothing else > there will change. You can read the thread as well as the summary; if > you disagree, you're welcome to join the discussion there. > > http://thread.gmane.org/gmane.linux.gentoo.kernel/697 > > http://thread.gmane.org/gmane.linux.gentoo.kernel/697/focus=730 > > But yes, apart from that, vanilla-sources will give you vanilla kernel. > > -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.19 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQIcBAEBAgAGBQJR0cHKAAoJEKXdFCfdEflKwOYQAIArPK7SsJ6M9BFCuhf/bFOg yZfVpSRR8UJgl7as6FsVg4q48NFuFh3lm8PQK792IraccXkYjk/kBGY6IyDt3rUY yXHberr53cBZKPC0PPVzDo7ER73GWkobrd+vDrtRHeTJEaPQknvOEmQFBnjY181o 8uTqQi2VtTnWPP2PeIkqdobkasUcf5lDDXdrvX+ipN+oOSSZ0VJK+XfdlstgrRnH o7sESjKm7RbvxQFizzGuE7gOh9GFtIY92zpQVUO4L4P/L5NrGz0yqyor0WYKuUhN dZ3k84FQ5SDyKCdCMq/JPKS8jj47gFIkZwfArwKNsRsxkBtcsFHQuj2VXSx0MEcp aKv5FfZCj4iSUAg2uwKVfVonyn5qt73Fm+XquxjfbEaT4oTq0FFCL5zjCuf1Zzpt 3/VOer5N5xu5gY0y6Yt5w7ionHLAFWqXgJF7s/sC7L9eJNHT3XiQtZSPxLGeAkG8 beM9PNt7fdYzQAudbi6NZiJ35ZwZQrQfKEAc16hbcH0qDd7ndTWqg0nELX3ulL2z FbeCMM0J+tmZ8lEgRriUL7Ki/een1DJX4eCQhYbKIMYKdxDeEMkbZ8N/T3dAWtxm IivIs4tTK9EWBF6+8kmUINszCQBsLI6P50TzocqHV+Tj/RyCmeJnOia7DLTD0HGg 53QUb/jcDjkTF34HbUdG =gsg9 -END PGP SIGNATURE-
Re: [gentoo-dev] Proper distribution integration of kernel *-sources, patches and configuration.
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Mon, 01 Jul 2013 12:20:09 -0400 "Rick \"Zero_Chaos\" Farina" wrote: > Some patches are reasonably easy to combine, such as genpatches and > aufs. Some patches are difficult to combine, such as hardened and *. > When you combine hardened patches and aufs (for example) you need > extra patches. I would be THRILLED to see the number of sources cut > down, but hardened-sources must be it's own thing (that said, I'll > personally maintain the aufs patches for hardened if they wanted to > add a USE=aufs flag). Yes, gave it as an quick example but I indeed remember from going through the sources ebuilds that hardened ebuilds do quite some things. I think the downside from extending genpatches is that hardened-sources can no longer rely on it, but we'll have to see that as we go forward. I don't think that apart from hardened the optional patches on their own are hard to combine; they each have their own separate goal, I don't see them conflict on anything. If it happens once in a while, we can still maintain them to work together. Also note that I do not plan to introduce any USE flags, since that would duplicate the options to be listed in the kernel menuconfig. > If users want a vanilla kernel, they want vanilla-sources. Nothing > about that should change. I don't feel that it would be honest to > add a vanilla use flag to gentoo-sources as in no reality are those > vanilla. Apart from the changes discussed on the gentoo-kernel ML, nothing else there will change. You can read the thread as well as the summary; if you disagree, you're welcome to join the discussion there. http://thread.gmane.org/gmane.linux.gentoo.kernel/697 http://thread.gmane.org/gmane.linux.gentoo.kernel/697/focus=730 But yes, apart from that, vanilla-sources will give you vanilla kernel. - -- With kind regards, Tom Wijsman (TomWij) Gentoo Developer E-mail address : tom...@gentoo.org GPG Public Key : 6D34E57D GPG Fingerprint : C165 AF18 AB4C 400B C3D2 ABF0 95B2 1FCD 6D34 E57D -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.20 (GNU/Linux) iQEcBAEBAgAGBQJR0b3UAAoJEJWyH81tNOV9UAMIAMSQy/EPWr9l4JCVMaB3x7bb ezv8vrmSDxFUB30mX9cPPAvfpaeWSFvRCpvPfEsKkHAcTJomSfP7PxgEVKLA4jT2 TwoovBQsADEowAIgUHFr9lqIrE9HQ0nik89bNz1HxkbUl4uQijnnfcLTu/6WpdGJ aMAxImTXQc1Py7w+RNcy9fR0pXh6zXx23CWNVMvn2hb+wGkQDbzy4gOeHFok2i9F 3B3ZjLNUdY2agEBpo0AqF0hwSPHUDbrteFVwJKjauYTyNA8Ofc5OhRnpGHEbq5/4 mB4J2R/MnsukXNeNJUgLzjqSykKChdNbImLl6OXXXl1PyinjuAdBeC7SoUV3DDg= =5OIS -END PGP SIGNATURE-
Re: [gentoo-dev] Proper distribution integration of kernel *-sources, patches and configuration.
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 07/01/2013 06:20 PM, Rick "Zero_Chaos" Farina wrote: > On 07/01/2013 10:41 AM, Tom Wijsman wrote: > >> What does a patch introducing new features really do? Or rather, >> what should it do when we add it? Let me summarize: > >> 1) The features should be disabled by default. 2) These feature >> should depend on a non-vanilla / experimental option. > If users want a vanilla kernel, they want vanilla-sources. > Nothing about that should change. I don't feel that it would be > honest to add a vanilla use flag to gentoo-sources as in no reality > are those vanilla. > Agreed, some of this sounds interesting (although I don't care as a user), but whatever you do... don't mess with vanilla-sources. I stopped using gentoo-sources, because some random hacks broke some network drivers once and I don't want a hackjob on vanilla-sources... optional or not. -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.20 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQEcBAEBAgAGBQJR0a5AAAoJEFpvPKfnPDWzf4wIAIhfHvIJmqU3jfIVvGlNL5QR Z8SM4jyw95ehtTMWDdpnEdLUuW7yu8pK9K9N9cXSchvQ3GJfjxJTI/v7RSI+WPBN NrTgfKYF1KGg6jNQXjuiyG5QwV9faE7zEZ8unDRyxX0EhWyiWACjuSzBdpzS6Nhm QcDCEzzuXNtPR44pqfDQ3DqqRU+aUAE7juM+Yd146x3CFDE8vvVuvuGYnXhVczQ8 vkfdqLNLMamIROWapV4HG8p2NOzDbPjbUcMB7uBsP8DPm/HjOQRNxyY0yCD38hq3 4RJFZz7RPTjWQ2p8PacHKZ4Rye6EY7W/x6JTVqmh3zbc+6tm6q7kKtMyn1L03Bw= =ds3y -END PGP SIGNATURE-
Re: [gentoo-dev] Proper distribution integration of kernel *-sources, patches and configuration.
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 07/01/2013 10:41 AM, Tom Wijsman wrote: > Hello > > > Please reply to gentoo-dev in case ML daemon changes Reply-To. > > > ### TL; DR ### > > By introducing feature patches which menu options are disabled by > default to genpatches, we can deduplicate *-sources maintainers as well > as large groups of users work. By introducing a distribution section > in the menuconfig, we can provide options that enable minimal Gentoo > requirements by default (DEVTMPFS, making Gentoo systems bootable since > an udev release a long time ago) and other distribution stuff. > > > ### Proper distribution integration of kernel *-sources, patches ... ### > > Gentoo is a distribution; but what is a distribution that doesn't > properly integrate what it provides, but instead expects its users to > do so, needlessly duplicating work amongst its maintainers and users. > > Let's say I want genpatches, aufs and TuxOnIce; closest candidates: > > - sys-kernel/aufs-sources: genpatches, aufs > - sys-kernel/pf-sources: genpatches, CK, BFQ, BFQ, TuxOnIce, UKSM > - sys-kernel/tuxonice-sources: genpatches, TuxOnIce > > What do I do? Take one (eg. aufs) and apply the other (eg. TuxOnIce)? > What if I want to add another common patchset to it? Hardened perhaps? Some patches are reasonably easy to combine, such as genpatches and aufs. Some patches are difficult to combine, such as hardened and *. When you combine hardened patches and aufs (for example) you need extra patches. I would be THRILLED to see the number of sources cut down, but hardened-sources must be it's own thing (that said, I'll personally maintain the aufs patches for hardened if they wanted to add a USE=aufs flag). > > You can see, some of these sources (like pf-sources) already attempt to > do so; with pf-sources in mind, why do we even have ck-sources, > tuxonice-sources in the Portage tree? Just to duplicate work? > > I think we should trim down on the amount of *-sources and combine > multiple patches into genpatches. Take an example of geek-sources > which does all this without a problem; I don't really like the approach > with USE flags used there, as the menuconfig can just cover that. > > https://github.com/init6/init_6/wiki/geek-sources > > What does a patch introducing new features really do? Or rather, what > should it do when we add it? Let me summarize: > > 1) The features should be disabled by default. > 2) These feature should depend on a non-vanilla / experimental option. If users want a vanilla kernel, they want vanilla-sources. Nothing about that should change. I don't feel that it would be honest to add a vanilla use flag to gentoo-sources as in no reality are those vanilla. Thanks, Zero > 3) The patch should not affect the build by default. > 4) The user can optionally enable the feature. > > So, in genpatches, since 3.9.7, BFQ was added to try this out (except > 2.). Ensured it to be disabled by default, did not affect the build for > anyone that does not enable it and the users have been enabling and > using it on their own. So, does it differentiate more from vanilla? No. > > This helps users as well as maintainers to not have to apply BFQ on > their own, they simply have to tick a config option and they are set. > If all patches that introduce new features are added in this fashion, > it spares large groups of users from having to do this on their own; we > can also deduplicate the work in the Portage tree this way. > > > ### ... and configuration. ### > > This problem is not only visible for patches, but also in the config. > > Meet CONFIG_DEVTMPFS; forget to enable it, greet a failing boot. We're > telling users to enable it in some places, in the handbook it's a single > line you must read, on the Wiki it's kind of missing unless you are > luckily on the right page, on the Quick Install book it is missing too. > > So, we are currently providing a configuration that expects anyone to > enable CONFIG_DEVTMPFS. But you have to remember that it need to make > sure you read about it or enable it from the udev upgrade a while ago > if you decide to start from a fresh config or are installing without > that handbook you kind of have memorized. > > Searching for CONFIG_DEVTMPFS in the forums and #gentoo logs shows that > this is quite often suggested as a fix and quite often actually fixes > an user's boot. Why duplicate telling users to do that if we can do it? > > There are a small set of other variables in this nature, mostly *TMPFS*. > > This is why I think it would be handy to add a Gentoo section to the > kernel, along the lines as described by Linus. > > https://lkml.org/lkml/2012/7/13/369 > > The same goes for asking the user to enable configuration options in the > kernel, why can't we just tell him to enable all option that toggles > support for the user. For example; in the Gentoo section, there would be > a "Init System Support" under which you can toggle
[gentoo-dev] Re: [gentoo-kernel] Proper distribution integration of kernel *-sources, patches and configuration.
On 1 July 2013 10:41, Tom Wijsman wrote: > Hello > > > Please reply to gentoo-dev in case ML daemon changes Reply-To. > > > ### TL; DR ### > > By introducing feature patches which menu options are disabled by > default to genpatches, we can deduplicate *-sources maintainers as well > as large groups of users work. By introducing a distribution section > in the menuconfig, we can provide options that enable minimal Gentoo > requirements by default (DEVTMPFS, making Gentoo systems bootable since > an udev release a long time ago) and other distribution stuff. I really like this idea. geek-sources has appealed to me massively the past few months, but i didn't want to risk stability and go with a kernel from a unofficial overlay. I can not wait to see this in gentoo-sources and I fully support this idea.
Re: [gentoo-dev] Proper distribution integration of kernel *-sources, patches and configuration.
On 1 July 2013 22:41, Tom Wijsman wrote: > > ### TL; DR ### > > By introducing feature patches which menu options are disabled by > default to genpatches, we can deduplicate *-sources maintainers as well > as large groups of users work. By introducing a distribution section > in the menuconfig, we can provide options that enable minimal Gentoo > requirements by default (DEVTMPFS, making Gentoo systems bootable since > an udev release a long time ago) and other distribution stuff. > Sounds like a great idea to me! -- Cheers, Ben | yngwin Gentoo developer
[gentoo-dev] Proper distribution integration of kernel *-sources, patches and configuration.
Hello Please reply to gentoo-dev in case ML daemon changes Reply-To. ### TL; DR ### By introducing feature patches which menu options are disabled by default to genpatches, we can deduplicate *-sources maintainers as well as large groups of users work. By introducing a distribution section in the menuconfig, we can provide options that enable minimal Gentoo requirements by default (DEVTMPFS, making Gentoo systems bootable since an udev release a long time ago) and other distribution stuff. ### Proper distribution integration of kernel *-sources, patches ... ### Gentoo is a distribution; but what is a distribution that doesn't properly integrate what it provides, but instead expects its users to do so, needlessly duplicating work amongst its maintainers and users. Let's say I want genpatches, aufs and TuxOnIce; closest candidates: - sys-kernel/aufs-sources: genpatches, aufs - sys-kernel/pf-sources: genpatches, CK, BFQ, BFQ, TuxOnIce, UKSM - sys-kernel/tuxonice-sources: genpatches, TuxOnIce What do I do? Take one (eg. aufs) and apply the other (eg. TuxOnIce)? What if I want to add another common patchset to it? Hardened perhaps? You can see, some of these sources (like pf-sources) already attempt to do so; with pf-sources in mind, why do we even have ck-sources, tuxonice-sources in the Portage tree? Just to duplicate work? I think we should trim down on the amount of *-sources and combine multiple patches into genpatches. Take an example of geek-sources which does all this without a problem; I don't really like the approach with USE flags used there, as the menuconfig can just cover that. https://github.com/init6/init_6/wiki/geek-sources What does a patch introducing new features really do? Or rather, what should it do when we add it? Let me summarize: 1) The features should be disabled by default. 2) These feature should depend on a non-vanilla / experimental option. 3) The patch should not affect the build by default. 4) The user can optionally enable the feature. So, in genpatches, since 3.9.7, BFQ was added to try this out (except 2.). Ensured it to be disabled by default, did not affect the build for anyone that does not enable it and the users have been enabling and using it on their own. So, does it differentiate more from vanilla? No. This helps users as well as maintainers to not have to apply BFQ on their own, they simply have to tick a config option and they are set. If all patches that introduce new features are added in this fashion, it spares large groups of users from having to do this on their own; we can also deduplicate the work in the Portage tree this way. ### ... and configuration. ### This problem is not only visible for patches, but also in the config. Meet CONFIG_DEVTMPFS; forget to enable it, greet a failing boot. We're telling users to enable it in some places, in the handbook it's a single line you must read, on the Wiki it's kind of missing unless you are luckily on the right page, on the Quick Install book it is missing too. So, we are currently providing a configuration that expects anyone to enable CONFIG_DEVTMPFS. But you have to remember that it need to make sure you read about it or enable it from the udev upgrade a while ago if you decide to start from a fresh config or are installing without that handbook you kind of have memorized. Searching for CONFIG_DEVTMPFS in the forums and #gentoo logs shows that this is quite often suggested as a fix and quite often actually fixes an user's boot. Why duplicate telling users to do that if we can do it? There are a small set of other variables in this nature, mostly *TMPFS*. This is why I think it would be handy to add a Gentoo section to the kernel, along the lines as described by Linus. https://lkml.org/lkml/2012/7/13/369 The same goes for asking the user to enable configuration options in the kernel, why can't we just tell him to enable all option that toggles support for the user. For example; in the Gentoo section, there would be a "Init System Support" under which you can toggle an option to support the minimal requirements for some init system. Feedback is very welcome. P.S.: Not everything in this mail has been acked by the kernel lead; only some thoughts, I was suggested to take this to ML for discussion. The usage of the word 'we' in this mail is therefore hypothetical. ### F.A.Q. ### Q: I don't want feature X? Please don't add the patch! A: It's optional, don't enable it in your menu config. Q: What about my stable server? I really don't want to run this stuff! A: These options would depend on !CONFIG_VANILLA or CONFIG_EXPERIMENTAL which would be disabled by default, therefore if you keep this option the way it is on your stable server; it won't affect you. In other words, genpatches stay as stable as before; unless you explicitly toggle options that intentionally make it unstable. Q: Genpatches used to be minimal, would it gain weight? A: If you don't enable those
