Re: [RFC] Provide in-kernel headers for making it easy to extend the kernel

2019-03-07 Thread Joel Fernandes
On Thu, Mar 07, 2019 at 03:40:37PM -0800, h...@zytor.com wrote: > On March 7, 2019 3:12:07 PM PST, Joel Fernandes > wrote: > >Enrico, > > > >On Thu, Mar 07, 2019 at 11:11:22PM +0100, Enrico Weigelt, metux IT > >consult wrote: > >> On 07.03.19 21:55, Greg KH wrote: > >> > >> > Ick, no, no more

Re: [RFC] Provide in-kernel headers for making it easy to extend the kernel

2019-03-07 Thread hpa
On March 7, 2019 3:12:07 PM PST, Joel Fernandes wrote: >Enrico, > >On Thu, Mar 07, 2019 at 11:11:22PM +0100, Enrico Weigelt, metux IT >consult wrote: >> On 07.03.19 21:55, Greg KH wrote: >> >> > Ick, no, no more squashfs please, let's just kill that mess once >and for >> > all :) >> >> okay,

Re: [RFC] Provide in-kernel headers for making it easy to extend the kernel

2019-03-07 Thread Joel Fernandes
Enrico, On Thu, Mar 07, 2019 at 11:11:22PM +0100, Enrico Weigelt, metux IT consult wrote: > On 07.03.19 21:55, Greg KH wrote: > > > Ick, no, no more squashfs please, let's just kill that mess once and for > > all :) > > okay, then: s/squashfs/whatever_fs_image_or_archive_you_like/; > > >

Re: [RFC] Provide in-kernel headers for making it easy to extend the kernel

2019-03-07 Thread Enrico Weigelt, metux IT consult
On 07.03.19 21:55, Greg KH wrote: > Ick, no, no more squashfs please, let's just kill that mess once and for > all :) okay, then: s/squashfs/whatever_fs_image_or_archive_you_like/; > Again, putting this in a simple compressed tar image allows anyone to do > whatever they need to with this. If

Re: [RFC] Provide in-kernel headers for making it easy to extend the kernel

2019-03-07 Thread Greg KH
On Thu, Mar 07, 2019 at 09:41:24PM +0100, Enrico Weigelt, metux IT consult wrote: > On 07.03.19 02:49, Daniel Colascione wrote: > > > Entirely FS-less operation is uncommon, granted. :-) I guess I've just > > spent too much time debugging emulators that refuse to mount their> root > >

[RFC] Provide in-kernel headers for making it easy to extend the kernel

2019-03-07 Thread Enrico Weigelt, metux IT consult
On 07.03.19 02:49, Daniel Colascione wrote: > Entirely FS-less operation is uncommon, granted. :-) I guess I've just > spent too much time debugging emulators that refuse to mount their> root > filesystems. :-) Fix these emulators ? > There are legitimate reasons why a device's> filesystems

Re: [RFC] Provide in-kernel headers for making it easy to extend the kernel

2019-03-07 Thread Enrico Weigelt, metux IT consult
On 07.03.19 02:48, Joel Fernandes wrote: >> I'm confused. > > Take a look at this thread: https://lkml.org/lkml/2019/2/28/634 Okay, replying to that mail: > > > There's no linux-headers That's the first fundamental problem. Actually, there's not even any decent package management at all (no,

Re: [RFC] Provide in-kernel headers for making it easy to extend the kernel

2019-03-07 Thread Enrico Weigelt, metux IT consult
On 07.03.19 02:42, Joel Fernandes wrote: Speaking as an embedded linux architect, who's daily job for a large part is to getting bootloader, kernel, userland and customer applications play nice together, in a stable, secure and easily maintainable way: > They would include the module because

Re: [RFC] Provide in-kernel headers for making it easy to extend the kernel

2019-03-06 Thread Daniel Colascione
On Wed, Mar 6, 2019 at 5:23 PM Enrico Weigelt, metux IT consult wrote: > > On 07.03.19 01:33, Daniel Colascione wrote: > > > *There* *may* *be* *no* *filesystem*. > > A Linux system w/o any filesystem at all ? Well, that's interesting. Entirely FS-less operation is uncommon, granted. :-) I guess

Re: [RFC] Provide in-kernel headers for making it easy to extend the kernel

2019-03-06 Thread Joel Fernandes
On Thu, Mar 07, 2019 at 01:42:45AM +0100, Enrico Weigelt, metux IT consult wrote: > On 07.03.19 00:09, Pavel Machek wrote: > > > So your licensing requirements prevent you from having headers in the > > filesystem, but allow module with the headers hidden inside on the > > filesystem? > > Maybe

Re: [RFC] Provide in-kernel headers for making it easy to extend the kernel

2019-03-06 Thread Joel Fernandes
On Wed, Mar 06, 2019 at 04:07:13PM -0800, H. Peter Anvin wrote: > On 3/6/19 3:37 PM, Daniel Colascione wrote: > > > > I just don't get the opposition to Joel's work. The rest of the thread > > already goes into detail about the problems with pure-filesystem > > solutions, and you and others are

Re: [RFC] Provide in-kernel headers for making it easy to extend the kernel

2019-03-06 Thread Enrico Weigelt, metux IT consult
On 07.03.19 01:33, Daniel Colascione wrote: > *There* *may* *be* *no* *filesystem*. A Linux system w/o any filesystem at all ? Well, that's interesting. > The only thing the kernel can really guarantee is its own> existence --- it > should be entire in itself. I vaguely recall some option

Re: [RFC] Provide in-kernel headers for making it easy to extend the kernel

2019-03-06 Thread Enrico Weigelt, metux IT consult
On 07.03.19 00:09, Pavel Machek wrote: > So your licensing requirements prevent you from having headers in the > filesystem, but allow module with the headers hidden inside on the > filesystem? Maybe it's just because I've missed most of the thread, but which license requirements exactly could

Re: [RFC] Provide in-kernel headers for making it easy to extend the kernel

2019-03-06 Thread Daniel Colascione
On Wed, Mar 6, 2019 at 4:33 PM H. Peter Anvin wrote: > > On 3/6/19 3:37 PM, Daniel Colascione wrote: > > > > I just don't get the opposition to Joel's work. The rest of the thread > > already goes into detail about the problems with pure-filesystem > > solutions, and you and others are just

Re: [RFC] Provide in-kernel headers for making it easy to extend the kernel

2019-03-06 Thread H. Peter Anvin
On 3/6/19 3:37 PM, Daniel Colascione wrote: > > I just don't get the opposition to Joel's work. The rest of the thread > already goes into detail about the problems with pure-filesystem > solutions, and you and others are just totally ignoring those > well-thought-out rationales for the module

Re: [RFC] Provide in-kernel headers for making it easy to extend the kernel

2019-03-06 Thread Daniel Colascione
On Wed, Mar 6, 2019 at 4:07 PM H. Peter Anvin wrote: > > On 3/6/19 3:37 PM, Daniel Colascione wrote: > > > > I just don't get the opposition to Joel's work. The rest of the thread > > already goes into detail about the problems with pure-filesystem > > solutions, and you and others are just

Re: [RFC] Provide in-kernel headers for making it easy to extend the kernel

2019-03-06 Thread H. Peter Anvin
On 3/6/19 3:37 PM, Daniel Colascione wrote: > > I just don't get the opposition to Joel's work. The rest of the thread > already goes into detail about the problems with pure-filesystem > solutions, and you and others are just totally ignoring those > well-thought-out rationales for the module

Re: [RFC] Provide in-kernel headers for making it easy to extend the kernel

2019-03-06 Thread Daniel Colascione
On Wed, Mar 6, 2019 at 3:09 PM Pavel Machek wrote: > > > > > >Ok, I'll look into LZMA. Thanks for checking the compression sizes. > > > > > > > >- Joel > > > > > > Don't use lzma, use xz if you are going to do something. > > > > Ok, sounds good. XZ is a file format for LZMA2. Everyone's right.

Re: [RFC] Provide in-kernel headers for making it easy to extend the kernel

2019-03-06 Thread H. Peter Anvin
On 3/6/19 3:09 PM, Pavel Machek wrote: > On Fri 2019-01-18 17:55:43, Joel Fernandes wrote: >> From: "Joel Fernandes (Google)" >> >> Introduce in-kernel headers and other artifacts which are made available >> as an archive through proc (/proc/kheaders.tgz file). This archive makes >> it possible

Re: [RFC] Provide in-kernel headers for making it easy to extend the kernel

2019-03-06 Thread Pavel Machek
On Fri 2019-01-18 17:55:43, Joel Fernandes wrote: > From: "Joel Fernandes (Google)" > > Introduce in-kernel headers and other artifacts which are made available > as an archive through proc (/proc/kheaders.tgz file). This archive makes > it possible to build kernel modules, run eBPF programs,

Re: [RFC] Provide in-kernel headers for making it easy to extend the kernel

2019-03-06 Thread Pavel Machek
> > >Ok, I'll look into LZMA. Thanks for checking the compression sizes. > > > > > >- Joel > > > > Don't use lzma, use xz if you are going to do something. > > Ok, sounds good. > > > However, it seems unlikely to me that someone not willing to spend the > > space in the filesystem will spend

Re: [RFC] Provide in-kernel headers for making it easy to extend the kernel

2019-01-26 Thread Norbert Lange
I have a similar problem, which is caused by an attempt to separate the kernel installation from the rootfs. updates of the kernel should not affect the (read-only) rootfs or initramfs. For technical reasons I am unable to built all modules static. - have multiple kernels #K and rootfs

Re: [RFC] Provide in-kernel headers for making it easy to extend the kernel

2019-01-25 Thread Joel Fernandes
On Fri, Jan 25, 2019 at 12:34:52PM -0800, Daniel Colascione wrote: > On Fri, Jan 25, 2019 at 12:28 PM wrote: > > > > On January 25, 2019 11:15:52 AM PST, Daniel Colascione > > wrote: > > >On Fri, Jan 25, 2019 at 11:01 AM wrote: > > >> > > >> On January 24, 2019 12:59:29 PM PST, Joel Fernandes

Re: [RFC] Provide in-kernel headers for making it easy to extend the kernel

2019-01-25 Thread Daniel Colascione
On Fri, Jan 25, 2019 at 12:28 PM wrote: > > On January 25, 2019 11:15:52 AM PST, Daniel Colascione > wrote: > >On Fri, Jan 25, 2019 at 11:01 AM wrote: > >> > >> On January 24, 2019 12:59:29 PM PST, Joel Fernandes > > wrote: > >> >On Thu, Jan 24, 2019 at 07:57:26PM +0100, Karim Yaghmour wrote:

Re: [RFC] Provide in-kernel headers for making it easy to extend the kernel

2019-01-25 Thread hpa
On January 25, 2019 11:15:52 AM PST, Daniel Colascione wrote: >On Fri, Jan 25, 2019 at 11:01 AM wrote: >> >> On January 24, 2019 12:59:29 PM PST, Joel Fernandes > wrote: >> >On Thu, Jan 24, 2019 at 07:57:26PM +0100, Karim Yaghmour wrote: >> >> >> >> On 1/23/19 11:37 PM, Daniel Colascione wrote:

Re: [RFC] Provide in-kernel headers for making it easy to extend the kernel

2019-01-25 Thread Joel Fernandes
On Fri, Jan 25, 2019 at 11:00:25AM -0800, h...@zytor.com wrote: > On January 24, 2019 12:59:29 PM PST, Joel Fernandes > wrote: > >On Thu, Jan 24, 2019 at 07:57:26PM +0100, Karim Yaghmour wrote: > >> > >> On 1/23/19 11:37 PM, Daniel Colascione wrote: > >[..] > >> > > Personally I advocated a

Re: [RFC] Provide in-kernel headers for making it easy to extend the kernel

2019-01-25 Thread Daniel Colascione
On Fri, Jan 25, 2019 at 11:01 AM wrote: > > On January 24, 2019 12:59:29 PM PST, Joel Fernandes > wrote: > >On Thu, Jan 24, 2019 at 07:57:26PM +0100, Karim Yaghmour wrote: > >> > >> On 1/23/19 11:37 PM, Daniel Colascione wrote: > >[..] > >> > > Personally I advocated a more aggressive approach

Re: [RFC] Provide in-kernel headers for making it easy to extend the kernel

2019-01-25 Thread hpa
On January 24, 2019 12:59:29 PM PST, Joel Fernandes wrote: >On Thu, Jan 24, 2019 at 07:57:26PM +0100, Karim Yaghmour wrote: >> >> On 1/23/19 11:37 PM, Daniel Colascione wrote: >[..] >> > > Personally I advocated a more aggressive approach with Joel in >private: >> > > just put the darn headers

Re: [RFC] Provide in-kernel headers for making it easy to extend the kernel

2019-01-24 Thread Joel Fernandes
On Thu, Jan 24, 2019 at 07:57:26PM +0100, Karim Yaghmour wrote: > > On 1/23/19 11:37 PM, Daniel Colascione wrote: [..] > > > Personally I advocated a more aggressive approach with Joel in private: > > > just put the darn headers straight into the kernel image, it's the > > > *only* artifact we're

Re: [RFC] Provide in-kernel headers for making it easy to extend the kernel

2019-01-24 Thread Karim Yaghmour
On 1/23/19 11:37 PM, Daniel Colascione wrote: While I think there's definitely a place for eBPF as part of the Android performance toolkit, I think most users will end up using it through rich front-end performance collection and analysis tools (of the sort I'm working on) rather than directly

Re: [RFC] Provide in-kernel headers for making it easy to extend the kernel

2019-01-24 Thread Joel Fernandes
On Wed, Jan 23, 2019 at 09:32:16PM -0500, Joel Fernandes wrote: > On Wed, Jan 23, 2019 at 02:37:47PM -0800, Daniel Colascione wrote: > > On Wed, Jan 23, 2019 at 1:29 PM Karim Yaghmour > > wrote: > [...] > > > Personally I advocated a more aggressive approach with Joel in private: > > > just put

Re: [RFC] Provide in-kernel headers for making it easy to extend the kernel

2019-01-23 Thread Joel Fernandes
On Wed, Jan 23, 2019 at 02:37:47PM -0800, Daniel Colascione wrote: > On Wed, Jan 23, 2019 at 1:29 PM Karim Yaghmour > wrote: [...] > > Personally I advocated a more aggressive approach with Joel in private: > > just put the darn headers straight into the kernel image, it's the > > *only* artifact

Re: [RFC] Provide in-kernel headers for making it easy to extend the kernel

2019-01-23 Thread Daniel Colascione
On Wed, Jan 23, 2019 at 1:29 PM Karim Yaghmour wrote: > > By the way, we can easily write a script to just extract the .ko directly - > > if the whole "load it as a module" thing bothers you. The kheaders.ko can > > just be thought of as a tarball. There's already a script to extract > >

Re: [RFC] Provide in-kernel headers for making it easy to extend the kernel

2019-01-23 Thread Karim Yaghmour
On 1/22/19 2:39 PM, Joel Fernandes wrote: [snip] On Sun, Jan 20, 2019 at 06:49:56PM -0800, h...@zytor.com wrote: [snip] My point is that if we're going to actually solve a problem, we need to make it so that the distro won't just disable it anyway, and it ought to be something scalable;

Re: [RFC] Provide in-kernel headers for making it easy to extend the kernel

2019-01-22 Thread Joel Fernandes
Hi Hans, Thanks for the discussion and sorry for my late reply due to the holidays. I replied inline below: On Sun, Jan 20, 2019 at 06:49:56PM -0800, h...@zytor.com wrote: > On January 20, 2019 5:45:53 PM PST, Joel Fernandes > wrote: > >On Sun, Jan 20, 2019 at 01:58:15PM -0800, h...@zytor.com

Re: [RFC] Provide in-kernel headers for making it easy to extend the kernel

2019-01-21 Thread hpa
On January 20, 2019 8:10:03 AM PST, Joel Fernandes wrote: >On Sat, Jan 19, 2019 at 11:01:13PM -0800, h...@zytor.com wrote: >> On January 19, 2019 2:36:06 AM PST, Greg KH > wrote: >> >On Sat, Jan 19, 2019 at 02:28:00AM -0800, Christoph Hellwig wrote: >> >> This seems like a pretty horrible idea

Re: [RFC] Provide in-kernel headers for making it easy to extend the kernel

2019-01-21 Thread hpa
On January 20, 2019 5:45:53 PM PST, Joel Fernandes wrote: >On Sun, Jan 20, 2019 at 01:58:15PM -0800, h...@zytor.com wrote: >> On January 20, 2019 8:10:03 AM PST, Joel Fernandes > wrote: >> >On Sat, Jan 19, 2019 at 11:01:13PM -0800, h...@zytor.com wrote: >> >> On January 19, 2019 2:36:06 AM PST,

Re: [RFC] Provide in-kernel headers for making it easy to extend the kernel

2019-01-20 Thread Sandeep Patil
On Sun, Jan 20, 2019 at 06:49:56PM -0800, h...@zytor.com wrote: > On January 20, 2019 5:45:53 PM PST, Joel Fernandes > wrote: > >On Sun, Jan 20, 2019 at 01:58:15PM -0800, h...@zytor.com wrote: > >> On January 20, 2019 8:10:03 AM PST, Joel Fernandes > > wrote: > >> >On Sat, Jan 19, 2019 at

Re: [RFC] Provide in-kernel headers for making it easy to extend the kernel

2019-01-20 Thread Joel Fernandes
On Sun, Jan 20, 2019 at 01:58:15PM -0800, h...@zytor.com wrote: > On January 20, 2019 8:10:03 AM PST, Joel Fernandes > wrote: > >On Sat, Jan 19, 2019 at 11:01:13PM -0800, h...@zytor.com wrote: > >> On January 19, 2019 2:36:06 AM PST, Greg KH > > wrote: > >> >On Sat, Jan 19, 2019 at 02:28:00AM

Re: [RFC] Provide in-kernel headers for making it easy to extend the kernel

2019-01-20 Thread Joel Fernandes
On Sat, Jan 19, 2019 at 11:01:13PM -0800, h...@zytor.com wrote: > On January 19, 2019 2:36:06 AM PST, Greg KH > wrote: > >On Sat, Jan 19, 2019 at 02:28:00AM -0800, Christoph Hellwig wrote: > >> This seems like a pretty horrible idea and waste of kernel memory. > > > >It's only a waste if you

Re: [RFC] Provide in-kernel headers for making it easy to extend the kernel

2019-01-20 Thread Joel Fernandes
On Sat, Jan 19, 2019 at 03:44:48PM -0800, h...@zytor.com wrote: > On January 19, 2019 3:25:03 PM PST, Joel Fernandes > wrote: > >On Sat, Jan 19, 2019 at 12:43:35PM -0500, Daniel Colascione wrote: > >> On Sat, Jan 19, 2019 at 11:27 AM Joel Fernandes > > wrote: > >> > > >> > On Sat, Jan 19, 2019

Re: [RFC] Provide in-kernel headers for making it easy to extend the kernel

2019-01-19 Thread hpa
On January 19, 2019 2:36:06 AM PST, Greg KH wrote: >On Sat, Jan 19, 2019 at 02:28:00AM -0800, Christoph Hellwig wrote: >> This seems like a pretty horrible idea and waste of kernel memory. > >It's only a waste if you want it to be a waste, i.e. if you load the >kernel module. > >This really isn't

Re: [RFC] Provide in-kernel headers for making it easy to extend the kernel

2019-01-19 Thread hpa
On January 19, 2019 3:25:03 PM PST, Joel Fernandes wrote: >On Sat, Jan 19, 2019 at 12:43:35PM -0500, Daniel Colascione wrote: >> On Sat, Jan 19, 2019 at 11:27 AM Joel Fernandes > wrote: >> > >> > On Sat, Jan 19, 2019 at 09:25:32AM +0100, Greg KH wrote: >> > > On Fri, Jan 18, 2019 at 05:55:43PM

Re: [RFC] Provide in-kernel headers for making it easy to extend the kernel

2019-01-19 Thread Joel Fernandes
On Sat, Jan 19, 2019 at 12:43:35PM -0500, Daniel Colascione wrote: > On Sat, Jan 19, 2019 at 11:27 AM Joel Fernandes > wrote: > > > > On Sat, Jan 19, 2019 at 09:25:32AM +0100, Greg KH wrote: > > > On Fri, Jan 18, 2019 at 05:55:43PM -0500, Joel Fernandes wrote: > > > > --- /dev/null > > > > +++

Re: [RFC] Provide in-kernel headers for making it easy to extend the kernel

2019-01-19 Thread Daniel Colascione
On Sat, Jan 19, 2019 at 11:27 AM Joel Fernandes wrote: > > On Sat, Jan 19, 2019 at 09:25:32AM +0100, Greg KH wrote: > > On Fri, Jan 18, 2019 at 05:55:43PM -0500, Joel Fernandes wrote: > > > --- /dev/null > > > +++ b/kernel/kheaders.c Thanks a ton for this work. It'll make it much easier to do

Re: [RFC] Provide in-kernel headers for making it easy to extend the kernel

2019-01-19 Thread Joel Fernandes
On Sat, Jan 19, 2019 at 09:25:32AM +0100, Greg KH wrote: > On Fri, Jan 18, 2019 at 05:55:43PM -0500, Joel Fernandes wrote: > > --- /dev/null > > +++ b/kernel/kheaders.c > > @@ -0,0 +1,74 @@ > > +// SPDX-License-Identifier: GPL-2.0 > > Nice, but: > > > > > +MODULE_LICENSE("GPL"); > > That

Re: [RFC] Provide in-kernel headers for making it easy to extend the kernel

2019-01-19 Thread Joel Fernandes
On Sat, Jan 19, 2019 at 09:26:04AM +0100, Greg KH wrote: > On Fri, Jan 18, 2019 at 05:55:43PM -0500, Joel Fernandes wrote: > > --- a/init/Kconfig > > +++ b/init/Kconfig > > @@ -549,6 +549,16 @@ config IKCONFIG_PROC > > This option enables access to the kernel configuration file > >

Re: [RFC] Provide in-kernel headers for making it easy to extend the kernel

2019-01-19 Thread Joel Fernandes
On Sat, Jan 19, 2019 at 11:36:06AM +0100, Greg KH wrote: > On Sat, Jan 19, 2019 at 02:28:00AM -0800, Christoph Hellwig wrote: > > This seems like a pretty horrible idea and waste of kernel memory. > > It's only a waste if you want it to be a waste, i.e. if you load the > kernel module. > > This

Re: [RFC] Provide in-kernel headers for making it easy to extend the kernel

2019-01-19 Thread Greg KH
On Sat, Jan 19, 2019 at 02:28:00AM -0800, Christoph Hellwig wrote: > This seems like a pretty horrible idea and waste of kernel memory. It's only a waste if you want it to be a waste, i.e. if you load the kernel module. This really isn't any different from how /proc/config.gz works. > Just add

Re: [RFC] Provide in-kernel headers for making it easy to extend the kernel

2019-01-19 Thread Christoph Hellwig
This seems like a pretty horrible idea and waste of kernel memory. Just add support to kbuild to store a compressed archive in initramfs and unpack it in the right place.

Re: [RFC] Provide in-kernel headers for making it easy to extend the kernel

2019-01-19 Thread Greg KH
On Fri, Jan 18, 2019 at 05:55:43PM -0500, Joel Fernandes wrote: > --- a/init/Kconfig > +++ b/init/Kconfig > @@ -549,6 +549,16 @@ config IKCONFIG_PROC > This option enables access to the kernel configuration file > through /proc/config.gz. > > +config IKHEADERS_PROC > +

Re: [RFC] Provide in-kernel headers for making it easy to extend the kernel

2019-01-19 Thread Greg KH
On Fri, Jan 18, 2019 at 05:55:43PM -0500, Joel Fernandes wrote: > --- /dev/null > +++ b/kernel/kheaders.c > @@ -0,0 +1,74 @@ > +// SPDX-License-Identifier: GPL-2.0 Nice, but: > +MODULE_LICENSE("GPL"); That means "GPL2+" (yeah, horrible, I know, we all get it wrong, look at

[RFC] Provide in-kernel headers for making it easy to extend the kernel

2019-01-18 Thread Joel Fernandes
From: "Joel Fernandes (Google)" Introduce in-kernel headers and other artifacts which are made available as an archive through proc (/proc/kheaders.tgz file). This archive makes it possible to build kernel modules, run eBPF programs, and other tracing programs that need to extend the kernel for