Re: [OE-core] RFC: Maintain backwards compatibility or not for module-base.bbclass
On Fri, Jan 17, 2014 at 8:53 AM, Peter Kjellerstedt wrote: > I will aggregate my responses to Koen, Bruce and Phil below. > >> -Original Message- >> From: Koen Kooi [mailto:k...@dominion.thruhere.net] >> Sent: den 16 januari 2014 20:19 >> To: Peter Kjellerstedt [snip] >> >> On Thu, Jan 16, 2014 at 8:58 AM, Peter Kjellerstedt >> wrote: >> > Anyway, I made a modified version that does maintain backwards >> > compatibility for module-base.bbclass here: >> > >> > http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=pkj/kernel-headers-backwards-compatible >> > >> > This time Richard complained about the extra class >> > (kernel-scripts-base.bbclass), and noted that there was no >> > way to win... He then suggested that I take the question of >> > whether we need to maintain backwards compatibility for >> > modules-base.bbclass to the mailing list. >> >> FWIW: I agree that have to many small, single purpose >> kernel-*.bbclasses is a pain, since it provides granularity, but >> more opportunity for varying behaviour during kernel builds. >> >> I have uses of the module-base.bbclass and an expectation that it >> will generate the scripts, largely around the SDK and some custom >> kernel recipes. So they only inherit module.bbclass, and would be >> impacted if that functionality was changed to require another inherit. > > As long as they inherit module (and not add task dependencies on > do_make_scripts) they should not be affected. It is only if you > inherit module-base directly and expect the do_make_scripts() > function to exist that you are affected. I've seen some direct inherits of module-base, and even one that referenced the task (to ensure that recordmcount and friends were in place before using them). > >> Speaking of that, we through something like this late last year >> with automatically restoring the scripts into the sysroot, which >> ended up being reverted: >> >> see b2c948d56241ff7cdea2e9e68b740f305c72f5ca in oe-core >> >> At least the module (and your scripts) class avoid the sstate >> problems and compiler dependencies that we hit with that solution. > > Interesting. It looks like it would have been a good solution > if it had worked... It worked :) But it introduced complexity when building from sstate, since the compiler was required to restore them, which meant that the sstate builds had to depend on gcc, which sort of defeats the purpose of a quick sstate startup and build .. so we decided to abandon it and educate people to include module-base. > >> What are the alternatives to more classes, isn't this something >> that could be a .inc routine ? And modules simply includes it, and >> you can do the same .. but I suppose a .inc versus a class inherit >> is largely semantics in the difference. > > Tbh, I think keeping it as a class makes it more straight forward. > And class or include file is merely semantics. Agreed. It really wouldn't help. > >> bottom line, my rambling says that backwards compatibility matters >> here, and that if we can avoid a new class, that would also be a >> good thing. > > But I think that it should be easy enough for any external party > to adopt to the new class if they do indeed have a direct > dependency on module-base and expecting to find do_make_scripts() > there. It is not like we would change it in the middle of Dora > anyway. Thus I advocate for the change above on the kernel-headers > branch that does not maintain backwards compatibility. I think we all agree that it really isn't a hard thing to switch (just as in the examples I sited, it wouldn't be that hard to fix them), it comes down to more of a perception thing about how often a layer needs to be adapted to a changing core structure. Developers understand changing interfaces, customers and larger companies .. not so much (but I'm stating what we all know). So in the end, the question is the extra class "clutter" versus compatibility. I lean towards the extra class with the compatibility maintained .. but I'm not adamant about the issue :) Cheers, Bruce > >> Cheers, >> >> Bruce > >> -Original Message- >> From: openembedded-core-boun...@lists.openembedded.org >> [mailto:openembedded-core-boun...@lists.openembedded.org] On Behalf Of >> Phil Blundell >> Sent: den 16 januari 2014 22:39 >> To: Peter Kjellerstedt >> Cc: OE Core (openembedded-core@lists.openembedded.org) >> Subject: Re: [OE-core] RFC: Maintain backwards compatibility or not for >> module-base.bbclass >> >> On Thu, 2014-01-16 a
Re: [OE-core] RFC: Maintain backwards compatibility or not for module-base.bbclass
I will aggregate my responses to Koen, Bruce and Phil below.
> -Original Message-
> From: Koen Kooi [mailto:k...@dominion.thruhere.net]
> Sent: den 16 januari 2014 20:19
> To: Peter Kjellerstedt
> Cc: OE Core (openembedded-core@lists.openembedded.org); Phil Blundell
> Subject: Re: [OE-core] RFC: Maintain backwards compatibility or not for
> module-base.bbclass
>
> Op 16 jan. 2014, om 14:58 heeft Peter Kjellerstedt
> het volgende geschreven:
>
> > Background: Back in September, Richard made a commit to
> > linux-libc-headers.inc describing why one should not fork the
> > linux-libc-headers recipe:
> >
> > http://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=babbf7a46acaefd9b36031483cafce053f607e66
> >
> > As a result I created a local bbclass for our layer called
> > kernel-headers, which provides a recipe with kernel headers
> > that match the actually used version of the Linux kernel.
> > This was needed for packages that need to access hardware
> > specific features that are not present in the generic kernel
> > headers provided by linux-libc-headers.
> >
> > My intention for this class was that it should be generic
> > enough to be able to upstream it to OE Core.
> >
> > Now, the other day a colleague of mine had a build failure due
> > to this class. It turned out that even though the class adds a
> > dependency on virtual/kernel and then uses the files that are
> > installed to ${STAGING_KERNEL_DIR} when running oe_runmake
> > headers_install, the command could fail because the
> > ${STAGING_KERNEL_DIR}/scripts was not populated. After asking
> > Richard about this, I learned that this is due to problems
> > with the sstate cache and not knowing whether a 32 bit host or
> > a 64 bit host was used to generate the files. Thus I also
> > learned that the scripts are actually built as a result of
> > building modules.
> >
> > Since I did not want my class to depend on modules having been
> > built, I looked into modules.bbclass and modules-base.bbclass.
> > There I found the function do_make_scripts() which is
> > responsible for building the kernel scripts. However, the
> > current setup doesn't lend itself very well to use the
> > modules-base.bbclass for something other than modules.
> >
> > My idea then was to break this part out into a separate class,
> > kernel-scripts, which I did. You can find both the
> > kernel-scripts and kernel-headers classes here:
> >
> > http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=pkj/kernel-headers
> >
> > When I showed this to Richard he noted that my change was not
> > backwards compatible (as I no longer provide the
> > do_make_scripts() function from the module-base.bbclass).
> > However, there is nothing besides module.bbclass in OE Core
> > and meta-oe that use the module-base.bbclass.
>
> Can't do_make_script run as an sstate post function?
I think that is exactly what the reverted change Bruce mentions
below did.
> regards,
>
> Koen
> -Original Message-
> From: Bruce Ashfield [mailto:bruce.ashfi...@gmail.com]
> Sent: den 16 januari 2014 19:41
> To: Peter Kjellerstedt
> Cc: OE Core (openembedded-core@lists.openembedded.org); Phil Blundell
> Subject: Re: [OE-core] RFC: Maintain backwards compatibility or not for
> module-base.bbclass
>
> On Thu, Jan 16, 2014 at 8:58 AM, Peter Kjellerstedt
> wrote:
> > Anyway, I made a modified version that does maintain backwards
> > compatibility for module-base.bbclass here:
> >
> > http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=pkj/kernel-headers-backwards-compatible
> >
> > This time Richard complained about the extra class
> > (kernel-scripts-base.bbclass), and noted that there was no
> > way to win... He then suggested that I take the question of
> > whether we need to maintain backwards compatibility for
> > modules-base.bbclass to the mailing list.
>
> FWIW: I agree that have to many small, single purpose
> kernel-*.bbclasses is a pain, since it provides granularity, but
> more opportunity for varying behaviour during kernel builds.
>
> I have uses of the module-base.bbclass and an expectation that it
> will generate the scripts, largely around the SDK and some custom
> kernel recipes. So they only inherit module.bbclass, and would be
> impacted if that functionality was changed to require another inherit.
As long as they inherit module (and not add task dependencies on
do_make_scripts) they should not be affected. It is only if you
inherit module-base directly and expect the do_make_scripts()
function to e
Re: [OE-core] RFC: Maintain backwards compatibility or not for module-base.bbclass
On Thu, 2014-01-16 at 14:58 +0100, Peter Kjellerstedt wrote: > So, here I am now. I do not know who else use the > do_make_scripts() function from module-base.bbclass and in what > way, and whether restructuring the functionality into the new > kernel-scripts.bbclass without maintaining backwards > compatibility would be a problem. If you know anything about > this, please let me know. I'm not entirely clear why you couldn't maintain compatibility by moving the task in question to a new class but keeping its name the same (i.e. refrain from renaming "do_make_scripts" to "do_kernel_scripts") and having module-base.bbclass simply inherit the newly-added class. That seems like it ought to be fairly straightforward and uncontroversial. That said, though, I don't think there is any pressing need to maintain backwards compatibility around module-base.bbclass. In fact, I think it would probably be fine for module-base.bbclass to simply go away altogether and have its functionality subsumed into module.bbclass; the split between those two classes is mostly a relic of yesteryear and I can't think of any good purpose that it serves nowadays. So I would be happy enough to see that (and the tangly mess that is the kernel classes in general) cleaned up irrespective of what happens with do_make_scripts. p. ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] RFC: Maintain backwards compatibility or not for module-base.bbclass
Op 16 jan. 2014, om 14:58 heeft Peter Kjellerstedt
het volgende geschreven:
> Background: Back in September, Richard made a commit to
> linux-libc-headers.inc describing why one should not fork the
> linux-libc-headers recipe:
>
> http://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=babbf7a46acaefd9b36031483cafce053f607e66
>
> As a result I created a local bbclass for our layer called
> kernel-headers, which provides a recipe with kernel headers
> that match the actually used version of the Linux kernel.
> This was needed for packages that need to access hardware
> specific features that are not present in the generic kernel
> headers provided by linux-libc-headers.
>
> My intention for this class was that it should be generic
> enough to be able to upstream it to OE Core.
>
> Now, the other day a colleague of mine had a build failure due
> to this class. It turned out that even though the class adds a
> dependency on virtual/kernel and then uses the files that are
> installed to ${STAGING_KERNEL_DIR} when running oe_runmake
> headers_install, the command could fail because the
> ${STAGING_KERNEL_DIR}/scripts was not populated. After asking
> Richard about this, I learned that this is due to problems
> with the sstate cache and not knowing whether a 32 bit host or
> a 64 bit host was used to generate the files. Thus I also
> learned that the scripts are actually built as a result of
> building modules.
>
> Since I did not want my class to depend on modules having been
> built, I looked into modules.bbclass and modules-base.bbclass.
> There I found the function do_make_scripts() which is
> responsible for building the kernel scripts. However, the
> current setup doesn't lend itself very well to use the
> modules-base.bbclass for something other than modules.
>
> My idea then was to break this part out into a separate class,
> kernel-scripts, which I did. You can find both the
> kernel-scripts and kernel-headers classes here:
>
> http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=pkj/kernel-headers
>
> When I showed this to Richard he noted that my change was not
> backwards compatible (as I no longer provide the
> do_make_scripts() function from the module-base.bbclass).
> However, there is nothing besides module.bbclass in OE Core
> and meta-oe that use the module-base.bbclass.
Can't do_make_script run as an sstate post function?
regards,
Koen
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] RFC: Maintain backwards compatibility or not for module-base.bbclass
On Thu, Jan 16, 2014 at 8:58 AM, Peter Kjellerstedt
wrote:
> Background: Back in September, Richard made a commit to
> linux-libc-headers.inc describing why one should not fork the
> linux-libc-headers recipe:
>
> http://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=babbf7a46acaefd9b36031483cafce053f607e66
>
> As a result I created a local bbclass for our layer called
> kernel-headers, which provides a recipe with kernel headers
> that match the actually used version of the Linux kernel.
> This was needed for packages that need to access hardware
> specific features that are not present in the generic kernel
> headers provided by linux-libc-headers.
>
> My intention for this class was that it should be generic
> enough to be able to upstream it to OE Core.
>
> Now, the other day a colleague of mine had a build failure due
> to this class. It turned out that even though the class adds a
> dependency on virtual/kernel and then uses the files that are
> installed to ${STAGING_KERNEL_DIR} when running oe_runmake
> headers_install, the command could fail because the
> ${STAGING_KERNEL_DIR}/scripts was not populated. After asking
> Richard about this, I learned that this is due to problems
> with the sstate cache and not knowing whether a 32 bit host or
> a 64 bit host was used to generate the files. Thus I also
> learned that the scripts are actually built as a result of
> building modules.
>
> Since I did not want my class to depend on modules having been
> built, I looked into modules.bbclass and modules-base.bbclass.
> There I found the function do_make_scripts() which is
> responsible for building the kernel scripts. However, the
> current setup doesn't lend itself very well to use the
> modules-base.bbclass for something other than modules.
>
> My idea then was to break this part out into a separate class,
> kernel-scripts, which I did. You can find both the
> kernel-scripts and kernel-headers classes here:
>
> http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=pkj/kernel-headers
>
> When I showed this to Richard he noted that my change was not
> backwards compatible (as I no longer provide the
> do_make_scripts() function from the module-base.bbclass).
> However, there is nothing besides module.bbclass in OE Core
> and meta-oe that use the module-base.bbclass.
>
> Anyway, I made a modified version that does maintain backwards
> compatibility for module-base.bbclass here:
>
> http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=pkj/kernel-headers-backwards-compatible
>
> This time Richard complained about the extra class
> (kernel-scripts-base.bbclass), and noted that there was no
> way to win... He then suggested that I take the question of
> whether we need to maintain backwards compatibility for
> modules-base.bbclass to the mailing list.
FWIW: I agree that have to many small, single purpose
kernel-*.bbclasses is a pain,
since it provides granularity, but more opportunity for varying behaviour during
kernel builds.
I have uses of the module-base.bbclass and an expectation that it will generate
the scripts, largely around the SDK and some custom kernel recipes. So they
only inherit module.bbclass, and would be impacted if that functionality was
changed to require another inherit.
Speaking of that, we through something like this late last year with
automatically
restoring the scripts into the sysroot, which ended up being reverted:
see b2c948d56241ff7cdea2e9e68b740f305c72f5ca in oe-core
At least the module (and your scripts) class avoid the sstate problems and
compiler dependencies that we hit with that solution.
What are the alternatives to more classes, isn't this something that could be
a .inc routine ? And modules simply includes it, and you can do the same ..
but I suppose a .inc versus a class inherit is largely semantics in
the difference.
bottom line, my rambling says that backwards compatibility matters here, and
that if we can avoid a new class, that would also be a good thing.
Cheers,
Bruce
>
> So, here I am now. I do not know who else use the
> do_make_scripts() function from module-base.bbclass and in what
> way, and whether restructuring the functionality into the new
> kernel-scripts.bbclass without maintaining backwards
> compatibility would be a problem. If you know anything about
> this, please let me know.
>
> Any comments appreciated.
>
> //Peter
>
> ___
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core
--
"Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end"
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] RFC: Maintain backwards compatibility or not for module-base.bbclass
Background: Back in September, Richard made a commit to
linux-libc-headers.inc describing why one should not fork the
linux-libc-headers recipe:
http://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=babbf7a46acaefd9b36031483cafce053f607e66
As a result I created a local bbclass for our layer called
kernel-headers, which provides a recipe with kernel headers
that match the actually used version of the Linux kernel.
This was needed for packages that need to access hardware
specific features that are not present in the generic kernel
headers provided by linux-libc-headers.
My intention for this class was that it should be generic
enough to be able to upstream it to OE Core.
Now, the other day a colleague of mine had a build failure due
to this class. It turned out that even though the class adds a
dependency on virtual/kernel and then uses the files that are
installed to ${STAGING_KERNEL_DIR} when running oe_runmake
headers_install, the command could fail because the
${STAGING_KERNEL_DIR}/scripts was not populated. After asking
Richard about this, I learned that this is due to problems
with the sstate cache and not knowing whether a 32 bit host or
a 64 bit host was used to generate the files. Thus I also
learned that the scripts are actually built as a result of
building modules.
Since I did not want my class to depend on modules having been
built, I looked into modules.bbclass and modules-base.bbclass.
There I found the function do_make_scripts() which is
responsible for building the kernel scripts. However, the
current setup doesn't lend itself very well to use the
modules-base.bbclass for something other than modules.
My idea then was to break this part out into a separate class,
kernel-scripts, which I did. You can find both the
kernel-scripts and kernel-headers classes here:
http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=pkj/kernel-headers
When I showed this to Richard he noted that my change was not
backwards compatible (as I no longer provide the
do_make_scripts() function from the module-base.bbclass).
However, there is nothing besides module.bbclass in OE Core
and meta-oe that use the module-base.bbclass.
Anyway, I made a modified version that does maintain backwards
compatibility for module-base.bbclass here:
http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=pkj/kernel-headers-backwards-compatible
This time Richard complained about the extra class
(kernel-scripts-base.bbclass), and noted that there was no
way to win... He then suggested that I take the question of
whether we need to maintain backwards compatibility for
modules-base.bbclass to the mailing list.
So, here I am now. I do not know who else use the
do_make_scripts() function from module-base.bbclass and in what
way, and whether restructuring the functionality into the new
kernel-scripts.bbclass without maintaining backwards
compatibility would be a problem. If you know anything about
this, please let me know.
Any comments appreciated.
//Peter
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
