Re: [OE-core] RFE: make the init manager an image feature (again)
Khem Raj raj.khem-re5jqeeqqe8avxtiumw...@public.gmane.org writes: On the specifics of the do_install_append, you've seen my comments about how we're not learning from past mistakes with the way the do_install in the class was written. I note Phil also agreed with them, both of us remembering some of the horrors we've dealt with in the past (and binconfig.bbclass is still around, sadly). So you prefer same code to copy same type of files all over the recipes? duplicating usecase is better even though it may sound copying here now think if you add this to a bbclass and works ok for a package where we glue the unitfiles fast forward 6mos and the next version of the package has added the unitfiles into the package itself since systemd is so cool. We will be silently installing our own unit files without knowing. The recent .bbappends with their own do_install_append have this problem too. But due to the code replication the problem must be fixed at a lot of different places instead of at a central one. The .bbclass can do some sanity checks (e.g. making the copy operation fail when the unit file already exists). Worst if the files from package are different. Having individual install append gives you an opportunity to this append I do not see how this is better than removing the local .service from SRC_URI. Enrico ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
Re: [OE-core] RFE: make the init manager an image feature (again)
Ross Burton ross.burton-ral2jqcrhueavxtiumw...@public.gmane.org writes: the source, so enabling systemd may well lead to libsystemd-* libraries sneaking into your rescue image. for socket activation and sd_notify(), only libsystemd-daemon is required which is -rwxr-xr-x1 root root 11644 Feb 25 09:27 /lib/libsystemd-daemon.so.0.0.7 ~ # ldd /lib/libsystemd-daemon.so.0.* libdl.so.2 = /lib/libdl.so.2 (0x441e) librt.so.1 = /lib/librt.so.1 (0x441c8000) libc.so.6 = /lib/libc.so.6 (0x4404) /lib/ld-linux.so.3 (0x4401) libpthread.so.0 = /lib/libpthread.so.0 (0x441a) ~ # ldd -u -r /lib/libsystemd-daemon.so.0.* Unused direct dependencies: /lib/libdl.so.2 Ok, it needs more space than nothing and small things like this can sum up easily. But it is still a significant difference between having libsystemd-daemon in the image or the whole systemd. The systemd libraries will have to be reviewed to verify that installing one of them doesn't pull in systemd itself. afair, libsystemd-daemon was designed to be as light weighted as possible and will never depend on the rest of systemd. Enrico ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
Re: [OE-core] RFE: make the init manager an image feature (again)
On Sun, Feb 24, 2013 at 2:37 AM, Ross Burton ross.bur...@intel.com wrote:
Hi,
Just brainstorming out loud, but here's a suggestion that might just please
everyone:
A virtual provider for the init manager, which can be overridden per-image
(for main / rescue images).
Yes I was thinking about it too but it has to be made one off case.
Since we wont be able to support all combinations
DISTRO_FEATURES contains the init script *style* that you want: sysvinit or
systemd. These are not mutually exclusive so specifying both will get you
both directly in packages that support both. I'm still not convinced we need
to split these out into separate packages, the size impact is practically
negligible and the dependencies are effectively redundant as you'll have
trouble booting without an init system. Instead the postinsts should wrap
the initscript fragments in checks.
systemd lives fine with initscripts. In some cases the adhoc unit
files just call these initscripts so in a way systemd does
expect the initscripts to be around. So systemd is sort of addon and
thats why ${PN}-systemd worked well. So we can have
a perfect sysvinit based system and then we replace it with systemd
and add the ${PN}-systemd packages into image and it becomes
a systemd based image. I think as porting happens this case of
wrapping initscripts into systemd service files will continue
unless individual packages get their own unit files which are
rewritten and do not use initscripts anymore.
Those distro features will also be used to enable/disable features in the
source, so enabling systemd may well lead to libsystemd-* libraries sneaking
into your rescue image. The systemd libraries will have to be reviewed to
verify that installing one of them doesn't pull in systemd itself.
Here's the fun bit - we can't have two builds of udev in the same distro, and
that's what could happen, so I think we'll need to cut down to one udev
instead of two.
well we can make a udev recipe which uses exact sources used by systemd.
I'm not entirely convinced this is a good plan yet myself though…
Ross
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
Re: [OE-core] RFE: make the init manager an image feature (again)
On (16/02/13 11:41), Otavio Salvador wrote: On Sat, Feb 16, 2013 at 10:53 AM, Richard Purdie richard.pur...@linuxfoundation.org wrote: On Sat, 2013-02-16 at 08:47 -0200, Otavio Salvador wrote: On Sat, Feb 16, 2013 at 7:15 AM, Richard Purdie richard.pur...@linuxfoundation.org wrote: On Fri, 2013-02-15 at 19:19 +0100, Enrico Scholz wrote: it would be nice when the decision to make the init manager a distribution feature will be reverted to the old oe-meta mechanism. Being a distribution feature means, that packages are created in such a way that it is impossible to split off unwanted and heavy weighted functionality at image creation time. E.g. on most of my systems, I create two kinds of images: a full featured, systemd based one and a very minimal rescue system with busybox and some filesystem utilities. With recent systemd packaging change, the rescue image size grow up from 5.9 MiB to 27 MiB because systemd dependencies are hardcoded in mandatory packages. Formerly, systemd dependencies could be avoided by adding the -systemd packages to BAD_RECOMMENDATIONS (e.g. due to busybox-syslog - busybox-syslog-systemd rrecommend). I am aware that initscripts were always part of the main package. But sysvinit was very lightweighted and the extra space either negligible or easy to recover by removing some files in IMAGE_PREPROCESS_COMMAND. Hence my recommendation: make the init manager an image feature again and create -systemd and -sysv packages with the corresponding scripts. OpenEmbedded is still for embedded devices where size matters. Of course, systemd can be still a distribution feature to enable things like socket activation as part of PACKAGE_CONFIG. But dependencies on init system packages should be RRECOMMENDS which can be overridden easily at image creation time. The trouble is that by making it an image feature, people will expect *everything* to work properly and to be able to have fully functional sysvinit and systemd variants of images. We already see this expectation. Trying to explain to people what the limitations are, what is expected to work and what isn't will be difficult. I know you understand this but going forward most people will simply not and I think it will be a nightmare for usability. You said you already see it so what is the change? For me less flexibility is more problem and the result are more ugly hacks. I've been using the systemd system in meta-oe in product for long time and it works fine. In same distro I build other product with is sysvinit based and it also works fine. Now I have a nightmare in upgrade path, a change in the way my customer work and as a plus the need to make two different distributions for same product. No sense to me. Final result? This product won't go out of danny as I won't be able to provide an usable upgrade path without forking OE or having an insane amount of bbappend hacks. When systemd was added to meta-oe, it was clearly mentioned that it was a proof of concept to experiment with integrating it and subject to change until it made it into OE-Core. The fact is is changing is therefore not a surprise. We do need to think about making sure we have equivalent functionality, we also need to think about migration but the fact changes happened with some experimental work should not surprise people. I appreciate that isn't ideal but it is the way it is. Sure; I'd expect a change for the better ... besides all the work done in meta-oe lead it to a proper and good implementation (having it being split and drop the contamination of images we had first implementation). Having it as a distro feature is a good option too but all package split, logic and implementation of rest were good and flexible. For that reason I'd rather see this done in a different way, for example blacklisting the problematic systemd dependencies at image generation time with some kind of stronger BAD_RECOMMENDS code. Yes, this means there would be some systemd config files lying around but you'd stop the main bulk of it coming in (and as you said, some small config files aren't an issue). Check the amount of changes for do_install_append has sent today. This was all not need with the code we had and which works fine. This is just duplicated code all over recipes. Seriously the systemd integration was done in complete wrong way in my POV. It broght up problems we have fixed and do major improvements in meta-oe ... at no profit. Sorry you feel that way. I'm equally frustrated with the way systemd was added to meta-oe and the damage it did to the overall usability of the OE ecosystem. I'm trying to take positive actions from that such as ensuring we never do something like it again. Agreed. This was fix when we moved it to meta-oe/meta-systemd. As for the
Re: [OE-core] RFE: make the init manager an image feature (again)
Hi, Just brainstorming out loud, but here's a suggestion that might just please everyone: A virtual provider for the init manager, which can be overridden per-image (for main / rescue images). DISTRO_FEATURES contains the init script *style* that you want: sysvinit or systemd. These are not mutually exclusive so specifying both will get you both directly in packages that support both. I'm still not convinced we need to split these out into separate packages, the size impact is practically negligible and the dependencies are effectively redundant as you'll have trouble booting without an init system. Instead the postinsts should wrap the initscript fragments in checks. Those distro features will also be used to enable/disable features in the source, so enabling systemd may well lead to libsystemd-* libraries sneaking into your rescue image. The systemd libraries will have to be reviewed to verify that installing one of them doesn't pull in systemd itself. Here's the fun bit - we can't have two builds of udev in the same distro, and that's what could happen, so I think we'll need to cut down to one udev instead of two. I'm not entirely convinced this is a good plan yet myself though… Ross ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
Re: [OE-core] RFE: make the init manager an image feature (again)
On Sun, Feb 24, 2013 at 5:50 AM, Khem Raj raj.k...@gmail.com wrote: On (16/02/13 11:41), Otavio Salvador wrote: On Sat, Feb 16, 2013 at 10:53 AM, Richard Purdie richard.pur...@linuxfoundation.org wrote: On Sat, 2013-02-16 at 08:47 -0200, Otavio Salvador wrote: On Sat, Feb 16, 2013 at 7:15 AM, Richard Purdie richard.pur...@linuxfoundation.org wrote: On Fri, 2013-02-15 at 19:19 +0100, Enrico Scholz wrote: it would be nice when the decision to make the init manager a distribution feature will be reverted to the old oe-meta mechanism. Being a distribution feature means, that packages are created in such a way that it is impossible to split off unwanted and heavy weighted functionality at image creation time. E.g. on most of my systems, I create two kinds of images: a full featured, systemd based one and a very minimal rescue system with busybox and some filesystem utilities. With recent systemd packaging change, the rescue image size grow up from 5.9 MiB to 27 MiB because systemd dependencies are hardcoded in mandatory packages. Formerly, systemd dependencies could be avoided by adding the -systemd packages to BAD_RECOMMENDATIONS (e.g. due to busybox-syslog - busybox-syslog-systemd rrecommend). I am aware that initscripts were always part of the main package. But sysvinit was very lightweighted and the extra space either negligible or easy to recover by removing some files in IMAGE_PREPROCESS_COMMAND. Hence my recommendation: make the init manager an image feature again and create -systemd and -sysv packages with the corresponding scripts. OpenEmbedded is still for embedded devices where size matters. Of course, systemd can be still a distribution feature to enable things like socket activation as part of PACKAGE_CONFIG. But dependencies on init system packages should be RRECOMMENDS which can be overridden easily at image creation time. The trouble is that by making it an image feature, people will expect *everything* to work properly and to be able to have fully functional sysvinit and systemd variants of images. We already see this expectation. Trying to explain to people what the limitations are, what is expected to work and what isn't will be difficult. I know you understand this but going forward most people will simply not and I think it will be a nightmare for usability. You said you already see it so what is the change? For me less flexibility is more problem and the result are more ugly hacks. I've been using the systemd system in meta-oe in product for long time and it works fine. In same distro I build other product with is sysvinit based and it also works fine. Now I have a nightmare in upgrade path, a change in the way my customer work and as a plus the need to make two different distributions for same product. No sense to me. Final result? This product won't go out of danny as I won't be able to provide an usable upgrade path without forking OE or having an insane amount of bbappend hacks. When systemd was added to meta-oe, it was clearly mentioned that it was a proof of concept to experiment with integrating it and subject to change until it made it into OE-Core. The fact is is changing is therefore not a surprise. We do need to think about making sure we have equivalent functionality, we also need to think about migration but the fact changes happened with some experimental work should not surprise people. I appreciate that isn't ideal but it is the way it is. Sure; I'd expect a change for the better ... besides all the work done in meta-oe lead it to a proper and good implementation (having it being split and drop the contamination of images we had first implementation). Having it as a distro feature is a good option too but all package split, logic and implementation of rest were good and flexible. For that reason I'd rather see this done in a different way, for example blacklisting the problematic systemd dependencies at image generation time with some kind of stronger BAD_RECOMMENDS code. Yes, this means there would be some systemd config files lying around but you'd stop the main bulk of it coming in (and as you said, some small config files aren't an issue). Check the amount of changes for do_install_append has sent today. This was all not need with the code we had and which works fine. This is just duplicated code all over recipes. Seriously the systemd integration was done in complete wrong way in my POV. It broght up problems we have fixed and do major improvements in meta-oe ... at no profit. Sorry you feel that way. I'm equally frustrated with the way systemd was added to meta-oe and the damage it did to the overall usability of the OE ecosystem. I'm trying to take positive actions from that such as ensuring we never do something like it again. Agreed. This
Re: [OE-core] RFE: make the init manager an image feature (again)
On Sun, Feb 24, 2013 at 10:04:42PM +, Ross Burton wrote: On Sunday, 24 February 2013 at 14:06, Otavio Salvador wrote: DISTRO_FEATURES contains the init script *style* that you want: sysvinit or systemd. These are not mutually exclusive so specifying both will get you both directly in packages that support both. I'm still not convinced we need to split these out into separate packages, the size impact is practically negligible and the dependencies are effectively redundant as you'll have trouble booting without an init system. Instead the postinsts should wrap the initscript fragments in checks. The size impact it not negligible; specially for initramfs images but what concerns me even more is the upgrade path from previous users of meta-oe systemd. I obviously didn't make myself clear - the size impact is negligible when you're talking about just the init script - the dependencies on systemd/updatercd could be recommends at most, as the postinst scripts could check what init system they have before calling any tools. Either way a rescue image that boots using busybox init shouldn't have systemd, clearly. I'd like to have an upgrade path. Well, strictly speaking oe-core itself doesn't have an upgrade path to consider… Why can't any distros that shipped with meta-systemd (or just in meta-systemd) have an include that injects the RPROVIDES/RREPLACES/RCONFLICTS for the packages that they enabled? That would force multiple distro layers to maintain many .bbappends with just RPROVIDES/RREPLACES/RCONFLICTS and PRINC bump maintaned forever. It was pain to rename .bbappends in meta-systemd after each .bb upgrade, so if we have to do that, then we should rather keep meta-systemd as layer so that we can share .bbappend maintenance for all distributions in one place. -- Martin 'JaMa' Jansa jabber: martin.ja...@gmail.com signature.asc Description: Digital signature ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
Re: [OE-core] RFE: make the init manager an image feature (again)
On Sun, Feb 24, 2013 at 11:04 PM, Ross Burton ross.bur...@intel.com wrote: The size impact it not negligible; specially for initramfs images but what concerns me even more is the upgrade path from previous users of meta-oe systemd. I obviously didn't make myself clear - the size impact is negligible when you're talking about just the init script - the dependencies on systemd/updatercd could be recommends at most, as the postinst scripts could check what init system they have before calling any tools. Either way a rescue image that boots using busybox init shouldn't have systemd, clearly. Maybe I missed something but I have seen some arguments for splitting initscripts into single packages - but I can't remember one for having all in one package. Maybe somebody can help me out here. Andreas ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
Re: [OE-core] RFE: make the init manager an image feature (again)
Burton, Ross ross.bur...@intel.com writes:
But it doesn't need to be as dangerous as binconfig.bbclass, because
we already list .service or .socket files in SYSTEMD_SERVICE so we
can improve that find call
Why is 'find' required at all? afaik, only files from $SRC_URI are
affected. So we can
1. create an (overridable) SYSTEMD_EXTRA_SERVICES variable
2. fill this variable in systemd.bbclass with .service, .target, .socket
+ .mount files from $SRC_URI
3. modify systemd's do_install so that files above are copied after
doing some some sanity checks (e.g. checks that no previous version
from 'make install' exists or that it are really systemd files).
As I've said before you'll need to be pre-processing these files
anyway,
oh... this means khem's meta-systemd: Append ${PN} to SYSTEMD_SERVICE
patch series is incomplete and all the do_install_append() need to get
yet more complicated
It is really time, to move the additional-service-file installation back
into the class.
so either the class gets *even more* logic or you just deal with it.
Over time more and more upstreams will get systemd support so we'll
be carrying overcomplicated logic for no reason. We're really just
talking about one conditional and a sed call here.
By applying some general rules (e.g. using autoconf like '@bindir@'
templates), this can/should be done by the class and not in every
recipe.
Enrico
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
Re: [OE-core] RFE: make the init manager an image feature (again)
On 21 February 2013 10:34, Enrico Scholz
enrico.sch...@sigma-chemnitz.de wrote:
oh... this means khem's meta-systemd: Append ${PN} to SYSTEMD_SERVICE
patch series is incomplete and all the do_install_append() need to get
yet more complicated
What was originally in meta-systemd, and what is there by simply
adding a do_install_append that just does install, is plain wrong.
The moment someone changes the directory structure. it's broken.
You only have appends where you don't do the work in the real class.
Remember that as oe-core supports systemd, all other layers can
support it too. meta-systemd can disappear, and more upstream over
time will also integrate systemd unit files directly.
Ross
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
Re: [OE-core] RFE: make the init manager an image feature (again)
Burton, Ross ross.burton-ral2jqcrhueavxtiumw...@public.gmane.org writes: more upstream over time will also integrate systemd unit files directly. When in 5 or 10 years everybody switched to systemd and installs its service files by itself, we can mark the relevant code in the class as deprecated and remove it. But now, the oe-core systemd implementation causes redundant implementations of non trivial code in several packages. Enrico ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
Re: [OE-core] RFE: make the init manager an image feature (again)
On Thu, Feb 21, 2013 at 8:34 AM, Enrico Scholz enrico.sch...@sigma-chemnitz.de wrote: Burton, Ross ross.burton-ral2jqcrhueavxtiumw...@public.gmane.org writes: more upstream over time will also integrate systemd unit files directly. When in 5 or 10 years everybody switched to systemd and installs its service files by itself, we can mark the relevant code in the class as deprecated and remove it. But now, the oe-core systemd implementation causes redundant implementations of non trivial code in several packages. I fully agree; we have many cases where classes workaround system issues/limitations to avoid code duplications so I see no reason why this needs to be different with systemd. -- Otavio Salvador O.S. Systems E-mail: ota...@ossystems.com.br http://www.ossystems.com.br Mobile: +55 53 9981-7854 http://projetos.ossystems.com.br ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
Re: [OE-core] RFE: make the init manager an image feature (again)
On Thu, 2013-02-21 at 08:50 -0300, Otavio Salvador wrote:
I fully agree; we have many cases where classes workaround system
issues/limitations to avoid code duplications so I see no reason why
this needs to be different with systemd.
If you wanted to propose an addition to the class that purely abstracted
out the duplication (without adding any potential bear-traps) then I
would have thought that should be fine.
For example, something along the approximate lines of:
install_systemd_collateral_file() {
if ${@base_contains('DISTRO_FEATURES', 'systemd', 'true', 'false')}; then
sed 's/@bindir@/${bindir}/' $1 $2
fi
}
seems like it would address most of the concerns about code duplication
without introducing any hairy heuristics or unpredictably spontaneous
behaviour in the class.
p.
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
Re: [OE-core] RFE: make the init manager an image feature (again)
Hi, This thread started sprawling, so I'll do my best to cover all the points. This is also mainly an attempt to get more information as to how people are using init managers, as it's still not very clear what people want beyond choice. With recent systemd packaging change, the rescue image size grow up from 5.9 MiB to 27 MiB because systemd dependencies are hardcoded in mandatory packages. This certainly can happen. core-image-minimal-initramfs went from 19M up to 35M if you turn on systemd with the (unmerged) busybox enabling turned on. Investigating this shows that the cause was busybox recommending busybox-syslog which depends on systemd, which isn't useful in the initramfs. My branch adds busybox-syslog to BAD_RECOMMENDS and it's back down to 19M. This works as this image has a custom /sbin/init so it doesn't use an init manager at all. When people are building images with and without systemd, how are the non-systemd images booting? If sysvinit, what's the rationale for this? Will you consider switching to systemd for everything if the disk increase was only a new MB, on the grounds of consistent booting behaviour? I can imagine many rescue or initramfs images are using a custom /init, but as these don't/shouldn't actually have any daemons installed this is practically a solved problem. systemd as integrated is currently too big for very constrained environments, and I'm not denying that. Using this core-image-minimal-initramfs as a test whilst pulling in systemd I've been cleaning up spurious dependencies and have managed to trim around 4MB from the footprint, but there's massive low-hanging fruit like a 4MB /lib/udev/hwdb.d. Rescue images likely don't need this, so we can split it out and not always install it (now down to a 5MB footprint compared to no init system). One massive problem with making init system an image time choice with packages is what happens when you have a feed with both pn-sysv and pn-systemd packages in. If after the initial construction you want to install a daemon, you'll have to know to manually install the right init package too. Supporting multiple init systems in the same packaging would be achievable, but would people who want this be happy with pulling the systemd libraries into sysvinit images? The alternative is that if sysvinit and systemd are enabled we can't do any real systemd enabling. The good thing is that we could certainly make systemd.bbclass inject a recommends instead of dependency on systemd and add guards around the systemd calls, ditto for update-rcd.bbclass. Regards, Ross ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
Re: [OE-core] RFE: make the init manager an image feature (again)
On Thu, Feb 21, 2013 at 12:35 PM, Burton, Ross ross.bur...@intel.com wrote: Hi, This thread started sprawling, so I'll do my best to cover all the points. This is also mainly an attempt to get more information as to how people are using init managers, as it's still not very clear what people want beyond choice. Awesome. I agree it is difficult to track in a such big thread. With recent systemd packaging change, the rescue image size grow up from 5.9 MiB to 27 MiB because systemd dependencies are hardcoded in mandatory packages. This certainly can happen. core-image-minimal-initramfs went from 19M up to 35M if you turn on systemd with the (unmerged) busybox enabling turned on. Investigating this shows that the cause was busybox recommending busybox-syslog which depends on systemd, which isn't useful in the initramfs. My branch adds busybox-syslog to BAD_RECOMMENDS and it's back down to 19M. This works as this image has a custom /sbin/init so it doesn't use an init manager at all. Right. However BAD_RECOMMENDS is a little *manual* ... more on this later ... When people are building images with and without systemd, how are the non-systemd images booting? If sysvinit, what's the rationale for Last time I tested no. systemd-udevd would need to be split from systemd package and have init scripts as an option. this? Will you consider switching to systemd for everything if the disk increase was only a new MB, on the grounds of consistent booting behaviour? I can imagine many rescue or initramfs images are using a custom /init, but as these don't/shouldn't actually have any daemons installed this is practically a solved problem. No; one of our products is build with an initramfs (using initramfs-framework) and it is much easier for this than the systemd. In some cases we use the init.d scripts provided by packages to allow reuse in initramfs-framework and it works fine. systemd as integrated is currently too big for very constrained environments, and I'm not denying that. Using this core-image-minimal-initramfs as a test whilst pulling in systemd I've been cleaning up spurious dependencies and have managed to trim around 4MB from the footprint, but there's massive low-hanging fruit like a 4MB /lib/udev/hwdb.d. Rescue images likely don't need this, so we can split it out and not always install it (now down to a 5MB footprint compared to no init system). Yes; this is a regression from the status we had in meta-oe *before* the merge in oe-core. It was more flexible. One massive problem with making init system an image time choice with packages is what happens when you have a feed with both pn-sysv and pn-systemd packages in. If after the initial construction you want to install a daemon, you'll have to know to manually install the right init package too. Yes; this is one problem. I think we might think about havig something in package manager that try to fullfil a virtual provide (as we had for locales). Supporting multiple init systems in the same packaging would be achievable, but would people who want this be happy with pulling the systemd libraries into sysvinit images? The alternative is that if sysvinit and systemd are enabled we can't do any real systemd enabling. The good thing is that we could certainly make systemd.bbclass inject a recommends instead of dependency on systemd and add guards around the systemd calls, ditto for update-rcd.bbclass. It depends. If I have 'systemd' enabled in the distro I am sure expecting it to happen. If I don't, I don't expect it to happen. -- Otavio Salvador O.S. Systems E-mail: ota...@ossystems.com.br http://www.ossystems.com.br Mobile: +55 53 9981-7854 http://projetos.ossystems.com.br ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
Re: [OE-core] RFE: make the init manager an image feature (again)
Burton, Ross ross.burton-ral2jqcrhueavxtiumw...@public.gmane.org writes: With recent systemd packaging change, the rescue image size grow up from 5.9 MiB to 27 MiB because systemd dependencies are hardcoded in mandatory packages. This certainly can happen. core-image-minimal-initramfs went from 19M up to 35M if you turn on systemd with the (unmerged) busybox enabling turned on. Investigating this shows that the cause was busybox recommending busybox-syslog which depends on systemd, which isn't useful in the initramfs. My branch adds busybox-syslog to BAD_RECOMMENDS and it's back down to 19M. I want busybox-syslogd in the rescue image. And an ntp + dhcp client and http server. Next might be tools from util-linux which depends on systemd too. When people are building images with and without systemd, how are the non-systemd images booting? Custom /init which executes busybox's /sbin/init finally. If sysvinit, what's the rationale for this? It allows modularization (starting new services without editing /init) with zero costs (busybox-init is already there). I need full control over things like attaching/detecting ubi volumes or sd cards and I feel much better, when every line of relevant code was written by me ;) Will you consider switching to systemd for everything if the disk increase was only a new MB, on the grounds of consistent booting behaviour? Definitively not. There is far too much logic in usual systemd/udev systems (e.g. accessing block devices and other hardware during udev scanning) which contradicts with purposes of a rescue system. One massive problem with making init system an image time choice with packages is what happens when you have a feed with both pn-sysv and pn-systemd packages in. I wrote some ideas here[1]. It starts with moving init script subpackages into separate 'all-initsys' feeds till implementing the BAD_RECOMMENDS mechanism in the depsolvers. For me, it would be also okay, when runtime package management gets ignored at all (rescue system are initramfs images without pkg management) and making the initmgr in DISTRO_FEATURES selecting the default RRECOMMENDS. The actual BAD_RECOMMENDATIONS mechanism works perfect in this case; adding interesting -sysv subpackages manually when creating the rescue image would be ok for me. The alternative is that if sysvinit and systemd are enabled we can't do any real systemd enabling. Why not? The functions in libsystemd are robust enough so that daemons are still working in non-systemd systems. Enrico Footnotes: [1] http://lists.linuxtogo.org/pipermail/openembedded-core/2013-February/035998.html ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
Re: [OE-core] RFE: make the init manager an image feature (again)
On 18 February 2013 10:17, Enrico Scholz enrico.sch...@sigma-chemnitz.de wrote: But it doesn't need to be as dangerous as binconfig.bbclass, because we already list .service or .socket files in SYSTEMD_SERVICE so we can improve that find call Why is 'find' required at all? afaik, only files from $SRC_URI are affected. So we can 1. create an (overridable) SYSTEMD_EXTRA_SERVICES variable 2. fill this variable in systemd.bbclass with .service, .target, .socket + .mount files from $SRC_URI 3. modify systemd's do_install so that files above are copied after doing some some sanity checks (e.g. checks that no previous version from 'make install' exists or that it are really systemd files). As I've said before you'll need to be pre-processing these files anyway, so either the class gets *even more* logic or you just deal with it. Over time more and more upstreams will get systemd support so we'll be carrying overcomplicated logic for no reason. We're really just talking about one conditional and a sed call here. Ross ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
Re: [OE-core] RFE: make the init manager an image feature (again)
Martin Jansa martin.jansa-re5jqeeqqe8avxtiumw...@public.gmane.org writes: On the specifics of the do_install_append, you've seen my comments about how we're not learning from past mistakes with the way the do_install in the class was written. I note Phil also agreed with them, both of us remembering some of the horrors we've dealt with in the past (and binconfig.bbclass is still around, sadly). But it doesn't need to be as dangerous as binconfig.bbclass, because we already list .service or .socket files in SYSTEMD_SERVICE so we can improve that find call Why is 'find' required at all? afaik, only files from $SRC_URI are affected. So we can 1. create an (overridable) SYSTEMD_EXTRA_SERVICES variable 2. fill this variable in systemd.bbclass with .service, .target, .socket + .mount files from $SRC_URI 3. modify systemd's do_install so that files above are copied after doing some some sanity checks (e.g. checks that no previous version from 'make install' exists or that it are really systemd files). Enrico ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
Re: [OE-core] RFE: make the init manager an image feature (again)
Richard Purdie richard.pur...@linuxfoundation.org writes: meta-oe earned a *horrendous* reputation because of the way systemd was implemented there. Can you point me to the corresponding discussion resp. which aspects of the meta-oe implementation were criticized? I can image only two problems with splitted -sysv/-systemd packages: 1. it enlarges the package database 2. there does not exist a good mechanism to select automatically the correct subpackage; e.g. so that when somebody makes 'opkg install foo', the -sysv subpackage gets installed on sysv systems and the -systemd subpackage on systemd systems. Ditto with rpm and dpkg package managers. meta-oe implementation defined a default with help of RRECOMMENDS which can be overridden at image build time with BAD_RECOMMENDS. For me, point 1 is negligible because pkg management is disabled for images where every byte matters. Point 2 needs some discussion but I am pretty sure that some clean technical solution can be found. The most trivial one might be the moving of -initsys packages into an own 'all-initsys' architecture whose package feed is enabled at image build time. This does not scale well and violates orthogonality of buildsystem somehow, but works now. Another approach might adding RRECOMMENDS on all supported -initsys packages and let the package manager choose this with the lowest costs. That's not trivial and requires probably changes in opkg. Blacklisting packages with a certain, configurable pattern in its name or (better) rprovides (e.g. implementing BAD_RECOMMENDS in opkg, smart, ...) is another way requiring changes in the package manager. I believe (as do a number of others) that it has damaged OE's reputation and usability and actively hurts new users. The current oe-core implementation hurts active, long time users. It was clear systemd needed to move into the core but also become more configurable to work for everyone. Exactly this configurability was taken away. I don't want to remove features, I do want to talk about whether there is a better way we can fulfil certain uses cases, particularly with a focus on usability. For me, a good usability of a build system is defined by a small, orthogonal and powerful set of tools and rules. Implementing hacks for specific use cases (e.g. magically remove the busybox or util-linux - systemd dep for rescue systems) violates this and will probably cause bad surprises (e.g. for people who want 'lighttp' in the rescue system and when the hack has not been applied to this package). I'm actually moderately annoyed that throughout the various discussions about systemd and how we'd get it into OE-Core nobody actually mentioned these specific requirements until very late in the implementation. That's a general problem with oe release model. Parts of systemd (in -core) were changed without adapting the other ones (in -meta). This made the whole oe unbuildable for me for one week and I did not saw the problem with the rescue image hence. Assuming we are able to break the hard dependencies, what is with package scripts which require programs, files or directories from these deps? Do we need a way to differ between good and bad script failures then? At the technical level there is little difference from packaging these things separately to avoid specific dependencies and explicitly telling the package manager to avoid that dependency instead. see Martin's comment about post/pre scripts Equally there are other cases where people do want to break specific dependencies so this could help us in that area too. Dependencies and recommends are good and moving things into subpackages is *the* way to break them. Enrico ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
Re: [OE-core] RFE: make the init manager an image feature (again)
On Sat, Feb 16, 2013 at 12:53:07PM +, Richard Purdie wrote: On the specifics of the do_install_append, you've seen my comments about how we're not learning from past mistakes with the way the do_install in the class was written. I note Phil also agreed with them, both of us remembering some of the horrors we've dealt with in the past (and binconfig.bbclass is still around, sadly). But it doesn't need to be as dangerous as binconfig.bbclass, because we already list .service or .socket files in SYSTEMD_SERVICE so we can improve that find call to install only stuff we know we need and error out if it's not found or if there are multiple versions of that file in WORKDIR. -- Martin 'JaMa' Jansa jabber: martin.ja...@gmail.com signature.asc Description: Digital signature ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
Re: [OE-core] RFE: make the init manager an image feature (again)
On Fri, 2013-02-15 at 19:19 +0100, Enrico Scholz wrote: it would be nice when the decision to make the init manager a distribution feature will be reverted to the old oe-meta mechanism. Being a distribution feature means, that packages are created in such a way that it is impossible to split off unwanted and heavy weighted functionality at image creation time. E.g. on most of my systems, I create two kinds of images: a full featured, systemd based one and a very minimal rescue system with busybox and some filesystem utilities. With recent systemd packaging change, the rescue image size grow up from 5.9 MiB to 27 MiB because systemd dependencies are hardcoded in mandatory packages. Formerly, systemd dependencies could be avoided by adding the -systemd packages to BAD_RECOMMENDATIONS (e.g. due to busybox-syslog - busybox-syslog-systemd rrecommend). I am aware that initscripts were always part of the main package. But sysvinit was very lightweighted and the extra space either negligible or easy to recover by removing some files in IMAGE_PREPROCESS_COMMAND. Hence my recommendation: make the init manager an image feature again and create -systemd and -sysv packages with the corresponding scripts. OpenEmbedded is still for embedded devices where size matters. Of course, systemd can be still a distribution feature to enable things like socket activation as part of PACKAGE_CONFIG. But dependencies on init system packages should be RRECOMMENDS which can be overridden easily at image creation time. The trouble is that by making it an image feature, people will expect *everything* to work properly and to be able to have fully functional sysvinit and systemd variants of images. We already see this expectation. Trying to explain to people what the limitations are, what is expected to work and what isn't will be difficult. I know you understand this but going forward most people will simply not and I think it will be a nightmare for usability. For that reason I'd rather see this done in a different way, for example blacklisting the problematic systemd dependencies at image generation time with some kind of stronger BAD_RECOMMENDS code. Yes, this means there would be some systemd config files lying around but you'd stop the main bulk of it coming in (and as you said, some small config files aren't an issue). Cheers, Richard ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
Re: [OE-core] RFE: make the init manager an image feature (again)
On Sat, Feb 16, 2013 at 7:15 AM, Richard Purdie richard.pur...@linuxfoundation.org wrote: On Fri, 2013-02-15 at 19:19 +0100, Enrico Scholz wrote: it would be nice when the decision to make the init manager a distribution feature will be reverted to the old oe-meta mechanism. Being a distribution feature means, that packages are created in such a way that it is impossible to split off unwanted and heavy weighted functionality at image creation time. E.g. on most of my systems, I create two kinds of images: a full featured, systemd based one and a very minimal rescue system with busybox and some filesystem utilities. With recent systemd packaging change, the rescue image size grow up from 5.9 MiB to 27 MiB because systemd dependencies are hardcoded in mandatory packages. Formerly, systemd dependencies could be avoided by adding the -systemd packages to BAD_RECOMMENDATIONS (e.g. due to busybox-syslog - busybox-syslog-systemd rrecommend). I am aware that initscripts were always part of the main package. But sysvinit was very lightweighted and the extra space either negligible or easy to recover by removing some files in IMAGE_PREPROCESS_COMMAND. Hence my recommendation: make the init manager an image feature again and create -systemd and -sysv packages with the corresponding scripts. OpenEmbedded is still for embedded devices where size matters. Of course, systemd can be still a distribution feature to enable things like socket activation as part of PACKAGE_CONFIG. But dependencies on init system packages should be RRECOMMENDS which can be overridden easily at image creation time. The trouble is that by making it an image feature, people will expect *everything* to work properly and to be able to have fully functional sysvinit and systemd variants of images. We already see this expectation. Trying to explain to people what the limitations are, what is expected to work and what isn't will be difficult. I know you understand this but going forward most people will simply not and I think it will be a nightmare for usability. You said you already see it so what is the change? For me less flexibility is more problem and the result are more ugly hacks. I've been using the systemd system in meta-oe in product for long time and it works fine. In same distro I build other product with is sysvinit based and it also works fine. Now I have a nightmare in upgrade path, a change in the way my customer work and as a plus the need to make two different distributions for same product. No sense to me. Final result? This product won't go out of danny as I won't be able to provide an usable upgrade path without forking OE or having an insane amount of bbappend hacks. For that reason I'd rather see this done in a different way, for example blacklisting the problematic systemd dependencies at image generation time with some kind of stronger BAD_RECOMMENDS code. Yes, this means there would be some systemd config files lying around but you'd stop the main bulk of it coming in (and as you said, some small config files aren't an issue). Check the amount of changes for do_install_append has sent today. This was all not need with the code we had and which works fine. This is just duplicated code all over recipes. Seriously the systemd integration was done in complete wrong way in my POV. It broght up problems we have fixed and do major improvements in meta-oe ... at no profit. I know it is not the better thing to read but I think it is nice when we receive feedback as it allows for rethink of our path of work. Regards, -- Otavio Salvador O.S. Systems E-mail: ota...@ossystems.com.br http://www.ossystems.com.br Mobile: +55 53 9981-7854 http://projetos.ossystems.com.br ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
Re: [OE-core] RFE: make the init manager an image feature (again)
Richard Purdie richard.pur...@linuxfoundation.org writes: it would be nice when the decision to make the init manager a distribution feature will be reverted to the old oe-meta mechanism. The trouble is that by making it an image feature, people will expect *everything* to work properly and to be able to have fully functional sysvinit and systemd variants of images. I do not see an obvious reason why fully functional sysvinit, systemd and perhaps upstart image variants based on the same distribution/package set are impossible. Of course, not everything will work. But initmgr being a distribution feature makes some things completely impossible. We already see this expectation. IMO, removal of features just to lower expectations is the completely wrong way. Trying to explain to people what the limitations are, what is expected to work and what isn't will be difficult. OpenEmbedded is not an end-user distribution but for people who are willing to invest some learning effort. Trying to limit ourself on the lowest common ground is not desirable imo. For that reason I'd rather see this done in a different way, for example blacklisting the problematic systemd dependencies at image generation time with some kind of stronger BAD_RECOMMENDS code. Assuming we are able to break the hard dependencies, what is with package scripts which require programs, files or directories from these deps? Do we need a way to differ between good and bad script failures then? Sounds extremely hacky and fragile... Enrico ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
Re: [OE-core] RFE: make the init manager an image feature (again)
On Sat, 2013-02-16 at 12:57 +0100, Enrico Scholz wrote: Richard Purdie richard.pur...@linuxfoundation.org writes: it would be nice when the decision to make the init manager a distribution feature will be reverted to the old oe-meta mechanism. The trouble is that by making it an image feature, people will expect *everything* to work properly and to be able to have fully functional sysvinit and systemd variants of images. I do not see an obvious reason why fully functional sysvinit, systemd and perhaps upstart image variants based on the same distribution/package set are impossible. Of course, not everything will work. But initmgr being a distribution feature makes some things completely impossible. We already see this expectation. IMO, removal of features just to lower expectations is the completely wrong way. meta-oe earned a *horrendous* reputation because of the way systemd was implemented there. I believe (as do a number of others) that it has damaged OE's reputation and usability and actively hurts new users. Yes, the people who use systemd and meta-oe were happy. People who didn't want systemd were not. There continues to be a fairly toxic mix of distro policy mingled in with the recipes in there although good progress is being made in fixing that and I'm grateful to the people who've noticed and taken on that work (and done previous work like the separation of meta-systemd). It was clear systemd needed to move into the core but also become more configurable to work for everyone. I don't want to remove features, I do want to talk about whether there is a better way we can fulfil certain uses cases, particularly with a focus on usability. Trying to explain to people what the limitations are, what is expected to work and what isn't will be difficult. OpenEmbedded is not an end-user distribution but for people who are willing to invest some learning effort. Trying to limit ourself on the lowest common ground is not desirable imo. I did not say we're not going to support your use case. I'm asking if we can summarise exactly what the problem is and whether there is another way we can get there which isn't going to surprise as many people and be easier to use. I'm actually moderately annoyed that throughout the various discussions about systemd and how we'd get it into OE-Core nobody actually mentioned these specific requirements until very late in the implementation. At the bare minimum, we actually need to document the usecases people are using and require. Yes, you know your usecase but you need to take some responsibility for ensuring its documented and known about else it will continue to get broken time and time again. For that reason I'd rather see this done in a different way, for example blacklisting the problematic systemd dependencies at image generation time with some kind of stronger BAD_RECOMMENDS code. Assuming we are able to break the hard dependencies, what is with package scripts which require programs, files or directories from these deps? Do we need a way to differ between good and bad script failures then? At the technical level there is little difference from packaging these things separately to avoid specific dependencies and explicitly telling the package manager to avoid that dependency instead. Equally there are other cases where people do want to break specific dependencies so this could help us in that area too. Perhaps there is a dependency we need to drop to a Recommends level, then use BAD_RECOMMENDS to handy that. We do need to document why we do that and how it is expected to be used. Sounds extremely hacky and fragile... It depends on the implementation. Having initscripts in individual packages and having to play tricks to ensure the right sets of packages get installed is rather hacky and fragile. Cheers, Richard ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
Re: [OE-core] RFE: make the init manager an image feature (again)
On Sat, 2013-02-16 at 08:47 -0200, Otavio Salvador wrote: On Sat, Feb 16, 2013 at 7:15 AM, Richard Purdie richard.pur...@linuxfoundation.org wrote: On Fri, 2013-02-15 at 19:19 +0100, Enrico Scholz wrote: it would be nice when the decision to make the init manager a distribution feature will be reverted to the old oe-meta mechanism. Being a distribution feature means, that packages are created in such a way that it is impossible to split off unwanted and heavy weighted functionality at image creation time. E.g. on most of my systems, I create two kinds of images: a full featured, systemd based one and a very minimal rescue system with busybox and some filesystem utilities. With recent systemd packaging change, the rescue image size grow up from 5.9 MiB to 27 MiB because systemd dependencies are hardcoded in mandatory packages. Formerly, systemd dependencies could be avoided by adding the -systemd packages to BAD_RECOMMENDATIONS (e.g. due to busybox-syslog - busybox-syslog-systemd rrecommend). I am aware that initscripts were always part of the main package. But sysvinit was very lightweighted and the extra space either negligible or easy to recover by removing some files in IMAGE_PREPROCESS_COMMAND. Hence my recommendation: make the init manager an image feature again and create -systemd and -sysv packages with the corresponding scripts. OpenEmbedded is still for embedded devices where size matters. Of course, systemd can be still a distribution feature to enable things like socket activation as part of PACKAGE_CONFIG. But dependencies on init system packages should be RRECOMMENDS which can be overridden easily at image creation time. The trouble is that by making it an image feature, people will expect *everything* to work properly and to be able to have fully functional sysvinit and systemd variants of images. We already see this expectation. Trying to explain to people what the limitations are, what is expected to work and what isn't will be difficult. I know you understand this but going forward most people will simply not and I think it will be a nightmare for usability. You said you already see it so what is the change? For me less flexibility is more problem and the result are more ugly hacks. I've been using the systemd system in meta-oe in product for long time and it works fine. In same distro I build other product with is sysvinit based and it also works fine. Now I have a nightmare in upgrade path, a change in the way my customer work and as a plus the need to make two different distributions for same product. No sense to me. Final result? This product won't go out of danny as I won't be able to provide an usable upgrade path without forking OE or having an insane amount of bbappend hacks. When systemd was added to meta-oe, it was clearly mentioned that it was a proof of concept to experiment with integrating it and subject to change until it made it into OE-Core. The fact is is changing is therefore not a surprise. We do need to think about making sure we have equivalent functionality, we also need to think about migration but the fact changes happened with some experimental work should not surprise people. I appreciate that isn't ideal but it is the way it is. For that reason I'd rather see this done in a different way, for example blacklisting the problematic systemd dependencies at image generation time with some kind of stronger BAD_RECOMMENDS code. Yes, this means there would be some systemd config files lying around but you'd stop the main bulk of it coming in (and as you said, some small config files aren't an issue). Check the amount of changes for do_install_append has sent today. This was all not need with the code we had and which works fine. This is just duplicated code all over recipes. Seriously the systemd integration was done in complete wrong way in my POV. It broght up problems we have fixed and do major improvements in meta-oe ... at no profit. Sorry you feel that way. I'm equally frustrated with the way systemd was added to meta-oe and the damage it did to the overall usability of the OE ecosystem. I'm trying to take positive actions from that such as ensuring we never do something like it again. As for the integration into the core, I haven't been too involved with that and perhaps I should be. The trouble is I alone don't scale and I'm trying to ensure we have more people taking on this kind of work. This means letting others take it on and learn. There is a requirement from the community to ensure people working on improvements like this have all the needed data and that is something which I don't think happened well here. My point is that I think there have been failings in various places and the community needs to take some responsibility too. On the specifics of the do_install_append, you've seen my comments about how we're not learning from past
Re: [OE-core] RFE: make the init manager an image feature (again)
On Sat, Feb 16, 2013 at 10:34 AM, Richard Purdie richard.pur...@linuxfoundation.org wrote: On Sat, 2013-02-16 at 12:57 +0100, Enrico Scholz wrote: Richard Purdie richard.pur...@linuxfoundation.org writes: it would be nice when the decision to make the init manager a distribution feature will be reverted to the old oe-meta mechanism. The trouble is that by making it an image feature, people will expect *everything* to work properly and to be able to have fully functional sysvinit and systemd variants of images. I do not see an obvious reason why fully functional sysvinit, systemd and perhaps upstart image variants based on the same distribution/package set are impossible. Of course, not everything will work. But initmgr being a distribution feature makes some things completely impossible. We already see this expectation. IMO, removal of features just to lower expectations is the completely wrong way. meta-oe earned a *horrendous* reputation because of the way systemd was implemented there. I believe (as do a number of others) that it has damaged OE's reputation and usability and actively hurts new users. Yes, the people who use systemd and meta-oe were happy. People who didn't want systemd were not. There continues to be a fairly toxic mix of distro policy mingled in with the recipes in there although good progress is being made in fixing that and I'm grateful to the people who've noticed and taken on that work (and done previous work like the separation of meta-systemd). In the begging yes. This has been fixed in meta-oe/meta-systemd split and it allowed for both Worlds to be in peace again. It was clear systemd needed to move into the core but also become more configurable to work for everyone. I don't want to remove features, I do want to talk about whether there is a better way we can fulfil certain uses cases, particularly with a focus on usability. Agreed; but removing the -systemd packages this removed features and broke upgrade path for everyone using it. Plus it enforces the deployment of systemd things in every image so now we cannot have images with sysv OR systemd anymore. Trying to explain to people what the limitations are, what is expected to work and what isn't will be difficult. OpenEmbedded is not an end-user distribution but for people who are willing to invest some learning effort. Trying to limit ourself on the lowest common ground is not desirable imo. I did not say we're not going to support your use case. I'm asking if we can summarise exactly what the problem is and whether there is another way we can get there which isn't going to surprise as many people and be easier to use. I'm actually moderately annoyed that throughout the various discussions about systemd and how we'd get it into OE-Core nobody actually mentioned these specific requirements until very late in the implementation. Sorry Richard, I talked to people about it. If you check the first version of patchset I complained about it but it was ignored. At the bare minimum, we actually need to document the usecases people are using and require. Yes, you know your usecase but you need to take some responsibility for ensuring its documented and known about else it will continue to get broken time and time again. The use-case were covered by meta-oe/meta-systemd very well. So it was documented and working. -- Otavio Salvador O.S. Systems E-mail: ota...@ossystems.com.br http://www.ossystems.com.br Mobile: +55 53 9981-7854 http://projetos.ossystems.com.br ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
Re: [OE-core] RFE: make the init manager an image feature (again)
On Sat, Feb 16, 2013 at 10:53 AM, Richard Purdie richard.pur...@linuxfoundation.org wrote: On Sat, 2013-02-16 at 08:47 -0200, Otavio Salvador wrote: On Sat, Feb 16, 2013 at 7:15 AM, Richard Purdie richard.pur...@linuxfoundation.org wrote: On Fri, 2013-02-15 at 19:19 +0100, Enrico Scholz wrote: it would be nice when the decision to make the init manager a distribution feature will be reverted to the old oe-meta mechanism. Being a distribution feature means, that packages are created in such a way that it is impossible to split off unwanted and heavy weighted functionality at image creation time. E.g. on most of my systems, I create two kinds of images: a full featured, systemd based one and a very minimal rescue system with busybox and some filesystem utilities. With recent systemd packaging change, the rescue image size grow up from 5.9 MiB to 27 MiB because systemd dependencies are hardcoded in mandatory packages. Formerly, systemd dependencies could be avoided by adding the -systemd packages to BAD_RECOMMENDATIONS (e.g. due to busybox-syslog - busybox-syslog-systemd rrecommend). I am aware that initscripts were always part of the main package. But sysvinit was very lightweighted and the extra space either negligible or easy to recover by removing some files in IMAGE_PREPROCESS_COMMAND. Hence my recommendation: make the init manager an image feature again and create -systemd and -sysv packages with the corresponding scripts. OpenEmbedded is still for embedded devices where size matters. Of course, systemd can be still a distribution feature to enable things like socket activation as part of PACKAGE_CONFIG. But dependencies on init system packages should be RRECOMMENDS which can be overridden easily at image creation time. The trouble is that by making it an image feature, people will expect *everything* to work properly and to be able to have fully functional sysvinit and systemd variants of images. We already see this expectation. Trying to explain to people what the limitations are, what is expected to work and what isn't will be difficult. I know you understand this but going forward most people will simply not and I think it will be a nightmare for usability. You said you already see it so what is the change? For me less flexibility is more problem and the result are more ugly hacks. I've been using the systemd system in meta-oe in product for long time and it works fine. In same distro I build other product with is sysvinit based and it also works fine. Now I have a nightmare in upgrade path, a change in the way my customer work and as a plus the need to make two different distributions for same product. No sense to me. Final result? This product won't go out of danny as I won't be able to provide an usable upgrade path without forking OE or having an insane amount of bbappend hacks. When systemd was added to meta-oe, it was clearly mentioned that it was a proof of concept to experiment with integrating it and subject to change until it made it into OE-Core. The fact is is changing is therefore not a surprise. We do need to think about making sure we have equivalent functionality, we also need to think about migration but the fact changes happened with some experimental work should not surprise people. I appreciate that isn't ideal but it is the way it is. Sure; I'd expect a change for the better ... besides all the work done in meta-oe lead it to a proper and good implementation (having it being split and drop the contamination of images we had first implementation). Having it as a distro feature is a good option too but all package split, logic and implementation of rest were good and flexible. For that reason I'd rather see this done in a different way, for example blacklisting the problematic systemd dependencies at image generation time with some kind of stronger BAD_RECOMMENDS code. Yes, this means there would be some systemd config files lying around but you'd stop the main bulk of it coming in (and as you said, some small config files aren't an issue). Check the amount of changes for do_install_append has sent today. This was all not need with the code we had and which works fine. This is just duplicated code all over recipes. Seriously the systemd integration was done in complete wrong way in my POV. It broght up problems we have fixed and do major improvements in meta-oe ... at no profit. Sorry you feel that way. I'm equally frustrated with the way systemd was added to meta-oe and the damage it did to the overall usability of the OE ecosystem. I'm trying to take positive actions from that such as ensuring we never do something like it again. Agreed. This was fix when we moved it to meta-oe/meta-systemd. As for the integration into the core, I haven't been too involved with that and perhaps I should be. The trouble is I alone don't scale and I'm Agreed.
Re: [OE-core] RFE: make the init manager an image feature (again)
On Sat, Feb 16, 2013 at 12:34:58PM +, Richard Purdie wrote: On Sat, 2013-02-16 at 12:57 +0100, Enrico Scholz wrote: Richard Purdie richard.pur...@linuxfoundation.org writes: it would be nice when the decision to make the init manager a distribution feature will be reverted to the old oe-meta mechanism. The trouble is that by making it an image feature, people will expect *everything* to work properly and to be able to have fully functional sysvinit and systemd variants of images. I do not see an obvious reason why fully functional sysvinit, systemd and perhaps upstart image variants based on the same distribution/package set are impossible. Of course, not everything will work. But initmgr being a distribution feature makes some things completely impossible. We already see this expectation. IMO, removal of features just to lower expectations is the completely wrong way. meta-oe earned a *horrendous* reputation because of the way systemd was implemented there. I believe (as do a number of others) that it has damaged OE's reputation and usability and actively hurts new users. Yes, the people who use systemd and meta-oe were happy. People who didn't want systemd were not. There continues to be a fairly toxic mix of distro policy mingled in with the recipes in there although good progress is being made in fixing that and I'm grateful to the people who've noticed and taken on that work (and done previous work like the separation of meta-systemd). I was using sysvinit images even with systemd in meta-oe, setting 2 VIRTUAL_RUNTIME variables and 1 .bbappend was enough to keep sysvinit images working. I agree that we were missing documentation for that and new users weren't able to figure it out for themselves. It was clear systemd needed to move into the core but also become more configurable to work for everyone. I don't want to remove features, I do want to talk about whether there is a better way we can fulfil certain uses cases, particularly with a focus on usability. I was talking with Ross about systemd as DISTRO_FEATURE, I agreed that it's fine to have it as DISTRO_FEATURE but I didn't expect that it also means that he wants to move PN-systemd to PN without upgrade path and possibility to create image without PN-systemd packages. My expectation was that systemd in DISTRO_FEATURES will enable systemd.bbclass functionality (same functionality as meta-systemd/system.bbclass had), not that it will force systemd and only systemd. Like with 3g in DISTRO_FEATURES we expect that DISTRO supports 3g but not that every possible image and MACHINE will provide 3g functionality. I can imagine distribution with sysvinit+upstart+systemd in DISTRO_FEATURES if they are careful enough to prepare images with only right packages. Trying to explain to people what the limitations are, what is expected to work and what isn't will be difficult. OpenEmbedded is not an end-user distribution but for people who are willing to invest some learning effort. Trying to limit ourself on the lowest common ground is not desirable imo. I did not say we're not going to support your use case. I'm asking if we can summarise exactly what the problem is and whether there is another way we can get there which isn't going to surprise as many people and be easier to use. I'm actually moderately annoyed that throughout the various discussions about systemd and how we'd get it into OE-Core nobody actually mentioned these specific requirements until very late in the implementation. I think we all replied on first patch to move from PN-systemd to PN. At the bare minimum, we actually need to document the usecases people are using and require. Yes, you know your usecase but you need to take some responsibility for ensuring its documented and known about else it will continue to get broken time and time again. For that reason I'd rather see this done in a different way, for example blacklisting the problematic systemd dependencies at image generation time with some kind of stronger BAD_RECOMMENDS code. Assuming we are able to break the hard dependencies, what is with package scripts which require programs, files or directories from these deps? Do we need a way to differ between good and bad script failures then? At the technical level there is little difference from packaging these things separately to avoid specific dependencies and explicitly telling the package manager to avoid that dependency instead. Equally there are other cases where people do want to break specific dependencies so this could help us in that area too. Perhaps there is a dependency we need to drop to a Recommends level, then use BAD_RECOMMENDS to handy that. We do need to document why we do that and how it is expected to be used. I think the problem Enrico describes is that systemctl calls were contained to PN-systemd postinst/prerm
Re: [OE-core] RFE: make the init manager an image feature (again)
On Sat, Feb 16, 2013 at 5:40 PM, Martin Jansa martin.ja...@gmail.com wrote: On Sat, Feb 16, 2013 at 12:34:58PM +, Richard Purdie wrote: On Sat, 2013-02-16 at 12:57 +0100, Enrico Scholz wrote: Richard Purdie richard.pur...@linuxfoundation.org writes: it would be nice when the decision to make the init manager a distribution feature will be reverted to the old oe-meta mechanism. The trouble is that by making it an image feature, people will expect *everything* to work properly and to be able to have fully functional sysvinit and systemd variants of images. I do not see an obvious reason why fully functional sysvinit, systemd and perhaps upstart image variants based on the same distribution/package set are impossible. Of course, not everything will work. But initmgr being a distribution feature makes some things completely impossible. We already see this expectation. IMO, removal of features just to lower expectations is the completely wrong way. meta-oe earned a *horrendous* reputation because of the way systemd was implemented there. I believe (as do a number of others) that it has damaged OE's reputation and usability and actively hurts new users. Yes, the people who use systemd and meta-oe were happy. People who didn't want systemd were not. There continues to be a fairly toxic mix of distro policy mingled in with the recipes in there although good progress is being made in fixing that and I'm grateful to the people who've noticed and taken on that work (and done previous work like the separation of meta-systemd). I was using sysvinit images even with systemd in meta-oe, setting 2 VIRTUAL_RUNTIME variables and 1 .bbappend was enough to keep sysvinit images working. I agree that we were missing documentation for that and new users weren't able to figure it out for themselves. It was clear systemd needed to move into the core but also become more configurable to work for everyone. I don't want to remove features, I do want to talk about whether there is a better way we can fulfil certain uses cases, particularly with a focus on usability. I was talking with Ross about systemd as DISTRO_FEATURE, I agreed that it's fine to have it as DISTRO_FEATURE but I didn't expect that it also means that he wants to move PN-systemd to PN without upgrade path and possibility to create image without PN-systemd packages. My expectation was that systemd in DISTRO_FEATURES will enable systemd.bbclass functionality (same functionality as meta-systemd/system.bbclass had), not that it will force systemd and only systemd. Like with 3g in DISTRO_FEATURES we expect that DISTRO supports 3g but not that every possible image and MACHINE will provide 3g functionality. I can imagine distribution with sysvinit+upstart+systemd in DISTRO_FEATURES if they are careful enough to prepare images with only right packages. Trying to explain to people what the limitations are, what is expected to work and what isn't will be difficult. OpenEmbedded is not an end-user distribution but for people who are willing to invest some learning effort. Trying to limit ourself on the lowest common ground is not desirable imo. I did not say we're not going to support your use case. I'm asking if we can summarise exactly what the problem is and whether there is another way we can get there which isn't going to surprise as many people and be easier to use. I'm actually moderately annoyed that throughout the various discussions about systemd and how we'd get it into OE-Core nobody actually mentioned these specific requirements until very late in the implementation. I think we all replied on first patch to move from PN-systemd to PN. At the bare minimum, we actually need to document the usecases people are using and require. Yes, you know your usecase but you need to take some responsibility for ensuring its documented and known about else it will continue to get broken time and time again. For that reason I'd rather see this done in a different way, for example blacklisting the problematic systemd dependencies at image generation time with some kind of stronger BAD_RECOMMENDS code. Assuming we are able to break the hard dependencies, what is with package scripts which require programs, files or directories from these deps? Do we need a way to differ between good and bad script failures then? At the technical level there is little difference from packaging these things separately to avoid specific dependencies and explicitly telling the package manager to avoid that dependency instead. Equally there are other cases where people do want to break specific dependencies so this could help us in that area too. Perhaps there is a dependency we need to drop to a Recommends level, then use BAD_RECOMMENDS to handy that. We do need to document why we do that and how it is expected to be used. I think the problem Enrico
Re: [OE-core] RFE: make the init manager an image feature (again)
On Fri, Feb 15, 2013 at 4:19 PM, Enrico Scholz enrico.sch...@sigma-chemnitz.de wrote: Hello, it would be nice when the decision to make the init manager a distribution feature will be reverted to the old oe-meta mechanism. Being a distribution feature means, that packages are created in such a way that it is impossible to split off unwanted and heavy weighted functionality at image creation time. E.g. on most of my systems, I create two kinds of images: a full featured, systemd based one and a very minimal rescue system with busybox and some filesystem utilities. With recent systemd packaging change, the rescue image size grow up from 5.9 MiB to 27 MiB because systemd dependencies are hardcoded in mandatory packages. Formerly, systemd dependencies could be avoided by adding the -systemd packages to BAD_RECOMMENDATIONS (e.g. due to busybox-syslog - busybox-syslog-systemd rrecommend). I am aware that initscripts were always part of the main package. But sysvinit was very lightweighted and the extra space either negligible or easy to recover by removing some files in IMAGE_PREPROCESS_COMMAND. Hence my recommendation: make the init manager an image feature again and create -systemd and -sysv packages with the corresponding scripts. OpenEmbedded is still for embedded devices where size matters. Of course, systemd can be still a distribution feature to enable things like socket activation as part of PACKAGE_CONFIG. But dependencies on init system packages should be RRECOMMENDS which can be overridden easily at image creation time. I fully support this! I also want this flexibility back (in fact I see no reason why it has been dropped). -- Otavio Salvador O.S. Systems E-mail: ota...@ossystems.com.br http://www.ossystems.com.br Mobile: +55 53 9981-7854 http://projetos.ossystems.com.br ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
Re: [OE-core] RFE: make the init manager an image feature (again)
On Fri, Feb 15, 2013 at 04:47:37PM -0200, Otavio Salvador wrote: On Fri, Feb 15, 2013 at 4:19 PM, Enrico Scholz enrico.sch...@sigma-chemnitz.de wrote: Hello, it would be nice when the decision to make the init manager a distribution feature will be reverted to the old oe-meta mechanism. Being a distribution feature means, that packages are created in such a way that it is impossible to split off unwanted and heavy weighted functionality at image creation time. E.g. on most of my systems, I create two kinds of images: a full featured, systemd based one and a very minimal rescue system with busybox and some filesystem utilities. With recent systemd packaging change, the rescue image size grow up from 5.9 MiB to 27 MiB because systemd dependencies are hardcoded in mandatory packages. Formerly, systemd dependencies could be avoided by adding the -systemd packages to BAD_RECOMMENDATIONS (e.g. due to busybox-syslog - busybox-syslog-systemd rrecommend). I am aware that initscripts were always part of the main package. But sysvinit was very lightweighted and the extra space either negligible or easy to recover by removing some files in IMAGE_PREPROCESS_COMMAND. Hence my recommendation: make the init manager an image feature again and create -systemd and -sysv packages with the corresponding scripts. OpenEmbedded is still for embedded devices where size matters. Of course, systemd can be still a distribution feature to enable things like socket activation as part of PACKAGE_CONFIG. But dependencies on init system packages should be RRECOMMENDS which can be overridden easily at image creation time. I fully support this! I also want this flexibility back (in fact I see no reason why it has been dropped). me too -- Martin 'JaMa' Jansa jabber: martin.ja...@gmail.com signature.asc Description: Digital signature ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
