Re: no systemd in containers: Requires -> Recommends

[Rex Dieter]

Jason L Tibbitts III wrote:

>> "RWMJ" == Richard W M Jones  writes:
> 
> RWMJ> Couldn't it use inotify (or whatever we're using these days to
> RWMJ> detect filesystem changes)?  So when you drop in the unit file,
> RWMJ> systemd notices and reloads.
> 
> Well, the point is that systemd isn't running or even installed when the
> daemons are installed.  The question is what happens when systemd comes
> up later.  (And yes, systemd could use rpm file triggers so that we can
> drop the scriptlets entirely.  That would be great, but it's an entirely
> separate discussion.)
> 
> I know this thread has gone in a completely different direction, but the
> original message was about modifying or removing the dependencies.  Can
> we drop dependencies on systemd and still have a system that works as
> expected if systemd gets installed later?

One way to help make that work would be to adjust existing systemd 
scriptlets to use triggers (on systemd)

(that's probably not practical though)

-- Rex
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: no systemd in containers: Requires -> Recommends

[Gerd Hoffmann]

  Hi,

> It doesn't help that there doesn't seem to be a way to use systemd's
> own container technologies to do these things in a more lightweight,
> yet compatible fashion. nspawn currently only does OS style
> containers, where you have another PID 1 inside.

Hmm?

I have a fedora 23 container on my rhel workstation, mostly for
testbuilds.

I can of course do a full-blown boot (then login, install/update
things, ...).

I can also run stuff directly in the container just fine, i.e.
"systemd-nspawn -D $root --bind /home make -C $HOME/src/$project".

cheers,
  Gerd
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: no systemd in containers: Requires -> Recommends

[Lennart Poettering]

On Thu, 17.12.15 22:22, Richard W.M. Jones (rjo...@redhat.com) wrote:

> On Thu, Dec 17, 2015 at 02:35:03PM -0600, Jason L Tibbitts III wrote:
> > > "HH" == Harald Hoyer  writes:
> > 
> > HH> The preset enablement would be missing.
> > 
> > Couldn't systemd simply apply presets when it is installed?  (Not
> > upgraded, but on a fresh install?)
> 
> Couldn't it use inotify (or whatever we're using these days to detect
> filesystem changes)?  So when you drop in the unit file, systemd
> notices and reloads.

It would neither solve this problem (since we are not running then),
nor is this somethign we can ever support. Watching config changes
with inotify is calling for trouble, as there's no concept for file
system transactions on Linux, and your daemon might end up rereading
the configuration at the wrong time, for example, while a file is
being moved around, and then you end up reading it twice under two
names, or not at all.

Lennart

-- 
Lennart Poettering, Red Hat
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: no systemd in containers: Requires -> Recommends

[Reindl Harald]

Am 18.12.2015 um 15:39 schrieb Neil Horman:

On Thu, Dec 17, 2015 at 10:18:10PM +0100, Reindl Harald wrote:

What I hear you saying is that on a system that has nothing better to do, the
primary monitoring process wakes up periodically to check on various system
aspects (cron jobs, journal rotates, etc).  That sounds like working as designed
to me, not a reason to throw the proverbial baby out with the bath water.  That
seems like a reason to do some tuning.


it is wasting ressources in a container running only one process - period -
what is there to discuss - in any serious setup there shou,ld be nothing
installed which is not *really* needed


You're making two assumptions here:
1) That there is only one process
2) That systemd (or a component thereof) isn't needed

While you're use case is certainly valid, it isn't the only use case by a long
shot.  And if the goal for your use case is to minimize the footprint of a
container, you can already do that (as I've pointed out previously)


i don't make *any* assumptions

since kernel-core pull systemd anyways there is not much for discussion, 
by default systemd is there - so everybody is happy


the only difference is *where you know* you don't need it you are *able* 
to remove it




signature.asc
Description: OpenPGP digital signature
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: no systemd in containers: Requires -> Recommends

[Matthew Miller]

On Fri, Dec 18, 2015 at 03:36:21PM -, g...@cfware.com wrote:
> I have no experience with containers so forgive me if I'm missing
> something. Why couldn't you just have a 'microinit.rpm' in a separate
> dnf repo, put 'Obsoletes: systemd' into that package? This way people
> who are building hundreds of containers that do not require systemd
> can use the repo containing microinit and it will take the place of
> systemd. RPM macro's could be provided by the microinit rpm, so it
> would provide reasonable replacements for %systemd_requires,
> %systemd_post, etc. I think this could solve your concern with
> minimum (no) impact to bare metal installs.

We actually experimented with this, and it was very, very messy. And
also more prone to interfering with real systems than the proposed
approach here.

-- 
Matthew Miller

Fedora Project Leader
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: no systemd in containers: Requires -> Recommends

[Neil Horman]

On Thu, Dec 17, 2015 at 10:18:10PM +0100, Reindl Harald wrote:
> 
> 
> Am 17.12.2015 um 22:11 schrieb Neil Horman:
> >On Thu, Dec 17, 2015 at 10:04:48PM +0100, Reindl Harald wrote:
> >>Am 17.12.2015 um 22:00 schrieb Neil Horman:
> >>>On Thu, Dec 17, 2015 at 03:39:17PM -0500, Colin Walters wrote:
> What you're arguing is that *build* convenience for our current 
> architecture
> outweighs the *runtime* cost.  That doesn't make sense long term - they're
> different problems.
> >>>What runtime cost are you referring to here?  The cost of running systemd 
> >>>in a
> >>>container?  Its miniscule, given that its only job is to start a handful of
> >>>units and get out of the way
> >>
> >>that's not true
> >>
> >>on virtual machines with no load all day long PID1 is often found as number
> >>1 in htop sorted by CPU usage - multiply that with 500 and you have a
> >>*useless* and well noticeable load on the host
> >>
> >What I hear you saying is that on a system that has nothing better to do, the
> >primary monitoring process wakes up periodically to check on various system
> >aspects (cron jobs, journal rotates, etc).  That sounds like working as 
> >designed
> >to me, not a reason to throw the proverbial baby out with the bath water.  
> >That
> >seems like a reason to do some tuning.
> 
> it is wasting ressources in a container running only one process - period -
> what is there to discuss - in any serious setup there shou,ld be nothing
> installed which is not *really* needed
> 
You're making two assumptions here:
1) That there is only one process
2) That systemd (or a component thereof) isn't needed

While you're use case is certainly valid, it isn't the only use case by a long
shot.  And if the goal for your use case is to minimize the footprint of a
container, you can already do that (as I've pointed out previously)
Neil



> --
> devel mailing list
> devel@lists.fedoraproject.org
> http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: no systemd in containers: Requires -> Recommends

[Reindl Harald]

Am 18.12.2015 um 15:44 schrieb Neil Horman:

On Fri, Dec 18, 2015 at 01:27:33AM +, Zbigniew Jędrzejewski-Szmek wrote:

For some packages "reduced capacity" because of lack of systemd.rpm
means "doesn't even get started as expected" or "crashes on
start with permission errors" or "cannot write logs" or similar.
Like Lennart and Neil said, utilities provided by systemd.rpm are the
basis which allows many things to "just work". This is so obvious
that it is assumed implicitly in this disussion, and it's hardly
"personal use cases".


I concur, this really seems like were forsaking full OS functionality to support
a specific container use case, which is wrong.  And the argument that its ok
because the kernel will pull in systemd, while currently accurate seems like bad
practice, as philisophically rpms always specify all their requires instead of
assuming some other package will do it for them


no idea what you are talking about

following "as philisophically rpms always specify all their requires" 
than wahtever package has to have a "Requires: systemd" when it requires 
systemd and since that is not happening they all rely anyways that 
something else does it


so following your logic the change should be done in case there is no 
kernel installed and the packages in the container don't need systemd to 
have the ability to remove it





signature.asc
Description: OpenPGP digital signature
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: no systemd in containers: Requires -> Recommends

[git]

I have no experience with containers so forgive me if I'm missing something.  
Why couldn't you just have a 'microinit.rpm' in a separate dnf repo, put 
'Obsoletes: systemd' into that package?  This way people who are building 
hundreds of containers that do not require systemd can use the repo containing 
microinit and it will take the place of systemd.  RPM macro's could be provided 
by the microinit rpm, so it would provide reasonable replacements for 
%systemd_requires, %systemd_post, etc.  I think this could solve your concern 
with minimum (no) impact to bare metal installs.
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: no systemd in containers: Requires -> Recommends

[Neil Horman]

On Fri, Dec 18, 2015 at 01:27:33AM +, Zbigniew Jędrzejewski-Szmek wrote:
> On Thu, Dec 17, 2015 at 04:13:06PM -0800, Brendan Conoboy wrote:
> > On 12/17/2015 01:43 AM, Harald Hoyer wrote:
> > >For docker containers, or containers, which don't want systemd, the current
> > >"Requires: systemd" in a lot of packages is preventing building a minimal 
> > >image.
> > >
> > >To improve the situation, we could make use of the new rpm weak 
> > >dependencies.
> > >So the
> > >
> > >Requires(post): systemd
> > >Requires(preun): systemd
> > >Requires(postun): systemd
> > >
> > >would become
> > >
> > >Recommends: systemd
> > >OrderWithRequires(post): systemd
> > >OrderWithRequires(preun): systemd
> > >OrderWithRequires(postun): systemd
> > >
> > >With this in place, kickstart files could omit systemd.
> > >
> > >The downside is:
> > >- if systemd is installed afterwards, the %post scripts do not trigger
> > >- packages, which need systemd-tmpfiles or systemd-sysusers could not be 
> > >converted
> > >
> > >If systemd is removed before the other packages, I don't see a problem.
> > >There are only leftovers in /etc/systemd.
> > >
> > >To prevent having a non-bootable system (not container), we could let the
> > >kernel.spec have a Requires on systemd.
> > >
> > >Comments? Please discuss.
> > 
> > I haven't seen a lot of downside brought up in this thread.  If the
> > only objections people have is that it doesn't facilitate their
> > personal use cases those don't seem like real objections.  Is
> > anybody going to be really negatively impacted by such a change?
> > 
> > For my part I'd like to see this happen, not just for packages
> > requiring systemd, but for all packages where "Requires" is really
> > stronger than necessary.  Now that we have soft dependencies it
> > would be nice to go through and move to Recommends where software
> > continues to function in some reduced capacity.
> 
> For some packages "reduced capacity" because of lack of systemd.rpm
> means "doesn't even get started as expected" or "crashes on
> start with permission errors" or "cannot write logs" or similar.
> Like Lennart and Neil said, utilities provided by systemd.rpm are the
> basis which allows many things to "just work". This is so obvious
> that it is assumed implicitly in this disussion, and it's hardly
> "personal use cases".
> 
> Zbyszek
I concur, this really seems like were forsaking full OS functionality to support
a specific container use case, which is wrong.  And the argument that its ok
because the kernel will pull in systemd, while currently accurate seems like bad
practice, as philisophically rpms always specify all their requires instead of
assuming some other package will do it for them

Neil
 
> --
> devel mailing list
> devel@lists.fedoraproject.org
> http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: no systemd in containers: Requires -> Recommends

[Reindl Harald]

Am 18.12.2015 um 16:36 schrieb g...@cfware.com:

I have no experience with containers so forgive me if I'm missing something.  
Why couldn't you just have a 'microinit.rpm' in a separate dnf repo, put 
'Obsoletes: systemd' into that package?  This way people who are building 
hundreds of containers that do not require systemd can use the repo containing 
microinit and it will take the place of systemd.  RPM macro's could be provided 
by the microinit rpm, so it would provide reasonable replacements for 
%systemd_requires, %systemd_post, etc.  I think this could solve your concern 
with minimum (no) impact to bare metal installs.


because in that case a dependency of a package which really requires 
systemd would no longer work and it's only a dirty hack


"minimum (no) impact to bare metal installs" - how much bare metal 
installs have you seen where kernel-core is not present?




signature.asc
Description: OpenPGP digital signature
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: no systemd in containers: Requires -> Recommends

[Harald Hoyer]

On 17.12.2015 11:15, Jason L Tibbitts III wrote:
>> "HH" == Harald Hoyer  writes:
> 
> HH> To improve the situation, we could make use of the new rpm weak
> HH> dependencies.  So the
> 
> I'm not sure I see the point.  The dependencies are there so that the
> scriptlets work.  If the scriptlets don't actually need to work then
> there's no point in having the Recommends: at all, especially of other
> essential parts of the system will pull in systemd if necessary.

Maybe.. yes. All you would want then is the OrderWithRequires()

> 
> And I'm not sure what harm would occur if systemd gets installed later.
> All that's missed for package install is the daemon-reload call which
> isn't a problem.  If systemd isn't installed at package upgrade or

The preset enablement would be missing.

> uninstall time, the scriptlets for stopping or restarting the daemon
> aren't relevant either.
> 
> So maybe I'm assuming too much, but I wonder if we couldn't drop just
> the systemd dependencies entirely.
> 
>  - J<
> 

--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: no systemd in containers: Requires -> Recommends

[Jason L Tibbitts III]

> "HH" == Harald Hoyer  writes:

HH> To improve the situation, we could make use of the new rpm weak
HH> dependencies.  So the

I'm not sure I see the point.  The dependencies are there so that the
scriptlets work.  If the scriptlets don't actually need to work then
there's no point in having the Recommends: at all, especially of other
essential parts of the system will pull in systemd if necessary.

And I'm not sure what harm would occur if systemd gets installed later.
All that's missed for package install is the daemon-reload call which
isn't a problem.  If systemd isn't installed at package upgrade or
uninstall time, the scriptlets for stopping or restarting the daemon
aren't relevant either.

So maybe I'm assuming too much, but I wonder if we couldn't drop just
the systemd dependencies entirely.

 - J<
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

no systemd in containers: Requires -> Recommends

[Harald Hoyer]

For docker containers, or containers, which don't want systemd, the current
"Requires: systemd" in a lot of packages is preventing building a minimal image.

To improve the situation, we could make use of the new rpm weak dependencies.
So the

Requires(post): systemd
Requires(preun): systemd
Requires(postun): systemd

would become

Recommends: systemd
OrderWithRequires(post): systemd
OrderWithRequires(preun): systemd
OrderWithRequires(postun): systemd

With this in place, kickstart files could omit systemd.

The downside is:
- if systemd is installed afterwards, the %post scripts do not trigger
- packages, which need systemd-tmpfiles or systemd-sysusers could not be 
converted

If systemd is removed before the other packages, I don't see a problem.
There are only leftovers in /etc/systemd.

To prevent having a non-bootable system (not container), we could let the
kernel.spec have a Requires on systemd.

Comments? Please discuss.
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: no systemd in containers: Requires -> Recommends

[Michal Sekletar]

On Thu, Dec 17, 2015 at 10:43 AM, Harald Hoyer  wrote:
> The downside is:
> - if systemd is installed afterwards, the %post scripts do not trigger
> - packages, which need systemd-tmpfiles or systemd-sysusers could not be 
> converted

IIRC, some time ago there was a proposal to split systemd-tmpfiles,
systemd-sysusers and other utilities to separate sub-package called
systemd-tools. We should probably revisit this idea.

Michal
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: no systemd in containers: Requires -> Recommends

[Colin Walters]

On Thu, Dec 17, 2015, at 08:28 AM, Neil Horman wrote:
>
> I would question why its necessecary to keep systemd out so ardently.  If you
> build your container layers properly, you can effectively put systemd in a 
> base
> container and layer other applications in child containers that inherit from 
> it.

If one is doing "micro" containers that only have typically one process in them,
having systemd managing it is unnecessary overhead. 

There are some outstanding issues with this model; need a way to tell
a pid namespace to auto-reap children in the kernel so pid 1 doesn't have to 
know
to do that.

But we should also obviously support systemd-in-container as a lot of
people are in a middle phase of "virt lite" with containers.
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: no systemd in containers: Requires -> Recommends

[Neil Horman]

On Thu, Dec 17, 2015 at 10:43:32AM +0100, Harald Hoyer wrote:
> For docker containers, or containers, which don't want systemd, the current
> "Requires: systemd" in a lot of packages is preventing building a minimal 
> image.
> 
> To improve the situation, we could make use of the new rpm weak dependencies.
> So the
> 
> Requires(post): systemd
> Requires(preun): systemd
> Requires(postun): systemd
> 
> would become
> 
> Recommends: systemd
> OrderWithRequires(post): systemd
> OrderWithRequires(preun): systemd
> OrderWithRequires(postun): systemd
> 
> With this in place, kickstart files could omit systemd.
> 
> The downside is:
> - if systemd is installed afterwards, the %post scripts do not trigger
> - packages, which need systemd-tmpfiles or systemd-sysusers could not be 
> converted
> 
> If systemd is removed before the other packages, I don't see a problem.
> There are only leftovers in /etc/systemd.
> 
> To prevent having a non-bootable system (not container), we could let the
> kernel.spec have a Requires on systemd.
> 
> Comments? Please discuss.
> --
> devel mailing list
> devel@lists.fedoraproject.org
> http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

I would question why its necessecary to keep systemd out so ardently.  If you
build your container layers properly, you can effectively put systemd in a base
container and layer other applications in child containers that inherit from it.
One systemd installation can be used by 1000's of child containers, making the
overhead negligible.

Neil
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: no systemd in containers: Requires -> Recommends

[Zbigniew Jędrzejewski-Szmek]

On Thu, Dec 17, 2015 at 02:36:05PM +, Zbigniew Jędrzejewski-Szmek wrote:
> On Thu, Dec 17, 2015 at 11:28:51AM +0100, Michal Sekletar wrote:
> > On Thu, Dec 17, 2015 at 10:43 AM, Harald Hoyer  wrote:
> > > The downside is:
> > > - if systemd is installed afterwards, the %post scripts do not trigger
> > > - packages, which need systemd-tmpfiles or systemd-sysusers could not be 
> > > converted
> > 
> > IIRC, some time ago there was a proposal to split systemd-tmpfiles,
> > systemd-sysusers and other utilities to separate sub-package called
> > systemd-tools. We should probably revisit this idea.

Pfff, I need to work on my reading comprehension skills :(

> Yes, I intend to push the split to rawhide real soon now:
> https://fedorahosted.org/fesco/ticket/1501
That's not the right ticket or the right proposal.

The (formerly rejected) proposal to split systemd-tools was
https://fedoraproject.org/wiki/Changes/SystemdPackageSplit

Between that (let's call it a) and Harald's OrderWithRequires approach
(call it b) the real difference is when systemd is not installed as part
of the initial rpm transaction, and later on it is added.
In case (a) presets and sysusers and similar work, in case (b) they
do not. (b) would translate to "if you want all systemd installation
facilities to work, make sure to install systemd as part of the first
transaction". The question is how often people would miss that
requirement and how much of a problem it would be if they did.

The (a) proposal would have be adjusted a bit. I think that with
rpm filetriggers and other recent changes, the list of binaries to
include in "systemd-tools" could become slightly smaller.

Zbyszek
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: no systemd in containers: Requires -> Recommends

[Zbigniew Jędrzejewski-Szmek]

On Thu, Dec 17, 2015 at 11:28:51AM +0100, Michal Sekletar wrote:
> On Thu, Dec 17, 2015 at 10:43 AM, Harald Hoyer  wrote:
> > The downside is:
> > - if systemd is installed afterwards, the %post scripts do not trigger
> > - packages, which need systemd-tmpfiles or systemd-sysusers could not be 
> > converted
> 
> IIRC, some time ago there was a proposal to split systemd-tmpfiles,
> systemd-sysusers and other utilities to separate sub-package called
> systemd-tools. We should probably revisit this idea.

Yes, I intend to push the split to rawhide real soon now:
https://fedorahosted.org/fesco/ticket/1501

Zbyszek
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: no systemd in containers: Requires -> Recommends

[Lennart Poettering]

On Thu, 17.12.15 10:27, Matthew Miller (mat...@fedoraproject.org) wrote:

> On Thu, Dec 17, 2015 at 04:18:12PM +0100, Lennart Poettering wrote:
> > > IIRC, some time ago there was a proposal to split systemd-tmpfiles,
> > > systemd-sysusers and other utilities to separate sub-package called
> > > systemd-tools. We should probably revisit this idea.
> > Why? Do you have any technical reasons?
> 
> This is the wrong question. Is there a technical reason all of this
> stuff must be on every system?

Nope, that's not the point to make. We ship tons of stuff you don't
always need, but why is this stuff that matters? Is it *that* large?
Does it have such heavy otherwise unneeded deps?

Also, why would you want to ship tmpfiles without systemd, or systemd
without tmpfiles? I don't get what this specific split would
accomplish. If you don't want systemd, then why would you want
tmpfiles? I mean, nothing would run tmpfiles in regular intervals,
hence it wouldn't work. Or is it that you want systemd, but not
tmpfiles? Why would that matter? 

Lennart

-- 
Lennart Poettering, Red Hat
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: no systemd in containers: Requires -> Recommends

[Reindl Harald]

Am 17.12.2015 um 16:57 schrieb Lennart Poettering:

On Thu, 17.12.15 10:50, Matthew Miller (mat...@fedoraproject.org) wrote:


On Thu, Dec 17, 2015 at 04:40:16PM +0100, Lennart Poettering wrote:

Nope, that's not the point to make. We ship tons of stuff you don't
always need, but why is this stuff that matters? Is it *that* large?


"Ship" and "require in the most minimal application-only install case"
are different. And "eh, it's not that large" is the approach that's
lead us to having a collective minimal set that is undeniably unwieldy.
If, instead, every package at the base level would take modularity as a
baseline principle, we'd be in a lot better and more flexible state.


Does it have such heavy otherwise unneeded deps?


In some cases, yes. In others, it's deps that don't seem individually
heavy but they add up.


I am not sure I can read this any other way than "Nope, I won't be
specific with numbers and stuff, I just have the 'feeling' that
systemd is large and has huge deps"


read it they way: *anything* which is *not missed* when it's not 
installed should not be installed - period - there is nothing to dicuss 
about




signature.asc
Description: OpenPGP digital signature
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: no systemd in containers: Requires -> Recommends

[Colin Walters]

On Thu, Dec 17, 2015, at 10:24 AM, Lennart Poettering wrote:

> Can you give realistic examples for these? Can you explain what you
> are intend to run as PID 1 in them instead?

Nothing, if the pid namespace did zombie collection in the kernel,
you don't need a separate init.

> What is cleaning up /tmp
> for those things? 

You bind mount the container's /tmp to a host /tmp/container-$uuid
for example.

> What is setting up the tmpfiles bits in /run for
> them, and so on?

One would carry this in the Dockerfile or equivalent if it applies,
although it doesn't for a lot of software.

Your broader point is very valid - we're going to need a lot
of software to run both on the host outside of a container,
underneath systemd, but we're also trying to enable a
fully container-only distributed/cluster model via Kubernetes/Docker,
and in the end microservice state, it just doesn't make sense
to have a systemd instance per microservice in a cluster.
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: no systemd in containers: Requires -> Recommends

[Matthew Miller]

On Thu, Dec 17, 2015 at 04:40:16PM +0100, Lennart Poettering wrote:
> Nope, that's not the point to make. We ship tons of stuff you don't
> always need, but why is this stuff that matters? Is it *that* large?

"Ship" and "require in the most minimal application-only install case"
are different. And "eh, it's not that large" is the approach that's
lead us to having a collective minimal set that is undeniably unwieldy.
If, instead, every package at the base level would take modularity as a
baseline principle, we'd be in a lot better and more flexible state.

> Does it have such heavy otherwise unneeded deps?

In some cases, yes. In others, it's deps that don't seem individually
heavy but they add up.


-- 
Matthew Miller

Fedora Project Leader
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: no systemd in containers: Requires -> Recommends

[Reindl Harald]

Am 17.12.2015 um 16:54 schrieb Lennart Poettering:

On Thu, 17.12.15 10:44, Colin Walters (walt...@verbum.org) wrote:

What is cleaning up /tmp
for those things?


You bind mount the container's /tmp to a host /tmp/container-$uuid
for example.


Well, and what sets up all the rest listed in tmpfiles snippets?


well, you missed the topic is about *tiny* containers for single 
applications - not everything has tmpfiles snippets



If you want to replace systemd functionality with Docker
functionality, then that's fine, but I think you better make sure you
actually have replacement functionally around. Because otherwise you
just give up on platform design


no, you missed the point: not every systemd functionality is needed for 
all usecases (not that i run containers because i prefer full 
virtualization) but if i would run some hundret containers with a single 
application running systemd in each of them would be waste of ressources




signature.asc
Description: OpenPGP digital signature
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: no systemd in containers: Requires -> Recommends

[Colin Walters]

On Thu, Dec 17, 2015, at 10:54 AM, Lennart Poettering wrote:

Because microservice containers are a valid goal, and allowing them
to be more minimal while still pulling in glibc etc. is useful (from
the start of this thread).

> Note that PID 1 is in more ways different than just reaping
> processes... For example for PID 1, SIGTERM usually is a request for
> reexecution, and SIGINT a request for reboot, while for non-PID1
> processes these are requests for termination and cancelling...

I don't think anyone really cares about the traditional PID 1 signal
handling in a Kubernetes cluster, it's just not used for administration.

> Well, that's certainly a opinion on this. I certainly disagree. It
> would essentially mean giving up on much what makes up an OS
> though, in particular, about half of whatthe packaging guidelines say
> what packages shall use and rely on.

Indeed, the current packaging model is designed for a world
where all software is installed on the host.  For a lot of
software (postgres, nginx), we need to support both.

But there's also a lot of software that is container-only, and
I expect this to increase.  This will be part of Dockerfile and
other container formats that we include as part of Fedora.

> If you want to replace systemd functionality with Docker functionality

Let's be clear - from my perspective systemd's design is awesome
for the *real* pid 1.  AFAIK no one here is talking about changing anything
related to that.  We're just talking about supporting microservice
containers without a pid 1 in the container namespace.

--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: no systemd in containers: Requires -> Recommends

[Mustafa Muhammad]

On Thu, Dec 17, 2015 at 6:24 PM, Lennart Poettering 
wrote:

> On Thu, 17.12.15 10:02, Colin Walters (walt...@verbum.org) wrote:
>
> > On Thu, Dec 17, 2015, at 08:28 AM, Neil Horman wrote:
> > >
> > > I would question why its necessecary to keep systemd out so ardently.
> If you
> > > build your container layers properly, you can effectively put systemd
> in a base
> > > container and layer other applications in child containers that
> inherit from it.
> >
> > If one is doing "micro" containers that only have typically one process
> in them,
> > having systemd managing it is unnecessary overhead.
>
> Can you give realistic examples for these? Can you explain what you
> are intend to run as PID 1 in them instead? What is cleaning up /tmp
> for those things? What is setting up the tmpfiles bits in /run for
> them, and so on?
>

The recommendation for docker containers is to only have one process, when
it dies, the whole container stops, this is how to manage a cluster and
know when something die.

Mustafa


> Lennart
>
> --
> Lennart Poettering, Red Hat
> --
> devel mailing list
> devel@lists.fedoraproject.org
> http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org
>
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: no systemd in containers: Requires -> Recommends

[Lennart Poettering]

On Thu, 17.12.15 10:44, Colin Walters (walt...@verbum.org) wrote:

> On Thu, Dec 17, 2015, at 10:24 AM, Lennart Poettering wrote:
> 
> > Can you give realistic examples for these? Can you explain what you
> > are intend to run as PID 1 in them instead?
> 
> Nothing, if the pid namespace did zombie collection in the kernel,
> you don't need a separate init.

hmm? but this is not how Linux works these days, so what's the point
discussing this now?

Note that PID 1 is in more ways different than just reaping
processes... For example for PID 1, SIGTERM usually is a request for
reexecution, and SIGINT a request for reboot, while for non-PID1
processes these are requests for termination and cancelling...

> > What is cleaning up /tmp
> > for those things? 
> 
> You bind mount the container's /tmp to a host /tmp/container-$uuid
> for example.

Well, and what sets up all the rest listed in tmpfiles snippets?

> > What is setting up the tmpfiles bits in /run for
> > them, and so on?
> 
> One would carry this in the Dockerfile or equivalent if it applies,
> although it doesn't for a lot of software.
> 
> Your broader point is very valid - we're going to need a lot
> of software to run both on the host outside of a container,
> underneath systemd, but we're also trying to enable a
> fully container-only distributed/cluster model via Kubernetes/Docker,
> and in the end microservice state, it just doesn't make sense
> to have a systemd instance per microservice in a cluster.

Well, that's certainly a opinion on this. I certainly disagree. It
would essentially mean giving up on much what makes up an OS
though, in particular, about half of whatthe packaging guidelines say
what packages shall use and rely on.

If you want to replace systemd functionality with Docker
functionality, then that's fine, but I think you better make sure you
actually have replacement functionally around. Because otherwise you
just give up on platform design.

Lennart

-- 
Lennart Poettering, Red Hat
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: no systemd in containers: Requires -> Recommends

[Matthew Miller]

On Thu, Dec 17, 2015 at 11:07:34AM -0500, Colin Walters wrote:
> Let's be clear - from my perspective systemd's design is awesome
> for the *real* pid 1.  AFAIK no one here is talking about changing anything
> related to that.  We're just talking about supporting microservice
> containers without a pid 1 in the container namespace.

I agree here as well. And, in fact, I'd like systemd to be the number 1
and primary choice for containers which _do_ need process management.
As it stands now, we are in real danger of losing that. Docker
recommends Supervisor. I know people are all excited about S6. And
there's good ol' Runit.


-- 
Matthew Miller

Fedora Project Leader
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: no systemd in containers: Requires -> Recommends

[Lennart Poettering]

On Thu, 17.12.15 11:28, Michal Sekletar (msekl...@redhat.com) wrote:

> On Thu, Dec 17, 2015 at 10:43 AM, Harald Hoyer  wrote:
> > The downside is:
> > - if systemd is installed afterwards, the %post scripts do not trigger
> > - packages, which need systemd-tmpfiles or systemd-sysusers could not be 
> > converted
> 
> IIRC, some time ago there was a proposal to split systemd-tmpfiles,
> systemd-sysusers and other utilities to separate sub-package called
> systemd-tools. We should probably revisit this idea.

Why? Do you have any technical reasons?

Lennart

-- 
Lennart Poettering, Red Hat
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: no systemd in containers: Requires -> Recommends

[Lennart Poettering]

On Thu, 17.12.15 14:36, Zbigniew Jędrzejewski-Szmek (zbys...@in.waw.pl) wrote:

> On Thu, Dec 17, 2015 at 11:28:51AM +0100, Michal Sekletar wrote:
> > On Thu, Dec 17, 2015 at 10:43 AM, Harald Hoyer  wrote:
> > > The downside is:
> > > - if systemd is installed afterwards, the %post scripts do not trigger
> > > - packages, which need systemd-tmpfiles or systemd-sysusers could not be 
> > > converted
> > 
> > IIRC, some time ago there was a proposal to split systemd-tmpfiles,
> > systemd-sysusers and other utilities to separate sub-package called
> > systemd-tools. We should probably revisit this idea.
> 
> Yes, I intend to push the split to rawhide real soon now:
> https://fedorahosted.org/fesco/ticket/1501

Hmm? I am not aware of any plan to split out tmpfiles and stuff... 

Lennart

-- 
Lennart Poettering, Red Hat
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: no systemd in containers: Requires -> Recommends

[Matthew Miller]

On Thu, Dec 17, 2015 at 04:18:12PM +0100, Lennart Poettering wrote:
> > IIRC, some time ago there was a proposal to split systemd-tmpfiles,
> > systemd-sysusers and other utilities to separate sub-package called
> > systemd-tools. We should probably revisit this idea.
> Why? Do you have any technical reasons?

This is the wrong question. Is there a technical reason all of this
stuff must be on every system?

-- 
Matthew Miller

Fedora Project Leader
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: no systemd in containers: Requires -> Recommends

[Matthew Miller]

On Thu, Dec 17, 2015 at 04:17:37PM +0100, Lennart Poettering wrote:
> So, this I think goes to the core of things: What is Fedora supposed
> to be? I always thought it was supposed to be an OS, meaning a
> provider of certain APIs, resources and functionality that apps can
> rely on. tmpfiles and things like that are part of that. At this point
> Fedora packages can rely on these things, and that's kinda what I
> think that Fedora is selling to packagers and app developers. If you
> now remove all that, what is precisely left? Do you want to go back to
> the point where nobody can depend on anything anymore? Where does this
> leave Fedora?

I think that's a reasonable-enough definition. However, "can rely on"
does not mean "always installed all the time". Packagers and app
developers — and people looking to build solutions with Fedora, like
our main editions but also spins and remixes — benefit from the ability
to deploy the parts of that support infrastructure they need and not
the parts they don't. And that, in turn, is benefited by more-flexible
systemd packaging.
 
> Are you proposing to deprecate use of tmpfiles/sysusers and all those
> things in packages now? What are you proposing shall take its place?

Not everything needs those tools. And not everything that needs _those_
tools needs systemd-delta and its dependencies.

And systemd-sysusers isn't even the standard Fedora way of adding
system users! Maybe it should be, but to my knowledge we've never had
that conversation. As an OS our standard API/functionality for that
shadow-utils as described in
.



-- 
Matthew Miller

Fedora Project Leader
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: no systemd in containers: Requires -> Recommends

[Lennart Poettering]

On Thu, 17.12.15 10:50, Matthew Miller (mat...@fedoraproject.org) wrote:

> On Thu, Dec 17, 2015 at 04:40:16PM +0100, Lennart Poettering wrote:
> > Nope, that's not the point to make. We ship tons of stuff you don't
> > always need, but why is this stuff that matters? Is it *that* large?
> 
> "Ship" and "require in the most minimal application-only install case"
> are different. And "eh, it's not that large" is the approach that's
> lead us to having a collective minimal set that is undeniably unwieldy.
> If, instead, every package at the base level would take modularity as a
> baseline principle, we'd be in a lot better and more flexible state.
> 
> > Does it have such heavy otherwise unneeded deps?
> 
> In some cases, yes. In others, it's deps that don't seem individually
> heavy but they add up.

I am not sure I can read this any other way than "Nope, I won't be
specific with numbers and stuff, I just have the 'feeling' that
systemd is large and has huge deps".

Lennart

-- 
Lennart Poettering, Red Hat
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: no systemd in containers: Requires -> Recommends

[Lennart Poettering]

On Thu, 17.12.15 10:43, Harald Hoyer (har...@redhat.com) wrote:

> For docker containers, or containers, which don't want systemd, the current
> "Requires: systemd" in a lot of packages is preventing building a
> minimal image.

What does this even mean? What are you actually "fixing" with this?
What's the sizes and deps you are talking of here, that you want to
keep out? Why does this even matter, why wouldn't he base layer be
shared anyway between all containers?

Is there any technical background for what you are trying to do? Or
are you just complicating things, to make things "feel" more minimal?
If so, can we pelase get back to technical reasoning, instead?

> - packages, which need systemd-tmpfiles or systemd-sysusers could not be 
> converted

So, this I think goes to the core of things: What is Fedora supposed
to be? I always thought it was supposed to be an OS, meaning a
provider of certain APIs, resources and functionality that apps can
rely on. tmpfiles and things like that are part of that. At this point
Fedora packages can rely on these things, and that's kinda what I
think that Fedora is selling to packagers and app developers. If you
now remove all that, what is precisely left? Do you want to go back to
the point where nobody can depend on anything anymore? Where does this
leave Fedora?

Are you proposing to deprecate use of tmpfiles/sysusers and all those
things in packages now? What are you proposing shall take its place?

To me this all sounds very much not thought to the end.

Lennart

-- 
Lennart Poettering, Red Hat
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: no systemd in containers: Requires -> Recommends

[Lennart Poettering]

On Thu, 17.12.15 10:02, Colin Walters (walt...@verbum.org) wrote:

> On Thu, Dec 17, 2015, at 08:28 AM, Neil Horman wrote:
> >
> > I would question why its necessecary to keep systemd out so ardently.  If 
> > you
> > build your container layers properly, you can effectively put systemd in a 
> > base
> > container and layer other applications in child containers that inherit 
> > from it.
> 
> If one is doing "micro" containers that only have typically one process in 
> them,
> having systemd managing it is unnecessary overhead. 

Can you give realistic examples for these? Can you explain what you
are intend to run as PID 1 in them instead? What is cleaning up /tmp
for those things? What is setting up the tmpfiles bits in /run for
them, and so on?

Lennart

-- 
Lennart Poettering, Red Hat
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: no systemd in containers: Requires -> Recommends

[Matthew Miller]

On Thu, Dec 17, 2015 at 08:28:13AM -0500, Neil Horman wrote:
> I would question why its necessecary to keep systemd out so ardently.
> If you build your container layers properly, you can effectively put
> systemd in a base container and layer other applications in child
> containers that inherit from it. One systemd installation can be used
> by 1000's of child containers, making the overhead negligible.

Initial deploy isn't insignificant, though, especially in the case
where you're bringing up and down multiple hosts in a cluster.

More importantly, rebuilding all of those child containers when there's
a systemd change isn't negligible at all.

And third, not all application container formats support Docker's
concept of layering, and I'd like to make sure we can adapt to those
too.

-- 
Matthew Miller

Fedora Project Leader
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: no systemd in containers: Requires -> Recommends

[Lennart Poettering]

On Thu, 17.12.15 10:25, Matthew Miller (mat...@fedoraproject.org) wrote:

> On Thu, Dec 17, 2015 at 08:28:13AM -0500, Neil Horman wrote:
> > I would question why its necessecary to keep systemd out so ardently.
> > If you build your container layers properly, you can effectively put
> > systemd in a base container and layer other applications in child
> > containers that inherit from it. One systemd installation can be used
> > by 1000's of child containers, making the overhead negligible.
> 
> Initial deploy isn't insignificant, though, especially in the case
> where you're bringing up and down multiple hosts in a cluster.
> 
> More importantly, rebuilding all of those child containers when there's
> a systemd change isn't negligible at all.

Well, how often are there systemd changes? Are you saying we update it
too often? That's news too me... I am pretty sure that systemd is
roughly updated as frequently as libc or so these days, i.e. maybe
once or twice per cycle... And you cannot get rid of the libc rebuilds
can you? So if you have to rebuild things once or twice per cycle
anyway, what are you trying to do?

Lennart

-- 
Lennart Poettering, Red Hat
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: no systemd in containers: Requires -> Recommends

[Matthew Miller]

On Thu, Dec 17, 2015 at 04:57:26PM +0100, Lennart Poettering wrote:
> > If, instead, every package at the base level would take modularity as a
> > baseline principle, we'd be in a lot better and more flexible state.
> > > Does it have such heavy otherwise unneeded deps?
> > In some cases, yes. In others, it's deps that don't seem individually
> > heavy but they add up.
> I am not sure I can read this any other way than "Nope, I won't be
> specific with numbers and stuff, I just have the 'feeling' that
> systemd is large and has huge deps".

The minimal image as a whole is large, and systemd is a large part of
it. There are plenty of specifics. systemd-delta brings in diffutils,
at over 1MB for a very specific niche use. systemd-localed brings in
libxkbcommon + xkeyboard-config at... what, more than 6MB? I think it's
very safe to say that not all containerized applications need to be
able to select a keyboard layout.

My point isn't that there _aren't_ specifics. It's that I don't think
it's productive to get in a quagmire arguing about each and every one,
every time. Let's instead just have "keep overhead down" as a
_baseline_ aim.

Honestly, I don't get the antagonism here. Systemd itself is basically
modular, and we have good tools for packaging things up in a modular
way already. Let's use them, so that Fedora is a good base for people
wanting to build application containers. 

-- 
Matthew Miller

Fedora Project Leader
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: no systemd in containers: Requires -> Recommends

[Reindl Harald]

Am 17.12.2015 um 17:03 schrieb Reindl Harald:



Am 17.12.2015 um 16:57 schrieb Lennart Poettering:

On Thu, 17.12.15 10:50, Matthew Miller (mat...@fedoraproject.org) wrote:


On Thu, Dec 17, 2015 at 04:40:16PM +0100, Lennart Poettering wrote:

Nope, that's not the point to make. We ship tons of stuff you don't
always need, but why is this stuff that matters? Is it *that* large?


"Ship" and "require in the most minimal application-only install case"
are different. And "eh, it's not that large" is the approach that's
lead us to having a collective minimal set that is undeniably unwieldy.
If, instead, every package at the base level would take modularity as a
baseline principle, we'd be in a lot better and more flexible state.


Does it have such heavy otherwise unneeded deps?


In some cases, yes. In others, it's deps that don't seem individually
heavy but they add up.


I am not sure I can read this any other way than "Nope, I won't be
specific with numbers and stuff, I just have the 'feeling' that
systemd is large and has huge deps"


read it they way: *anything* which is *not missed* when it's not
installed should not be installed - period - there is nothing to dicuss
about


OK, you want numbers

full featured VM running authoritative DNS for hundrets
of zones while bind and rsyslog would be enough
__

whole operating system: 795 MB

systemd: 24 MB
systemd-libs: 1.1 MB
__

(100 * 25.1) / 795 = 3,16%

sorry, 25 MB in each container *is bloat* and you pull deps
like dbus and it's dep-chain not needed to run a single process
which has no business for IPC since there is no other process

not so long ago systemd unconditionally pulled "libmicrohttpd"
as dependency - most deps have chains and when you get rid of
one package you can get rid of their deps too

yes, many would now argue "but that's not much overhead", but
that overhead on 100, 200, 500 containers makes a total sum





signature.asc
Description: OpenPGP digital signature
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: no systemd in containers: Requires -> Recommends

[Neal Gompa]

On Thu, Dec 17, 2015 at 11:20 AM, Matthew Miller
 wrote:
> On Thu, Dec 17, 2015 at 11:07:34AM -0500, Colin Walters wrote:
>> Let's be clear - from my perspective systemd's design is awesome
>> for the *real* pid 1.  AFAIK no one here is talking about changing anything
>> related to that.  We're just talking about supporting microservice
>> containers without a pid 1 in the container namespace.
>
> I agree here as well. And, in fact, I'd like systemd to be the number 1
> and primary choice for containers which _do_ need process management.
> As it stands now, we are in real danger of losing that. Docker
> recommends Supervisor. I know people are all excited about S6. And
> there's good ol' Runit.
>
>
> --
> Matthew Miller
> 
> Fedora Project Leader
> --
> devel mailing list
> devel@lists.fedoraproject.org
> http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

It doesn't help that there doesn't seem to be a way to use systemd's
own container technologies to do these things in a more lightweight,
yet compatible fashion. nspawn currently only does OS style
containers, where you have another PID 1 inside. If something that
leveraged systemd as the service manager existed that also allowed
microservices to work very well, then you've got a recipe for some
interesting solutions.

-- 
真実はいつも一つ！/ Always, there's only one truth!
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: no systemd in containers: Requires -> Recommends

[Neil Horman]

On Thu, Dec 17, 2015 at 05:18:21PM +0100, Reindl Harald wrote:
> 
> 
> Am 17.12.2015 um 17:03 schrieb Reindl Harald:
> >
> >
> >Am 17.12.2015 um 16:57 schrieb Lennart Poettering:
> >>On Thu, 17.12.15 10:50, Matthew Miller (mat...@fedoraproject.org) wrote:
> >>
> >>>On Thu, Dec 17, 2015 at 04:40:16PM +0100, Lennart Poettering wrote:
> Nope, that's not the point to make. We ship tons of stuff you don't
> always need, but why is this stuff that matters? Is it *that* large?
> >>>
> >>>"Ship" and "require in the most minimal application-only install case"
> >>>are different. And "eh, it's not that large" is the approach that's
> >>>lead us to having a collective minimal set that is undeniably unwieldy.
> >>>If, instead, every package at the base level would take modularity as a
> >>>baseline principle, we'd be in a lot better and more flexible state.
> >>>
> Does it have such heavy otherwise unneeded deps?
> >>>
> >>>In some cases, yes. In others, it's deps that don't seem individually
> >>>heavy but they add up.
> >>
> >>I am not sure I can read this any other way than "Nope, I won't be
> >>specific with numbers and stuff, I just have the 'feeling' that
> >>systemd is large and has huge deps"
> >
> >read it they way: *anything* which is *not missed* when it's not
> >installed should not be installed - period - there is nothing to dicuss
> >about
> 
> OK, you want numbers
> 
> full featured VM running authoritative DNS for hundrets
> of zones while bind and rsyslog would be enough
> __
> 
> whole operating system: 795 MB
> 
> systemd: 24 MB
> systemd-libs: 1.1 MB
> __
> 
> (100 * 25.1) / 795 = 3,16%
> 
> sorry, 25 MB in each container *is bloat* and you pull deps
So dont put it in each container, put it in a parent container that can be
inherited via btrfs subvolumes or unionfs.  The only reason this seems like a
significant number is because you seem to be insisting that it has to be copied
into each container. If you share it (as you should), the space requirements
become negligible very quickly at scales of hundreds of containers.

Neil





> --
> devel mailing list
> devel@lists.fedoraproject.org
> http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: no systemd in containers: Requires -> Recommends

[J.C. Cleaver]

On Thu, December 17, 2015 8:08 am, Mustafa Muhammad wrote:
> On Thu, Dec 17, 2015 at 6:24 PM, Lennart Poettering 
> wrote:
>
>> On Thu, 17.12.15 10:02, Colin Walters (walt...@verbum.org) wrote:
>>
>> > On Thu, Dec 17, 2015, at 08:28 AM, Neil Horman wrote:
>> > >
>> > > I would question why its necessecary to keep systemd out so
>> ardently.
>> If you
>> > > build your container layers properly, you can effectively put
>> systemd
>> in a base
>> > > container and layer other applications in child containers that
>> inherit from it.
>> >
>> > If one is doing "micro" containers that only have typically one
>> process
>> in them,
>> > having systemd managing it is unnecessary overhead.
>>
>> Can you give realistic examples for these? Can you explain what you
>> are intend to run as PID 1 in them instead? What is cleaning up /tmp
>> for those things? What is setting up the tmpfiles bits in /run for
>> them, and so on?
>>
>
> The recommendation for docker containers is to only have one process, when
> it dies, the whole container stops, this is how to manage a cluster and
> know when something die.
>


The larger point is that -- despite best efforts -- there are still
paradigms where the init process doesn't need to be heavy or complex in a
system. Lightweight zombie processing and perhaps dispatching telinit
signals to something else, if that.

Modularized into distinctly-installed packages or not, there's no need for
systemd in that environment when something smaller will do for PID 1.


-jc
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: no systemd in containers: Requires -> Recommends

[Neil Horman]

On Thu, Dec 17, 2015 at 04:54:39PM +0100, Lennart Poettering wrote:
> On Thu, 17.12.15 10:44, Colin Walters (walt...@verbum.org) wrote:
> 
> > On Thu, Dec 17, 2015, at 10:24 AM, Lennart Poettering wrote:
> > 
> > > Can you give realistic examples for these? Can you explain what you
> > > are intend to run as PID 1 in them instead?
> > 
> > Nothing, if the pid namespace did zombie collection in the kernel,
> > you don't need a separate init.
> 
> hmm? but this is not how Linux works these days, so what's the point
> discussing this now?
> 
> Note that PID 1 is in more ways different than just reaping
> processes... For example for PID 1, SIGTERM usually is a request for
> reexecution, and SIGINT a request for reboot, while for non-PID1
> processes these are requests for termination and cancelling...
> 
> > > What is cleaning up /tmp
> > > for those things? 
> > 
> > You bind mount the container's /tmp to a host /tmp/container-$uuid
> > for example.
> 
> Well, and what sets up all the rest listed in tmpfiles snippets?
> 
> > > What is setting up the tmpfiles bits in /run for
> > > them, and so on?
> > 
> > One would carry this in the Dockerfile or equivalent if it applies,
> > although it doesn't for a lot of software.
> > 
> > Your broader point is very valid - we're going to need a lot
> > of software to run both on the host outside of a container,
> > underneath systemd, but we're also trying to enable a
> > fully container-only distributed/cluster model via Kubernetes/Docker,
> > and in the end microservice state, it just doesn't make sense
> > to have a systemd instance per microservice in a cluster.
> 
> Well, that's certainly a opinion on this. I certainly disagree. It
> would essentially mean giving up on much what makes up an OS
> though, in particular, about half of whatthe packaging guidelines say
> what packages shall use and rely on.
> 
> If you want to replace systemd functionality with Docker
> functionality, then that's fine, but I think you better make sure you
> actually have replacement functionally around. Because otherwise you
> just give up on platform design.
> 
> Lennart

FWIW, In freight[1], I assume systemd is a member of every container, and that
services are started via unit files in the container.  Its made generating
containers unbelievably easy, in that it doesn't require any additional
knoweldge about an application - the way it runs in a container is exactly the
way it runs outside of a container, its really nice.

[1] http://freightagent.github.io/freight-tools/

> 
> -- 
> Lennart Poettering, Red Hat
> --
> devel mailing list
> devel@lists.fedoraproject.org
> http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: no systemd in containers: Requires -> Recommends

[Neil Horman]

On Thu, Dec 17, 2015 at 10:25:17AM -0500, Matthew Miller wrote:
> On Thu, Dec 17, 2015 at 08:28:13AM -0500, Neil Horman wrote:
> > I would question why its necessecary to keep systemd out so ardently.
> > If you build your container layers properly, you can effectively put
> > systemd in a base container and layer other applications in child
> > containers that inherit from it. One systemd installation can be used
> > by 1000's of child containers, making the overhead negligible.
> 
> Initial deploy isn't insignificant, though, especially in the case
> where you're bringing up and down multiple hosts in a cluster.
> 
Perhaps, but in the grand scheme of things, we're talking about a single large
install per host for potentially thousands of child containers.  That seems like
a reasonable trade off.

> More importantly, rebuilding all of those child containers when there's
> a systemd change isn't negligible at all.
> 
I would disagree.  Its what I'm doing in freight, and it works pretty well.  If
I update my base container with a new systemd, the child containers regenerate
in seconds.

> And third, not all application container formats support Docker's
> concept of layering, and I'd like to make sure we can adapt to those
> too.
> 
So we're talking about container formats that are both minimal, and don't
support the concept of file system inheritance?  Is seems like the latter would
be a integral component to support of the former, and more a problem for those
formats to fix, rather than for a container developer to work around.

Neil

> -- 
> Matthew Miller
> 
> Fedora Project Leader
> --
> devel mailing list
> devel@lists.fedoraproject.org
> http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: no systemd in containers: Requires -> Recommends

[Neil Horman]

On Thu, Dec 17, 2015 at 10:02:43AM -0500, Colin Walters wrote:
> On Thu, Dec 17, 2015, at 08:28 AM, Neil Horman wrote:
> >
> > I would question why its necessecary to keep systemd out so ardently.  If 
> > you
> > build your container layers properly, you can effectively put systemd in a 
> > base
> > container and layer other applications in child containers that inherit 
> > from it.
> 
> If one is doing "micro" containers that only have typically one process in 
> them,
> having systemd managing it is unnecessary overhead. 
> 
Well, thats true, but if thats the use case you're focused on, one would imagine
you either:

1) Writing your container install script to simply copy in only the files needed
for the process that you want

2) Writing your container install script to minimize filesystem contents after
then install

In either case, you're going to wind up butchering a fair amount of what the rpm
is going to be doing anyway.  If its so important to minimize that storage, rpm
dependencies shouldn't really be a big deal, because you know you're going to
have to either do some FS surgery anyway.  At that point all your saving is the
time to download systemd.  It seems like this is making alot of tradeoffs to do
something that you'll need to do anyway.

Neil

> There are some outstanding issues with this model; need a way to tell
> a pid namespace to auto-reap children in the kernel so pid 1 doesn't have to 
> know
> to do that.
> 
Yeah, thats the argument behind doing a properly layered container.  Spend the
300Mb to get systemd and its dependencies once, and make all your other
contaienrs children of that.  The children inherit the systemd code, they
temselves are still minimized, and you can use systemd to do process management
in the container.  

Yes, its more than one process.  I think thats the better tradeoff though.
Neil

> But we should also obviously support systemd-in-container as a lot of
> people are in a middle phase of "virt lite" with containers.
> --
> devel mailing list
> devel@lists.fedoraproject.org
> http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: no systemd in containers: Requires -> Recommends

[Reindl Harald]

Am 17.12.2015 um 21:47 schrieb Craig Garner:

I usually don't say anything and just read...it's not really my place.
I just like to keep up with things going on.  But, by the time you
finish worrying about all the overhead and things get finalized, there's
going to be so damn much RAM and processing power no one will care.
Kind of like developing one white blood cell to defend your entire body.


this is pure nonsense and the attitude "going to be so damn much RAM and 
processing power no one will care" is the reason that machines these 
days are not nearby as fast as they could and should be compared with 
machines 10 years ago - brainless ressource wasting


when you have real production load and a ton of virtual machines and 
containers running on the same host and every one of them is wasting 
ressources the summary of all the overhead is something you care or 
should care about




On Thu, Dec 17, 2015 at 3:39 PM, Colin Walters > wrote:

On Thu, Dec 17, 2015, at 01:19 PM, Neil Horman wrote:
>
> In either case, you're going to wind up butchering a fair amount of what 
the rpm
> is going to be doing anyway.  If its so important to minimize that 
storage, rpm
> dependencies shouldn't really be a big deal, because you know you're 
going to
> have to either do some FS surgery anyway.

What I'm actually arguing long term is to rearchitect the model of
the subset
of Fedora that is for server containers to support this - something
like a "just the binaries"
data blob, and then *optionally* turn that intermediate into an RPM
that would
have a systemd unit file.  It could really be an RPM, just without
the %post
scripts relating to systemd and the unit files.

In practice though, it's not a big deal as long as shared libraries
don't
end up pulling in systemd or other components.  And for software that's
container-only, the build process (Dockerfile or something better in the
future) for containers just won't require it.

> Yes, its more than one process.  I think thats the better tradeoff though.

What you're arguing is that *build* convenience for our current
architecture
outweighs the *runtime* cost.  That doesn't make sense long term -
they're
different problems.




signature.asc
Description: OpenPGP digital signature
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: no systemd in containers: Requires -> Recommends

[Reindl Harald]

Am 17.12.2015 um 19:43 schrieb Neil Horman:

On Thu, Dec 17, 2015 at 05:18:21PM +0100, Reindl Harald wrote:

OK, you want numbers

full featured VM running authoritative DNS for hundrets
of zones while bind and rsyslog would be enough
__

whole operating system: 795 MB

systemd: 24 MB
systemd-libs: 1.1 MB
__

(100 * 25.1) / 795 = 3,16%

sorry, 25 MB in each container *is bloat* and you pull deps

So dont put it in each container, put it in a parent container that can be
inherited via btrfs subvolumes or unionfs.  The only reason this seems like a
significant number is because you seem to be insisting that it has to be copied
into each container. If you share it (as you should), the space requirements
become negligible very quickly at scales of hundreds of containers


bla - i have also zero understanding for 25 MB in every full 
virtualization VM and when you look at the sizes systemd grows with each 
fedora release




signature.asc
Description: OpenPGP digital signature
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: no systemd in containers: Requires -> Recommends

[Jason L Tibbitts III]

> "HH" == Harald Hoyer  writes:

HH> The preset enablement would be missing.

Couldn't systemd simply apply presets when it is installed?  (Not
upgraded, but on a fresh install?)

 - J<
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: no systemd in containers: Requires -> Recommends

[Colin Walters]

On Thu, Dec 17, 2015, at 01:19 PM, Neil Horman wrote:
>
> In either case, you're going to wind up butchering a fair amount of what the 
> rpm
> is going to be doing anyway.  If its so important to minimize that storage, 
> rpm
> dependencies shouldn't really be a big deal, because you know you're going to
> have to either do some FS surgery anyway. 

What I'm actually arguing long term is to rearchitect the model of the subset
of Fedora that is for server containers to support this - something like a 
"just the binaries"
data blob, and then *optionally* turn that intermediate into an RPM that would
have a systemd unit file.  It could really be an RPM, just without the %post
scripts relating to systemd and the unit files.

In practice though, it's not a big deal as long as shared libraries don't
end up pulling in systemd or other components.  And for software that's
container-only, the build process (Dockerfile or something better in the
future) for containers just won't require it.

> Yes, its more than one process.  I think thats the better tradeoff though.

What you're arguing is that *build* convenience for our current architecture
outweighs the *runtime* cost.  That doesn't make sense long term - they're
different problems.
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: no systemd in containers: Requires -> Recommends

[Craig Garner]

I usually don't say anything and just read...it's not really my place.  I
just like to keep up with things going on.  But, by the time you finish
worrying about all the overhead and things get finalized, there's going to
be so damn much RAM and processing power no one will care.  Kind of like
developing one white blood cell to defend your entire body.



On Thu, Dec 17, 2015 at 3:39 PM, Colin Walters  wrote:

> On Thu, Dec 17, 2015, at 01:19 PM, Neil Horman wrote:
> >
> > In either case, you're going to wind up butchering a fair amount of what
> the rpm
> > is going to be doing anyway.  If its so important to minimize that
> storage, rpm
> > dependencies shouldn't really be a big deal, because you know you're
> going to
> > have to either do some FS surgery anyway.
>
> What I'm actually arguing long term is to rearchitect the model of the
> subset
> of Fedora that is for server containers to support this - something like a
> "just the binaries"
> data blob, and then *optionally* turn that intermediate into an RPM that
> would
> have a systemd unit file.  It could really be an RPM, just without the
> %post
> scripts relating to systemd and the unit files.
>
> In practice though, it's not a big deal as long as shared libraries don't
> end up pulling in systemd or other components.  And for software that's
> container-only, the build process (Dockerfile or something better in the
> future) for containers just won't require it.
>
> > Yes, its more than one process.  I think thats the better tradeoff
> though.
>
> What you're arguing is that *build* convenience for our current
> architecture
> outweighs the *runtime* cost.  That doesn't make sense long term - they're
> different problems.
> --
> devel mailing list
> devel@lists.fedoraproject.org
> http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org
>
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: no systemd in containers: Requires -> Recommends

[Colin Walters]

On Thu, Dec 17, 2015, at 04:00 PM, Neil Horman wrote:
> If its so important to not use up that small
> additional amount of ram and cpu, so be it, but that seems like a different
> question than the one being addressed.

That is primarily what I'm talking about indeed.  

The disk usage does matter, but less so - there are many ways to optimize that
that don't necessarily require changing the container content.
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: no systemd in containers: Requires -> Recommends

[Richard W.M. Jones]

On Thu, Dec 17, 2015 at 02:35:03PM -0600, Jason L Tibbitts III wrote:
> > "HH" == Harald Hoyer  writes:
> 
> HH> The preset enablement would be missing.
> 
> Couldn't systemd simply apply presets when it is installed?  (Not
> upgraded, but on a fresh install?)

Couldn't it use inotify (or whatever we're using these days to detect
filesystem changes)?  So when you drop in the unit file, systemd
notices and reloads.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
Fedora Windows cross-compiler. Compile Windows programs, test, and
build Windows installers. Over 100 libraries supported.
http://fedoraproject.org/wiki/MinGW
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: no systemd in containers: Requires -> Recommends

[Neil Horman]

On Thu, Dec 17, 2015 at 10:04:48PM +0100, Reindl Harald wrote:
> 
> 
> Am 17.12.2015 um 22:00 schrieb Neil Horman:
> >On Thu, Dec 17, 2015 at 03:39:17PM -0500, Colin Walters wrote:
> >>What you're arguing is that *build* convenience for our current architecture
> >>outweighs the *runtime* cost.  That doesn't make sense long term - they're
> >>different problems.
> >What runtime cost are you referring to here?  The cost of running systemd in 
> >a
> >container?  Its miniscule, given that its only job is to start a handful of
> >units and get out of the way
> 
> that's not true
> 
> on virtual machines with no load all day long PID1 is often found as number
> 1 in htop sorted by CPU usage - multiply that with 500 and you have a
> *useless* and well noticeable load on the host
> 
What I hear you saying is that on a system that has nothing better to do, the
primary monitoring process wakes up periodically to check on various system
aspects (cron jobs, journal rotates, etc).  That sounds like working as designed
to me, not a reason to throw the proverbial baby out with the bath water.  That
seems like a reason to do some tuning.

Neil




> --
> devel mailing list
> devel@lists.fedoraproject.org
> http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: no systemd in containers: Requires -> Recommends

[Craig Garner]

3D XPoint

On Thu, Dec 17, 2015 at 4:20 PM, Josh Boyer 
wrote:

> On Thu, Dec 17, 2015 at 3:54 PM, Reindl Harald 
> wrote:
> >
> >
> > Am 17.12.2015 um 21:47 schrieb Craig Garner:
> >>
> >> I usually don't say anything and just read...it's not really my place.
> >> I just like to keep up with things going on.  But, by the time you
> >> finish worrying about all the overhead and things get finalized, there's
> >> going to be so damn much RAM and processing power no one will care.
> >> Kind of like developing one white blood cell to defend your entire body.
> >
> >
> > this is pure nonsense and the attitude "going to be so damn much RAM and
> > processing power no one will care" is the reason that machines these days
> > are not nearby as fast as they could and should be compared with
> machines 10
> > years ago - brainless ressource wasting
>
> Please, please stop calling other people's ideas nonsense.  There's no
> need for it.  You would do much better to get your point across by
> just providing your details and specifics.  This entire paragraph is
> unnecessarily antagonistic.  It immediately puts people on the
> defensive.
>
> > when you have real production load and a ton of virtual machines and
> > containers running on the same host and every one of them is wasting
> > ressources the summary of all the overhead is something you care or
> should
> > care about
>
> This alone would have gone much farther in furthering the discussion.
>
> josh
> --
> devel mailing list
> devel@lists.fedoraproject.org
> http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org
>
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: no systemd in containers: Requires -> Recommends

[Neil Horman]

On Thu, Dec 17, 2015 at 03:39:17PM -0500, Colin Walters wrote:
> On Thu, Dec 17, 2015, at 01:19 PM, Neil Horman wrote:
> >
> > In either case, you're going to wind up butchering a fair amount of what 
> > the rpm
> > is going to be doing anyway.  If its so important to minimize that storage, 
> > rpm
> > dependencies shouldn't really be a big deal, because you know you're going 
> > to
> > have to either do some FS surgery anyway. 
> 
> What I'm actually arguing long term is to rearchitect the model of the subset
> of Fedora that is for server containers to support this - something like a 
> "just the binaries"
> data blob, and then *optionally* turn that intermediate into an RPM that would
> have a systemd unit file.  It could really be an RPM, just without the %post
> scripts relating to systemd and the unit files.
> 

sooo, you're advocating for an intermediate RPM that is just what you need for
your use case, to which you could then add additional metadata/tags/etc later?
That...seems like alot of work for very little value.

Lets try a example: httpd.  If you want to run a webserver in a microcontainer,
under this model, you would wind up packaging /usr/sbin/apachectl,
/ur/sbin/fcgistarter, /usr/sbin/httpd, /usr/sbin/suexec, /usr/sbin/rotatelogs,
dozens of module files, and a host of default page images.
Presuming you want your web server to do a bit more, your docker file still
needs to:

1) remove at least 3 of those binaries, as well as the default image files

2) remove any and all of the modules that you don't plan to use

3) add a config file

4) write a startup script so docker knows how to start this thing

So, even if you have 'just the binaries' packaged, you still have to do a metric
butload of file system surgery to arrive at your minimal microcontainer goal.
So packaging all the binaries alone hasn't bought you anything. At the above
point you've downloaded, installed and erased about 6Mb of data.  At that point
the other 24Mb for systemd really isn't much of a headache to deal with,
especially if the cure is making a major overhaul to our packaging tool.

Just out of curiosity, if you wish to avoid all these dependencies, why not just
writ a simple plugin to dnf to not depsolve anything, so you only install
specifically what you ask for?  It might even be as simple as passing -x systemd
to the operation, so dnf exludes systemd from the operation.


> In practice though, it's not a big deal as long as shared libraries don't
> end up pulling in systemd or other components.  And for software that's
> container-only, the build process (Dockerfile or something better in the
> future) for containers just won't require it.
> 
> > Yes, its more than one process.  I think thats the better tradeoff though.
> 
> What you're arguing is that *build* convenience for our current architecture
> outweighs the *runtime* cost.  That doesn't make sense long term - they're
> different problems.
What runtime cost are you referring to here?  The cost of running systemd in a
container?  Its miniscule, given that its only job is to start a handful of
units and get out of the way.  If its so important to not use up that small
additional amount of ram and cpu, so be it, but that seems like a different
question than the one being addressed.  Initially we were talking about
minimzing container size, not runtime speed.  If thats what you want, perhaps
the solution is to provide tunables (if they don't already exist), to ensure
that systemd optimizes its runtime for memory size and cpu usage, but its alreay
not that big in those areas.

Neil

> --
> devel mailing list
> devel@lists.fedoraproject.org
> http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: no systemd in containers: Requires -> Recommends

[Reindl Harald]

Am 17.12.2015 um 22:00 schrieb Neil Horman:

On Thu, Dec 17, 2015 at 03:39:17PM -0500, Colin Walters wrote:

What you're arguing is that *build* convenience for our current architecture
outweighs the *runtime* cost.  That doesn't make sense long term - they're
different problems.

What runtime cost are you referring to here?  The cost of running systemd in a
container?  Its miniscule, given that its only job is to start a handful of
units and get out of the way


that's not true

on virtual machines with no load all day long PID1 is often found as 
number 1 in htop sorted by CPU usage - multiply that with 500 and you 
have a *useless* and well noticeable load on the host




signature.asc
Description: OpenPGP digital signature
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: no systemd in containers: Requires -> Recommends

[Reindl Harald]

Am 17.12.2015 um 22:11 schrieb Neil Horman:

On Thu, Dec 17, 2015 at 10:04:48PM +0100, Reindl Harald wrote:

Am 17.12.2015 um 22:00 schrieb Neil Horman:

On Thu, Dec 17, 2015 at 03:39:17PM -0500, Colin Walters wrote:

What you're arguing is that *build* convenience for our current architecture
outweighs the *runtime* cost.  That doesn't make sense long term - they're
different problems.

What runtime cost are you referring to here?  The cost of running systemd in a
container?  Its miniscule, given that its only job is to start a handful of
units and get out of the way


that's not true

on virtual machines with no load all day long PID1 is often found as number
1 in htop sorted by CPU usage - multiply that with 500 and you have a
*useless* and well noticeable load on the host


What I hear you saying is that on a system that has nothing better to do, the
primary monitoring process wakes up periodically to check on various system
aspects (cron jobs, journal rotates, etc).  That sounds like working as designed
to me, not a reason to throw the proverbial baby out with the bath water.  That
seems like a reason to do some tuning.


it is wasting ressources in a container running only one process - 
period - what is there to discuss - in any serious setup there shou,ld 
be nothing installed which is not *really* needed




signature.asc
Description: OpenPGP digital signature
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: no systemd in containers: Requires -> Recommends

[Josh Boyer]

On Thu, Dec 17, 2015 at 3:54 PM, Reindl Harald  wrote:
>
>
> Am 17.12.2015 um 21:47 schrieb Craig Garner:
>>
>> I usually don't say anything and just read...it's not really my place.
>> I just like to keep up with things going on.  But, by the time you
>> finish worrying about all the overhead and things get finalized, there's
>> going to be so damn much RAM and processing power no one will care.
>> Kind of like developing one white blood cell to defend your entire body.
>
>
> this is pure nonsense and the attitude "going to be so damn much RAM and
> processing power no one will care" is the reason that machines these days
> are not nearby as fast as they could and should be compared with machines 10
> years ago - brainless ressource wasting

Please, please stop calling other people's ideas nonsense.  There's no
need for it.  You would do much better to get your point across by
just providing your details and specifics.  This entire paragraph is
unnecessarily antagonistic.  It immediately puts people on the
defensive.

> when you have real production load and a ton of virtual machines and
> containers running on the same host and every one of them is wasting
> ressources the summary of all the overhead is something you care or should
> care about

This alone would have gone much farther in furthering the discussion.

josh
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: no systemd in containers: Requires -> Recommends

[Michal Sekletar]

On Thu, Dec 17, 2015 at 4:18 PM, Lennart Poettering
 wrote:
> On Thu, 17.12.15 11:28, Michal Sekletar (msekl...@redhat.com) wrote:
>
>> On Thu, Dec 17, 2015 at 10:43 AM, Harald Hoyer  wrote:
>> > The downside is:
>> > - if systemd is installed afterwards, the %post scripts do not trigger
>> > - packages, which need systemd-tmpfiles or systemd-sysusers could not be 
>> > converted
>>
>> IIRC, some time ago there was a proposal to split systemd-tmpfiles,
>> systemd-sysusers and other utilities to separate sub-package called
>> systemd-tools. We should probably revisit this idea.
>
> Why? Do you have any technical reasons?

Reason to have those tools separate was to satisfy dependencies of rpm
macros in various packages w/o a need to install systemd package.

Michal
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: no systemd in containers: Requires -> Recommends

[Jason L Tibbitts III]

> "RWMJ" == Richard W M Jones  writes:

RWMJ> Couldn't it use inotify (or whatever we're using these days to
RWMJ> detect filesystem changes)?  So when you drop in the unit file,
RWMJ> systemd notices and reloads.

Well, the point is that systemd isn't running or even installed when the
daemons are installed.  The question is what happens when systemd comes
up later.  (And yes, systemd could use rpm file triggers so that we can
drop the scriptlets entirely.  That would be great, but it's an entirely
separate discussion.)

I know this thread has gone in a completely different direction, but the
original message was about modifying or removing the dependencies.  Can
we drop dependencies on systemd and still have a system that works as
expected if systemd gets installed later?  If so, then dropping the
dependency doesn't actually hurt anything since the kernel will still
pull it in, and thus the other arguments about this become kind of
pointless.

 - J<
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: no systemd in containers: Requires -> Recommends

[Brendan Conoboy]

On 12/17/2015 01:43 AM, Harald Hoyer wrote:

For docker containers, or containers, which don't want systemd, the current
"Requires: systemd" in a lot of packages is preventing building a minimal image.

To improve the situation, we could make use of the new rpm weak dependencies.
So the

Requires(post): systemd
Requires(preun): systemd
Requires(postun): systemd

would become

Recommends: systemd
OrderWithRequires(post): systemd
OrderWithRequires(preun): systemd
OrderWithRequires(postun): systemd

With this in place, kickstart files could omit systemd.

The downside is:
- if systemd is installed afterwards, the %post scripts do not trigger
- packages, which need systemd-tmpfiles or systemd-sysusers could not be 
converted

If systemd is removed before the other packages, I don't see a problem.
There are only leftovers in /etc/systemd.

To prevent having a non-bootable system (not container), we could let the
kernel.spec have a Requires on systemd.

Comments? Please discuss.


I haven't seen a lot of downside brought up in this thread.  If the 
only objections people have is that it doesn't facilitate their 
personal use cases those don't seem like real objections.  Is anybody 
going to be really negatively impacted by such a change?


For my part I'd like to see this happen, not just for packages 
requiring systemd, but for all packages where "Requires" is really 
stronger than necessary.  Now that we have soft dependencies it would 
be nice to go through and move to Recommends where software continues 
to function in some reduced capacity.  Everything would still go into 
the composes as before and for people who like things the way they 
are, there isn't much downside.  Meanwhile, for people who want to 
trim their package set to the utmost, they would be able to do so 
without creating fake stub packages or using hacks to get around requires.


--
Brendan Conoboy / RHEL Development Coordinator / Red Hat, Inc.
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: no systemd in containers: Requires -> Recommends

[Josh Boyer]

On Thu, Dec 17, 2015 at 4:43 AM, Harald Hoyer  wrote:
> To prevent having a non-bootable system (not container), we could let the
> kernel.spec have a Requires on systemd.

The kernel has had a Requires on systemd for quite some time.  It is
now in kernel-core, but you cannot boot a system without kernel-core
either.

josh
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: no systemd in containers: Requires -> Recommends

[Matthew Miller]

On Thu, Dec 17, 2015 at 01:23:01PM -0500, Neil Horman wrote:
> FWIW, In freight[1], I assume systemd is a member of every container,
> and that services are started via unit files in the container. Its
> made generating containers unbelievably easy, in that it doesn't
> require any additional knoweldge about an application - the way it
> runs in a container is exactly the way it runs outside of a
> container, its really nice.
> 

And to be clear, this is an awesome, valid use case too which we should
also support.

-- 
Matthew Miller

Fedora Project Leader
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: no systemd in containers: Requires -> Recommends

[Zbigniew Jędrzejewski-Szmek]

On Thu, Dec 17, 2015 at 05:34:31PM -0800, Brendan Conoboy wrote:
> On 12/17/2015 05:27 PM, Zbigniew Jędrzejewski-Szmek wrote:
> >On Thu, Dec 17, 2015 at 04:13:06PM -0800, Brendan Conoboy wrote:
> >>On 12/17/2015 01:43 AM, Harald Hoyer wrote:
> >>>For docker containers, or containers, which don't want systemd, the current
> >>>"Requires: systemd" in a lot of packages is preventing building a minimal 
> >>>image.
> >>>
> >>>To improve the situation, we could make use of the new rpm weak 
> >>>dependencies.
> >>>So the
> >>>
> >>>Requires(post): systemd
> >>>Requires(preun): systemd
> >>>Requires(postun): systemd
> >>>
> >>>would become
> >>>
> >>>Recommends: systemd
> >>>OrderWithRequires(post): systemd
> >>>OrderWithRequires(preun): systemd
> >>>OrderWithRequires(postun): systemd
> >>>
> >>>With this in place, kickstart files could omit systemd.
> >>>
> >>>The downside is:
> >>>- if systemd is installed afterwards, the %post scripts do not trigger
> >>>- packages, which need systemd-tmpfiles or systemd-sysusers could not be 
> >>>converted
> >>>
> >>>If systemd is removed before the other packages, I don't see a problem.
> >>>There are only leftovers in /etc/systemd.
> >>>
> >>>To prevent having a non-bootable system (not container), we could let the
> >>>kernel.spec have a Requires on systemd.
> >>>
> >>>Comments? Please discuss.
> >>
> >>I haven't seen a lot of downside brought up in this thread.  If the
> >>only objections people have is that it doesn't facilitate their
> >>personal use cases those don't seem like real objections.  Is
> >>anybody going to be really negatively impacted by such a change?
> >>
> >>For my part I'd like to see this happen, not just for packages
> >>requiring systemd, but for all packages where "Requires" is really
> >>stronger than necessary.  Now that we have soft dependencies it
> >>would be nice to go through and move to Recommends where software
> >>continues to function in some reduced capacity.
> >
> >For some packages "reduced capacity" because of lack of systemd.rpm
> >means "doesn't even get started as expected" or "crashes on
> >start with permission errors" or "cannot write logs" or similar.
> >Like Lennart and Neil said, utilities provided by systemd.rpm are the
> >basis which allows many things to "just work". This is so obvious
> >that it is assumed implicitly in this disussion, and it's hardly
> >"personal use cases".
> 
> If the software crashes on start with permission error that's not
> really working in a reduced capacity.

Exactly. So Required(post/pre/preun):systemd cannot currently be
changed to Recommmends, at least in the general case.

Zbyszek
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: no systemd in containers: Requires -> Recommends

[Brendan Conoboy]

On 12/17/2015 05:46 PM, Zbigniew Jędrzejewski-Szmek wrote:

On Thu, Dec 17, 2015 at 05:34:31PM -0800, Brendan Conoboy wrote:

On 12/17/2015 05:27 PM, Zbigniew Jędrzejewski-Szmek wrote:

On Thu, Dec 17, 2015 at 04:13:06PM -0800, Brendan Conoboy wrote:

On 12/17/2015 01:43 AM, Harald Hoyer wrote:

For docker containers, or containers, which don't want systemd, the current
"Requires: systemd" in a lot of packages is preventing building a minimal image.

To improve the situation, we could make use of the new rpm weak dependencies.
So the

Requires(post): systemd
Requires(preun): systemd
Requires(postun): systemd

would become

Recommends: systemd
OrderWithRequires(post): systemd
OrderWithRequires(preun): systemd
OrderWithRequires(postun): systemd

With this in place, kickstart files could omit systemd.

The downside is:
- if systemd is installed afterwards, the %post scripts do not trigger
- packages, which need systemd-tmpfiles or systemd-sysusers could not be 
converted

If systemd is removed before the other packages, I don't see a problem.
There are only leftovers in /etc/systemd.

To prevent having a non-bootable system (not container), we could let the
kernel.spec have a Requires on systemd.

Comments? Please discuss.


I haven't seen a lot of downside brought up in this thread.  If the
only objections people have is that it doesn't facilitate their
personal use cases those don't seem like real objections.  Is
anybody going to be really negatively impacted by such a change?

For my part I'd like to see this happen, not just for packages
requiring systemd, but for all packages where "Requires" is really
stronger than necessary.  Now that we have soft dependencies it
would be nice to go through and move to Recommends where software
continues to function in some reduced capacity.


For some packages "reduced capacity" because of lack of systemd.rpm
means "doesn't even get started as expected" or "crashes on
start with permission errors" or "cannot write logs" or similar.
Like Lennart and Neil said, utilities provided by systemd.rpm are the
basis which allows many things to "just work". This is so obvious
that it is assumed implicitly in this disussion, and it's hardly
"personal use cases".


If the software crashes on start with permission error that's not
really working in a reduced capacity.


Exactly. So Required(post/pre/preun):systemd cannot currently be
changed to Recommmends, at least in the general case.


It's not clear the cases you have in mind are the general case.  There 
is doubtless a happy medium here, though.  Perhaps it is opening up 
the policy to promote Recommends, but leave it to packager discretion. 
 Recommends largely behaves like requires, so it's fairly low risk to 
err on the side of recommends.


--
Brendan Conoboy / RHEL Development Coordinator / Red Hat, Inc.
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: no systemd in containers: Requires -> Recommends

[Zbigniew Jędrzejewski-Szmek]

On Thu, Dec 17, 2015 at 05:54:52PM -0800, Brendan Conoboy wrote:
> On 12/17/2015 05:46 PM, Zbigniew Jędrzejewski-Szmek wrote:
> >On Thu, Dec 17, 2015 at 05:34:31PM -0800, Brendan Conoboy wrote:
> >>On 12/17/2015 05:27 PM, Zbigniew Jędrzejewski-Szmek wrote:
> >>>On Thu, Dec 17, 2015 at 04:13:06PM -0800, Brendan Conoboy wrote:
> On 12/17/2015 01:43 AM, Harald Hoyer wrote:
> >For docker containers, or containers, which don't want systemd, the 
> >current
> >"Requires: systemd" in a lot of packages is preventing building a 
> >minimal image.
> >
> >To improve the situation, we could make use of the new rpm weak 
> >dependencies.
> >So the
> >
> >Requires(post): systemd
> >Requires(preun): systemd
> >Requires(postun): systemd
> >
> >would become
> >
> >Recommends: systemd
> >OrderWithRequires(post): systemd
> >OrderWithRequires(preun): systemd
> >OrderWithRequires(postun): systemd
> >
> >With this in place, kickstart files could omit systemd.
> >
> >The downside is:
> >- if systemd is installed afterwards, the %post scripts do not trigger
> >- packages, which need systemd-tmpfiles or systemd-sysusers could not be 
> >converted
> >
> >If systemd is removed before the other packages, I don't see a problem.
> >There are only leftovers in /etc/systemd.
> >
> >To prevent having a non-bootable system (not container), we could let the
> >kernel.spec have a Requires on systemd.
> >
> >Comments? Please discuss.
> 
> I haven't seen a lot of downside brought up in this thread.  If the
> only objections people have is that it doesn't facilitate their
> personal use cases those don't seem like real objections.  Is
> anybody going to be really negatively impacted by such a change?
> 
> For my part I'd like to see this happen, not just for packages
> requiring systemd, but for all packages where "Requires" is really
> stronger than necessary.  Now that we have soft dependencies it
> would be nice to go through and move to Recommends where software
> continues to function in some reduced capacity.
> >>>
> >>>For some packages "reduced capacity" because of lack of systemd.rpm
> >>>means "doesn't even get started as expected" or "crashes on
> >>>start with permission errors" or "cannot write logs" or similar.
> >>>Like Lennart and Neil said, utilities provided by systemd.rpm are the
> >>>basis which allows many things to "just work". This is so obvious
> >>>that it is assumed implicitly in this disussion, and it's hardly
> >>>"personal use cases".
> >>
> >>If the software crashes on start with permission error that's not
> >>really working in a reduced capacity.
> >
> >Exactly. So Required(post/pre/preun):systemd cannot currently be
> >changed to Recommmends, at least in the general case.
> 
> It's not clear the cases you have in mind are the general case.
> There is doubtless a happy medium here, though.  Perhaps it is
> opening up the policy to promote Recommends, but leave it to
> packager discretion.  Recommends largely behaves like requires, so
> it's fairly low risk to err on the side of recommends.

Currently service enablement is very much centralized. With the
recent changes (https://fedoraproject.org/wiki/Packaging:DefaultServices)
and the move of presets to fedora-release this is even more true than
before. Various editions and spins modify the presets to disable/enable
stuff they need, and this hinges on all packages behaving uniformly.
Letting individual packages make the choice would break this
centralization; packages are not the right level to make this choice.

Zbyszek
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: no systemd in containers: Requires -> Recommends

[Haïkel]

2015-12-18 0:58 GMT+01:00 Jason L Tibbitts III :
>> "RWMJ" == Richard W M Jones  writes:
>
> RWMJ> Couldn't it use inotify (or whatever we're using these days to
> RWMJ> detect filesystem changes)?  So when you drop in the unit file,
> RWMJ> systemd notices and reloads.
>
> Well, the point is that systemd isn't running or even installed when the
> daemons are installed.  The question is what happens when systemd comes
> up later.  (And yes, systemd could use rpm file triggers so that we can
> drop the scriptlets entirely.  That would be great, but it's an entirely
> separate discussion.)
>
> I know this thread has gone in a completely different direction, but the
> original message was about modifying or removing the dependencies.  Can
> we drop dependencies on systemd and still have a system that works as
> expected if systemd gets installed later?  If so, then dropping the
> dependency doesn't actually hurt anything since the kernel will still
> pull it in, and thus the other arguments about this become kind of
> pointless.
>
>  - J<

We could waste time bickering but Jason made a point.
kernel-core requires systemd, so all running instances of Fedora
(except for containers) have systemd installed,
and since our systemd scriptlets are non failing, we could safely drop
the requirements in other packages.

I'm in favor of updating systemd guidelines to drop that requirements.

Regards,
H.
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: no systemd in containers: Requires -> Recommends

[Adam Williamson]

On Thu, 2015-12-17 at 16:13 -0800, Brendan Conoboy wrote:
> On 12/17/2015 01:43 AM, Harald Hoyer wrote:
> > For docker containers, or containers, which don't want systemd, the current
> > "Requires: systemd" in a lot of packages is preventing building a minimal 
> > image.
> > 
> > To improve the situation, we could make use of the new rpm weak 
> > dependencies.
> > So the
> > 
> > Requires(post): systemd
> > Requires(preun): systemd
> > Requires(postun): systemd
> > 
> > would become
> > 
> > Recommends: systemd
> > OrderWithRequires(post): systemd
> > OrderWithRequires(preun): systemd
> > OrderWithRequires(postun): systemd
> > 
> > With this in place, kickstart files could omit systemd.
> > 
> > The downside is:
> > - if systemd is installed afterwards, the %post scripts do not trigger
> > - packages, which need systemd-tmpfiles or systemd-sysusers could not be 
> > converted
> > 
> > If systemd is removed before the other packages, I don't see a problem.
> > There are only leftovers in /etc/systemd.
> > 
> > To prevent having a non-bootable system (not container), we could let the
> > kernel.spec have a Requires on systemd.
> > 
> > Comments? Please discuss.
> 
> I haven't seen a lot of downside brought up in this thread.  If the 
> only objections people have is that it doesn't facilitate their 
> personal use cases those don't seem like real objections.  Is anybody 
> going to be really negatively impacted by such a change?
> 
> For my part I'd like to see this happen, not just for packages 
> requiring systemd, but for all packages where "Requires" is really 
> stronger than necessary.  Now that we have soft dependencies it would 
> be nice to go through and move to Recommends where software continues 
> to function in some reduced capacity.  Everything would still go into 
> the composes as before and for people who like things the way they 
> are, there isn't much downside.  Meanwhile, for people who want to 
> trim their package set to the utmost, they would be able to do so 
> without creating fake stub packages or using hacks to get around requires.

I'm OK with this *so long as* people don't start using Recommends: for
optional and heavy things. As long as we have a strong convention that
Recommends: is for things you're almost certainly going to want but
aren't technically *required*, and Suggests: is for things that are
pretty optional, it should work OK. What would concern me is if we had
a mix of packages using Recommends: for nearly-required things and
packages using Recommends: for very-optional things, because you'd then
be more or less stuck with the very-optional stuff (since if you
skipped 'Recommends' universally, you'd miss lots of important stuff
from other packages).
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net

--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: no systemd in containers: Requires -> Recommends

[Zbigniew Jędrzejewski-Szmek]

On Thu, Dec 17, 2015 at 04:19:44PM +0100, Lennart Poettering wrote:
> On Thu, 17.12.15 14:36, Zbigniew Jędrzejewski-Szmek (zbys...@in.waw.pl) wrote:
> 
> > On Thu, Dec 17, 2015 at 11:28:51AM +0100, Michal Sekletar wrote:
> > > On Thu, Dec 17, 2015 at 10:43 AM, Harald Hoyer  wrote:
> > > > The downside is:
> > > > - if systemd is installed afterwards, the %post scripts do not trigger
> > > > - packages, which need systemd-tmpfiles or systemd-sysusers could not 
> > > > be converted
> > > 
> > > IIRC, some time ago there was a proposal to split systemd-tmpfiles,
> > > systemd-sysusers and other utilities to separate sub-package called
> > > systemd-tools. We should probably revisit this idea.
> > 
> > Yes, I intend to push the split to rawhide real soon now:
> > https://fedorahosted.org/fesco/ticket/1501
> 
> Hmm? I am not aware of any plan to split out tmpfiles and stuff... 
Please see my reply to my owm mail. Ticket 1501 is a different split,
the one we discussed recently on systemd-devel. I pasted the wrong ticket.

Zbyszek
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: no systemd in containers: Requires -> Recommends

[Zbigniew Jędrzejewski-Szmek]

On Thu, Dec 17, 2015 at 04:13:06PM -0800, Brendan Conoboy wrote:
> On 12/17/2015 01:43 AM, Harald Hoyer wrote:
> >For docker containers, or containers, which don't want systemd, the current
> >"Requires: systemd" in a lot of packages is preventing building a minimal 
> >image.
> >
> >To improve the situation, we could make use of the new rpm weak dependencies.
> >So the
> >
> >Requires(post): systemd
> >Requires(preun): systemd
> >Requires(postun): systemd
> >
> >would become
> >
> >Recommends: systemd
> >OrderWithRequires(post): systemd
> >OrderWithRequires(preun): systemd
> >OrderWithRequires(postun): systemd
> >
> >With this in place, kickstart files could omit systemd.
> >
> >The downside is:
> >- if systemd is installed afterwards, the %post scripts do not trigger
> >- packages, which need systemd-tmpfiles or systemd-sysusers could not be 
> >converted
> >
> >If systemd is removed before the other packages, I don't see a problem.
> >There are only leftovers in /etc/systemd.
> >
> >To prevent having a non-bootable system (not container), we could let the
> >kernel.spec have a Requires on systemd.
> >
> >Comments? Please discuss.
> 
> I haven't seen a lot of downside brought up in this thread.  If the
> only objections people have is that it doesn't facilitate their
> personal use cases those don't seem like real objections.  Is
> anybody going to be really negatively impacted by such a change?
> 
> For my part I'd like to see this happen, not just for packages
> requiring systemd, but for all packages where "Requires" is really
> stronger than necessary.  Now that we have soft dependencies it
> would be nice to go through and move to Recommends where software
> continues to function in some reduced capacity.

For some packages "reduced capacity" because of lack of systemd.rpm
means "doesn't even get started as expected" or "crashes on
start with permission errors" or "cannot write logs" or similar.
Like Lennart and Neil said, utilities provided by systemd.rpm are the
basis which allows many things to "just work". This is so obvious
that it is assumed implicitly in this disussion, and it's hardly
"personal use cases".

Zbyszek
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: no systemd in containers: Requires -> Recommends

[Brendan Conoboy]

On 12/17/2015 05:27 PM, Zbigniew Jędrzejewski-Szmek wrote:

On Thu, Dec 17, 2015 at 04:13:06PM -0800, Brendan Conoboy wrote:

On 12/17/2015 01:43 AM, Harald Hoyer wrote:

For docker containers, or containers, which don't want systemd, the current
"Requires: systemd" in a lot of packages is preventing building a minimal image.

To improve the situation, we could make use of the new rpm weak dependencies.
So the

Requires(post): systemd
Requires(preun): systemd
Requires(postun): systemd

would become

Recommends: systemd
OrderWithRequires(post): systemd
OrderWithRequires(preun): systemd
OrderWithRequires(postun): systemd

With this in place, kickstart files could omit systemd.

The downside is:
- if systemd is installed afterwards, the %post scripts do not trigger
- packages, which need systemd-tmpfiles or systemd-sysusers could not be 
converted

If systemd is removed before the other packages, I don't see a problem.
There are only leftovers in /etc/systemd.

To prevent having a non-bootable system (not container), we could let the
kernel.spec have a Requires on systemd.

Comments? Please discuss.


I haven't seen a lot of downside brought up in this thread.  If the
only objections people have is that it doesn't facilitate their
personal use cases those don't seem like real objections.  Is
anybody going to be really negatively impacted by such a change?

For my part I'd like to see this happen, not just for packages
requiring systemd, but for all packages where "Requires" is really
stronger than necessary.  Now that we have soft dependencies it
would be nice to go through and move to Recommends where software
continues to function in some reduced capacity.


For some packages "reduced capacity" because of lack of systemd.rpm
means "doesn't even get started as expected" or "crashes on
start with permission errors" or "cannot write logs" or similar.
Like Lennart and Neil said, utilities provided by systemd.rpm are the
basis which allows many things to "just work". This is so obvious
that it is assumed implicitly in this disussion, and it's hardly
"personal use cases".


If the software crashes on start with permission error that's not 
really working in a reduced capacity.  There is a degree of 
subjectivity and arguing every corner case is not useful.  The default 
behavior of dnf is that if a package is recommended and available, it 
will be installed just like if it were required.  The only difference 
is that if it isn't available an install can still succeed.


For those who are curious, Fedora's documentation on weak dependencies 
is here:


https://fedoraproject.org/wiki/Packaging:WeakDependencies

--
Brendan Conoboy / RHEL Development Coordinator / Red Hat, Inc.
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: no systemd in containers: Requires -> Recommends

[Brendan Conoboy]

On 12/17/2015 04:22 PM, Adam Williamson wrote:

On Thu, 2015-12-17 at 16:13 -0800, Brendan Conoboy wrote:

On 12/17/2015 01:43 AM, Harald Hoyer wrote:

For docker containers, or containers, which don't want systemd, the current
"Requires: systemd" in a lot of packages is preventing building a minimal image.

To improve the situation, we could make use of the new rpm weak dependencies.
So the

Requires(post): systemd
Requires(preun): systemd
Requires(postun): systemd

would become

Recommends: systemd
OrderWithRequires(post): systemd
OrderWithRequires(preun): systemd
OrderWithRequires(postun): systemd

With this in place, kickstart files could omit systemd.

The downside is:
- if systemd is installed afterwards, the %post scripts do not trigger
- packages, which need systemd-tmpfiles or systemd-sysusers could not be 
converted

If systemd is removed before the other packages, I don't see a problem.
There are only leftovers in /etc/systemd.

To prevent having a non-bootable system (not container), we could let the
kernel.spec have a Requires on systemd.

Comments? Please discuss.


I haven't seen a lot of downside brought up in this thread.  If the
only objections people have is that it doesn't facilitate their
personal use cases those don't seem like real objections.  Is anybody
going to be really negatively impacted by such a change?

For my part I'd like to see this happen, not just for packages
requiring systemd, but for all packages where "Requires" is really
stronger than necessary.  Now that we have soft dependencies it would
be nice to go through and move to Recommends where software continues
to function in some reduced capacity.  Everything would still go into
the composes as before and for people who like things the way they
are, there isn't much downside.  Meanwhile, for people who want to
trim their package set to the utmost, they would be able to do so
without creating fake stub packages or using hacks to get around requires.


I'm OK with this *so long as* people don't start using Recommends: for
optional and heavy things. As long as we have a strong convention that
Recommends: is for things you're almost certainly going to want but
aren't technically *required*, and Suggests: is for things that are
pretty optional, it should work OK. What would concern me is if we had
a mix of packages using Recommends: for nearly-required things and
packages using Recommends: for very-optional things, because you'd then
be more or less stuck with the very-optional stuff (since if you
skipped 'Recommends' universally, you'd miss lots of important stuff
from other packages).


Totally with you there.  We don't want to cut dependencies out at any 
cost.  Rather we want to make packages suitable to a wider audience, 
useful in more scenarios, without forcing the pulling in superfluous 
pieces unnecessarily.


--
Brendan Conoboy / RHEL Development Coordinator / Red Hat, Inc.
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: no systemd in containers: Requires -> Recommends

[Zbigniew Jędrzejewski-Szmek]

On Thu, Dec 17, 2015 at 02:35:03PM -0600, Jason L Tibbitts III wrote:
> > "HH" == Harald Hoyer  writes:
> 
> HH> The preset enablement would be missing.
> 
> Couldn't systemd simply apply presets when it is installed?  (Not
> upgraded, but on a fresh install?)

No. It simply doesn't have enough information. If you install some service,
and manually change it's enablement, this local configuration is preserved.
If systemd would reset enablement on installation, it would be step back.

Zbyszek
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: no systemd in containers: Requires -> Recommends

[Zbigniew Jędrzejewski-Szmek]

On Fri, Dec 18, 2015 at 01:24:27AM +0100, Haïkel wrote:
> 2015-12-18 0:58 GMT+01:00 Jason L Tibbitts III :
> >> "RWMJ" == Richard W M Jones  writes:
> >
> > RWMJ> Couldn't it use inotify (or whatever we're using these days to
> > RWMJ> detect filesystem changes)?  So when you drop in the unit file,
> > RWMJ> systemd notices and reloads.

This is answered below (systemd is not running), but let me add one
more reason why inotify wouldn't work: you cannot blindly reload
units, because sometimes a set of units have dependencies between
them, so you need to first install the full set, and then reload.

> > Well, the point is that systemd isn't running or even installed when the
> > daemons are installed.  The question is what happens when systemd comes
> > up later.  (And yes, systemd could use rpm file triggers so that we can
> > drop the scriptlets entirely.  That would be great, but it's an entirely
> > separate discussion.)

No, we can't. Using file triggers allows us to reduce the rpm
scriptlets in packages a lot, but not entirely. Only the package
scriptlets know whether some service is installed or upgraded and have
enough information to properly support presets and restarts.

> > I know this thread has gone in a completely different direction, but the
> > original message was about modifying or removing the dependencies.  Can
> > we drop dependencies on systemd and still have a system that works as
> > expected if systemd gets installed later?  If so, then dropping the
> > dependency doesn't actually hurt anything since the kernel will still
> > pull it in, and thus the other arguments about this become kind of
> > pointless.
> 
> We could waste time bickering but Jason made a point.
> kernel-core requires systemd, so all running instances of Fedora
> (except for containers) have systemd installed,
> and since our systemd scriptlets are non failing, we could safely drop
> the requirements in other packages.

You seem to be missing normal containers: in those there is not kernel
but systemd is and pretty much all services are running.

Zbyszek
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org