Re: [systemd-devel] How to create minimal portable services environments?

[Lennart Poettering]

On Mi, 29.08.18 12:24, Wojtek Swiatek (w...@swtk.info) wrote:

> Some of these are existing directories, some new ones and then  there are
> files (new, but possibly existing as well).
> 
> *How should I create a filesystem which has only the files required by the
> packages (= the ones it brings in, as well as all the
> dependencies)?*

Ideally your distriution would solve this or help you with it. This is
after all not a problem specific to portable services, but having a
minimal system is useful in almost all cases.

One idea is to use an initrd builder to put these images together. For
example, in systemd's set of tests we have something based on the
"dracut" initrd builder that puts together minimal testing images and
runs them in qemu and nspawn (see test/TEST-*/ in the
sources). something like this could also be used to put together
portable services. initrd builders usually start with a minimal set of
files you specify and then have limited logic built in to add
necessary dependencies of these files (usually by following ELF .so
deps and such).

It certainly would be a cool project to maybe generalize and beef up
the dracut image logic for building such portable service images!

Lennart

-- 
Lennart Poettering, Red Hat
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] How to create minimal portable services environments?

[Steve Dodd]

On 29 August 2018 at 16:14, Wojtek Swiatek  wrote:
> Le mer. 29 août 2018 à 17:11, Steve Dodd  a écrit :

>> Shouldn't be that hard to adapt one of the above for nspawn?

> nspawn is not the problem - portable services are. I use a minimal image
> with nspawn which is OK but portable services are intended to be even
> thinner: no need to bring in the OS, just the files needed to run the
> service (which are the program itself, dependencies not found in the host
> and unit files)

My bad, portable services are lumped in with nspawn in my head :)

Dependencies will still need duplicating in an image though, as even a
portable service will be chrooted and therefore need its own copy of
its libraries. Hence my remark about page cache sharing. The /usr
merge process and the possibility of read-only root host filesystems
mentioned in various systemd docs suggests it may one day be
possible/sane for containers to share /usr or even / ..

S.
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] How to create minimal portable services environments?

[Wojtek Swiatek]

Le mer. 29 août 2018 à 17:11, Steve Dodd  a écrit :

> On 29 August 2018 at 15:43, Steve Dodd  wrote:
>
> Shouldn't be that hard to adapt one of the above for nspawn?
>
>
nspawn is not the problem - portable services are. I use a minimal image
with nspawn which is OK but portable services are intended to be even
thinner: no need to bring in the OS, just the files needed to run the
service (which are the program itself, dependencies not found in the host
and unit files)
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] How to create minimal portable services environments?

[Steve Dodd]

On 29 August 2018 at 15:43, Steve Dodd  wrote:

> I'm kind of surprised a tool for this hasn't crossed my path already, [..] I 
> also wonder if there might be any
> Docker based tools that do this sort of inspection.

Bingo:

- https://github.com/djosephsen/skinnywhale/blob/master/skinnywhale.sh
- https://github.com/mvanholsteijn/strip-docker-image
- 
https://xebia.com/blog/how-to-create-the-smallest-possible-docker-container-of-any-image/
- https://github.com/docker-slim/docker-slim

Shouldn't be that hard to adapt one of the above for nspawn?

One other thing I'm interested in is page cache sharing between the
host and the container .. seems like a messier problem.

S.
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] How to create minimal portable services environments?

[Jérémy Rosen]

I know that this is not the usual answer to that sort of question but

you should try buildroot, rather than base yourself on debian.

When you deactivate the kernel and busybox (it's done through a 
menuconfig type interface) you will get the minimal type of system you want.




On 29/08/2018 12:24, Wojtek Swiatek wrote:

Hello everyone,

v239 brought in portable services (a good description is at
http://0pointer.net/blog/walkthrough-for-portable-services.html) and while
I still cannot make it work (I do not have a /usr/lib/systemd/portablectl
despite having systemd --version reporting 239, but this is going to be a
separate question) I wanted to understand how to build a minimal portable
service.

A practical example could be dnsmasq. It is provided as a package and
depends (apt show dnsmasq) on netbase, dnsmasq-base, init-system-helpers
(>= 1.18~), lsb-base (>= 3.0-6). These packages may further depend on
something else.
It also brings in a set of files into the system:

root@srv ~# dpkg-query -L dnsmasq
/.
/etc
/etc/default
/etc/default/dnsmasq
/etc/dnsmasq.conf
/etc/dnsmasq.d
/etc/dnsmasq.d/README
/etc/init.d
/etc/init.d/dnsmasq
/etc/insserv.conf.d
/etc/insserv.conf.d/dnsmasq
/etc/resolvconf
/etc/resolvconf/update.d
/etc/resolvconf/update.d/dnsmasq
/lib
/lib/systemd
/lib/systemd/system
/lib/systemd/system/dnsmasq.service
/usr
/usr/lib
/usr/lib/resolvconf
/usr/lib/resolvconf/dpkg-event.d
/usr/lib/resolvconf/dpkg-event.d/dnsmasq
/usr/share
/usr/share/dnsmasq
/usr/share/dnsmasq/installed-marker
/usr/share/doc
/usr/share/doc/dnsmasq

Some of these are existing directories, some new ones and then  there are
files (new, but possibly existing as well).

*How should I create a filesystem which has only the files required by the
packages (= the ones it brings in, as well as all the dependencies)?*

I know that I can dbootstrap a new system and install there dnsmasq - and
then hopefully use this as the tree to be attached via portablectl. This
however means that I do not have a "dnsmasq" portable service but rather a
"bionic install with dnsmasq installed on top". The main reason for me to
use portable service is to create small packages which encompass my
service, to be mounted on a more or less base core OS install (say, minimal
bionic). This would allow to get rid of several nspawn containers which do
exactly this (base OS + a package installed on top).

One of the ideas I had (but which seems very wrong) is to have a copy of
the core OS, then in another copy install the required packages, and then
make a diff of the two directories. But this looks horrible and is hardly
maintainable.

I would very much  appreciate any pointers or hints on how to approach this.

Wojtek



___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel


--
SMILE 

20 rue des Jardins
92600 Asnières-sur-Seine


*Jérémy ROSEN*
Architecte technique
Responsable de l'expertise Smile-ECS

email jeremy.ro...@smile.fr 
phone +33141402967
url http://www.smile.eu

Twitter  Facebook 
 LinkedIn 
 Github 




Découvrez l’univers Smile, rendez-vous sur smile.eu 



eco Pour la planète, n'imprimez ce mail que si c'est nécessaire
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel