* Ed W <li...@wildgooses.com> schrieb:

> I'm just building some embedded devices on the side using
> gentoo and my minimal builds are only a few MB? 

How to do you get out all the buildtime stuff (portage, toolchain, etc) ?

> Seems like your complaint is that you have gentoo installs which are 
> full featured with a toolchain and portage, which you are comparing to 
> an installation you built with a different tool that doesn't have a 
> toolchain installed? 

Well, with Briegel, I'm strictly differenciating between the build
host and the target image. Everything's crosscompiled by design.
So I can explicitly define which packages need to be in the final
image (and only those are built). The build host can by any fairly
recent GNU/Linux system (happens to be Gentoo at my site)

> 1) Build your base images without a toolchain or portage and use a 
> minimal package installer to install pre-built binary packages.
> This seems fraught with issues long term though...

How can this be done w/ Gentoo ?
AFAIK it always has a system set containing things like portage,
toolchain, etc, etc.

> 2) Build your base images without a toolchain, but with portage (and 
> perhaps a very minimal python). This gives you full dependency tracking 
> and obviously bind mount/nfs mount the actual portage tree to avoid 
> space used there. This seems workable and minimal?

Would be still too much for my goals: I don't wanna have anything in 
the final system that's not necessary for runtime. Building and even
installation should be driven from outside (actually, the final system
could even have /bin, /usr, etc mounted ro and /var w/o noexec)

> 3) If we are talking virtual machines then who cares if your containers 
> are individually quite large, if the files in them are duplicated across 
> all containers?  Simply use an appropriate de-duplication strategy to 
> coalesce the space and most of the disadvantages disappear?

Well, that might be an option, if the *is* enough duplication
(within an storage enclousure). But storage space is not the only
reason. My personal background is from medical and industrial
appliances, so I have some more rigid rules ;-o

> Personally I think option 3) is quite interesting from the medium number 
> of virtual machines, ie in the 10s to hundreds, ie simply don't worry 
> about it, let the OS do the work.  In the hundreds to thousands plus 
> level I guess you have unique challenges and I would be wrong to try and 
> suggest a solution from the comfort of a laptop without having that 
> responsibility, but I would have thought there was some advantage in a 
> very rigidly deployed base OS generated and updated very precisely?

I'm thinking of several thousands of vm's for dedicated purposes,
which fall under similar requirements as industrial embedded
systems (eg. there will be practically no human operator).

> >For this we need a different approach (strictly separating build
> >and production environments). Binary distros (eg. Debian) might
> >be one option, but they're lacking the configurability and mostly
> >are still too large. So I'm going a different route using my own
> >buildsystem - called Briegel - which originally was designed for
> >embedded/small-device targets.
> >
> >For now I didn't have the spare time to port all the packages
> >required for complete server systems (most of it is making
> >them all cleanly crosscompile'able, as this is a fundamental
> >concept of Briegel). But maybe you'd like to join in and try it :)
> 
> Sounds like an interesting challenge, but I'm unconvinced you can't 
> solve 90% of your problem within the constraints of Gentoo? This saves 
> you a bunch of time that could be invested in the last 10% through more 
> traditional means?

Maybe. But I need 100%. (cant explain all the reasons here right now).
I'm well aware that this all is far out of Gentoo's scope.

> >I'm not sure if Gentoo really is the right distro for that purpose,
> >as it's targeted to very different systems (i.g. Gentoo boxes are
> >expected to be quite unique, beginning with different per-package
> >useflags, even down to cflags, etc). But it might still be a good
> >basis for building specific system images (let's call them stage5 ;-))
> 
> I won't disagree on your "where it's targeted", but just to re-iterate 
> why I think Gentoo works well is that it does have a very workable 
> binary package feature!

Note that Gentoo's binary packages (which are practically snapshots
of what will be copied in by merge stage) are alway bound to specific
domains, eg. the full configuration (make.conf, useflags, etc, etc),
and there may be a lot of interdependencies between installed packages
(and versions), so it takes some care to use it in stable ways.

This might be very fine for you (eg. if you have dozens of really
equal systems), but you'll have to be aware of the limitations.

> My way of working is to use (several) shared binary package repos and 
> the guests largely pull from those shared package directories.  In fact 
> what I do is have a minimal number of shared "/usr/portage/package" 
> directories and I mount an appropriate one to the guest type at boot 
> time.  At the moment my main two options are "32bit" and "64bit" for the 
> package mounts, but I recently introduced a new machine type which is 
> held back to perl 5.8 and that guest gets it's own package mount since 
> it's obviously linking a lot of binaries differently

As long as you have the same make.conf/useflag/etc settings for all
systems (within a guest type), this might work well.

> Now, the icing is that this works extremely well even once you decide to 
> lightly customise machine types.  So for example my binary packages are 
> very high level (eg 32/64bit), my "profiles" would be fairly high level, 
> eg I have www-apache and www-nginx base profiles.  However, a specific 
> virtual machine running say nginx might itself need a specific PHP 
> application installed, and that itself might need some dependencies, 
> which in turn might require a specific set of customisation of use flags 
> and versions.

At this point, Gentoo's binary packages could become tricky.
The big questions eg. are: can it handle multiple binpkgs of the
same (source) package with differing useflags ? How are version-
specific dependencies handled, etc, etc.

> >An setup for 100 equal webserver vm's could look like this:
> >
> >* run a normal Gentoo vm (tailored for the webserver appliance),
> >   where do you do regular updates (emerge, revdep-rebuild, etc, etc)
> >* from time to time take a snapshot, strip off the buildtime-only
> >   stuff (hmm, could turn out to be a bit tricky ;-o)
> >* this stripped snapshot now goes into testing vm's
> >* when approved, the individual production vm's are switched over
> >   to the new image (maybe using some mount magic, unionfs, etc)
> 
> This could work and perhaps for 100 identical Vms you have enough meat 
> to work on something quite customised anyway?

Yes, in this case, all the VMs have to have identical build configuration.
Once you need things like differing useflag, that approach wont fit anymore.
 
> >At this point I've got a question for to the other folks here:
> >
> >emerge has an --root option which allows to (un)merge in a separate
> >system image. So it should be possible to unmerge a lot of system
> >packages which are just required for updating/building (even
> >portage itself), but this still will be manual - what about
> >dependency handling ?
> 
> This is correct.  In fact this is how you build a stage 1,2,3 etc and 
> how catalyst works!
> 
> The information is a bit spread out over several out of date wiki 
> articles, but perhaps start with:
>     http://en.gentoo-wiki.com/wiki/Tiny_Gentoo
> 
> Roughly speaking you could "freshen" your current installation with 
> (from memory):
>     ROOT="/tmp/new_build" emerge -av world

Is this just the installation root or also the sysroot for the
toolchain ?

<snip>

> Further, if you wish you can make those VMs have a reduced or missing 
> toolchain etc. 

hmm, do I have to unmerge certain packages explicitly or can I
tell it to only install those packages required for runtime in
the first place ?

> >Is there some way to drop at least parts of the standard system set,
> >so eg. portage, python, gcc, etc, etc get unmerged by --depclean
> >if nobody else (in world set) doesn't explicitly require them ?
> 
> You are almost thinking about it all wrong.  ("There is no spoon...")
> 
> This is gentoo, so at this more advanced level, stop thinking about 
> "standard system set" and instead free your mind to start with 
> "nothing".

With "standard system set" I mean those packages which are in @system
on a particular arch. In my vision, @system would be simply empty,
instead @world tells which packages you want and the dependencies
are pulled in automatically.

No idea if that's valid anymore, but some (long?) time ago, there
was the rule that individual packages weren't allowed to explicitly
DEPEND= on system packages. Obviously this wouldn't work with an
empty system set (you'd have to find out the deps and add them
to world set manually).

> Largely the portage dependency tracker will help you pull in the minimal 
> needed dependencies, but beware that system packages arent generally 
> explicitly tracked so you may stumble across some deps when you are 
> going really basic and omiting standard system packages (just use common 
> sense: it should be fairly obvious if an application requires a compiler 
> and you didn't install one then you have a conflict of interest...)

Well, that's where it gets complicated: if some packages depend on some
library libfoo in @system (which I would empty in this case), portage
lacks information to compute the full dependency tree, so I'd have to
find them out and add them to @world manually. Exactly what I do not want.


OTOH, Briegel doenst have that issue, as all dependencies are stated
explicitly in each package (well, with one exception: libc).
(of course, Briegel yet lacks a large package repository as I'm
still working mostly alone on it)


cu
-- 
----------------------------------------------------------------------
 Enrico Weigelt, metux IT service -- http://www.metux.de/

 phone:  +49 36207 519931  email: weig...@metux.de
 mobile: +49 151 27565287  icq:   210169427         skype: nekrad666
----------------------------------------------------------------------
 Embedded-Linux / Portierung / Opensource-QM / Verteilte Systeme
----------------------------------------------------------------------

Reply via email to