On 26/02/2011 15:57, Enrico Weigelt wrote:
* Ed W<li...@wildgooses.com> schrieb:
I'm just building some embedded devices on the side using
gentoo and my minimal builds are only a few MB?
How to do you get out all the buildtime stuff (portage, toolchain, etc) ?
Seems like your complaint is that you have gentoo installs which are
full featured with a toolchain and portage, which you are comparing to
an installation you built with a different tool that doesn't have a
toolchain installed?
Well, with Briegel, I'm strictly differenciating between the build
host and the target image. Everything's crosscompiled by design.
So I can explicitly define which packages need to be in the final
image (and only those are built). The build host can by any fairly
recent GNU/Linux system (happens to be Gentoo at my site)
I think you are missing the point that the gentoo build system is quite
happily building you a destination build with as little as one package
in it? Whether that destination boots is a function of careful package
choice, but the point is you are definitely not constrained from
building a highly minimal target if you wish?
Just from the top of my head I should imagine this would likely boot:
mkdir -p /newroot/{proc,dev,sys}
cp -a /dev/{null,zero} /newroot/dev/
emerge —root /newroot glibc shadow baselayout openrc util-linux udev
If you want to go more minimal then I have a uclibc chroot that I use as
a build environment, the first line of my build script for my minimal
embedded device is simply:
emerge --root /embedded baselayout uclibc busybox
I haven't checked, but give or take creating /proc, /sys and some
minimal devices in /dev, the above should boot fine.
If you were booting into a "container" based virtualisation then you
don't need to manage /dev and mounting devices and so you can get even
more minimal than the top example I posted (ie I don't think you need
udev, etc)
Obviously such minimal builds will have minimal package management, but
this may not be a problem if you have a super fixed configuration?
Simply construct your environment such that the data is separate from
the OS and also make sure any OS customisation is scriptable and now you
would have a way to occasionally build a new image which you rolled out
to all virtual machines
1) Build your base images without a toolchain or portage and use a
minimal package installer to install pre-built binary packages.
This seems fraught with issues long term though...
How can this be done w/ Gentoo ?
AFAIK it always has a system set containing things like portage,
toolchain, etc, etc.
"System" is just a statement of some defaults. At the level you are
working at you can largely ignore it.
Build what you like into the target build....
Would be still too much for my goals: I don't wanna have anything in
the final system that's not necessary for runtime. Building and even
installation should be driven from outside (actually, the final system
could even have /bin, /usr, etc mounted ro and /var w/o noexec)
OK, so sounds like a simple build script like the one above would suit you?
For more advanced use consider learning to use catalyst or Funtoos build
system? These are full on scriptable ways to repeatably build non
trivial target builds. (remember they can build as little as single
package targets or more complicated builds)
Note that Gentoo's binary packages (which are practically snapshots
of what will be copied in by merge stage) are alway bound to specific
domains, eg. the full configuration (make.conf, useflags, etc, etc),
and there may be a lot of interdependencies between installed packages
(and versions), so it takes some care to use it in stable ways.
This might be very fine for you (eg. if you have dozens of really
equal systems), but you'll have to be aware of the limitations.
Well, your requirements suggest that you don't need a package manager
and you will instead build a new read-only OS build every so often and
deploy that. However, in the interests of exploring the idea further:
- By definition binary packages only work where you have equal systems?
This is true of gentoo AND rpm/deb.
- If you need binary packages for non equal systems then you need some
kind of namespacing method to separate the binaries. Gentoo has no
automated way to do this, but it's reasonably workable to have binary
package repos and simply tell your guest machine to use the appropriate
binary repo
- Gentoo's binaries are slightly cleverer than some notice in that they
have some awareness of when they don't match the target and the operator
has the option to either ignore that and continue the install, OR to
ignore the binary package and generate it's own binaries using the local
compiler, etc.
So, I don't really see the problem, in fact Gentoo binaries can be seen
as slightly more flexible than RPMs to some extent if you allow them to
be "optional".
As long as you have the same make.conf/useflag/etc settings for all
systems (within a guest type), this might work well.
But that's true of Redhat/Debian/Gentoo?
RPMs don't allow you to have a single binary repo where some of the
packages are linked against perl 5.8 and some against 5.12? I guess it
would be possible to enhance the packaging format to cater for that, but
largely the whole premise of binary packages is that you lock down your
target options and build all the systems identically? There is no easy
way around this? (In fact it's what you want)
At this point, Gentoo's binary packages could become tricky.
The big questions eg. are: can it handle multiple binpkgs of the
same (source) package with differing useflags ? How are version-
specific dependencies handled, etc, etc.
As it stands, no it can't. However, neither can RPM/DEB...
Remember the premise of binaries is that they are all the same... Pick a
system configuration and build all your binaries with that
configuration. If you need to support multiple configurations then setup
multiple binary repositories
articles, but perhaps start with:
http://en.gentoo-wiki.com/wiki/Tiny_Gentoo
Roughly speaking you could "freshen" your current installation with
(from memory):
ROOT="/tmp/new_build" emerge -av world
Is this just the installation root or also the sysroot for the
toolchain ?
I think you need to read through the tiny_gentoo type wiki pages. The
general build process is:
1) Start with a working machine
2) Build a build environment
3) Jump into the build environment and repeat step 2) to build a target
environment
4) Jump into the target environment
Now in the absence of cross compiling you can often skip step 2) and
build the target environment directly, but it's normally useful to have
that build environment so that you can customise things a bit.
Also this will help explain some of the catalyst recipes and why you see
everything built twice, once to get a build environment, then again to
build the target build
Further, if you wish you can make those VMs have a reduced or missing
toolchain etc.
hmm, do I have to unmerge certain packages explicitly or can I
tell it to only install those packages required for runtime in
the first place ?
Well your really simplest "build" would be to take a stage3/4 and simply
"unmerge" anything you don't need... bit crude though
However, from the email above you can see that it's more desirable to
instead "build up", starting with an empty build and from there add the
packages you actually want.
You are almost thinking about it all wrong. ("There is no spoon...")
This is gentoo, so at this more advanced level, stop thinking about
"standard system set" and instead free your mind to start with
"nothing".
With "standard system set" I mean those packages which are in @system
on a particular arch. In my vision, @system would be simply empty,
instead @world tells which packages you want and the dependencies
are pulled in automatically.
Seriously, "there is no spoon"...
If you want to make your own build script then there isn't even an
@system...
No idea if that's valid anymore, but some (long?) time ago, there
was the rule that individual packages weren't allowed to explicitly
DEPEND= on system packages. Obviously this wouldn't work with an
empty system set (you'd have to find out the deps and add them
to world set manually).
I believe this is correct. However, there aren't many packages in system
and so whilst it's kind of buyer beware once you start chopping stuff
out of system, the point is only if you don't install gzip or gcc then
you need to beware stuff which depends on either.
I think in practice manually managing those dependencies is going to be
very straightforward, ie you are more going to fix the applications to
not require the dependencies. After all if you don't want gcc, then you
really don't want gcc, simple really... eg, if some simple application
uses make to update it's config files then you will likely look at some
workaround rather than installing make...
Well, that's where it gets complicated: if some packages depend on some
library libfoo in @system (which I would empty in this case), portage
lacks information to compute the full dependency tree, so I'd have to
find them out and add them to @world manually. Exactly what I do not want.
Oh come on. You are working at the level of building your OS image from
scratch and you don't know if any of your applications require (at
runtime) gcc, bison, m4, wget, man, os-headers, portage or ssh?
It seems likely that you can quickly determine if you can live without
most of the above (at runtime)...?!!
OTOH, Briegel doenst have that issue, as all dependencies are stated
explicitly in each package (well, with one exception: libc).
So, you can probably already use that information to figure out your
minimal set of system packages that you likely need (for runtime).
Just checking that Briegel tracks both runtime and build time
dependencies separately? eg you might need gzip to build something, but
not to run it?
So far I think you could easily convert your briegel recipes to a simple
ebuild script and run them side by side? It's actually such an easy
thing to do, to build a new stage2/3/4 that it's a shame folks don't do
it more!
Good luck
Ed W
