Ian P. Christian wrote: > I would like to move to an automated way of keeping these servers up to > date. > > Firstly, I would like to use binary packages. I initially thought about > having a repository pre architecture, and standardising on cflags and use > flags - however this isn't feasable if you need to alter the use flags on > just one server - installing from the binary package will result in the > wrong package being installed.
I use a central server to build and host all binary packages that are used to install the various server types. In a nutshell, this is how it works. One hoss server contains chroot build hosts for every individual server type in my farm (web servers, dns servers, mail exchangers, database servers, etc...) This server does all of the compiling of binary packages for each arch it can support (pentium4 or below). A "build host" logically consists of a specific portage snapshot and a specific set of config files (make.conf, /etc/portage/*, etc..). Each individual build host contains a flag file - /etc/conf.d/buildhost, which is used to store the snapshot version and as a method to change the behavior during the emerging of certain packages (kernel, config files), etc.. Each portage snapshot (local and overlays) is shared via rsync from the central server. GENTOO_MIRRORS on the target servers syncs from there. Each build host /usr/portage/packages directory is symlinked into the apache document root on the central server. PORTAGE_BINHOST on the target servers points there. An example directory structure looks like this: /buildhosts/stage3-p3 -> chroot environment to build a pentium3 stage3 /buildhosts/stage3-p4 -> chroot environment to build a pentium4 stage3 /buildhosts/web-servers -> chroot environment to build web servers /buildhosts/dns-servers -> chroot environment to build dns servers /buildhosts/mysql-servers -> chroot environment to build mysql servers Each server type uses a stage3 generated from the corresponding stage3 build host. The portage directory within each buildhost is bind mounted to the corresponding portage snapshot in /buildhosts/shared/snapshots/YYYYMMDD/portage (see below). Each chroot build host has /dev and /proc bind mounted. /buildhosts/shared/snapshots/20051112/portage -> portage snapshot /buildhosts/shared/snapshots/20060201/portage -> portage snapshot These are the actual portage snapshots, extracted, from the specified date. These directories will be bind mounted within the chroot build hosts that use them. /buildhosts/shared/configs/stage3-p3/20060201 /buildhosts/shared/configs/stage3-p4/20060201 /buildhosts/shared/configs/web-servers/20060201 /buildhosts/shared/configs/dns-servers/20051112 Config files for each build host. I use a script that sources the buildhost etc/conf.d/buildhost flag file, determines the value of SNAPSHOT_DATE, and if CONFIGS_COPIED is not defined it will copy everything over to the chroot environment from the corresponding config directory. This is how each build host gets its needed make.conf, resolv.conf, /etc/portage/*, etc... /buildhosts/shared/distfiles This is bind mounted to each chroot build host /usr/portage/distfiles so that a common source of distfiles can be used. /buildhosts/shared/packages/stage3-p3/20060201 /buildhosts/shared/packages/web-servers/20060201 /buildhosts/shared/packages/dns-servers/20051112 Each of these is bind mounted to the corresponding chroot build host /usr/portage/packages directory. When the binary packages are built they end up here. These are the directories that will be symlinked into the apache document tree and obtained on the target servers via the PORTAGE_BINHOST variable... /buildhosts/shared/overlays/web-servers /buildhosts/shared/overlays/dns-servers Where I keep portage overlays that are bind mounted to the corresponding chroot build host. I use these overlay trees to create kernel ebuilds, config file ebuilds, web packages, dns zone file packages, etc... I keep track of these with the etc/conf.d/buildhost OVERLAYS variable. /buildhosts/shared/stages Where I keep stage1's and custom stage tarballs that are extracted if STAGE is set in etc/conf.d/buildhost. So basically, all emerges are done within each chroot build host. The resulting portage snapshot and binary packages wind up being available via rsync and apache to the servers to be installed. Each server type (web-servers, dns-servers, etc..) gets its own meta package via the overlays that determines what apps are compiled. Each individual server gets its own unique config file package within the overlays also. These config packages work much like the baselayout packages, the actual files are installed to /usr/share/server-configs/web-servers/20060201, in pkg_postinst() if there is no /etc/conf.d/buildhost, a cp -r is done that copies the config files over into the / filesystem.... Got all that :) -- [email protected] mailing list
