Jeroen T. Vermeulen has proposed merging lp:~jtv/orchestra/odev-without-apt-get-update into lp:orchestra.
Requested reviews: orchestra (orchestra) For more details, see: https://code.launchpad.net/~jtv/orchestra/odev-without-apt-get-update/+merge/96514 Installing the required packages was agonizingly slow and error-prone for Julian (who is no longer very close to the data centre). This should speed it up. The trick I use to check of missing packages works on Lucid and Precise, which are the only systems I tried it on. -- https://code.launchpad.net/~jtv/orchestra/odev-without-apt-get-update/+merge/96514 Your team orchestra is requested to review the proposed merge of lp:~jtv/orchestra/odev-without-apt-get-update into lp:orchestra.
=== added file 'HOWTO' --- HOWTO 1970-01-01 00:00:00 +0000 +++ HOWTO 2012-03-08 06:52:22 +0000 @@ -0,0 +1,41 @@ +#! /bin/bash -e +# +# This file documents how to get odev running on your system. But it's also +# a script; you may find that you can just run it and get a working setup. + +## System-level setup. This needs to be done only once. +./bin/system-setup + +## Build a zimmer image in this branch. +pushd zimmer-build +./build zimmer-disk0.img --import-keys=auto +popd + +## Get zimmer and cobbler running. +./bin/start-odev + [email protected] +cat <<EOF +While we're waiting for the server to come up, let's set up ssh login to +the cobbler server at $cobblerlogin. + +Please enter your Launchpad login name to import your ssh keys from Launchpad, +or an asterisk ("*") to import your local public ssh keys. Enter nothing to +skip this step. + +(If the server prompts you for a password, the default is "passw0rd") +EOF +read keyowner +./bin/authorize-ssh $cobblerlogin $keyowner + +## populate the nodes into the cobbler server +./setup.py cobbler-setup + +## Listen for libvirt requests from the Cobbler server. +VIRSH_LISTENER_DEBUG=1 ./bin/virsh-listener & + + +## at this point you may want to modify zimmer to provide a proxy +## other than itself to things installing from it (LP: #914202). +## ssh to zimmer, and then edit : +## /var/lib/cobbler/snippets/orchestra_proxy === added file 'HOWTO.juju' --- HOWTO.juju 1970-01-01 00:00:00 +0000 +++ HOWTO.juju 2012-03-08 06:52:22 +0000 @@ -0,0 +1,68 @@ +# http://askubuntu.com/questions/65359/how-do-i-configure-juju-for-local-usage + +pkgs="libzookeeper-java zookeeper juju bzr" + +JUJU_D=$HOME/juju +JUJU_ORIGIN="lp:juju" +JUJU_SERIES="precise" + +REPO="$HOME/charms" +CHARMS_D="$CHARMS_D/$JUJU_SERIES" + +ZIMMER_IP=192.168.123.2 + +id_rsa="$HOME/.ssh/id_rsa" +[ -f "$id_rsa" ] || ssh-keygen -t rsa -N '' -f "$id_rsa" +read x y z < "$id_rsa" +grep -q "$y" ~/.ssh/authorized_keys || + cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys + +sudo apt-get --assume-yes install $pkgs </dev/null + +mkdir -p "${JUJU_D%/*}" +#( cd ${JUJU_D%/*} && bzr branch lp:juju ) +( cd ${JUJU_D%/*} && bzr branch $JUJU_ORIGIN juju ) + +mkdir -p "$CHARMS_D" +( cd "$CHARMS_D" && bzr branch lp:charm/mysql && bzr branch lp:charm/wordpress ) + +ENAME="odev" + +mkdir ~/.juju/ +cat > ~/.juju/environments.yaml <<EOF +environments: + $ENAME: + type: orchestra + juju-origin: $JUJU_ORIGIN + orchestra-server: $ZIMMER_IP + orchestra-user: cobbler + orchestra-pass: xcobbler + acquired-mgmt-class: orchestra-juju-acquired + available-mgmt-class: orchestra-juju-available + admin-secret: SEEKRIT + storage-url: http://$ZIMMER_IP/webdav + authorized-keys: $(cat ~/.ssh/id_rsa.pub) + data-dir: $HOME/juju-data/$ENAME + default-series: $JUJU_SERIES +EOF + +export PATH="$JUJU_D/bin:$HOME/bin:/usr/sbin:/usr/bin:/sbin:/bin" PYTHONPATH=$JUJU_D + +# now start your juju bootstrap node. this will take some time, as we're +# doing a full install into the VM. +juju bootstrap --environment $ENAME + +# now create the mysql and wordpress units +# this takes quite a while as full VM install of each +juju deploy --environment $ENAME --repository $REPO local:mysql +juju deploy --environment $ENAME --repository $REPO local:wordpress + +# now link the two +juju add-relation --environment $ENAME wordpress mysql + +# juju status: +# FIXME: resolution will try to use dns and will not work for nodes +# workaround: can add 192.168.123.1 to /etc/resolv.conf 'server' line +# FIXME: juju status hangs "connecting to environment" during bootstrap +# node installation. The post should call home and indicate done. so +# juju could/should know that its still installing. === added file 'README.txt' --- README.txt 1970-01-01 00:00:00 +0000 +++ README.txt 2012-03-08 06:52:22 +0000 @@ -0,0 +1,5 @@ +This allows you to create a VM cobbler provisioning environment in +a single system. That allows development working with cobbler or the +API without need for lots of hardware. + + === added file 'TODO' --- TODO 1970-01-01 00:00:00 +0000 +++ TODO 2012-03-08 06:52:22 +0000 @@ -0,0 +1,14 @@ +- prefix names with 'odev' (or some prefix) +- settings.cfg: add 'cobbler' section for auth +- improve the Domain objects +- document + - vinagre $(virsh vncdisplay node01) + - ssh -L 5901:localhost:5901 -L 8000:192.168.123.2:80 + - start ssh connection to remote system with a bunch of ports + forwarded for vnc connections and http to the zimmer box + ssh -C home-jimbo \ + $(t=98; for((i=0;i<5;i++)); do p=$(printf "%02d" "$i"); echo -L $t$p:localhost:59$p; done ; echo -L${t}80:192.168.123.2:80) +- tell orchestra to point to a different proxy server +- document or fix annoying ssh key entries (juju prompt for add and change) +- get serial consoles to log file for domains +- support i386 (for i386 installs of ubuntu) === added file 'api-list.py' --- api-list.py 1970-01-01 00:00:00 +0000 +++ api-list.py 2012-03-08 06:52:22 +0000 @@ -0,0 +1,36 @@ +#!/usr/bin/python + +import xmlrpclib +import sys + +host = "192.168.123.2" +user = "cobbler" +password = "cobbler" +if len(sys.argv) >= 2: + host = sys.argv[1] +if len(sys.argv) >= 3: + user = sys.argv[2] +if len(sys.argv) >= 4: + password = sys.argv[3] + +if not host.startswith('http://'): + host = "http://%s/cobbler_api"; % host + +server = xmlrpclib.Server(host) +token = server.login(user, password) + +distros = server.get_distros() +print "::::::::::: distros :::::::::::" +for d in server.get_distros(): + print("%s: breed=%s, os_version=%s, mgmt_classes=%s" % + (d['name'], d['breed'], d['os_version'], d['mgmt_classes'])) + +profiles = server.get_profiles() +print "\n::::::::::: profiles :::::::::::" +for d in server.get_profiles(): + print("%s: distro=%s parent=%s kickstart=%s" % + (d['name'], d['distro'], d['parent'], d['kickstart'])) + +print "\n::::::::::: servers :::::::::::" +for s in server.get_systems(): + print s['interfaces'] === added directory 'bin' === renamed directory 'bin' => 'bin.moved' === added file 'bin/authorize-ssh' --- bin/authorize-ssh 1970-01-01 00:00:00 +0000 +++ bin/authorize-ssh 2012-03-08 06:52:22 +0000 @@ -0,0 +1,39 @@ +#! /bin/bash -e +# +# Wait for the virtual cobbler instance's ssh server to start up, and set up +# passwordless login if desired. +# +# Usage: +# authorize-ssh <cobbler-ssh-login> <key-owner> +# +# Where: +# * cobbler-ssh-login is an ssh user/hostname, e.g. [email protected] +# * key-owner is a Launchpad login name, or * to use local keys, or nothing. +# +# If a Launchpad login name is given, import the associated ssh keys into the +# cobbler instance. If key-owner is an asterisk, import the local public ssh +# keys from ~/.ssh/id_*.pub + +cobblerlogin=$1 +keyowner=$2 + +if test -z "$keyowner" +then + echo "Not setting up ssh keys." + echo "I'll still test a login to Cobbler though." + inputfiles=/dev/null + remotecmd="uptime" +elif test "$keyowner" = "*" +then + inputfiles=`ls ~/.ssh/id_*.pub` + echo "Copying public key(s): $inputfiles" + remotecmd="tee .ssh/authorized_keys" +else + inputfiles=/dev/null + remotecmd="ssh-import-id $keyowner" +fi + +while ! cat $inputfiles | ssh $cobblerlogin -o StrictHostKeyChecking=no $remotecmd +do + sleep 5 +done === added file 'bin/start-odev' --- bin/start-odev 1970-01-01 00:00:00 +0000 +++ bin/start-odev 2012-03-08 06:52:22 +0000 @@ -0,0 +1,26 @@ +#! /bin/bash -e +# +# Get zimmer and cobbler running, assuming that zimmer has already been set up. + +## create libvirt xml files for nodes, zimmer, network +./setup.py libvirt-setup + +## start odev-net network +virsh -c qemu:///system net-start odev-net + +## create zimmer disk image qcow backing against pristine version +qemu-img create -f qcow2 -b zimmer-build/zimmer-disk0.img zimmer-disk0.img + +## start zimmer instance / orchestra server +virsh -c qemu:///system start zimmer + +cat <<EOF +Starting orchestra server. +You can now ssh [email protected] (password: passw0rd). +If you do that, you may run 'ssh-import-id' to import your ssh key. + +Access the cobbler UI on http://192.168.123.2/cobbler_web +and log in with 'cobbler:xcobbler'. +EOF + + === added file 'bin/system-setup' --- bin/system-setup 1970-01-01 00:00:00 +0000 +++ bin/system-setup 2012-03-08 06:52:22 +0000 @@ -0,0 +1,50 @@ +#! /bin/bash -e +# +# System-wide setup for odev. This requires sudo. + +## install some dependencies +pkgs="" +pkgs="$pkgs genisoimage coreutils" # for cloud-init's 'make-iso' +pkgs="$pkgs python-libvirt libvirt-bin" # for libvirt interaction +pkgs="$pkgs socat" # for libvirt-> cobbler +pkgs="$pkgs python-cheetah" # for setup.py +pkgs="$pkgs qemu-utils qemu-kvm" # needed generally + + +need_packages_installed() { + # Do any of the packages in $pkgs need installing? + # (Show all installed versions for $pkgs, one per line, and grep for + # blank lines in the output.) + dpkg-query -f '${Version}\n' --show $pkgs | grep -q '^$' +} + + +if need_packages_installed +then + sudo apt-get install -y $pkgs </dev/null +fi + +new_groups="" +for group in libvirtd kvm +do + groups $USER | grep -q $group && continue + sudo adduser $USER $group + new_groups="${new_groups:+${new_groups} }${group}" +done + +if [ -n "$new_groups" ] +then + cat <<EOF +Done. + +The script just added you to the system group[s] $new_groups + +If you were not previously in these groups, you will need to log out and +log back in again to make the changes take effect. +EOF + + # The user may need to log out at this point. + echo "Abort with ctrl-C if you need to log out first." + echo "Otherwise, press <enter>." + read +fi === added file 'bin/virsh-listener' --- bin/virsh-listener 1970-01-01 00:00:00 +0000 +++ bin/virsh-listener 2012-03-08 06:52:22 +0000 @@ -0,0 +1,27 @@ +#!/bin/bash -e + +## * libvirt from the cobbler system: +## after 'cobbler-setup' above is done, the cobbler system will know about +## all the nodes and it will believe it can control them via the 'virsh' +## power module. It is configured +## to talk to qemu+tcp://192.168.123.1:65001/system . In order to allow +## that to be valid we have to make libvirt listen on that port/interface. +## This can be done moderately securely with 'socat'. Below, we tell socat +## to forward tcp connections on 192.168.123.1:65001 to the libvirt unix +## socket . It restricts connections to zimmer's IP address. + +sock="/var/run/libvirt/libvirt-sock" + +[ "${VIRSH_LISTENER_DEBUG:-0}" != "0" ] && cat <<EOF +Starting virsh listener. + +You can verify this is working by powering a sytem on from the web-ui or +the following on the cobbler server: + +zimmmer$ virsh -c qemu+tcp://192.168.123.1:65001/system +EOF + +echo "Listening for libvirt requests on $sock." +exec socat -d -d \ + TCP4-LISTEN:65001,bind=192.168.123.1,range=192.168.123.2/32,fork \ + UNIX-CONNECT:$sock === added directory 'cobbler-server' === added file 'cobbler-server/README.txt' --- cobbler-server/README.txt 1970-01-01 00:00:00 +0000 +++ cobbler-server/README.txt 2012-03-08 06:52:22 +0000 @@ -0,0 +1,8 @@ +Build a cobbler image: + $ time ./build-image -vv --preseed preseed.cfg oneiric amd64 8G + +to use a proxy, edit preseed.cfg, add something like: +# Specifying the Mirror. +d-i mirror/http/proxy string http://10.155.1.249:8000/ + +With correct IP and port. This will work well with squid-deb-proxy === added file 'cobbler-server/build-image' --- cobbler-server/build-image 1970-01-01 00:00:00 +0000 +++ cobbler-server/build-image 2012-03-08 06:52:22 +0000 @@ -0,0 +1,170 @@ +#!/bin/bash + +VERBOSITY=0 +TEMP_D="" +DEF_PRESEED="http://bit.ly/uquick"; +DEF_SIZE=4 +DEF_MIRROR="http://archive.ubuntu.com/ubuntu"; +DEF_ARCH=$(uname -m) +MY_DIR=${0%/*} +PATH=$MYDIR:$PATH + +[ "$DEF_ARCH" = "x86_64" ] && DEF_ARCH=amd64 + +error() { echo "$@" 1>&2; } +errorp() { printf "$@" 1>&2; } +fail() { [ $# -eq 0 ] || error "$@"; exit 1; } +failp() { [ $# -eq 0 ] || errorp "$@"; exit 1; } + +Usage() { + cat <<EOF +Usage: ${0##*/} [ options ] release [ arch [ size ] ] + + Do an install of Ubuntu for release. + arch : the arch to use (amd64 i386). Default: ${DEF_ARCH} + size : size of the image (in GigaBytes). Default: ${DEF_SIZE} + + options: + -o | --output IMAGE_FILE write the image to IMAGE_FILE + default: <release>-<arch>.img + -s | --preseed PRESEED use the preseed at preseed + default: ${DEF_PRESEED} + -m | --mirror MIRROR mirror to download iso from MIRROR + default: ${DEF_MIRROR} + --iso ISO use ISO file rather than downloading +EOF +} + +bad_Usage() { Usage 1>&2; [ $# -eq 0 ] || error "$@"; exit 1; } +cleanup() { + [ -z "${TEMP_D}" -o ! -d "${TEMP_D}" ] || rm -Rf "${TEMP_D}" +} + +dl() { + local url="$1" out="${2}" opts="" + [ "$url" = "$out" ] && return + [ $VERBOSITY -lt 1 ] && opts="-q" + case "$url" in + http://*|ftp://*) wget $opts "$url" -O "${out}";; + *) cp -a "${url}" "${out}";; + esac +} + +debug() { + local level=${1}; shift; + [ "${level}" -ge "${VERBOSITY}" ] && return + error "${@}" +} +update_latecommand() { + local infile=$1 program=$2 replace=$3 base64="" + base64=$(base64 --wrap 0 "${program}") + # the '&' have to be escaped through sed + str='f=$1; shift ; echo $0 | base64 --decode > "$f" \&\& chmod u+x "$f" \&\& "$f" "$@"' + sed -e "s,$replace,in-target sh -c '$str' ${base64} /root/late-command," "${infile}" +} + +short_opts="hm:o:p:v" +long_opts="help,iso:,mirror:,output:,preseed:,verbose" +getopt_out=$(getopt --name "${0##*/}" \ + --options "${short_opts}" --long "${long_opts}" -- "$@") && + eval set -- "${getopt_out}" || + bad_Usage + +release="${DEF_RELEASE}" +preseed="${DEF_PRESEED}" +output="" +mirror="${DEF_MIRROR}" +arch="${DEF_ARCH}" +iso="" + +while [ $# -ne 0 ]; do + cur=${1}; next=${2}; + case "$cur" in + -h|--help) Usage ; exit 0;; + -i|--iso) iso=${2}; shift;; + -m|--mirror) mirror=${2}; shift;; + -o|--output) output=${2}; shift;; + -p|--preseed) preseed=${2}; shift;; + -v|--verbose) VERBOSITY=$((${VERBOSITY}+1));; + --) shift; break;; + esac + shift; +done + +[ $# -ne 0 ] || bad_Usage "must provide arguments" +[ $# -lt 1 -o $# -gt 3 ] && bad_Usage "must provide 1,2, or 3 args" +release=$1 +arch=${2:-${DEF_ARCH}} +size=${3:-${DEF_SIZE}} +size=${size%G} + +[ -n "$output" ] || output="${release}-${arch}.img" + +TEMP_D=$(mktemp -d "${TMPDIR:-/tmp}/${0##*/}.XXXXXX") || + fail "failed to make tempdir" +trap cleanup EXIT + +pre="${release}-${arch}" +if [ -z "$iso" ]; then + iso="${release}-${arch}-mini.iso" + if [ -f "$iso" ]; then + debug 1 "using existing iso ${iso}" + else + url="${mirror}/dists/$release/main/installer-$arch/current/images/netboot/mini.iso" + debug 1 "downloading ${url}" + dl "${url}" "${iso}" || fail "failed to download $iso from $url" + fi +fi + +dl "$preseed" $pre-preseed.cfg || + fail "failed to download preseed: ${preseed}" + +update_latecommand $pre-preseed.cfg \ + late_command.sh __LATE_COMMAND__ > "${TEMP_D}/preseed.cfg" || fail "failed to update_latecommand" + +debug 1 "extracting kernel and ramdisk" +for f in /linux /initrd.gz ; do + targ="${pre}-${f##*/}.dist" + if [ ! "$targ" -nt "$iso" ]; then + isoinfo -RJ -x "$f" -i "$iso" > "$targ" || + fail "failed to extract $f from $iso" + else + debug 1 "skipping $targ, newer than $iso" + fi +done + +kernel="${pre}-linux.dist" +initrd="${pre}-initrd.gz" + +if [ "$pre-preseed.cfg" -nt "$initrd" ]; then + debug 1 "repacking initramfs" + zcat "$initrd.dist" > "${TEMP_D}/initrd" && + ( cd "${TEMP_D}" && echo "./preseed.cfg" | + cpio -o --format=newc --append -F initrd && + gzip -9 initrd -c ) > "${initrd}" || + fail "failed to repack initrd" +else + debug 1 "not repacking $initrd, it is newer than $pre-preseed.cfg" +fi + +debug 1 "creating a ${size}G disk in ${output}" +rm -f "$output" +qemu-img create -f qcow2 "${output}" "${size}G" || + fail "failed to create image" + +time ${KVM:-kvm} -kernel "$kernel" -initrd "$initrd" \ + -append "priority=critical locale=en_US" \ + -drive "file=${output},if=virtio,cache=unsafe" \ + -cdrom "${iso}" \ + -m 1024 -boot d -no-reboot \ + -net nic,model=virtio -net user || + fail "kvm failed" + +debug 1 "restarting for first boot" +time ${KVM:-kvm} \ + -drive "file=${output},if=virtio,cache=unsafe" \ + -m 512 -boot c -no-reboot \ + -net nic,model=virtio -net user || + fail "failed to reboot" + +# vi: ts=4 noexpandtab === added file 'cobbler-server/late_command.sh' --- cobbler-server/late_command.sh 1970-01-01 00:00:00 +0000 +++ cobbler-server/late_command.sh 2012-03-08 06:52:22 +0000 @@ -0,0 +1,170 @@ +#!/bin/bash + +{ +fb_d="/root/first-boot.d" +mkdir -p "$fb_d" +sed -i '/^exit 0/d' /etc/rc.local +cat >> /etc/rc.local <<EOF +## first boot finish stuff +if [ ! -f "$fb_d.done" ]; then + run-parts "$fb_d" 2>&1 | tee "${fb_d}.log" + touch "$fb_d.done" +fi +EOF + +cat >"$fb_d/10-addl-pkgs" <<"EOF" +#!/bin/sh +pkgs="ubuntu-orchestra-provisioning-server cman" +[ -t 1 ] || { + unset DEBIAN_HAS_FRONTEND DEBIAN_FRONTEND DEBCONF_REDIR DEBCONF_OLD_FD_BASE; + export DEBIAN_FRONTEND=noninteractive; +} +if [ "$(lsb_release --codename --short)" = "natty" ]; then + for p in ppa:orchestra/ppa ppa:dotdee/ppa; do apt-add-repository $p; done +fi +apt-get update +apt-get install --assume-yes ${pkgs} +EOF + +cat > "$fb_d/99-halt" <<EOF +#!/bin/sh +touch "$fb_d.done" +/sbin/poweroff +EOF + +cat >"$fb_d/50-setup-cobbler" <<"EOF" +#!/bin/sh + +cp -a /etc/cobbler/settings /etc/cobbler/settings.dist +sed -i 's,^next_server: .*,next_server: cobbler,' /etc/cobbler/settings +sed -i 's,^server: .*,server: cobbler,' /etc/cobbler/settings + +# https://fedorahosted.org/cobbler/wiki/CobblerWebInterface +# htdigest /etc/cobbler/users.digest "Cobbler" cobbler +cat > /etc/cobbler/users.digest <<ENDUSERDIGEST +cobbler:Cobbler:a2d6bae81669d707b72c0bd9806e01f3 +ENDUSERDIGEST + +seed="/var/lib/cobbler/kickstarts/ensemble.preseed" +cat > "$seed" <<"ENDPRESEED" +# Ubuntu Server Quick Install for Orchestra deployed systems +# by Dustin Kirkland <[email protected]> +# * Documentation: http://bit.ly/uquick-doc + +d-i debian-installer/locale string en_US.UTF-8 +d-i debian-installer/splash boolean false +d-i console-setup/ask_detect boolean false +d-i console-setup/layoutcode string us +d-i console-setup/variantcode string +d-i netcfg/get_nameservers string +d-i netcfg/get_ipaddress string +d-i netcfg/get_netmask string 255.255.255.0 +d-i netcfg/get_gateway string +d-i netcfg/confirm_static boolean true +d-i clock-setup/utc boolean true +d-i partman-auto/method string regular +d-i partman-lvm/device_remove_lvm boolean true +d-i partman-lvm/confirm boolean true +d-i partman/confirm_write_new_label boolean true +d-i partman/choose_partition select Finish partitioning and write changes to disk +d-i partman/confirm boolean true +d-i partman/confirm_nooverwrite boolean true +d-i partman/default_filesystem string ext4 +d-i clock-setup/utc boolean true +d-i clock-setup/ntp boolean true +d-i clock-setup/ntp-server string ntp.ubuntu.com +d-i base-installer/kernel/image string linux-server +d-i passwd/root-login boolean false +d-i passwd/make-user boolean true +d-i passwd/user-fullname string ubuntu +d-i passwd/username string ubuntu +d-i passwd/user-password-crypted password $6$.1eHH0iY$ArGzKX2YeQ3G6U.mlOO3A.NaL22Ewgz8Fi4qqz.Ns7EMKjEJRIW2Pm/TikDptZpuu7I92frytmk5YeL.9fRY4. +d-i passwd/user-uid string +d-i user-setup/allow-password-weak boolean false +d-i user-setup/encrypt-home boolean false +d-i passwd/user-default-groups string adm cdrom dialout lpadmin plugdev sambashare +d-i apt-setup/services-select multiselect security +d-i apt-setup/security_host string security.ubuntu.com +d-i apt-setup/security_path string /ubuntu +d-i debian-installer/allow_unauthenticated string false +d-i pkgsel/upgrade select safe-upgrade +d-i pkgsel/language-packs multiselect +d-i pkgsel/update-policy select none +d-i pkgsel/updatedb boolean true +d-i grub-installer/skip boolean false +d-i lilo-installer/skip boolean false +d-i grub-installer/only_debian boolean true +d-i grub-installer/with_other_os boolean true +d-i finish-install/keep-consoles boolean false +d-i finish-install/reboot_in_progress note +d-i cdrom-detect/eject boolean true +d-i debian-installer/exit/halt boolean false +d-i debian-installer/exit/poweroff boolean false +d-i pkgsel/include string ubuntu-orchestra-client $getVar('EXTRA_PACKAGES','') +byobu byobu/launch-by-default boolean true +d-i preseed/late_command string true && \ + $SNIPPET('ensemble_late_command') && \ + $SNIPPET('disable_pxe') && \ + true # add your late_commands here +ENDPRESEED + +mkdir -p /var/lib/cobbler/snippets/ +cat > /var/lib/cobbler/snippets/disable_pxe <<"ENDSNIP" +wget "http://$http_server:$http_port/cblr/svc/op/nopxe/system/$system_name"; -O /dev/null +ENDSNIP + +cat > /var/lib/cobbler/snippets/ensemble_late_command <<"ENDSNIP" +$getVar('ENSEMBLE_LATE_COMMAND', 'true') +ENDSNIP + + +mkdir -p /var/lib/cobbler/isos +cd /var/lib/cobbler/isos +set -- ${DISTS:-natty:i386 natty:amd64 oneiric:i386 oneiric:amd64} +mirror="${MIRROR:-http://archive.ubuntu.com/ubuntu}"; +for t in "$@"; do + rel=${t%:*}; arch=${t#*:} + iso=$rel-$arch-mini.iso + [ -f "$iso" ] && continue + u=$mirror/dists/$rel/main/installer-$arch/current/images/netboot/mini.iso + wget -O "$iso" "$u" +done + +for t in "$@"; do + rel=${t%:*}; arch=${t#*:} + xa=$arch; [ "$arch" = "amd64" ] && xa=x86_64 + mount -o loop $rel-$arch-mini.iso /mnt + cobbler import --name=$rel-$arch --path=/mnt --breed=ubuntu --os-version=$rel --arch=$xa + umount /mnt + name=$rel-$xa + ## cobbler wants to name distro and the default profile as <version>-<xa> (x86_64, not amd64) + ## so, we just let it be. if we renamed, we'd have to do both profile and distribution + ## [ "$xa" != "$arch" ] && + ## cobbler profile rename --name $name --newname $rel-$arch && + ## cobbler distro rename --name $name --newname $rel-$arch && + ## name=$rel-$arch && + ## cobbler profile edit --name $name --distro $name + #fi + cobbler profile edit --name $name --kopts="priority=critical locale=en_US" + cobbler profile add --parent $name --name $name-ensemble --kickstart=$seed +done + +# set up the webdav host +a2enmod dav +a2enmod dav_fs +service apache2 restart +cat > /etc/apache2/conf.d/dav.conf <<ENDWEBDAV +Alias /webdav /var/lib/webdav + +<Directory /var/lib/webdav> +Order allow,deny +allow from all +Dav On +</Directory> +ENDWEBDAV +#EOF + +EOF + +chmod u+x "$fb_d"/* +} 2>&1 | tee /root/late.log === added file 'cobbler-server/preseed.cfg' --- cobbler-server/preseed.cfg 1970-01-01 00:00:00 +0000 +++ cobbler-server/preseed.cfg 2012-03-08 06:52:22 +0000 @@ -0,0 +1,61 @@ +d-i debian-installer/locale string en_US.UTF-8 +d-i debian-installer/splash boolean false +d-i console-setup/ask_detect boolean false +d-i console-setup/layoutcode string us +d-i console-setup/variantcode string +d-i netcfg/get_nameservers string +d-i netcfg/get_ipaddress string +d-i netcfg/get_netmask string 255.255.255.0 +d-i netcfg/get_gateway string +d-i netcfg/confirm_static boolean true +d-i netcfg/get_hostname string cobbler +d-i netcfg/get_hostname seen true +#d-i netcfg/get_domain string unassigned-domain +#d-i netcfg/get_domain seen true +d-i clock-setup/utc boolean true +d-i partman-auto/method string regular +d-i partman-lvm/device_remove_lvm boolean true +d-i partman-lvm/confirm boolean true +d-i partman/confirm_write_new_label boolean true +d-i partman/choose_partition select Finish partitioning and write changes to disk +d-i partman/confirm boolean true +d-i partman/confirm_nooverwrite boolean true +d-i partman/default_filesystem string ext3 +d-i clock-setup/utc boolean true +d-i clock-setup/ntp boolean true +d-i clock-setup/ntp-server string ntp.ubuntu.com +d-i base-installer/kernel/image string linux-server +d-i passwd/root-login boolean false +d-i passwd/make-user boolean true +d-i passwd/user-fullname string ubuntu +d-i passwd/username string ubuntu +d-i passwd/user-password-crypted password $6$.1eHH0iY$ArGzKX2YeQ3G6U.mlOO3A.NaL22Ewgz8Fi4qqz.Ns7EMKjEJRIW2Pm/TikDptZpuu7I92frytmk5YeL.9fRY4. +d-i passwd/user-uid string +d-i user-setup/allow-password-weak boolean false +d-i user-setup/encrypt-home boolean false +d-i passwd/user-default-groups string adm cdrom dialout lpadmin plugdev sambashare +d-i apt-setup/services-select multiselect security +d-i apt-setup/security_host string security.ubuntu.com +d-i apt-setup/security_path string /ubuntu +d-i debian-installer/allow_unauthenticated string false +d-i pkgsel/upgrade select safe-upgrade +d-i pkgsel/language-packs multiselect +d-i pkgsel/update-policy select none +d-i pkgsel/updatedb boolean true +# additional packages to install +d-i pkgsel/include string vim openssh-server python-software-properties libvirt-bin screen +d-i grub-installer/skip boolean false +d-i lilo-installer/skip boolean false +d-i grub-installer/only_debian boolean true +d-i grub-installer/with_other_os boolean true +d-i finish-install/keep-consoles boolean false +d-i finish-install/reboot_in_progress note +d-i cdrom-detect/eject boolean true +d-i debian-installer/exit/halt boolean false +d-i debian-installer/exit/poweroff boolean false +## Specifying the Mirror. +#d-i mirror/http/proxy string http://192.168.1.102:8000/ +## Finish specifying Mirror. + +byobu byobu/launch-by-default boolean true +d-i preseed/late_command string __LATE_COMMAND__ === added file 'cobbler-server/update-latecmd' --- cobbler-server/update-latecmd 1970-01-01 00:00:00 +0000 +++ cobbler-server/update-latecmd 2012-03-08 06:52:22 +0000 @@ -0,0 +1,21 @@ +#!/bin/sh + +# replace 'replace' in file 'infile' with a sh snippit that will execute the +# file/shell-script 'program' as a late-command. +infile="${1}" +program="$2" +replace="${3:-__LATE_COMMAND__}" +gzip=1 + +if [ "${gzip}" != "0" ]; then + base64=$(gzip -c "${program}" | base64 --wrap 0) + pipe_gunzip="| gunzip" +else + base64=$(base64 --wrap 0 "${program}" ) + pipe_gunzip="" +fi +# the '&' have to be escaped through sed +str='f=$1; shift ; echo $0 | base64 --decode '"$pipe_gunzip"' > "$f" \&\& chmod u+x "$f" \&\& "$f" "$@"' + +val="in-target sh -c '$str' ${base64} /root/late-command" +sed -e "s,$replace,$val," "${infile}" === added file 'libvirt-domain.tmpl' --- libvirt-domain.tmpl 1970-01-01 00:00:00 +0000 +++ libvirt-domain.tmpl 2012-03-08 06:52:22 +0000 @@ -0,0 +1,56 @@ +<domain type='kvm'> + <name>$name</name> + <memory>$mem</memory> + <currentMemory>$mem</currentMemory> + <vcpu>1</vcpu> + <os> + <type arch='x86_64' machine='pc-0.12'>hvm</type> + <boot dev='network' /> + <boot dev='hd' /> + </os> + <features> + <acpi/> + <apic/> + <pae/> + </features> + <clock offset='utc'/> + <on_poweroff>destroy</on_poweroff> + <on_reboot>restart</on_reboot> + <on_crash>restart</on_crash> + <devices> + <emulator>/usr/bin/kvm</emulator> + <disk type='file' device='disk'> + <driver name='qemu' type='qcow2'/> + <source file='$disk0'/> + <target dev='vda' bus='virtio'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/> + </disk> + <controller type='ide' index='0'> + <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/> + </controller> + <interface type='network'> + <!-- <boot order='1'/> --> + <source network='$network'/> + <target dev='vnet1'/> + <model type='virtio'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/> + <mac address='$mac'/> + </interface> + <serial type='pty'> + <source path='/dev/pts/5'/> + <target port='0'/> + </serial> + <console type='pty'> + <target type='serial' port='0'/> + </console> + <input type='mouse' bus='ps2'/> + <graphics type='vnc' autoport='yes' keymap='en-us'/> + <video> + <model type='cirrus' vram='9216' heads='1'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/> + </video> + <memballoon model='virtio'> + <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/> + </memballoon> + </devices> +</domain> === added file 'libvirt-network.tmpl' --- libvirt-network.tmpl 1970-01-01 00:00:00 +0000 +++ libvirt-network.tmpl 2012-03-08 06:52:22 +0000 @@ -0,0 +1,23 @@ +<network> + <name>$name</name> + <forward mode='nat'/> + <bridge name='$bridge' stp='off' delay='0' /> + <dns> + <host ip='$ip_pre.1'> + <hostname>host-system</hostname> + </host> + <host ip='$ip_pre.2'> + <hostname>zimmer-server</hostname> + </host> + </dns> + <ip address='$ip_pre.1' netmask='$netmask'> + <dhcp> + <range start='$ip_pre.$dhcp.range.start' end='$ip_pre.$dhcp.range.end' /> + <bootp server="$all_systems.zimmer.ipaddr" file="pxelinux.0" /> + #for $sys in $all_systems.itervalues() + <host mac="$sys.mac" name="$sys.name" ip="$sys.ipaddr" /> + #end for + </dhcp> + </ip> +</network> + === added file 'misc.txt' --- misc.txt 1970-01-01 00:00:00 +0000 +++ misc.txt 2012-03-08 06:52:22 +0000 @@ -0,0 +1,67 @@ +### apt-get install kvm-pxe ### + +## guest packages: + ## squid-deb-proxy-client + +http://blog.dustinkirkland.com/2011/03/ubuntu-server-quick-install-no.html + +qemu-img create -f qcow2 disk.img 8G +kvm -m 1024 -drive file=disk.img,if=virtio,cache=unsafe -cdrom ../iso/natty-release/ubuntu-11.04-server-amd64.iso -boot d -no-reboot +priority=critical locale=en_US url=http://bit.ly/uquick + +mv disk.img disk.img.dist +qemu-img create -f qcow2 -b disk.img.dist disk.img + +kvm -m 1024 -drive file=disk.img,if=virtio,cache=unsafe -cdrom ../iso/natty-release/ubuntu-11.04-server-amd64.iso -boot d + + + +sudo apt-get install python-software-properties +for p in ppa:orchestra/ppa ppa:dotdee/ppa; do sudo apt-add-repository $p; done +sudo apt-get update +sudo apt-get install ubuntu-orchestra-provisioning-server +sudo apt-get dist-upgrade +sudo apt-get clean +sudo apt-get update +sudo poweroff + + +mv disk.img disk.installed.img +qemu-img create -f qcow2 -b disk.installed.img trash.img +qemu-img create -f qcow2 -b disk.installed.img source.img +qemu-img create -f qcow2 target.img 8G + +kvm -m 1024 \ + -drive file=trash.img,if=virtio,cache=unsafe,index=0 \ + -drive file=source.img,if=virtio,cache=unsafe,index=1 \ + -drive file=target.img,if=virtio,cache=unsafe,index=2 \ + -net nic,model=virtio -net user,hostfwd=tcp:127.0.0.1:2222-:22 \ + +ssh ubuntu@localhost -p 2222 + +sudo sfdisk -uS -d /dev/vdb > out +sed -i 's,/dev/vdb,/dev/vdc,' out +sudo sfdisk -uS /dev/vdc --force < out +sudo dd if=/dev/vdb of=/dev/vdc bs=512 count=2048 +sudo mkdir /mnt/src /mnt/target +sudo mount -o ro /dev/vdb1 /mnt/src +out=$(sudo blkid /dev/vdb1 | awk -F: '{print $2}') +eval ${out} +sudo mkfs.${TYPE} -U ${UUID} ${LABEL:+-L ${LABEL}} /dev/vdc1 +out=$(sudo blkid /dev/vdb5 | awk -F: '{print $2}') +eval ${out} +sudo mkswap ${UUID:+-U ${UUID} } ${LABEL:+-L ${LABEL}} /dev/vdb5 +sudo mount /dev/vdc1 /mnt/target +sudo umount +sudo rsync -aXHAS /mnt/src/ /mnt/target +echo "cobbler" | sudo tee /mnt/target/etc/hostname +sudo poweroff + +virsh -c qemu:///system net-dumpxml default # get the default, modify a bit +virsh net-create libvirt-network-cobbler.xml +virsh net-define libvirt-network-cobbler.xml +virsh net-start cobbler + +virsh destroy cobbler-server; virsh undefine cobbler-server; virsh define libvirt-cobbler.xml +nn="cobbler"; virsh net-destroy $nn; virsh net-undefine $nn; virsh net-define libvirt-network-cobbler.xml; virsh net-start $nn + === added directory 'ref' === added file 'ref/cobbler.xml' --- ref/cobbler.xml 1970-01-01 00:00:00 +0000 +++ ref/cobbler.xml 2012-03-08 06:52:22 +0000 @@ -0,0 +1,61 @@ +<domain type='kvm'> + <name>cobbler-server</name> + <uuid>ea9c555b-f860-6fff-d4d3-cae66cc76715</uuid> + <memory>524288</memory> + <currentMemory>524288</currentMemory> + <vcpu>1</vcpu> + <os> + <type arch='x86_64' machine='pc-0.12'>hvm</type> + <boot dev='hd'/> + </os> + <features> + <acpi/> + <apic/> + <pae/> + </features> + <clock offset='utc'/> + <on_poweroff>destroy</on_poweroff> + <on_reboot>restart</on_reboot> + <on_crash>restart</on_crash> + <devices> + <emulator>/usr/bin/kvm</emulator> + <disk type='file' device='disk'> + <driver name='qemu' type='qcow2'/> + <source file='/home/download/orchestra-demo/cobbler-disk0.img'/> + <target dev='vda' bus='virtio'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/> + </disk> + <disk type='file' device='cdrom'> + <driver name='qemu' type='raw'/> + <source file='/home/download/orchestra-demo/cdrom-0.iso'/> + <target dev='hdc' bus='ide'/> + <readonly/> + <address type='drive' controller='0' bus='1' unit='0'/> + </disk> + <controller type='ide' index='0'> + <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/> + </controller> + <interface type='network'> + <mac address='00:16:3e:3e:a9:1a'/> + <source network='cobbler'/> + <model type='virtio'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/> + </interface> + <serial type='pty'> + <target port='0'/> + </serial> + <console type='pty'> + <target type='serial' port='0'/> + </console> + <input type='mouse' bus='ps2'/> + <graphics type='vnc' port='-1' autoport='yes' keymap='en-us'/> + <video> + <model type='cirrus' vram='9216' heads='1'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/> + </video> + <memballoon model='virtio'> + <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/> + </memballoon> + </devices> +</domain> + === added file 'ref/net.xml' --- ref/net.xml 1970-01-01 00:00:00 +0000 +++ ref/net.xml 2012-03-08 06:52:22 +0000 @@ -0,0 +1,12 @@ +<network> + <name>default</name> + <uuid>ab2138ca-73d2-37d8-aeee-d89a557cfccb</uuid> + <forward mode='nat'/> + <bridge name='virbr0' stp='on' delay='0' /> + <ip address='192.168.122.1' netmask='255.255.255.0'> + <dhcp> + <range start='192.168.122.2' end='192.168.122.254' /> + </dhcp> + </ip> +</network> + === added file 'ref/node01.xml' --- ref/node01.xml 1970-01-01 00:00:00 +0000 +++ ref/node01.xml 2012-03-08 06:52:22 +0000 @@ -0,0 +1,54 @@ +<domain type='kvm'> + <name>node01</name> + <uuid>91ac2f39-b6c0-f7fa-1d6f-751c18eab318</uuid> + <memory>524288</memory> + <currentMemory>524288</currentMemory> + <vcpu>1</vcpu> + <os> + <type arch='x86_64' machine='pc-0.12'>hvm</type> + <boot dev='network'/> + </os> + <features> + <acpi/> + <apic/> + <pae/> + </features> + <clock offset='utc'/> + <on_poweroff>destroy</on_poweroff> + <on_reboot>restart</on_reboot> + <on_crash>restart</on_crash> + <devices> + <emulator>/usr/bin/kvm</emulator> + <disk type='file' device='disk'> + <driver name='qemu' type='qcow2'/> + <source file='/home/download/orchestra-demo/node01.img'/> + <target dev='vda' bus='virtio'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/> + </disk> + <controller type='ide' index='0'> + <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/> + </controller> + <interface type='network'> + <mac address='52:54:00:ed:6c:d3'/> + <source network='cobbler'/> + <model type='virtio'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/> + </interface> + <serial type='pty'> + <target port='0'/> + </serial> + <console type='pty'> + <target type='serial' port='0'/> + </console> + <input type='mouse' bus='ps2'/> + <graphics type='vnc' port='-1' autoport='yes' keymap='en-us'/> + <video> + <model type='cirrus' vram='9216' heads='1'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/> + </video> + <memballoon model='virtio'> + <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/> + </memballoon> + </devices> +</domain> + === added file 'settings.cfg' --- settings.cfg 1970-01-01 00:00:00 +0000 +++ settings.cfg 2012-03-08 06:52:22 +0000 @@ -0,0 +1,24 @@ +network: + name: odev-net + bridge: virbr1 + ip_pre: 192.168.123 + ip: 1 + netmask: 255.255.255.0 + dhcp: + range: + start: 2 + end: 254 + template: libvirt-network.tmpl + +systems: + zimmer: + ip: 2 # ip address must be in dhcp range + mac: 00:16:3e:3e:a9:1a + template: libvirt-domain.tmpl + mem: 512 + +nodes: + prefix: odev-node + mac_pre: 00:16:3e:3e:aa + mem: 512 + template: libvirt-domain.tmpl === added file 'setup.py' --- setup.py 1970-01-01 00:00:00 +0000 +++ setup.py 2012-03-08 06:52:22 +0000 @@ -0,0 +1,217 @@ +#!/usr/bin/python + +import yaml +import os +import re +import sys +import libvirt +from Cheetah.Template import Template +import subprocess +import xmlrpclib + +NODES_RANGE = range(1,4) + +def yaml_loadf(fname): + fp = open(fname) + ret = yaml.load(fp) + fp.close() + return(ret) + +class Domain: + def __init__(self, syscfg, ident, basedir=None): + self.ip_pre = syscfg['network']['ip_pre'] + if basedir == None: + basedir = os.path.abspath(os.curdir) + self.basedir = basedir + self._setcfg(syscfg,ident) + self.network = syscfg['network']['name'] + + def __repr__(self): + return("== %s ==\n ip: %s\n mac: %s\n template: %s\n" % + (self.name, self.ipaddr, self.mac, self.template)) + + @property + def ipaddr(self): + return("%s.%s" % (self.ip_pre, self.ipnum)) + + @property + def disk0(self): + return("%s/%s-disk0.img" % (self.basedir, self.name)) + + def dictInfo(self): + ret = vars(self) + # have to add the getters + for prop in ( "ipaddr", "disk0" ): + ret[prop] = getattr(self,prop) + return ret + + def toLibVirtXml(self): + template = Template(file=self.template, searchList=[self.dictInfo()]) + return template.respond() + +class Node(Domain): + def _setcfg(self, cfg, num): + cfg = cfg['nodes'] + self.name = "%s%02i" % (cfg['prefix'],num) + self.mac = "%s:%02x" % (cfg['mac_pre'],num) + self.ipnum = num + 100 + self.template = cfg['template'] + self.mem = cfg['mem'] * 1024 + return + +class System(Domain): + def _setcfg(self, cfg, ident): + cfg = cfg['systems'][ident] + self.name = ident + self.mac = cfg['mac'] + self.ipnum = cfg['ip'] + self.template = cfg['template'] + self.mem = cfg['mem'] * 1024 + +def renderSysDom(config, syscfg, stype="node"): + template = Template(file=syscfg['template'], searchList=[config, syscfg]) + return template.respond() + +# cobbler: +# ip: 2 # ip address must be in dhcp range +# mac: 00:16:3e:3e:a9:1a +# template: libvirt-system.tmpl +# mem: 524288 +# +#nodes: +# prefix: node +# mac_pre: 00:16:3e:3e:aa +# mam: 256 + +def writeDomXmlFile(dom, outpre=""): + fname="%s%s.xml" % (outpre, dom.name) + output = open(fname,"w") + output.write(dom.toLibVirtXml()) + output.close() + return fname + +def libvirt_setup(config): + conn = libvirt.open("qemu:///system") + netname = config['network']['name'] + if netname in conn.listDefinedNetworks() or netname in conn.listNetworks(): + net = conn.networkLookupByName(netname) + if net.isActive(): + net.destroy() + net.undefine() + + allsys = {} + for system in config['systems']: + d = System(config, system) + allsys[d.name]=d.dictInfo() + for num in NODES_RANGE: + d = Node(config, num) + allsys[d.name]=d.dictInfo() + + conn.networkDefineXML(Template(file=config['network']['template'], + searchList=[config['network'], + {'all_systems': allsys }]).respond()) + + print "defined network %s " % netname + + cob = System(config, "zimmer") + systems = [ cob ] + + for node in NODES_RANGE: + systems.append(Node(config, node)) + + qcow_create = "qemu-img create -f qcow2 %s 2G" + defined_systems = conn.listDefinedDomains() + for system in systems: + if system.name in defined_systems: + dom = conn.lookupByName(system.name) + if dom.isActive(): + dom.destroy() + dom.undefine() + conn.defineXML(system.toLibVirtXml()) + if isinstance(system,Node): + subprocess.check_call(qcow_create % system.disk0, shell=True) + print "defined domain %s" % system.name + +def cobbler_addsystem(server, token, system, profile, hostip): + eth0 = { + "macaddress-eth0" : system.mac, + "ipaddress-eth0" : system.ipaddr, + "static-eth0" : False, + } + items = { + 'name': system.name, + 'hostname': system.name, + 'power_address': "qemu+tcp://%s:65001" % hostip, + 'power_id': system.name, + 'power_type': "virsh", + 'profile': profile, + 'netboot_enabled': True, + 'modify_interface': eth0, + 'mgmt_classes': ['orchestra-juju-available'], + } + + if len(server.find_system({"name": system.name})): + server.remove_system(system.name,token) + server.update() + print "removed existing %s" % system.name + + sid = server.new_system(token) + for key, val in items.iteritems(): + ret = server.modify_system(sid, key, val, token) + if not ret: + raise Exception("failed for %s [%s]: %s, %s" % + (system.name, ret, key, val)) + ret = server.save_system(sid,token) + if not ret: + raise Exception("failed to save %s" % system.name) + print "added %s" % system.name + + +def get_profile_arch(): + """Get the system architecture for use in the cobbler setup profile.""" + # This should, for any given system, match what the zimmer-build + # script does to determine the right architecture. + arch_text = subprocess.check_output(['/bin/uname', '-m']).strip() + if re.match('i.86', arch_text): + return 'i386' + else: + return arch_text + + +def cobbler_setup(config): + hostip = "%s.1" % config['network']['ip_pre'] + arch = get_profile_arch() + profile = "precise-%s-juju" % arch + + cob = System(config, "zimmer") + cobbler_url = "http://%s/cobbler_api"; % cob.ipaddr + print("Connecting to %s." % cobbler_url) + server = xmlrpclib.Server(cobbler_url) + token = server.login("cobbler","xcobbler") + + systems = [Node(config, node) for node in NODES_RANGE] + + for system in systems: + cobbler_addsystem(server, token, system, profile, hostip) + +def main(): + outpre = "libvirt-cobbler-" + cfg_file = "settings.cfg" + + if len(sys.argv) == 1: + print( + "Usage: setup.py action\n" + "action one of: libvirt-setup, cobbler-setup") + sys.exit(1) + + config = yaml_loadf(cfg_file) + + if sys.argv[1] == "libvirt-setup": + libvirt_setup(config) + elif sys.argv[1] == "cobbler-setup": + cobbler_setup(config) + +if __name__ == '__main__': + main() + +# vi: ts=4 noexpandtab === added directory 'zimmer-build' === added file 'zimmer-build/build' --- zimmer-build/build 1970-01-01 00:00:00 +0000 +++ zimmer-build/build 2012-03-08 06:52:22 +0000 @@ -0,0 +1,271 @@ +#!/bin/bash + +# This should mirror what's in odev's setup.py, except here x86_64 is called +# amd64, not x86_64 because that is Ubuntu's selected name for the arch. +GUEST_ARCHITECTURE=$(uname -m) +case "$GUEST_ARCHITECTURE" in + i?86) GUEST_ARCHITECTURE="i386" ;; + x86_64) GUEST_ARCHITECTURE="amd64" ;; +esac + + +DEF_ZIMG="http://cloud-images.ubuntu.com/server/precise/current/precise-server-cloudimg-${GUEST_ARCHITECTURE}-disk1.img"; +DEF_SAVE_D="pristine" +DEF_UD_FILE="ud-build.txt" +ZIMMER_SSH_FORWARD="" +#ZIMMER_SSH_FORWARD=${ZIMMER_SSH_FORWARD:-"hostfwd=tcp::2222-:22"} +ZIMMER_MEM="${ZIMMER_MEM:-1024}" +KVM_PID="" +TAIL_PID="" +LOG="output.log" + +case $(uname -m) in + i?86) DEF_ZIMG=${DEF_ZIMG//amd64/i386};; +esac + +VERBOSITY=0 +TEMP_D="" + +error() { echo "$@" 1>&2; } +errorp() { printf "$@" 1>&2; } +fail() { [ $# -eq 0 ] || error "$@"; exit 1; } +failp() { [ $# -eq 0 ] || errorp "$@"; exit 1; } + +Usage() { + cat <<EOF +Usage: ${0##*/} [ options ] output + + build a zimmer server from a cloud image, and put it in 'output' + + options: + --zimg Z url or path to compressed cloud image + (will be uncompressed) + def: $DEF_ZIMG + --img I url or path to uncompressed cloud image + expected to be uncompressed. + default: create from zimg + --log L log items to LOG + default: $LOG + --save D put pristine copies of things in D + default: $DEF_SAVE_D + --ud-file F use user-data file F + default: $DEF_USER_DATA + --import-keys K import ssh keys + values are 'auto', 'lp:<id>', or path to file +EOF +} + +bad_Usage() { Usage 1>&2; [ $# -eq 0 ] || error "$@"; exit 1; } +cleanup() { + [ -z "${TEMP_D}" -o ! -d "${TEMP_D}" ] || rm -Rf "${TEMP_D}" + [ -z "$KVM_PID" ] || kill "$KVM_PID" + [ -z "$TAIL_PID" ] || kill "$TAIL_PID" +} + +log() { + [ -n "$LOG" ] || return + echo "$(date -R):" "$@" >> "$LOG" +} +debug() { + local level=${1}; shift; + log "$@" + [ "${level}" -gt "${VERBOSITY}" ] && return + error "${@}" +} + +# Download image file. +# Parameters: source URL, filename to save to +download() { + local src="$1" dest="$2" + + debug 0 "downloading $src to $dest" + wget --progress=dot:mega "$src" -O "$dest.partial" && + mv -- "$dest.partial" "$dest" || + fail "failed to get $src" +} + +short_opts="ho:v" +long_opts="help,img:,import-keys:,log:,ud-file:,verbose,zimg:" +getopt_out=$(getopt --name "${0##*/}" \ + --options "${short_opts}" --long "${long_opts}" -- "$@") && + eval set -- "${getopt_out}" || + bad_Usage + +img="" +zimg="" +save_d="$DEF_SAVE_D" +ud_file="$DEF_UD_FILE" +import_keys="" + +while [ $# -ne 0 ]; do + cur=${1}; next=${2}; + case "$cur" in + -h|--help) Usage ; exit 0;; + --img) img=${2}; shift;; + --log) LOG=${2}; shift;; + --save) save_d=${2}; shift;; + --ud-file) ud_file=${2}; shift;; + -v|--verbose) VERBOSITY=$((${VERBOSITY}+1));; + --zimg) zimg=${2}; shift;; + --import-keys) import_keys=${2}; shift;; + --) shift; break;; + esac + shift; +done + +## check arguments here +## how many args do you expect? +[ $# -gt 1 ] && bad_Usage "too many arguments" +[ $# -eq 0 ] && bad_Usage "need an output argument" +output="$1" +[ "${output%.zimg}" = "${output}" ] || fail "do not name output with .zimg" + +command -v genisoimage >/dev/null || + fail "you do not have genisoimage installed. install genisoimage package" + + +[ -f "$ud_file" ] || + fail "user data file $ud_file" is not a file + +TEMP_D=$(mktemp -d "${TMPDIR:-/tmp}/${0##*/}.XXXXXX") || + fail "failed to make tempdir" +trap cleanup EXIT + +mkdir -p "$save_d" || fail "failed to mkdir $save_d" + +# if --import-keys was specified, get the keys into a local file +keyf="$TEMP_D/keys" +if [ "$import_keys" = "auto" ]; then + ssh-add -L > "${keyf}" 2>/dev/null || + cat $HOME/.ssh/id*.pub > "$keyf" 2>/dev/null || + error "Warning: unable to find 'auto' keys" +elif [ -f "$import_keys" ]; then + cat "$import_keys" > "$keyf" +elif [ "${import_keys#lp:}" != "${import_keys}" ]; then + ssh-import-id -o - ${import_keys#lp:} > "$keyf" 2>/dev/null || + error "Warning: failed to ssh-import ${import_keys#lp:}" +fi + +if [ -n "$img" ]; then + # if img was given, then we assume good, its the backing image + [ -f "$img" ] || fail "$img (--img) is not a file" + debug 0 "using $img as uncompressed image" +else + if [ -z "$zimg" ]; then + zimg="$DEF_ZIMG" + fi + case "$zimg" in + http://*|https://*) + o_zimg="${zimg}" + zimg=${save_d}/$(basename "$o_zimg" ".img").zimg + [ -f "$zimg" ] && + fail "please delete $destfirst or use --zimg|--img" + download "$o_zimg" "$zimg" + ;; + file://*) + o_zimg=${zimg} + zimg=${zimg#file://} + debug 0 "using file $o_zimg as zimg" + [ -f "$zimg" ] || fail "$zimg is not a file" + ;; + *) [ -f "$zimg" ] || fail "$zimg is not a file" + debug 0 "using file $zimg as zimg" + ;; + esac + img=${zimg%.zimg}.img + debug 0 "creating uncompressed img $img from $zimg" + qemu-img convert -O qcow2 "$zimg" "$img" +fi + +debug 0 "making nocloud data source in iso" +seed_d="$TEMP_D/seed" +mkdir "$seed_d" || fail "failed to make 'seed' in tempdir" + +cp "$ud_file" "$seed_d/user-data" || fail "failed to copy $ud_file to $seed_d" +cat > "$seed_d/meta-data" <<EOF +instance-id: i-zimmer-build +local-hostname: zimmer-build +EOF + +# if keys were specified, dump them into meta-data +if [ -s "$keyf" ]; then + { + echo "public-keys:" + echo " zimmer-build:" + while read line; do + echo " - \"$line\"" + done < "$keyf" + } >> "$seed_d/meta-data" +fi + +( cd "$seed_d" && + genisoimage -output "$TEMP_D/build.iso" \ + -volid cidata -joliet -rock user-data meta-data 2>/dev/null ) || + fail "failed to create iso for user-data from $ud_file" + +build0="$TEMP_D/build0.img" +img_fp=$(readlink -f "$img") || fail "failed to get fullpath for $img" +qemu-img create -f qcow2 -b "$img_fp" "${build0}" || + fail "failed to create qcow image backed by $img" + +## on precise, you do do not need 'boot=on' in kvm commanad line +[ "$(lsb_release -sc)" = "precise" ] && bton="" || bton="boot=on" + +serial_out="$TEMP_D/serial.output" +monitor="${TEMP_D}/monitor.fifo" && mkfifo "$monitor" || + fail "failed to mkfifo for monitor" + +debug 0 "booting kvm guest to turn cloud-image into zimmer" +kvm_start=$SECONDS +MONITOR="-monitor null" +NOGRAPHIC="-nographic" +kvm \ + -drive file=${build0},if=virtio,cache=unsafe${bton:+,${bton}} \ + -boot c -cdrom "$TEMP_D/build.iso" \ + -net nic,model=virtio \ + -net user${ZIMMER_SSH_FORWARD:+,${ZIMMER_SSH_FORWARD}} \ + -m "${ZIMMER_MEM}" \ + $NOGRAPHIC \ + $MONITOR \ + -serial "file:$serial_out" \ + 2>&1 & + +KVM_PID=$! +tail -F "$serial_out" 2>/dev/null & +TAIL_PID=$! + +sleep 20 +[ -s "$serial_out" ] || + fail "no output in serial console output after 20 seconds" + +wait $KVM_PID +ret=$? +KVM_PID="" + +{ kill $TAIL_PID ; } >/dev/null 2>&1 +TAIL_PID="" + +{ + echo ===== begin serial console ==== + cat "$serial_out" + echo ===== end serial console ==== +} >> "$LOG" +[ $ret -eq 0 ] || fail "failed to build via kvm guest" +grep -q "ZIMMER BUILD FINISHED" "$serial_out" || + fail "did not find finished message in $serial_out" + +debug 0 "kvm image built in $(($SECONDS-$kvm_start))s" +debug 0 "creating dist image in $output" +## create a re-shrunk image of build0.img into 'zimmer-disk0.img.dist' +[ ! -f "$output" ] || rm -f "$output" || + fail "failed to remove existing $output" +qemu-img convert -O qcow2 "$TEMP_D/build0.img" "$output" && + chmod 444 "$output" || + fail "failed to create $output from build0.img" + +debug 0 "creating pristine compressed zimmer-disk0.zimg" +## optionally create a zip'd image for transmission +qemu-img convert -f qcow2 -O qcow2 -c "$output" "${output%.img}.zimg" + +debug 0 "done. took $SECONDS seconds" +# vi: ts=4 noexpandtab === added file 'zimmer-build/ud-build.txt' --- zimmer-build/ud-build.txt 1970-01-01 00:00:00 +0000 +++ zimmer-build/ud-build.txt 2012-03-08 06:52:22 +0000 @@ -0,0 +1,70 @@ +#cloud-config +password: passw0rd +chpasswd: { expire: False } +ssh_pwauth: True + +#apt_proxy: "http://local-proxy:3128/"; +#apt_mirror: "http://us.archive.ubuntu.com/ubuntu"; +#ssh_import_id: smoser + +bucket: + - &setup | + cd /root + ( + #ONE_TIME_PROXY=http://local-proxy:3128/ + + echo === $(date) ==== + debconf-set-selections <<EOF + ubuntu-orchestra-provisioning-server ubuntu-orchestra-provisioning-server/import-isos boolean false + ubuntu-orchestra-provisioning-server ubuntu-orchestra-provisioning-server/dnsmasq-dhcp-range string 10.10.10.2,10.10.10.254 + ubuntu-orchestra-provisioning-server ubuntu-orchestra-provisioning-server/dnsmasq-enabled boolean false + cobbler cobbler/server_and_next_server string zimmer-server + cobbler cobbler/password password xcobbler + cloud-init cloud-init/datasources multiselect NoCloud, OVF + + EOF + + [ -n "$ONE_TIME_PROXY" ] && export http_proxy="$ONE_TIME_PROXY" + export DEBIAN_FRONTEND=noninteractive; + dpkg-reconfigure cloud-init + + read oldhost < /etc/hostname + sed -i "/$oldhost/d;/zimmer/d" /etc/hosts + echo zimmer > /etc/hostname + hostname zimmer + + echo "127.0.1.2 zimmer-server" >> /etc/hosts + + echo === $(date): starting apt ==== + apt_get() { + DEBIAN_FRONTEND=noninteractive apt-get \ + --option "Dpkg::Options::=--force-confold" --assume-yes "$@" + } + apt_get update + apt_get install ubuntu-orchestra-provisioning-server libvirt-bin cobbler-web + + case $(uname -m) in + i?86) arches="i386";; + *) arches="amd64";; + esac + cat >> /etc/orchestra/import_isos <<END + RELEASES="oneiric precise" + ARCHES="${arches}" + END + + echo === $(date): starting import ==== + orchestra-import-isos + + sed -i '/zimmer-server/d' /etc/hosts + + echo === $(date): starting cleanup ==== + apt_get clean + time sh -c 'dd if=/dev/zero of=/out.img; rm /out.img' + + echo === $(date): poweroff === + echo === ZIMMER BUILD FINISHED === + ) 2>&1 | tee out.log + +runcmd: + - [ sh, -c, *setup ] + - [ /sbin/poweroff ]
-- Mailing list: https://launchpad.net/~orchestra Post to : [email protected] Unsubscribe : https://launchpad.net/~orchestra More help : https://help.launchpad.net/ListHelp
