Repository: couchdb-ci Updated Branches: refs/heads/master 1dec43917 -> b918cfb69 (forced update)
Updated images for non-root Jenkins+Docker builds Project: http://git-wip-us.apache.org/repos/asf/couchdb-ci/repo Commit: http://git-wip-us.apache.org/repos/asf/couchdb-ci/commit/b918cfb6 Tree: http://git-wip-us.apache.org/repos/asf/couchdb-ci/tree/b918cfb6 Diff: http://git-wip-us.apache.org/repos/asf/couchdb-ci/diff/b918cfb6 Branch: refs/heads/master Commit: b918cfb69020876b745e45938da63df8a6927d37 Parents: b19c411 Author: Joan Touzet <jo...@atypical.net> Authored: Thu Jul 20 22:31:19 2017 -0700 Committer: Joan Touzet <woh...@apache.org> Committed: Fri Jul 21 00:15:51 2017 -0700 ---------------------------------------------------------------------- README.md | 119 +++++++++++++++++++ ansible/centos-6-base.yml | 1 + ansible/roles/common/tasks/main.yml | 16 +++ ansible/roles/debian-8/tasks/main.yml | 7 ++ .../dependencies-debian/files/libmozjs185-fake | 7 ++ .../roles/dependencies-debian/tasks/main.yml | 18 +++ ansible/roles/fakejsrpm/files/js-1.8.5.spec | 20 ++++ ansible/roles/fakejsrpm/tasks/main.yml | 16 +++ bin/rebuild-all.sh | 3 + dockerfiles/centos-6-erlang-18.3 | 2 - dockerfiles/centos-6-erlang-default | 2 - dockerfiles/centos-7-erlang-18.3 | 2 - dockerfiles/centos-7-erlang-default | 2 - dockerfiles/debian-8-erlang-18.3 | 2 - dockerfiles/debian-8-erlang-default | 2 - dockerfiles/ubuntu-12.04-erlang-18.3 | 2 - dockerfiles/ubuntu-12.04-erlang-default | 2 - dockerfiles/ubuntu-14.04-erlang-18.3 | 2 - dockerfiles/ubuntu-14.04-erlang-default | 2 - dockerfiles/ubuntu-16.04-erlang-18.3 | 2 - dockerfiles/ubuntu-16.04-erlang-default | 2 - readme.markdown | 119 ------------------- 22 files changed, 207 insertions(+), 143 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/couchdb-ci/blob/b918cfb6/README.md ---------------------------------------------------------------------- diff --git a/README.md b/README.md new file mode 100644 index 0000000..4d9727f --- /dev/null +++ b/README.md @@ -0,0 +1,119 @@ +# CouchDB Continuous Integration (CI) support repo + +The main purpose of this repository is to provide a number of Docker containers, Ansible roles/tasks and other configuration functionality so that the ASF Jenkins CI server (https://builds.apache.org/) is capable of building (and eventually packaging) CouchDB for a number of platforms. It intends to cover a range of both operating systems (Linux, macOS, BSD, Windows) and Erlang versions (OS default, 18.x, 19.x, etc.) + +The current configuration builds CouchDB, Fauxton, its documentation, and runs the Erlang and JS test suites for each combination of OS and Erlang revision. + +## Background + +See: +* this [thread](https://www.mail-archive.com/dev%40couchdb.apache.org/msg43591.html) on the couchdb-dev mailing list and +* this [ASF Infra ticket](https://issues.apache.org/jira/browse/INFRA-10126). +for the origins of this work. + +## Supported Configurations (updated 2017-03-22) + +**OS/Erlang** | **default ** | **18.3** +--------------------|--------------|-------------- +**Ubuntu 12.04** | â (R14B04) | â +**Ubuntu 14.04** | â (16B03-1) | â +**Ubuntu 16.04** | â (18.3) | â +**Debian 8** | â (17.3) | â +**Debian 9** | - | - +**CentOS 6** | â (R14B04) | â +**CentOS 7** | â (16B03-1) | â +**macOS 10.12** | - | - +**FreeBSD** | - | - +**Windows** | - | - + +Builds marked with an â are skipped due to the version of Erlang being too old to build CouchDB >= 2.0.0. + + +## Open questions and TODOs +* Right now we run a CouchDB build on all combinations on each commit, but perhaps we don't need to run Jenkins this often. (We also have Travis CI for a single OS and a few Erlang revisions.) Should we just build them once a day? +* Right now we only trigger Jenkins on the master branch. The images support building from whatever branch $GIT_BRANCH is set to (c.f. Jenkins environment variable.) Travis CI currently handles PRs. ASF hasn't set up a Jenkins/Gitbox bridge yet, but if they do we can consider having Jenkins build PRs as well. +* Ideally, we'd also like to build convenience packages for some of these platforms on a regular basis, i.e. nightly - especially the platforms where building the software is harder (Windows). +* Currently, when changes occur to the base images (bugfixes, new OS/Erlang combinations added, etc.) new images are built manually using the scripts in `bin/`. TODO: automate this process in a way that avoids forcibly rebuilding every VM/Erlang combination with every checkin. + +--- + +# Docker + +For those OSes that support Docker, we run builds inside of Docker containers. The Dockerfiles typically use Ansible to handle all of their configuration and setup. This allows us to reuse the Ansible tasks for those OSes not supported by Docker for native apps (such as macOS and FreeBSD) where we must use VMs or bare hardware. The ASF has Windows Jenkins clients as an alternative to running Jenkins on Windows; a decision has not yet been reached here. + +## Building a container + +Run the `bin/<image-name>/create-container.sh` script. + +## Interactively working in a built container + +Run the `bin/<image-name>/enter-container.sh` script. + +## Running the CouchDB build in a locally built (but not published) container + +After using the `create-container.sh` script, run the command `docker run -it couchdbdev/<imagename>`. The build should immediately start. + +## Publishing a container + +1. You need a Docker Cloud account with access to the `couchdbdev` organization. Ask the CouchDB PMC for assistance with this. +2. `export DOCKER_ID_USER="username"` +3. `docker login` and enter your password. +4. Run the `bin/<image-name>/publish-container.sh` script. + +## Running the CouchDB build in a published container + +To pull down the latest image and run the CouchDB build in the container, run the `bin/<image-name>/run-build-in-container.sh` script. This does not work with `-base` images, which lack the full toolchain necessary to build CouchDB. + + +Vagrant +------- + +Note: This section on creating the build machines on Vagrant might be outdated. Either we bring it up to date to have both possibilities (Docker & Vagrant) or we remove it completely and just go with Docker for now. + +### Prerequisites + +See the readme files in folder `baseboxes` for docs on building the base boxes. + +You need to have [Vagrant](https://www.vagrantup.com/) and [VirtualBox](https://www.virtualbox.org/) installed. + +When Vagrant is installed you need to install an additional plug-in: +```bash +vagrant plugin install vagrant-hosts +``` + +plus an additional Ansible role +```bash +[sudo] ansible-galaxy install nodesource.node +``` + +Also, you might need to run +```bash +VBoxManage dhcpserver remove --netname HostInterfaceNetworking-vboxnet0 +``` +before doing +```bash +vagrant up +``` +because of <https://github.com/mitchellh/vagrant/issues/3083>. + +### Building and Registering the Base Box + +Each vagrant configuration requires its respective base box. For example, the configuration jenkins-master requires the base box `couchdb-ci-ubuntu-14.04`. You can build the base boxes locally with veewee (see `baseboxes/readme.markdown`). To make things easier we might upload the base boxes somewhere so people do not have to build them theirselves but can just download them but we are not there yet. + +Either way, you have to add the image to the vagrant base image registry with the following command: +``` +vagrant box add couchdb-ci-ubuntu-14.04 /path/to/your/base-box-file.box +``` + +When this has happened, `vagrant box list` should list the base box's name (`couchdb-ci-ubuntu-14.04` in our example). + +The base box built with veewee is just a fresh minimal install of the OS. All relevant packages and configurations are provisioned with Ansible. Vagrant is used to make creating, provisioning and destroying these virtual machines easier. + +### Launching and Provisioning the Boxes + +Just execute `vagrant up`. This will start all the virtual machine and provision it using the scripts in `ansible`. That's it :-) + +You can also selectively launch a single box, for example with +```bash +vagrant up couchdb-jenkins-master +``` http://git-wip-us.apache.org/repos/asf/couchdb-ci/blob/b918cfb6/ansible/centos-6-base.yml ---------------------------------------------------------------------- diff --git a/ansible/centos-6-base.yml b/ansible/centos-6-base.yml index 6b93f03..8712a5e 100644 --- a/ansible/centos-6-base.yml +++ b/ansible/centos-6-base.yml @@ -24,3 +24,4 @@ - dependencies-centos - common - spidermonkey + - fakejsrpm http://git-wip-us.apache.org/repos/asf/couchdb-ci/blob/b918cfb6/ansible/roles/common/tasks/main.yml ---------------------------------------------------------------------- diff --git a/ansible/roles/common/tasks/main.yml b/ansible/roles/common/tasks/main.yml index 56ac10a..a210471 100644 --- a/ansible/roles/common/tasks/main.yml +++ b/ansible/roles/common/tasks/main.yml @@ -25,3 +25,19 @@ group: root mode: 0440 +# uid/gid here came from ASF Infra +- name: create jenkins group + group: + name: jenkins + gid: 12036 + state: present + +- name: create jenkins user + user: + name: jenkins + comment: "Jenkins user" + uid: 10025 + group: jenkins + createhome: yes + state: present + http://git-wip-us.apache.org/repos/asf/couchdb-ci/blob/b918cfb6/ansible/roles/debian-8/tasks/main.yml ---------------------------------------------------------------------- diff --git a/ansible/roles/debian-8/tasks/main.yml b/ansible/roles/debian-8/tasks/main.yml new file mode 100644 index 0000000..3637c90 --- /dev/null +++ b/ansible/roles/debian-8/tasks/main.yml @@ -0,0 +1,7 @@ +# dependencies for making Debian/RPM repositories +- name: install packages required to build Debian/RPM repositories + apt: name={{item}} state=present install_recommends=no + with_items: + - reprepro + - createrepo + http://git-wip-us.apache.org/repos/asf/couchdb-ci/blob/b918cfb6/ansible/roles/dependencies-debian/files/libmozjs185-fake ---------------------------------------------------------------------- diff --git a/ansible/roles/dependencies-debian/files/libmozjs185-fake b/ansible/roles/dependencies-debian/files/libmozjs185-fake new file mode 100644 index 0000000..b5b1c77 --- /dev/null +++ b/ansible/roles/dependencies-debian/files/libmozjs185-fake @@ -0,0 +1,7 @@ +Section: misc +Priority: optional +Standards-Version: 3.9.2 + +Package: libmozjs185-fake +Provides: libmozjs185-dev, libmozjs185-1.0 +Description: Fake libmozjs185 packages for builds http://git-wip-us.apache.org/repos/asf/couchdb-ci/blob/b918cfb6/ansible/roles/dependencies-debian/tasks/main.yml ---------------------------------------------------------------------- diff --git a/ansible/roles/dependencies-debian/tasks/main.yml b/ansible/roles/dependencies-debian/tasks/main.yml index 6555e47..178670d 100644 --- a/ansible/roles/dependencies-debian/tasks/main.yml +++ b/ansible/roles/dependencies-debian/tasks/main.yml @@ -74,6 +74,7 @@ - libwww-perl - devscripts - quilt + - equivs # Hooray deMorgan's theorem - name: install dh-systemd package, if available @@ -86,3 +87,20 @@ with_items: - vim-tiny - screen + +# install libmozjs185-equivs file +- name: install libmozjs185-equivs file + copy: + src: libmozjs185-fake + dest: /tmp/libmozjs185-fake + owner: root + group: root + mode: 0755 + +# build and install fake libmozjs185 packages +- name: install fake libmozjs185 packages + shell: | + cd /tmp + equivs-build libmozjs185-fake + dpkg -i libmozjs185-fake_1.0_all.deb + rm libmozjs185* http://git-wip-us.apache.org/repos/asf/couchdb-ci/blob/b918cfb6/ansible/roles/fakejsrpm/files/js-1.8.5.spec ---------------------------------------------------------------------- diff --git a/ansible/roles/fakejsrpm/files/js-1.8.5.spec b/ansible/roles/fakejsrpm/files/js-1.8.5.spec new file mode 100644 index 0000000..6ab37de --- /dev/null +++ b/ansible/roles/fakejsrpm/files/js-1.8.5.spec @@ -0,0 +1,20 @@ +Name: js +Version: 1.8.5 +Release: 7 +Summary: Fake JavaScript package +Group: Development/Languages +License: GPLv2+ or LGPLv2+ or MPLv1.1 +BuildArch: noarch + +%description +Fake JavaScript package + +%prep + +%build + +%install + +%files + +%changelog http://git-wip-us.apache.org/repos/asf/couchdb-ci/blob/b918cfb6/ansible/roles/fakejsrpm/tasks/main.yml ---------------------------------------------------------------------- diff --git a/ansible/roles/fakejsrpm/tasks/main.yml b/ansible/roles/fakejsrpm/tasks/main.yml new file mode 100644 index 0000000..d50a5b5 --- /dev/null +++ b/ansible/roles/fakejsrpm/tasks/main.yml @@ -0,0 +1,16 @@ +# install fake specfile +- name: install fake specfile + copy: + src: js-1.8.5.spec + dest: /tmp/js-1.8.5.spec + owner: root + group: root + mode: 0755 + +# build and install fake js-1.8.5 package +- name: install fake js-1.8.5 package + shell: | + cd /tmp + rpmbuild -ba js-1.8.5.spec + rpm -Uvh /root/rpmbuild/RPMS/noarch/js-1.8.5-7.noarch.rpm + rm -rf /root/rpmbuild http://git-wip-us.apache.org/repos/asf/couchdb-ci/blob/b918cfb6/bin/rebuild-all.sh ---------------------------------------------------------------------- diff --git a/bin/rebuild-all.sh b/bin/rebuild-all.sh index 8207ab3..f0b4bb0 100755 --- a/bin/rebuild-all.sh +++ b/bin/rebuild-all.sh @@ -20,6 +20,9 @@ SCRIPT=$(readlink -f "$0") SCRIPTPATH=$(dirname "$SCRIPT") +( find $SCRIPTPATH/*-base -type d -printf "couchdbdev/%f\n" | xargs docker rmi) >/dev/null 2>&1 +( find $SCRIPTPATH/*erlang* -type d -printf "couchdbdev/%f\n" | xargs docker rmi) >/dev/null 2>&1 + find $SCRIPTPATH/*-base -type d -exec {}/create-container.sh \; find $SCRIPTPATH/*erlang* -type d -exec {}/create-container.sh \; exec $SCRIPTPATH/cleanup-docker.sh http://git-wip-us.apache.org/repos/asf/couchdb-ci/blob/b918cfb6/dockerfiles/centos-6-erlang-18.3 ---------------------------------------------------------------------- diff --git a/dockerfiles/centos-6-erlang-18.3 b/dockerfiles/centos-6-erlang-18.3 index bf67102..2dcd2ab 100644 --- a/dockerfiles/centos-6-erlang-18.3 +++ b/dockerfiles/centos-6-erlang-18.3 @@ -30,5 +30,3 @@ RUN ansible-playbook centos-6-erlang-18.3.yml \ --inventory-file=./inventory/couchdb-ci-worker USER build - -CMD ["/home/build/build-ci.sh"] http://git-wip-us.apache.org/repos/asf/couchdb-ci/blob/b918cfb6/dockerfiles/centos-6-erlang-default ---------------------------------------------------------------------- diff --git a/dockerfiles/centos-6-erlang-default b/dockerfiles/centos-6-erlang-default index 0c62c61..8b12634 100644 --- a/dockerfiles/centos-6-erlang-default +++ b/dockerfiles/centos-6-erlang-default @@ -30,5 +30,3 @@ RUN ansible-playbook centos-6-erlang-default.yml \ --inventory-file=./inventory/couchdb-ci-worker USER build - -CMD ["/home/build/build-ci.sh"] http://git-wip-us.apache.org/repos/asf/couchdb-ci/blob/b918cfb6/dockerfiles/centos-7-erlang-18.3 ---------------------------------------------------------------------- diff --git a/dockerfiles/centos-7-erlang-18.3 b/dockerfiles/centos-7-erlang-18.3 index f85c5d3..3729279 100644 --- a/dockerfiles/centos-7-erlang-18.3 +++ b/dockerfiles/centos-7-erlang-18.3 @@ -30,5 +30,3 @@ RUN ansible-playbook centos-7-erlang-18.3.yml \ --inventory-file=./inventory/couchdb-ci-worker USER build - -CMD ["/home/build/build-ci.sh"] http://git-wip-us.apache.org/repos/asf/couchdb-ci/blob/b918cfb6/dockerfiles/centos-7-erlang-default ---------------------------------------------------------------------- diff --git a/dockerfiles/centos-7-erlang-default b/dockerfiles/centos-7-erlang-default index 351275d..8071548 100644 --- a/dockerfiles/centos-7-erlang-default +++ b/dockerfiles/centos-7-erlang-default @@ -30,5 +30,3 @@ RUN ansible-playbook centos-7-erlang-default.yml \ --inventory-file=./inventory/couchdb-ci-worker USER build - -CMD ["/home/build/build-ci.sh"] http://git-wip-us.apache.org/repos/asf/couchdb-ci/blob/b918cfb6/dockerfiles/debian-8-erlang-18.3 ---------------------------------------------------------------------- diff --git a/dockerfiles/debian-8-erlang-18.3 b/dockerfiles/debian-8-erlang-18.3 index 2c7e8aa..445eb3d 100644 --- a/dockerfiles/debian-8-erlang-18.3 +++ b/dockerfiles/debian-8-erlang-18.3 @@ -30,5 +30,3 @@ RUN ansible-playbook debian-8-erlang-18.3.yml \ --inventory-file=./inventory/couchdb-ci-worker USER build - -CMD ["/home/build/build-ci.sh"] http://git-wip-us.apache.org/repos/asf/couchdb-ci/blob/b918cfb6/dockerfiles/debian-8-erlang-default ---------------------------------------------------------------------- diff --git a/dockerfiles/debian-8-erlang-default b/dockerfiles/debian-8-erlang-default index 4d4ef4c..d547b70 100644 --- a/dockerfiles/debian-8-erlang-default +++ b/dockerfiles/debian-8-erlang-default @@ -30,5 +30,3 @@ RUN ansible-playbook debian-8-erlang-default.yml \ --inventory-file=./inventory/couchdb-ci-worker USER build - -CMD ["/home/build/build-ci.sh"] http://git-wip-us.apache.org/repos/asf/couchdb-ci/blob/b918cfb6/dockerfiles/ubuntu-12.04-erlang-18.3 ---------------------------------------------------------------------- diff --git a/dockerfiles/ubuntu-12.04-erlang-18.3 b/dockerfiles/ubuntu-12.04-erlang-18.3 index e7ccc35..51d8fc3 100644 --- a/dockerfiles/ubuntu-12.04-erlang-18.3 +++ b/dockerfiles/ubuntu-12.04-erlang-18.3 @@ -30,5 +30,3 @@ RUN ansible-playbook ubuntu-12.04-erlang-18.3.yml \ --inventory-file=./inventory/couchdb-ci-worker USER build - -CMD ["/home/build/build-ci.sh"] http://git-wip-us.apache.org/repos/asf/couchdb-ci/blob/b918cfb6/dockerfiles/ubuntu-12.04-erlang-default ---------------------------------------------------------------------- diff --git a/dockerfiles/ubuntu-12.04-erlang-default b/dockerfiles/ubuntu-12.04-erlang-default index 7107075..0c98669 100644 --- a/dockerfiles/ubuntu-12.04-erlang-default +++ b/dockerfiles/ubuntu-12.04-erlang-default @@ -30,5 +30,3 @@ RUN ansible-playbook ubuntu-12.04-erlang-default.yml \ --inventory-file=./inventory/couchdb-ci-worker USER build - -CMD ["/home/build/build-ci.sh"] http://git-wip-us.apache.org/repos/asf/couchdb-ci/blob/b918cfb6/dockerfiles/ubuntu-14.04-erlang-18.3 ---------------------------------------------------------------------- diff --git a/dockerfiles/ubuntu-14.04-erlang-18.3 b/dockerfiles/ubuntu-14.04-erlang-18.3 index 1573a7f..179f9ce 100644 --- a/dockerfiles/ubuntu-14.04-erlang-18.3 +++ b/dockerfiles/ubuntu-14.04-erlang-18.3 @@ -30,5 +30,3 @@ RUN ansible-playbook ubuntu-14.04-erlang-18.3.yml \ --inventory-file=./inventory/couchdb-ci-worker USER build - -CMD ["/home/build/build-ci.sh"] http://git-wip-us.apache.org/repos/asf/couchdb-ci/blob/b918cfb6/dockerfiles/ubuntu-14.04-erlang-default ---------------------------------------------------------------------- diff --git a/dockerfiles/ubuntu-14.04-erlang-default b/dockerfiles/ubuntu-14.04-erlang-default index bbb2d80..507cc38 100644 --- a/dockerfiles/ubuntu-14.04-erlang-default +++ b/dockerfiles/ubuntu-14.04-erlang-default @@ -30,5 +30,3 @@ RUN ansible-playbook ubuntu-14.04-erlang-default.yml \ --inventory-file=./inventory/couchdb-ci-worker USER build - -CMD ["/home/build/build-ci.sh"] http://git-wip-us.apache.org/repos/asf/couchdb-ci/blob/b918cfb6/dockerfiles/ubuntu-16.04-erlang-18.3 ---------------------------------------------------------------------- diff --git a/dockerfiles/ubuntu-16.04-erlang-18.3 b/dockerfiles/ubuntu-16.04-erlang-18.3 index 3c98421..fcf6b08 100644 --- a/dockerfiles/ubuntu-16.04-erlang-18.3 +++ b/dockerfiles/ubuntu-16.04-erlang-18.3 @@ -30,5 +30,3 @@ RUN ansible-playbook ubuntu-16.04-erlang-18.3.yml \ --inventory-file=./inventory/couchdb-ci-worker USER build - -CMD ["/home/build/build-ci.sh"] http://git-wip-us.apache.org/repos/asf/couchdb-ci/blob/b918cfb6/dockerfiles/ubuntu-16.04-erlang-default ---------------------------------------------------------------------- diff --git a/dockerfiles/ubuntu-16.04-erlang-default b/dockerfiles/ubuntu-16.04-erlang-default index bdf8472..85335a8 100644 --- a/dockerfiles/ubuntu-16.04-erlang-default +++ b/dockerfiles/ubuntu-16.04-erlang-default @@ -30,5 +30,3 @@ RUN ansible-playbook ubuntu-16.04-erlang-default.yml \ --inventory-file=./inventory/couchdb-ci-worker USER build - -CMD ["/home/build/build-ci.sh"] http://git-wip-us.apache.org/repos/asf/couchdb-ci/blob/b918cfb6/readme.markdown ---------------------------------------------------------------------- diff --git a/readme.markdown b/readme.markdown deleted file mode 100644 index 166d485..0000000 --- a/readme.markdown +++ /dev/null @@ -1,119 +0,0 @@ -# CouchDB Continuous Integration (CI) support repo - -The main purpose of this repository is to provide a number of Docker containers, Ansible roles/tasks and other configuration functionality so that the ASF Jenkins CI server (https://builds.apache.org/) is capable of building (and eventually packaging) CouchDB for a number of platforms. It intends to cover a range of both operating systems (Linux, macOS, BSD, Windows) and Erlang versions (OS default, 18.x, 19.x, etc.) - -The current configuration builds CouchDB, Fauxton, its documentation, and runs the Erlang and JS test suites for each combination of OS and Erlang revision. - -## Background - -See: -* this [thread](https://www.mail-archive.com/dev%40couchdb.apache.org/msg43591.html) on the couchdb-dev mailing list and -* this [ASF Infra ticket](https://issues.apache.org/jira/browse/INFRA-10126). -for the origins of this work. - -## Supported Configurations (updated 2017-03-22) - -**OS/Erlang** | **default ** | **18.3** ---------------------|--------------|-------------- -**Ubuntu 12.04** | â (R14B04) | â -**Ubuntu 14.04** | â (16B03-1) | â -**Ubuntu 16.04** | â (18.3) | â -**Debian 8** | â (17.3) | â -**Debian 9** | unreleased | unreleased -**CentOS 6** | â (R14B04) | â -**CentOS 7** | â (16B03-1) | â -**macOS 10.12** | - | - -**FreeBSD** | - | - -**Windows** | - | - - -Builds marked with an â are skipped due to the version of Erlang being too old to build CouchDB >= 2.0.0. - - -## Open questions and TODOs -* Right now we run a CouchDB build on all combinations on each commit, but perhaps we don't need to run Jenkins this often. (We also have Travis CI for a single OS and a few Erlang revisions.) Should we just build them once a day? -* Right now we only trigger Jenkins on the master branch. The images support building from whatever branch $GIT_BRANCH is set to (c.f. Jenkins environment variable.) Travis CI currently handles PRs. ASF hasn't set up a Jenkins/Gitbox bridge yet, but if they do we can consider having Jenkins build PRs as well. -* Ideally, we'd also like to build convenience packages for some of these platforms on a regular basis, i.e. nightly - especially the platforms where building the software is harder (Windows). -* Currently, when changes occur to the base images (bugfixes, new OS/Erlang combinations added, etc.) new images are built manually using the scripts in `bin/`. TODO: automate this process in a way that avoids forcibly rebuilding every VM/Erlang combination with every checkin. - ---- - -# Docker - -For those OSes that support Docker, we run builds inside of Docker containers. The Dockerfiles typically use Ansible to handle all of their configuration and setup. This allows us to reuse the Ansible tasks for those OSes not supported by Docker for native apps (such as macOS and FreeBSD) where we must use VMs or bare hardware. The ASF has Windows Jenkins clients as an alternative to running Jenkins on Windows; a decision has not yet been reached here. - -## Building a container - -Run the `bin/<image-name>/create-container.sh` script. - -## Interactively working in a built container - -Run the `bin/<image-name>/enter-container.sh` script. - -## Running the CouchDB build in a locally built (but not published) container - -After using the `create-container.sh` script, run the command `docker run -it couchdbdev/<imagename>`. The build should immediately start. - -## Publishing a container - -1. You need a Docker Cloud account with access to the `couchdbdev` organization. Ask the CouchDB PMC for assistance with this. -2. `export DOCKER_ID_USER="username"` -3. `docker login` and enter your password. -4. Run the `bin/<image-name>/publish-container.sh` script. - -## Running the CouchDB build in a published container - -To pull down the latest image and run the CouchDB build in the container, run the `bin/<image-name>/run-build-in-container.sh` script. This does not work with `-base` images, which lack the full toolchain necessary to build CouchDB. - - -Vagrant -------- - -Note: This section on creating the build machines on Vagrant might be outdated. Either we bring it up to date to have both possibilities (Docker & Vagrant) or we remove it completely and just go with Docker for now. - -### Prerequisites - -See the readme files in folder `baseboxes` for docs on building the base boxes. - -You need to have [Vagrant](https://www.vagrantup.com/) and [VirtualBox](https://www.virtualbox.org/) installed. - -When Vagrant is installed you need to install an additional plug-in: -```bash -vagrant plugin install vagrant-hosts -``` - -plus an additional Ansible role -```bash -[sudo] ansible-galaxy install nodesource.node -``` - -Also, you might need to run -```bash -VBoxManage dhcpserver remove --netname HostInterfaceNetworking-vboxnet0 -``` -before doing -```bash -vagrant up -``` -because of <https://github.com/mitchellh/vagrant/issues/3083>. - -### Building and Registering the Base Box - -Each vagrant configuration requires its respective base box. For example, the configuration jenkins-master requires the base box `couchdb-ci-ubuntu-14.04`. You can build the base boxes locally with veewee (see `baseboxes/readme.markdown`). To make things easier we might upload the base boxes somewhere so people do not have to build them theirselves but can just download them but we are not there yet. - -Either way, you have to add the image to the vagrant base image registry with the following command: -``` -vagrant box add couchdb-ci-ubuntu-14.04 /path/to/your/base-box-file.box -``` - -When this has happened, `vagrant box list` should list the base box's name (`couchdb-ci-ubuntu-14.04` in our example). - -The base box built with veewee is just a fresh minimal install of the OS. All relevant packages and configurations are provisioned with Ansible. Vagrant is used to make creating, provisioning and destroying these virtual machines easier. - -### Launching and Provisioning the Boxes - -Just execute `vagrant up`. This will start all the virtual machine and provision it using the scripts in `ansible`. That's it :-) - -You can also selectively launch a single box, for example with -```bash -vagrant up couchdb-jenkins-master -```
