Hey everyone,
I had a brief private exchange with Garren about Docker, and figured it
would be better if everyone could benefit from the email chain. So here
it is!
The couchdb-docker repository has a self-contained, ready-to-run CouchDB
configuration. The intent for this is to be used in development
scenarios, for instance when the Fauxton team is improving Fauxton and
needs a backing CouchDB against which to test everything. The details
are here:
https://github.com/apache/couchdb-docker
Currently we use Docker a little differently for the Jenkins CI runs.
There is some useful overlap between the two approaches.
Current our CI setup is split across Travis and Jenkins. We use Travis
for (slightly faster) smoke tests across 4 varieties of Erlang, and one
variety of Linux (Ubuntu). The builds run "make check". Each of these
builds takes about 15 minutes, and they tend to run in parallel.
Jenkins, on the other hand, tests within custom-built Docker images
across 2 varieties of Erlang (right now the OS default, and 18.3) and
across multiple OS variants: Debian x1, Ubuntu x3, CentOS x2) and will
soon include macOS and FreeBSD builds. Builds on Jenkins are more
involved - we run 'make dist' to create a tarball, then run 'make check'
from that tarball. The intent is to simulate the full release process.
Builds take about 20 minutes as a result.
The Docker images are provisioned using Ansible to keep setup steps
consistent across different OSes. The details are here:
https://github.com/apache/couchdb-ci
Images are pushed up to the couchdbdev/ namespace at Docker Cloud by
hand, since they don't change very often. I considered having these
under the apache/ Docker namespace but since they are more
dev-consumable rather than end-user consumable it didn't make much
sense. Garren, klaemo and I all have access to the couchdbdev/
organization.
Also, I've added the packaging tools to the images for Debian & Ubuntu,
with CentOS builds not far behind. Most developers won't need these,
either, but we definitely need them if we intend to build packages for
each check-in -- which is a direction I'd like to go.
Note that the Docker remote build stuff is transitioning to a paid
model, see https://docs.docker.com/docker-cloud/builds/automated-build/
and specifically the text "This hosted build service is free while it is
in Beta." (Docker Hub is being replaced by Docker Cloud.) We need to be
careful about leveraging a service that is about to disappear; we may
have to start building those Docker images and publishing them from
Apache Jenkins in the same way that I intend to start publishing daily
packages as well.
I'm very concerned about pushing forward with building anything
automatically (Docker images, snap packages, Debian/Ubuntu/CentOS
packages, etc.) until the test suite is fixed. I've been begging for
help on the mailing lists to no avail. I don't feel right with
suggesting anyone run anything "nightly" or "direct from master" if any
tests fail. We need to eject the failing tests, or better yet, fix them
so they are deterministic. We can't keep limping along as a project with
a test suite that no one trusts.
Apache has a namespace at the Docker Cloud repository. Eventually, we'd
love to see the end-user Docker image show up at apache/couchdb. Infra
will be happy to help with this once we're ready to go with a semi-
official image.
P.S. There is actually a *third* docker, and that's the Dockerfile
contained within the actual couchdb repository itself. klaemo originally
intended this to be used for development work on CouchDB itself. My
proposal is to remove this Dockerfile in favour of the couchdb-ci
images.
Thanks for reading my little essay!
-Joan
