Oops, I've just realised that I accidentally PM'd this reply to Jakub
rather than the group (sorry Jakub, finger trouble!)
Re: "I think the small images have their pros and cons. When I run into
problems I find it always great to have something like CentOS, Fedora or
Ubuntu as the base image. You can easily install many tools to debug the
problems etc."
Sure there are trades. With what I did the build system Dockerfile
actually creates an executable qpidd too so you *could* use that and do
the sort of things you allude to too, but I prefer to create minimal
images as I feel it's a cleaner approach and I'd much rather operational
images expose as minimal a surface as possible.
I'd generally go with Alpine as a base OS if I had to use a base OS, but
I've not got Qpid compiling on Alpine yet and as I managed to get a
minimal OS-less Image for qpidd working I didn't pursue it.
IMHO there are far too many fat, bloaty and "kind-of-random" images
floating around on Docker Hub and have a hunch all those miscellaneous,
and usually unmaintained and unpatched base images sitting atop
ostensibly official application images is an accident waiting to happen.
As I said in my reply to Irina I think that there's a lot to be said for
having well understood provenance and good versioning, which is
something I see precious little of on Docker Hub, call me old fashioned .
Frase
On 13/06/17 20:39, Jakub Scholz wrote:
I think the small images have their pros and cons. When I run into problems
I find it always great to have something like CentOS, Fedora or Ubuntu as
the base image. You can easily install many tools to debug the problems
etc. And often the size doesn't have to be that big. It depends how minimal
the base image is.
BTW: I have also created my own build system (
https://github.com/scholzj/build-qpid-cpp and
https://github.com/scholzj/build-qpid-dispatch). The great thing is that
with Docker you can run the build anywhere including TravisCI or CircleCI.
I think this is another area which we might improve on as community. To
have also some nightly RPMs etc.
Regards
Jakub
On Tue, Jun 13, 2017 at 5:36 PM, Fraser Adams <[email protected]
wrote:
Hi Irina,
I think that this is an excellent idea and I totally agree with the
sentiment "Availability of Qpid images will make it easier to
consume/deploy Qpid components and promote Qpid visibility." I think that
in 2017 it's really important for the project to be providing official
maintained images as otherwise we'll continue to see a fragmented
miscellany of images of unknown provenance growing up out of necessity. To
be fair that seems to be a general trend on Docker Hub for many projects,
but I personally tend to find it a bit frustrating and would prefer a bit
more control over provenance and versioning for things used on production
systems.
I'd make a few comments though.
1. It's probably better to propose this on the users list as that tends to
have more traffic than the dev list. If you choose to repost on the users
list feel free to include this response (if it's useful).
2. I do wonder about the comment "We can maintain docker scripts creating
these images from the base OS images and using Qpid installation methods
consistent with the OS distribution" and "I would look to support
Fedora/CentOS latest images" and "expand OS coverage for debian/Ubuntu/etc".
I can see why this might be the /obvious/ approach, but IMHO I think that
it's a somewhat poor anti-pattern for containerisation that is propagated
way too often on Docker Hub and tends to result in large bloaty images with
larger potential attack surfaces than necessary and in addition reinforces
a model that containers are somehow "lightweight VMs" rather than
containerised appliances. I very much prefer a micro-containerised approach
using an approach similar to that descrived here:
http://blog.oddbit.com/2015/02/05/creating-minimal-docker-images/ though
acknowledging that this approach adds some complexity to the build process
and requires a little care.
Ultimately reducing image size from the 400-500MB range to something like
55MB for qpidd feels like a good thing for a number of reasons, the
following article contains some interesting musings on the subject
https://opensolitude.com/2015/05/13/docker-images-should-be-small.html.
It's a bit more complex for Python and Java based components, but I think
that basing components on lighter weight Alpine Linux based Java or Python
runtimes is probably the best way to go, I note that Docker Hub is (slowly)
moving towards Alpine and http://www.iron.io/ have built some uber tiny
Docker images for a wide range of languages see
https://github.com/iron-io/dockers and https://hub.docker.com/u/iron/
3. One question to ask is which Qpid versions would be supported by the
build system. Obviously one advantage of containerisation is the ability to
easily use multiple versions of an application in order to support
regression testing and potentially legacy capabilities in an operational
environment, but of course there is a price to be paid as older versions
run into difficulties with newer compilers, boost versions etc.
I had a bit of a go at creating a build system for qpidd, my intention has
been to do the same for other components, but I've been stuck for time, see
https://github.com/fadams/docker-qpid though the qpid-cpp link is the
only one with any useful content.
The approach that I took was to create a Dockerised build system based on
Debian so that the only dependency to actually build the component is a
working Docker install and an Internet connection, the build system
installs the necessary dev packages then installs the requested qpid-cpp
source, runs the Qpid build then it exports the relevant shared libraries
etc. and dynamic linker in order that a separate Dockerfile can use these
to create a small image containing only what is necessary for qpidd to
execute.
I created two alternative build systems, one based on Debian wheezy the
other based on jessie, the former was somewhat simpler as wheezy has
somewhat older gcc and boost packages so less "wrangling" was necessary to
build really old Qpid versions, with jessie I had to build gcc and boost
from source. If you only intent to build recent Qpid versions much of that
complexity is unnecessary as recent Qpid versions compile with recent gcc
and boost versions, but I'm a little bit of a glutton for punishment and it
seemed a bit more of a challenge to try and create something that could
build every qpidd version from 0.5 to trunk. I suspect that the best
approach might be to use jessie with the default compiler for recent Qpids
and only use wheezy for older ones to provide legacy support.
I'm no bash expert so the build.sh used by the Dockerfiles is a little bit
monolithic and could do with something of a refactor into functions, but
hopefully what it's doing should be fairly clear.
Hope this is useful.
Best regards,
Frase
On 13/06/17 15:13, Irina Boverman wrote:
Hi everyone,
I would like to propose creating Docker images for Qpid components hosted
in Docker Hub, updated upon component release and maintained by the
project, and I would like to contribute to doing this.
Availability of Qpid images will make it easier to consume/deploy Qpid
components and promote Qpid visibility.
We can maintain docker scripts creating these images from the base OS
images and using Qpid installation methods consistent with the OS
distribution. A possible naming convention might be qpid/<component>/<OS>.
I registered the 'qpid' user on DockerHub to use if this seems reasonable.
For example, we could create qpid/dispatch/<OS> image, qpid/<broker>/<OS>
image, qpid/<client(s)>/<OS> image, etc. Initially I would look to support
Fedora/CentOS latest images and Qpid components as RPMs for them, then aim
to expand OS coverage for debian/Ubuntu/etc in the future.
The goal would be to update Qpid images within a few days upon component
release (either directly or indirectly using yum/dnf from public
repositories). We could ask the Docker team to grant Qpid "official"
status
when images have been stabilized.
--
Regards, Irina.
