Thanks Roman for the detailed information! Great suggestion to extract the setup logic into a separate script. This will make the script usable in other environment and also make the dockerfile easier to maintain.
Very glad to have a formal hawq account on docker hub. Thank you for this. Best Regards Richard On Thu, Dec 29, 2016 at 3:43 AM, Roman Shaposhnik <[email protected]> wrote: > Hi! > > On Tue, Dec 27, 2016 at 9:55 PM, Richard Guo <[email protected]> > wrote: > > Hi everyone, > > > > HAWQ team is building HAWQ dev docker images. The purpose is to provide > an > > out-of-box way for developers to setup build and test environment for > HAWQ. > > First of all I think it is a great idea and will go a long way of > simplifying the developer > experience. > > Based on my observations of how Docker containers are used for build > and development > purposes by various other projects at ASF I'd like to share some of my > experience: > * while it may seem counterintuitive, Dockerfiles (for various > supported platforms) are > much more important than a docker container on Docker Hub. This > is partially because > despite all the deafening hype not everybody is in love with > Docker as much as > the tech press would make you believe. Having a Dockerfile lets > folks who are not > using Docker recreate a build environment based on what's in > there as part of RUN, etc. > Which brings me to my next point: > > * extract your actual environment setup logic into a script that > can easily be run outside > of Docker environment, then copy this script into a Docker > container and RUN it. See > this example from Docker best practices: > https://docs.docker.com/engine/userguide/eng-image/ > dockerfile_best-practices/#/add-or-copy > > * make sure to use your containers for actual builds on > builds.apache.org otherwise they > tend to bitrot. > > > A demo could be found on github hawq-docker > > <https://github.com/guofengrichard/hawq-docker> . Currently only CentOS > 7 > > is supported. CentOS 6 will be supported soon. It is based on Zhanwei's > > work. > > This is a good start that needs to be folded into the HAWQ code base itself > and ideally hooked to the top level Make logic. > > Also, if you could consider separating the logic into a separate script > that'd > be super awesome and would allow us for a much better integration with > downstream consumers such as Linux distributions and Bigtop. > > > The idea is to predefine all the environment setup steps in the > dockerfile > > and then build the image from dockerfile with tools provided by docker. > > After that, users can simply create containers with the docker image and > > then do the HAWQ build and test jobs. > > Also a Makefile is provided to simplify this process. Please refer to the > > README in the github repository for more details. > > > > Regarding the place to host the docker images and dockerfiles, does > anyone > > have any idea? Comments and discussions are welcomed. > > These types of build environments are full of GPL things and thus need to > be > kept separately from ASF artifacts. I suggest the same approach we used in > Bigtop: a community owned, public GitHub repo. E.g.: > https://hub.docker.com/u/bigtop/ > > Initially I'd suggest a pretty tight ownership of who can modify > containers in your > account. The best way we found so far is to have a single volunteer > who establishes > credentials and makes sure to push containers into the account everytime > there's > a change. If you couple it with setting the email on the account to > [email protected] > you've got yourself a pretty secure AND community friendly setup. > > In fact, while checking if hawq was available on Docker hub I took the > liberty of creating > an account for you ;-) Let me know if you like it: > https://hub.docker.com/u/hawq/ > > Thanks, > Roman. >
