Hi Isuru, +1 for the suggested approach. I think wso2base image will only complement the model that you have described. We can definitely provide guidelines to users so that they can extend from our product specific images (excluding Oracle JDK).
If we have the right layered model, i.e.; wso2base -> wso2product -> wso2product-customized then we can cut down the network traffic when push/pull'ing images to/from Docker registries or even locally save/load'ing images. This is because Docker will pull only the changed top level layer since bottom layers are do not change for a specific Dockerfiles release. Since wso2base is common across all products, this will cut down push/pull time in a multi-product deployment. wdyt? Thanks. On Tue, Jun 21, 2016 at 3:19 PM, Isuru Haththotuwa <isu...@wso2.com> wrote: > Hi Akila, > > If we use a base image, true that we can avoid the time taken to run > apt-get update and software installation. But still the users will have to > run the provisioning again and again when doing development/testing. The > model I suggest is a way to eliminate apt-get update and also the > provisioning over and over again. > > To summarize: > > - Publish a wso2 product specific docker image to a local docker > repository (we can even push to dockerhub, without the JDK distribution and > ask the users to provide the JDK when spinning up a container from the > image). > - Guide the users/developers to create their own Dockerfile extending > from the original wso2 product Docker image. > - Use the extension provided to copy artifacts/configurations to the > container [1]. At development time, ideally the developer would only need > to modify the configurations/copy artifacts to the carbon server. > - After the development phase, collect all the changes and apply them > to the provisioning method (puppet, etc.) that is used, and build the final > production ready Docker image. > > I'm +1 to push a Docker image with a configured product to Dockerhub, > excluding the JDK and ask the user to provide a JDK at runtime/creating a > new Dockerfile extending from the original product Dockerfile. > > [1]. > https://docs.wso2.com/display/DF110/Running+WSO2+Docker+Images#RunningWSO2DockerImages-Copyingartifactsfromhostmachinetothecontainer > > On Mon, Jun 20, 2016 at 12:37 PM, Akila Ravihansa Perera < > raviha...@wso2.com> wrote: > >> Hi Isuru, >> >> Do you have any specific concerns about using a wso2base image? On >> average there is a spike of 100MB when using a wso2base image, but that's a >> small price to pay when considering benefits users would gain out of this. >> Note that users can use "default" provisioning method to package any >> jars/configs to the image. The main problem here is the time taken to run >> apt-get update and package installation. Especially having to download >> files over I'net every time a Docker image is built which I think is >> unnecessary. wdyt? >> >> Thanks. >> >> On Wed, Jun 1, 2016 at 10:31 PM, Isuru Haththotuwa <isu...@wso2.com> >> wrote: >> >>> Hi Akila and Imesh, >>> >>> Without creating a base image, if we package everything in a product >>> specific image, host it in a local registry, and provide instructions to >>> create a local Dockerfile extending from that image, the time taken to >>> build the local image would be lesser. This will speedup the >>> development/testing procedure even more. The developers can package any >>> apps/jars/configs via the local Dockerfile (using volume mounts, etc.), for >>> testing. I have seen this pattern used in some Docker images, for an >>> example in the haproxy Docker image [1]. >>> >>> One downside is, as you mentioned, we cannot push the image to Dockerhub >>> with Oracle JDK distribution included. Maybe we can create the product >>> Docker image with the relevant utilities (wget, zip, unzip, etc.) + the >>> product distribution, and push it to Docker hub. Then provide an option to >>> package the JDK via a local Dockerfile extending from the product >>> Dockerfile. >>> >>> WDYT? >>> >>> [1]. https://hub.docker.com/_/haproxy/ >>> >>> On Mon, May 30, 2016 at 12:48 PM, Akila Ravihansa Perera < >>> raviha...@wso2.com> wrote: >>> >>>> Hi, >>>> >>>> Currently we don't have a base image for WSO2 Docker images due to >>>> image size issues we had with it. But later we found out that this was >>>> mainly due to multiple RUN commands, which caused Docker to add unnecessary >>>> layers. >>>> >>>> I think we should bring back the base image for couple of reasons; >>>> >>>> 1. wso2base image with minimum packages installed (wget, puppet etc) >>>> will eliminate the need for having to run apt-get commands (which are very >>>> expensive, time-wise) each and every time >>>> >>>> 2. wso2base image can be hosted publicly (without Oracle JDK of course) >>>> so that users don't have to build wso2base but rather pull it from >>>> DockerHub. >>>> >>>> 3. wso2base image provides a Docker layer which reduces the amount of >>>> traffic carried when pulling images from a Docker registry over a long >>>> period of time. This is because base image is very unlikely to be changed, >>>> but product images can change very rapidly (due to artifacts or >>>> configuration change, product version etc.). It is unnecessary to carry the >>>> information in the lower level layers each time a product image is changed. >>>> >>>> If you must use a single image, I found this cool Docker tool written >>>> in Python which can be used to squash Docker layers called docker-squash >>>> [1]. Following is a comparison of Docker images built with/without wso2base >>>> and squashed images; >>>> >>>> [image: Inline image 3] >>>> >>>> In summary; >>>> wso2am-1.10.0 with base image: 1.079GB >>>> wso2am-1.10.0 with base image and *squashed*: 948MB >>>> wso2am-1.10.0 *without* base image: 969MB >>>> >>>> wso2esb-4.9.0 with base image: 794MB >>>> wso2esb-4.9.0 with base image and *squashed*: 664MB >>>> wso2esb-4.9.0 *without* base image: 685MB >>>> >>>> In conclusion, introducing a base image with minimum packages has a >>>> image size overhead of around 100+MB, but having individual images without >>>> a hierarchy can cause a significant amount of unnecessary information being >>>> transferred over the network when pull/push'ing to/from a registry. >>>> >>>> I've created a PR for this feature at [2]. Also used >>>> "--no-install-recommends" option to apt-get when installing packages to >>>> prevent unnecessary packages being installed. Please share your thoughts. >>>> >>>> [1] https://github.com/goldmann/docker-squash >>>> [2] https://github.com/wso2/dockerfiles/pull/46/files >>>> >>>> Thanks. >>>> -- >>>> Akila Ravihansa Perera >>>> WSO2 Inc.; http://wso2.com/ >>>> >>>> Blog: http://ravihansa3000.blogspot.com >>>> >>>> _______________________________________________ >>>> Dev mailing list >>>> Dev@wso2.org >>>> http://wso2.org/cgi-bin/mailman/listinfo/dev >>>> >>>> >>> >>> >>> -- >>> Thanks and Regards, >>> >>> Isuru H. >>> +94 716 358 048* <http://wso2.com/>* >>> >>> >>> >> >> >> -- >> Akila Ravihansa Perera >> WSO2 Inc.; http://wso2.com/ >> >> Blog: http://ravihansa3000.blogspot.com >> > > > > -- > Thanks and Regards, > > Isuru H. > +94 716 358 048* <http://wso2.com/>* > > > -- Akila Ravihansa Perera WSO2 Inc.; http://wso2.com/ Blog: http://ravihansa3000.blogspot.com
_______________________________________________ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
