I am not sure about this approach. I would keep a separate script which compile our code and create a docker image and keep a clean docker image, Not git, No source, No maven, only .jar files and required configuration files. Or use a maven plugin to create the binary for us.
On Mon, Jul 6, 2015 at 11:35 AM, Shameera Rathnayaka <[email protected] > wrote: > Hi Suresh, > > Shall we create an docker module like distribution and put all dockers to > that module as Lahiru suggested? > > Hi Lahiru/Suresh, > This is how i am thinking this docker releases works, we will have > snapshot dockerfiles which based on ubuntu latest and have all tools(git , > maven , etc) installed on it, we download from git and build airavata > source and start airavata server. Once we come to release point, let's say > 16 release (next release from master, here I assume 0.15 has been released > with it's own docker files) then with release process we will > release(publish) official docker images(based on Ubuntu 14.04 or suitable) > and docker files which don't have git , maven , etc...installation commands > and download released airavata distribution from one of Appache mirror > server. We will maintain snapshot version of dockerfile with Airavata > source code. WDYT? > > Hi Pankaj, > > You may have confused with docker maven plugin which we are talking above, > that is only for building purpose. you don't need to worry about this. I > have few comments about your docker files let me reply it separately. > > Thanks, > Shameera. > > On Mon, Jul 6, 2015 at 2:14 PM Shameera Rathnayaka <[email protected]> > wrote: > >> Hi Lahiru, >> >> I am ok to use any official docker images, but I couldn't find any official >> java 8 image released by Oracle. There are set of official images for open >> jdk but not for oracle jdk. In that case I would like to suggest to use >> above steps to install java8. It is clean and no any unofficial >> dependencies. I am not much familiar how docker world works, this is my 2 >> cents. >> >> Thanks, >> Shameera. >> >> On Mon, Jul 6, 2015 at 1:43 PM Lahiru Ginnaliya Gamathige < >> [email protected]> wrote: >> >>> I have use sbt docker because we use sbt instead of maven to build java >>> code. I think we can use this[1] for maven. >>> >>> [1]https://github.com/rhuss/docker-maven-plugin >>> >>> I think its worth exploring this since we already have a maven build. >>> >>> Lahiru >>> >>> On Mon, Jul 6, 2015 at 10:39 AM, Suresh Marru <[email protected]> wrote: >>> >>>> Lahiru, >>>> >>>> Did you come across any maven plugins like assembly ones so we can >>>> bundle all docker things with the main build? There seems to be few github >>>> projects but could not weigh in which one is good. >>>> >>>> Suresh >>>> >>>> >>>> On Mon, Jul 6, 2015 at 1:26 PM Lahiru Ginnaliya Gamathige < >>>> [email protected]> wrote: >>>> >>>>> If we use docker for deployment purpose there is no requirement for >>>>> bundling source code in to docker image. We usually bundle minimum amount >>>>> of data (everything required in runtime) but nothing other than that. >>>>> During development we use a script which build the docker image for us >>>>> using exiting source in our local file system. Once we do a release we >>>>> have >>>>> docker images with proper naming which will not touch. >>>>> >>>>> Shameera, >>>>> >>>>> I am not sure why we need to install java or maven without using >>>>> existing docker image provided by Oracle. >>>>> >>>>> Lahiru >>>>> >>>>> On Mon, Jul 6, 2015 at 9:52 AM, Suresh Marru <[email protected]> >>>>> wrote: >>>>> >>>>>> Hi Pankaj, >>>>>> >>>>>> Thinking naively, I am considering docker images same as maven >>>>>> artifacts. Users will consume the published images. We will need to >>>>>> publish >>>>>> released docker images and also nightly builds. >>>>>> >>>>>> From developers perspective, I am guessing they will again work >>>>>> similar to maven. We do not want to push to a docker hub, but create and >>>>>> use the image locally. >>>>>> >>>>>> Let me defer to Lahiru to update us how docker is used in real-world >>>>>> by developers. I am guessing as long as we can build docker images with >>>>>> the >>>>>> maven build (through ant commands probably), it should probably help both >>>>>> the cases. >>>>>> >>>>>> Suresh >>>>>> >>>>>> >>>>>> On Mon, Jul 6, 2015 at 12:25 PM Pankaj Saha <[email protected]> >>>>>> wrote: >>>>>> >>>>>>> Hi Suresh >>>>>>> >>>>>>> Our initial plan was to make docker for the airavata developers so >>>>>>> that the initial hurdle of environment set up can be reduced. >>>>>>> If we don't include the source code and only keep the executable >>>>>>> binaries then how can we fulfill that goal? >>>>>>> do we need two set of docker images 1. with source code (dev >>>>>>> edition) and 2. without source code (end user edition) ? >>>>>>> please suggest. >>>>>>> >>>>>>> Thanks >>>>>>> Pankaj >>>>>>> >>>>>>> >>>>>>> On Mon, Jul 6, 2015 at 12:10 PM, Pankaj Saha <[email protected]> >>>>>>> wrote: >>>>>>> >>>>>>>> Hey Shameera, >>>>>>>> what about the PATH variable? If you see my dockerfile I have >>>>>>>> commented the ENV for the JAVA_HOME and PATH. It did not work for me. >>>>>>>> Lahiru suggested to use existing jdk8+maven base image which is >>>>>>>> better idea , what do you think? >>>>>>>> >>>>>>>> On Mon, Jul 6, 2015 at 12:03 PM, Shameera Rathnayaka < >>>>>>>> [email protected]> wrote: >>>>>>>> >>>>>>>>> Hi Pankaj, >>>>>>>>> >>>>>>>>> you can use following steps on your Dockerfile to install java. >>>>>>>>> After that java will be available anywhere and you don't need to set >>>>>>>>> PATH >>>>>>>>> variable , But you need to set JAVA_HOME variable which is used by >>>>>>>>> airavata >>>>>>>>> start script. >>>>>>>>> >>>>>>>>> >>>>>>>>> # Install prerequisites >>>>>>>>> RUN apt-get update >>>>>>>>> RUN apt-get install -y software-properties-common >>>>>>>>> >>>>>>>>> # Install java8 >>>>>>>>> RUN add-apt-repository -y ppa:webupd8team/java >>>>>>>>> RUN apt-get update >>>>>>>>> RUN echo oracle-java8-installer >>>>>>>>> shared/accepted-oracle-license-v1-1 select true | sudo >>>>>>>>> /usr/bin/debconf-set-selections >>>>>>>>> RUN apt-get install -y oracle-java8-installer >>>>>>>>> >>>>>>>>> # set JAVA_HOME >>>>>>>>> ENV JAVA_HOME /usr/lib/jvm/java-8-oracle >>>>>>>>> >>>>>>>>> >>>>>>>>> Regards, >>>>>>>>> Shameera. >>>>>>>>> >>>>>>>>> On Mon, Jul 6, 2015 at 11:56 AM Pankaj Saha <[email protected]> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> Hi Lahiru, >>>>>>>>>> >>>>>>>>>> 1. So you suggest to use the "java8 with maven" as the base >>>>>>>>>> image. >>>>>>>>>> dockerfile/java:oracle-java8 is not based on ubuntu image and >>>>>>>>>> also its not official image from oracle, can we use still use that? >>>>>>>>>> >>>>>>>>>> 2. Do you suggest to put only the executables for airavata inside >>>>>>>>>> the docker image? Does that mean docker file will copy the local host >>>>>>>>>> machine's airavata executable files to the docker image through ADD >>>>>>>>>> directive? >>>>>>>>>> So I will build the airavata source code locally through maven >>>>>>>>>> and then put that inside the container image. >>>>>>>>>> >>>>>>>>>> 3. One thing that I could not able to do was changing the >>>>>>>>>> properties values for different port and server ips when the >>>>>>>>>> containers >>>>>>>>>> starts. I was trying to run a startup shell script which can do the >>>>>>>>>> required changes and then kick start the airavat-server.sh file. >>>>>>>>>> problem was airavata-server.properties need apiserver.server.host >>>>>>>>>> = current running containers ip address. Can you suggest some better >>>>>>>>>> way to >>>>>>>>>> achieve this through shell script. >>>>>>>>>> >>>>>>>>>> Thanks >>>>>>>>>> Pankaj >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Mon, Jul 6, 2015 at 11:30 AM, Lahiru Ginnaliya Gamathige < >>>>>>>>>> [email protected]> wrote: >>>>>>>>>> >>>>>>>>>>> Hi Pankaj, >>>>>>>>>>> >>>>>>>>>>> I had a quick look. I think you can replace all the code to >>>>>>>>>>> install java and maven if you use this image [1] or at least java-8 >>>>>>>>>>> image. >>>>>>>>>>> >>>>>>>>>>> I think its not necessary to build airavata inside the >>>>>>>>>>> dockerfile, you can just assume its built in host machine and you >>>>>>>>>>> just have >>>>>>>>>>> to put required executables in to the docker image. >>>>>>>>>>> When you create he image maven and source code it not necessary >>>>>>>>>>> at all. Docker image should only contain required binaries, source >>>>>>>>>>> code >>>>>>>>>>> doesn't have to in the image. >>>>>>>>>>> >>>>>>>>>>> [1] >>>>>>>>>>> https://registry.hub.docker.com/u/jamesdbloom/docker-java8-maven/dockerfile/ >>>>>>>>>>> >>>>>>>>>>> On Mon, Jul 6, 2015 at 8:13 AM, Pankaj Saha < >>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>> >>>>>>>>>>>> I am so sorry , I have no intention to hide any docker code.I >>>>>>>>>>>> am attaching the docker files here.I have never done git check in >>>>>>>>>>>> and >>>>>>>>>>>> trying to upload files and its not working. I am trying to >>>>>>>>>>>> understand the >>>>>>>>>>>> git pull request/setting up git directory etc. >>>>>>>>>>>> >>>>>>>>>>>> My apology if I unnecessary delayed it. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On Mon, Jul 6, 2015 at 10:51 AM, Suresh Marru < >>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> Hi Pankaj, >>>>>>>>>>>>> >>>>>>>>>>>>> Open source software can never be final. Once the software is >>>>>>>>>>>>> final, the community is dead, because there is nothing for anyone >>>>>>>>>>>>> to do. An >>>>>>>>>>>>> important aspect for all the GSoC students to embrace is, these >>>>>>>>>>>>> are not >>>>>>>>>>>>> academic projects where you submit the final code. You are >>>>>>>>>>>>> supposed to >>>>>>>>>>>>> commit every few hours of code, you need to do the development in >>>>>>>>>>>>> open. We >>>>>>>>>>>>> have been lenient on this so far, but you have been holding off >>>>>>>>>>>>> the docker >>>>>>>>>>>>> work little too long. Please do commit the docker files in as in >>>>>>>>>>>>> condition. >>>>>>>>>>>>> Thats the only way you can engage others, otherwise you will >>>>>>>>>>>>> loose everyone >>>>>>>>>>>>> interest. >>>>>>>>>>>>> >>>>>>>>>>>>> Suresh >>>>>>>>>>>>> >>>>>>>>>>>>> On Mon, Jul 6, 2015 at 10:42 AM, Lahiru Ginnaliya Gamathige < >>>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> Not sure what is the big deal here, Why don't you create a >>>>>>>>>>>>>> pull request and every one can review it. It doesn't have to be >>>>>>>>>>>>>> finalized >>>>>>>>>>>>>> at all. People will comment and you can follow the comments and >>>>>>>>>>>>>> after >>>>>>>>>>>>>> multiple reviews we can move ti to the the code base. >>>>>>>>>>>>>> >>>>>>>>>>>>>> I propose to create a folder struction like this and put the >>>>>>>>>>>>>> docker files. >>>>>>>>>>>>>> >>>>>>>>>>>>>> airavata/deploy/thrift, airavata/deploy/rabbitmq, >>>>>>>>>>>>>> airavata/deploy/zookeeper. Create a directory for every image >>>>>>>>>>>>>> you build >>>>>>>>>>>>>> with proper naming and add the Dockerfile and other scripts if >>>>>>>>>>>>>> there's any. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Lahiru >>>>>>>>>>>>>> >>>>>>>>>>>>>> On Mon, Jul 6, 2015 at 7:03 AM, Pankaj Saha < >>>>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>>> This docker file is not final yet as we have to automate the >>>>>>>>>>>>>>> environment variable to replace the properties file. >>>>>>>>>>>>>>> I believe in a hangout session we can walk through and fix >>>>>>>>>>>>>>> this. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On Mon, Jul 6, 2015 at 9:58 AM, Shameera Rathnayaka < >>>>>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Hi Pankaj, >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Let't do this via Apache way, upload these files to >>>>>>>>>>>>>>>> publicly accessible place (as i mentioned in my previous >>>>>>>>>>>>>>>> reply) then invite >>>>>>>>>>>>>>>> everyone to review it. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Regards, >>>>>>>>>>>>>>>> Shameera. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> On Mon, Jul 6, 2015 at 9:44 AM Pankaj Saha < >>>>>>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Hi Shameera, >>>>>>>>>>>>>>>>> Please find the attached Dockerfile for Airavata Server. >>>>>>>>>>>>>>>>> Please suggest changes. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> On Thu, Jul 2, 2015 at 5:50 PM, Shameera Rathnayaka < >>>>>>>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Hi Pankaj, >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Yes please share your works with community. Do you have >>>>>>>>>>>>>>>>>> specific location to commit your works? If not you can use >>>>>>>>>>>>>>>>>> this location >>>>>>>>>>>>>>>>>> https://github.com/apache/airavata-sandbox. Let's try >>>>>>>>>>>>>>>>>> to track the changes with VCS. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> @Suresh do we have specific location for this time gsoc >>>>>>>>>>>>>>>>>> students? >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>> Shameera. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> On Thu, Jul 2, 2015 at 5:24 PM Pankaj Saha < >>>>>>>>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Hi Shameera >>>>>>>>>>>>>>>>>>> Yes we can see the containers ip they way you suggested. >>>>>>>>>>>>>>>>>>> What I mentioned is another way in case you are already >>>>>>>>>>>>>>>>>>> inside the >>>>>>>>>>>>>>>>>>> container. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Regarding java path variable yes we can do that with ENV >>>>>>>>>>>>>>>>>>> in docker file. It was throwing some errors so i just >>>>>>>>>>>>>>>>>>> commented that in the >>>>>>>>>>>>>>>>>>> docker file. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> I will send you the docker file in another email then we >>>>>>>>>>>>>>>>>>> can go through together to improve it. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> I was trying mesos today and didnt have time to fix >>>>>>>>>>>>>>>>>>> that. We can do it later as we are already running late for >>>>>>>>>>>>>>>>>>> the mesos >>>>>>>>>>>>>>>>>>> implementation. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Thanks >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Pankaj >>>>>>>>>>>>>>>>>>> On 02-Jul-2015 5:12 PM, "Shameera Rathnayaka" < >>>>>>>>>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Hi Pankaj, >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> See my comments inline. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> docker attach server >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> awk 'NR==1 {print $1}' /etc/hosts ---- note the ip >>>>>>>>>>>>>>>>>>>>>> address of the running container >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> can't we use "docker inspect --format '{{ >>>>>>>>>>>>>>>>>>>> .NetworkSettings.IPAddress }}' server " to get airavata >>>>>>>>>>>>>>>>>>>> server docker >>>>>>>>>>>>>>>>>>>> container ip ? >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> pico airavata-server.properties >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Replaces the below text as suggested below >>>>>>>>>>>>>>>>>>>>> apiserver.server.host= *<current containers ip >>>>>>>>>>>>>>>>>>>>> address>* >>>>>>>>>>>>>>>>>>>>> default.registry.gateway=*default* >>>>>>>>>>>>>>>>>>>>> rabbitmq.broker.url=amqp://*<rabbit MQ's ip address>* >>>>>>>>>>>>>>>>>>>>> :5672 >>>>>>>>>>>>>>>>>>>>> save changes. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> pico /etc/bash.bashrc >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> add below lines at the end >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> #JAVA_HOME >>>>>>>>>>>>>>>>>>>>>> JAVA_HOME=/opt/jdk/jdk1.8.0_05 >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> You can improve you docker file to add this variables >>>>>>>>>>>>>>>>>>>> using ENV command. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> export JAVA_HOME >>>>>>>>>>>>>>>>>>>>>> PATH=$JAVA_HOME/bin:$PATH >>>>>>>>>>>>>>>>>>>>>> export PATH >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> save changes. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> How you install java 8?, There is a way to install java >>>>>>>>>>>>>>>>>>>> on docker which automatically available in PATH variable. >>>>>>>>>>>>>>>>>>>> You don't have to >>>>>>>>>>>>>>>>>>>> set it here. It is good if you can show us the Dockerfiles >>>>>>>>>>>>>>>>>>>> then we can >>>>>>>>>>>>>>>>>>>> review those files. is it available online ? >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>> Shameera. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>> >>>>>>> >>>>> >>>
