Repository: zeppelin Updated Branches: refs/heads/branch-0.7 7382c1c62 -> e3071d66f
[ZEPPELIN-2492] Use single Dockerfile for each tag (branch-0.7) ### What is this PR for? **This PR is created for branch-0.7** **https://github.com/apache/zeppelin/pull/2318 was already merged into master** Use single `Dockerfile` for each release since [apache infra uses tag pushes](https://issues.apache.org/jira/browse/INFRA-12781) to build an image. - https://issues.apache.org/jira/browse/INFRA-12781 After release process finishes, dockerhub will build using the pushed tag. ### What type of PR is it? [Improvement] ### What is the Jira issue? [ZEPPELIN-2492](https://issues.apache.org/jira/browse/ZEPPELIN-2492) ### How should this be tested? 1. `./dev/change_zeppelin_version.sh 0.8.0-SNAPSHOT 0.7.1` 2. Check that version is properly set: `vi scrtips/docker/zeppelin/bin/Dockerfile` 3. Build docker image `cd scripts/docker/zeppelin/bin; docker build -t zeppelin:0.7.1 ./` 4. Run the image: `docker run -p 8080:8080 --rm --name zeppelin zeppelin:0.7.1` ### Screenshots (if appropriate) NONE ### Questions: * Does the licenses files need update? - NO * Is there breaking changes for older versions? - NO * Does this needs documentation? - NO Author: 1ambda <[email protected]> Closes #2388 from 1ambda/ZEPPELIN-2492/use-single-dockerfile-for-each-tag-branch07 and squashes the following commits: 64b3fe06 [1ambda] docs: Update docker.md 14710963 [1ambda] feat: Add dockerfile for tag push Project: http://git-wip-us.apache.org/repos/asf/zeppelin/repo Commit: http://git-wip-us.apache.org/repos/asf/zeppelin/commit/e3071d66 Tree: http://git-wip-us.apache.org/repos/asf/zeppelin/tree/e3071d66 Diff: http://git-wip-us.apache.org/repos/asf/zeppelin/diff/e3071d66 Branch: refs/heads/branch-0.7 Commit: e3071d66f0acdfe62ad62c794e53e14956ab2a63 Parents: 7382c1c Author: 1ambda <[email protected]> Authored: Fri Jun 2 15:24:38 2017 +0900 Committer: Jongyoul Lee <[email protected]> Committed: Mon Jun 5 02:07:00 2017 +0900 ---------------------------------------------------------------------- dev/change_zeppelin_version.sh | 3 + docs/install/docker.md | 45 +++++------- scripts/docker/zeppelin-base/Dockerfile | 42 ----------- scripts/docker/zeppelin/bin/Dockerfile | 106 +++++++++++++++++++++++++++ 4 files changed, 127 insertions(+), 69 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/zeppelin/blob/e3071d66/dev/change_zeppelin_version.sh ---------------------------------------------------------------------- diff --git a/dev/change_zeppelin_version.sh b/dev/change_zeppelin_version.sh index c12dd81..bbca9d7 100755 --- a/dev/change_zeppelin_version.sh +++ b/dev/change_zeppelin_version.sh @@ -58,6 +58,9 @@ sed -i '' 's/-'"${FROM_VERSION}"'.jar",/-'"${TO_VERSION}"'.jar",/g' zeppelin-exa sed -i '' 's/"version": "'"${FROM_VERSION}"'",/"version": "'"${TO_VERSION}"'",/g' zeppelin-web/src/app/tabledata/package.json sed -i '' 's/"version": "'"${FROM_VERSION}"'",/"version": "'"${TO_VERSION}"'",/g' zeppelin-web/src/app/visualization/package.json +# Change version in Dockerfile +sed -i '' 's/Z_VERSION="'"${FROM_VERSION}"'"/Z_VERSION="'"${TO_VERSION}"'"/g' scripts/docker/zeppelin/bin/Dockerfile + # When preparing new dev version from release tag, doesn't need to change docs version if is_dev_version "${FROM_VERSION}" || ! is_dev_version "${TO_VERSION}"; then # When prepare new rc for the maintenance release http://git-wip-us.apache.org/repos/asf/zeppelin/blob/e3071d66/docs/install/docker.md ---------------------------------------------------------------------- diff --git a/docs/install/docker.md b/docs/install/docker.md index c7b6f69..a7e8516 100644 --- a/docs/install/docker.md +++ b/docs/install/docker.md @@ -19,7 +19,7 @@ limitations under the License. --> {% include JB/setup %} -# Apache Zeppelin Releases Docker Images +# Docker Image for Apache Zeppelin Releases <div id="toc"></div> @@ -27,44 +27,35 @@ limitations under the License. This document contains instructions about making docker containers for Zeppelin. It mainly provides guidance into how to create, publish and run docker images for zeppelin releases. ## Quick Start + ### Installing Docker You need to [install docker](https://docs.docker.com/engine/installation/) on your machine. -### Creating and Publishing Zeppelin docker image -* In order to be able to create and/or publish an image, you need to set the **DockerHub** credentials `DOCKER_USERNAME, DOCKER_PASSWORD, DOCKER_EMAIL` variables as environment variables. - -* To create an image for some release use : -`create_release.sh <release-version> <git-tag>`. -* To publish the created image use : -`publish_release.sh <release-version> <git-tag>` - -### Running a Zeppelin docker image +### Running docker image -* To start Zeppelin, you need to pull the zeppelin release image: ``` -docker pull ${DOCKER_USERNAME}/zeppelin-release:<release-version> - -docker run --rm -it -p 7077:7077 -p 8080:8080 ${DOCKER_USERNAME}/zeppelin-release:<release-version> -c bash +docker run -p 8080:8080 --rm --name zeppelin apache/zeppelin:<release-version> ``` -* Then a docker container will start with a Zeppelin release on path : -`/usr/local/zeppelin/` -* Run zeppelin inside docker: -``` -/usr/local/zeppelin/bin/zeppelin.sh -``` +* Zeppelin will run at `http://localhost:8080`. -* To Run Zeppelin in daemon mode -Mounting logs and notebooks zeppelin to folders on your host machine +If you want to specify `logs` and `notebook` dir, ``` -docker run -p 7077:7077 -p 8080:8080 --privileged=true -v $PWD/logs:/logs -v $PWD/notebook:/notebook \ --e ZEPPELIN_NOTEBOOK_DIR='/notebook' \ +docker run -p 8080:8080 --rm \ +-v $PWD/logs:/logs \ +-v $PWD/notebook:/notebook \ -e ZEPPELIN_LOG_DIR='/logs' \ --d ${DOCKER_USERNAME}/zeppelin-release:<release-version> \ -/usr/local/zeppelin/bin/zeppelin.sh +-e ZEPPELIN_NOTEBOOK_DIR='/notebook' \ +--name zeppelin apache/zeppelin:<release-version> # e.g '0.7.2' ``` +### Building dockerfile locally -* Zeppelin will run at `http://localhost:8080`. +``` +cd $ZEPPELIN_HOME +cd scripts/docker/zeppelin/bin + +docker build -t my-zeppelin:my-tag ./ +``` http://git-wip-us.apache.org/repos/asf/zeppelin/blob/e3071d66/scripts/docker/zeppelin-base/Dockerfile ---------------------------------------------------------------------- diff --git a/scripts/docker/zeppelin-base/Dockerfile b/scripts/docker/zeppelin-base/Dockerfile deleted file mode 100644 index 3084d3e..0000000 --- a/scripts/docker/zeppelin-base/Dockerfile +++ /dev/null @@ -1,42 +0,0 @@ -# Licensed to the Apache Software Foundation (ASF) under one or more -# contributor license agreements. See the NOTICE file distributed with -# this work for additional information regarding copyright ownership. -# The ASF licenses this file to You under the Apache License, Version 2.0 -# (the "License"); you may not use this file except in compliance with -# the License. You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. -# - -FROM alpine:3.4 -MAINTAINER Apache Software Foundation <[email protected]> - -ENV JAVA_HOME /usr/lib/jvm/java-1.7-openjdk -ENV PATH $PATH:$JAVA_HOME/bin - -RUN apk add --update bash curl openjdk7-jre wget ca-certificates python build-base make gcc g++ java-cacerts openssl && \ - rm /usr/lib/jvm/java-1.7-openjdk/jre/lib/security/cacerts && \ - ln -s /etc/ssl/certs/java/cacerts /usr/lib/jvm/java-1.7-openjdk/jre/lib/security/cacerts && \ - curl --silent \ - --location https://github.com/sgerrand/alpine-pkg-R/releases/download/3.3.1-r0/R-3.3.1-r0.apk --output /var/cache/apk/R-3.3.1-r0.apk && \ - apk add --update --allow-untrusted /var/cache/apk/R-3.3.1-r0.apk && \ - curl --silent \ - --location https://github.com/sgerrand/alpine-pkg-R/releases/download/3.3.1-r0/R-dev-3.3.1-r0.apk --output /var/cache/apk/R-dev-3.3.1-r0.apk && \ - apk add --update --allow-untrusted /var/cache/apk/R-dev-3.3.1-r0.apk && \ - R -e "install.packages('knitr', repos = 'http://cran.us.r-project.org')" && \ - apk del curl build-base make gcc g++ && \ - rm -rf /var/cache/apk/* - -RUN wget -O /usr/local/bin/dumb-init https://github.com/Yelp/dumb-init/releases/download/v1.1.3/dumb-init_1.1.3_amd64 -RUN chmod +x /usr/local/bin/dumb-init - -# ports for zeppelin -EXPOSE 8080 7077 - -ENTRYPOINT ["/usr/local/bin/dumb-init", "--"] http://git-wip-us.apache.org/repos/asf/zeppelin/blob/e3071d66/scripts/docker/zeppelin/bin/Dockerfile ---------------------------------------------------------------------- diff --git a/scripts/docker/zeppelin/bin/Dockerfile b/scripts/docker/zeppelin/bin/Dockerfile new file mode 100644 index 0000000..8808b8a --- /dev/null +++ b/scripts/docker/zeppelin/bin/Dockerfile @@ -0,0 +1,106 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +FROM ubuntu:16.04 +MAINTAINER Apache Software Foundation <[email protected]> + +# `Z_VERSION` will be updated by `dev/change_zeppelin_version.sh` +ENV Z_VERSION="0.7.2-SNAPSHOT" +ENV LOG_TAG="[ZEPPELIN_${Z_VERSION}]:" \ + Z_HOME="/zeppelin" \ + LANG=en_US.UTF-8 \ + LC_ALL=en_US.UTF-8 + +RUN echo "$LOG_TAG update and install basic packages" && \ + apt-get -y update && \ + apt-get install -y locales && \ + locale-gen $LANG && \ + apt-get install -y software-properties-common && \ + apt -y autoclean && \ + apt -y dist-upgrade && \ + apt-get install -y build-essential + +RUN echo "$LOG_TAG install tini related packages" && \ + apt-get install -y curl grep sed dpkg && \ + TINI_VERSION=`curl https://github.com/krallin/tini/releases/latest | grep -o "/v.*\"" | sed 's:^..\(.*\).$:\1:'` && \ + curl -L "https://github.com/krallin/tini/releases/download/v${TINI_VERSION}/tini_${TINI_VERSION}.deb"; > tini.deb && \ + dpkg -i tini.deb && \ + rm tini.deb + +ENV JAVA_HOME=/usr/lib/jvm/java-8-oracle +RUN echo "$LOG_TAG Install java8" && \ + echo oracle-java8-installer shared/accepted-oracle-license-v1-1 select true | debconf-set-selections && \ + add-apt-repository -y ppa:webupd8team/java && \ + apt-get -y update && \ + apt-get install -y oracle-java8-installer && \ + rm -rf /var/lib/apt/lists/* && \ + rm -rf /var/cache/oracle-jdk8-installer + +# should install conda first before numpy, matploylib since pip and python will be installed by conda +RUN echo "$LOG_TAG Install miniconda2 related packages" && \ + apt-get -y update && \ + apt-get install -y bzip2 ca-certificates \ + libglib2.0-0 libxext6 libsm6 libxrender1 \ + git mercurial subversion && \ + echo 'export PATH=/opt/conda/bin:$PATH' > /etc/profile.d/conda.sh && \ + wget --quiet https://repo.continuum.io/miniconda/Miniconda2-4.3.11-Linux-x86_64.sh -O ~/miniconda.sh && \ + /bin/bash ~/miniconda.sh -b -p /opt/conda && \ + rm ~/miniconda.sh +ENV PATH /opt/conda/bin:$PATH + +RUN echo "$LOG_TAG Install python related packages" && \ + apt-get -y update && \ + apt-get install -y python-dev python-pip && \ + apt-get install -y gfortran && \ + # numerical/algebra packages + apt-get install -y libblas-dev libatlas-dev liblapack-dev && \ + # font, image for matplotlib + apt-get install -y libpng-dev libfreetype6-dev libxft-dev && \ + # for tkinter + apt-get install -y python-tk libxml2-dev libxslt-dev zlib1g-dev && \ + pip install numpy && \ + pip install matplotlib + +RUN echo "$LOG_TAG Install R related packages" && \ + echo "deb http://cran.rstudio.com/bin/linux/ubuntu xenial/" | tee -a /etc/apt/sources.list && \ + gpg --keyserver keyserver.ubuntu.com --recv-key E084DAB9 && \ + gpg -a --export E084DAB9 | apt-key add - && \ + apt-get -y update && \ + apt-get -y install r-base r-base-dev && \ + R -e "install.packages('knitr', repos='http://cran.us.r-project.org')" && \ + R -e "install.packages('ggplot2', repos='http://cran.us.r-project.org')" && \ + R -e "install.packages('googleVis', repos='http://cran.us.r-project.org')" && \ + R -e "install.packages('data.table', repos='http://cran.us.r-project.org')" && \ + # for devtools, Rcpp + apt-get -y install libcurl4-gnutls-dev libssl-dev && \ + R -e "install.packages('devtools', repos='http://cran.us.r-project.org')" && \ + R -e "install.packages('Rcpp', repos='http://cran.us.r-project.org')" && \ + Rscript -e "library('devtools'); library('Rcpp'); install_github('ramnathv/rCharts')" + +RUN echo "$LOG_TAG Download Zeppelin binary" && \ + wget -O /tmp/zeppelin-${Z_VERSION}-bin-all.tgz http://archive.apache.org/dist/zeppelin/zeppelin-${Z_VERSION}/zeppelin-${Z_VERSION}-bin-all.tgz && \ + tar -zxvf /tmp/zeppelin-${Z_VERSION}-bin-all.tgz && \ + rm -rf /tmp/zeppelin-${Z_VERSION}-bin-all.tgz && \ + mv /zeppelin-${Z_VERSION}-bin-all ${Z_HOME} + +RUN echo "$LOG_TAG Cleanup" && \ + apt-get autoclean && \ + apt-get clean + +EXPOSE 8080 + +ENTRYPOINT [ "/usr/bin/tini", "--" ] +WORKDIR ${Z_HOME} +CMD ["bin/zeppelin.sh"] \ No newline at end of file
