GitHub user 1ambda opened a pull request:
https://github.com/apache/zeppelin/pull/1761
[ZEPPELIN-1711] Create Released Binary Zeppelin Docker Images
### What is this PR for?
This PR
- added Dockerfiles for released zeppelin binaries (0.6.0, 0.6.1, 0.6.2)
- refactored Dockerfiles `scripts/docker/zeppelin-base` so that we can keep
small size base images
- added missing R, python packages to run tutorials properly
- also included `scripts/docker/create-dockerfile.sh` to create docker
images for newly released zeppelin binaries
#### For Reviewers
I have things to be discussed for this PR
1. **Support other linux (centos) or not:** Alpine linux is designed for
light weight OS so it doesn't have graphic devices. But usually people use
zeppelin to render graph like things. In this case Alpine is not enough
2. **Extracting script for installing required packages into
`testing/install_external_dependencies.sh` or not:** Alpine linux has it's own
command to install packages like `apk add`. I think this is not reusable. And
extracting scripts from Dockerfile causes other problems. For example, we need
to add script just for building base images for python, r to copy scripts for
external dependencies because Docker command `COPY` and `ADD` doens't support
relative paths. It means, we have to copy external dependency scripts before
building into the path where Dockerfile is built.
3. **Adding Binary Zeppelin Image to
[docker-library/official-images](https://github.com/docker-library/official-images/)**:
I haven't experiences about this.
### What type of PR is it?
[Feature]
### What is the Jira issue?
[ZEPPELIN-1711](https://issues.apache.org/jira/browse/ZEPPELIN-1711)
### How should this be tested?
#### Building Base Images
```bash
$ cd scripts/docker/zeppelin-base/alpine/java
$ docker build . -t zeppelin:alpine-base_java
$ cd scripts/docker/zeppelin-base/alpine/python
$ docker build . -t zeppelin:alpine-base_python
$ cd scripts/docker/zeppelin-base/alpine/r
$ docker build . -t zeppelin:alpine-base_r
```
#### Building Zeppelin Images
```bash
$ scripts/docker/zeppelin-bin-all/alpine/0.6.2_java
$ docker build . -t zeppelin:alpine-0.6.2_java
$ scripts/docker/zeppelin-bin-all/alpine/0.6.2_python
$ docker build . -t zeppelin:alpine-0.6.2_python
$ scripts/docker/zeppelin-bin-all/alpine/0.6.2_r
$ docker build . -t zeppelin:alpine-0.6.2_r
# build 0.6.1, 0.6.0 images too
```
#### Running Zeppelin Images
Make sure you have zeppelin docker images before running containers
```bash
$ docker images
REPOSITORY TAG IMAGE ID CREATED
SIZE
zeppelin alpine-base_java 5a24572968d6 5 hours
ago 145 MB
```
Then, running containers by replacing tags (0.6.2, 0.6.1, 0.6.0)
```
$ docker run -it --name zeppelin --rm -p 8080:8080 -p 7077:7077
zeppelin:alpine-0.6.2_java
$ docker run -it --name zeppelin --rm -p 8080:8080 -p 7077:7077
zeppelin:alpine-0.6.2_python
$ docker run -it --name zeppelin --rm -p 8080:8080 -p 7077:7077
zeppelin:alpine-0.6.2_r
# run 0.6.1, 0.6.0 also
```
#### Testing Zeppelin Tutorials
Here are things you need to know before testing.
- `zeppelin:alpine-$TAG_java` images can run **Zeppelin Tutorial** since it
has openjdk7
- `zeppelin:alpine-$TAG_python` images can run **Zeppelin Tutorial: Python
- matplotlib basic** as well as **Zeppelin Tutorial** since it has python
related packages
- `zeppelin:alpine-$TAG_r` images can run **R Tutorial** as well as
**Zeppelin Tutorial** since it has python related packages
- Since alpine linux doesn't have graphical device, some function may not
work (e.g plot in R). Some R examples throw invalid images like chunked.png
- For the same reason, we have to change some code of **Python - matplotlib
basic** before testing otherwise it will throw tkinter errors in the python
interpreter
```python
// before
%python
import numpy as np
import matplotlib.pyplot as plt
// should be
%python
import numpy as np
import matplotlib
matplotlib.use('Agg')
import matplotlib.pyplot as pet
```
Then, run each zeppelin container and run tutorial while replacing tags by
accessing localhost:8080 in your browsers
```
# run: Zeppelin Tutorial
$ docker run -it --name zeppelin --rm -p 8080:8080 -p 7077:7077
zeppelin:alpine-0.6.2_java
# run: Zeppelin Tutorial, Python - matplotlib basic
# and modify the code mentioned above before running paragraphs
$ docker run -it --name zeppelin --rm -p 8080:8080 -p 7077:7077
zeppelin:alpine-0.6.2_python
# run: Zeppelin Tutorial, R Tutorial
$ docker run -it --name zeppelin --rm -p 8080:8080 -p 7077:7077
zeppelin:alpine-0.6.2_r
# test 0.6.1, 0.6.0 containers too
```
#### Testing `create-dockerfile.sh`
```bash
$ scripts/docker/zeppelin-bin-all/create-dockerfile.sh -h
USAGE: ./create-dockerfile.sh version linux platform
* version: 0.6.2 (released zeppelin binary version)
* linux: [alpine]
* platform: [java, python, r]
# For example,
# This command will create
`scripts/docker/zeppelin-bin-all/alpine/0.7.0_java/Dockerfile
$ ./create-dockerfile.sh 0.7.0 alpine java
```
### Screenshots (if appropriate)
#### Image Sizes
zeppelin base image size added in #1538: 301.3 MB
```
REPOSITORY TAG IMAGE ID CREATED
SIZE
zeppelin alpine-base_java 5a24572968d6 5 hours
ago 145 MB
zeppelin alpine-base_python e41dcf04760b 4 hours
ago 684.9 MB
zeppelin alpine-base_r 80c2d7aa7156 4 hours
ago 538.9 MB
zeppelin alpine-0.6.0_java 85b4502fd255 2 hours
ago 737.7 MB
zeppelin alpine-0.6.0_python 709dc585b94c 2 hours
ago 1.278 GB
zeppelin alpine-0.6.0_r f6fba0c66563 2 hours
ago 1.131 GB
zeppelin alpine-0.6.1_java ed95ca6204a9 3 hours
ago 751.3 MB
zeppelin alpine-0.6.1_python 34ceea2aeba8 3 hours
ago 1.291 GB
zeppelin alpine-0.6.1_r 3886596f3759 3 hours
ago 1.145 GB
zeppelin alpine-0.6.2_java c65b2f6a1128 3 hours
ago 786.4 MB
zeppelin alpine-0.6.2_python 16c2c3630545 3 hours
ago 1.326 GB
zeppelin alpine-0.6.2_r 544311ec1e6a 3 hours
ago 1.18 GB
```
### Questions:
* Does the licenses files need update? - NO
* Is there breaking changes for older versions? - NO
* Does this needs documentation? - YES, I updated `docs/install/docker.md`
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/1ambda/zeppelin feat/docker-images
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/zeppelin/pull/1761.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1761
----
commit 9f5968339cee5f8c3e03fa73cf5e5b1e10af9ff9
Author: 1ambda <[email protected]>
Date: 2016-12-14T07:17:29Z
feat: Install dumb-init using package
commit d9b8421499551500a7db9d23fdb504f84cb0c3d3
Author: 1ambda <[email protected]>
Date: 2016-12-14T07:35:04Z
feat: Install openjdk7
ref -
https://github.com/1ambda/docker-zeppelin/blob/master/alpine/base/Dockerfile
commit a9b3d67ec0df1a0b05627550e84f91b9549392ba
Author: 1ambda <[email protected]>
Date: 2016-12-14T08:29:18Z
chore: Use --no-cache not to remove cache dir
commit 085527d85c50533fccf089662fdf7c44fa3cccd2
Author: 1ambda <[email protected]>
Date: 2016-12-14T08:33:05Z
chore: Add logging
commit 24e15ef207dea4fe064978cbab826e131d2ce641
Author: 1ambda <[email protected]>
Date: 2016-12-14T08:33:22Z
fix: Remove exposed ports in base image
commit e8d2f5345f010bd5a866224825a351e0881ab01b
Author: 1ambda <[email protected]>
Date: 2016-12-14T09:13:50Z
feat: Add base_python, base_r
commit d7623f76e91182c0c570edf5515235ccce81f32a
Author: 1ambda <[email protected]>
Date: 2016-12-14T09:53:49Z
chore: Use zeppelin-base/alpine directory structure
commit 0730e72fe437248994bcef8e0a5b828d21adb38a
Author: 1ambda <[email protected]>
Date: 2016-12-14T10:24:19Z
feat: Add create-dockerfile.sh
commit ce7b2e42123bf05726dd2000ab37eafd03f73417
Author: 1ambda <[email protected]>
Date: 2016-12-14T12:24:51Z
fix: _java postfix
commit a10c304984b91af2fc820f9ea4a1917033b16d88
Author: 1ambda <[email protected]>
Date: 2016-12-14T13:22:44Z
fix: Base image directory structure
commit 03d32d93acd5bb5a03344904d1d7e15348171d27
Author: 1ambda <[email protected]>
Date: 2016-12-14T13:23:04Z
docs: Update docker docs
commit 0c297af9ff7bb83b10378b13427a4929ccdafbf3
Author: 1ambda <[email protected]>
Date: 2016-12-14T13:35:44Z
fix: Update create-dockerfile usage message
commit 5df6b7359ef0398a26cab478c1f1ab876fa9574d
Author: 1ambda <[email protected]>
Date: 2016-12-14T13:36:13Z
feat: Add 0.6.2 bin images
commit d960ffd2cba62ee9214100355a1d57502c146da9
Author: 1ambda <[email protected]>
Date: 2016-12-14T13:36:40Z
feat: Add 0.6.1 bin images
commit 4af7cebe25ec748ca1b04de086970888e6a42abb
Author: 1ambda <[email protected]>
Date: 2016-12-14T13:37:00Z
feat: Add 0.6.0 bin images
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
