[
https://issues.apache.org/jira/browse/PIO-86?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16644566#comment-16644566
]
ASF GitHub Bot commented on PIO-86:
-----------------------------------
marevol closed pull request #462: [PIO-86] Docker support
URL: https://github.com/apache/predictionio/pull/462
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/docker/README.md b/docker/README.md
new file mode 100644
index 000000000..f1bc5a08f
--- /dev/null
+++ b/docker/README.md
@@ -0,0 +1,249 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one or more
+contributor license agreements. See the NOTICE file distributed with
+this work for additional information regarding copyright ownership.
+The ASF licenses this file to You under the Apache License, Version 2.0
+(the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+
+Apache PredictionIO Docker
+==========================
+
+## Overview
+
+PredictionIO Docker provides Docker image for use in development and
production environment.
+
+
+## Usage
+
+### Run PredictionIO with Selectable docker-compose Files
+
+You can choose storages for event/meta/model to select docker-compose.yml.
+
+```
+docker-compose -f docker-compose.yml -f ... up
+```
+
+Supported storages are as below:
+
+| Type | Storage |
+|:-----:|:---------------------------------|
+| Event | Postgresql, MySQL, Elasticsearch |
+| Meta | Postgresql, MySQL, Elasticsearch |
+| Model | Postgresql, MySQL, LocalFS |
+
+If you run PredictionIO with Postgresql, run as below:
+
+```
+docker-compose -f docker-compose.yml \
+ -f pgsql/docker-compose.base.yml \
+ -f pgsql/docker-compose.meta.yml \
+ -f pgsql/docker-compose.event.yml \
+ -f pgsql/docker-compose.model.yml \
+ up
+```
+
+To use localfs as model storage, change as below:
+
+```
+docker-compose -f docker-compose.yml \
+ -f pgsql/docker-compose.base.yml \
+ -f pgsql/docker-compose.meta.yml \
+ -f pgsql/docker-compose.event.yml \
+ -f localfs/docker-compose.model.yml \
+ up
+```
+
+## Tutorial
+
+In this demo, we will show you how to build a recommendation template.
+
+### Run PredictionIO environment
+
+The following command starts PredictionIO with an event server.
+PredictionIO docker image mounts ./templates directory to /templates.
+
+```
+$ docker-compose -f docker-compose.yml \
+ -f pgsql/docker-compose.base.yml \
+ -f pgsql/docker-compose.meta.yml \
+ -f pgsql/docker-compose.event.yml \
+ -f pgsql/docker-compose.model.yml \
+ up
+```
+
+We provide `pio-docker` command as an utility for `pio` command.
+`pio-docker` invokes `pio` command in PredictionIO container.
+
+```
+$ export PATH=`pwd`/bin:$PATH
+$ pio-docker status
+...
+[INFO] [Management$] Your system is all ready to go.
+```
+
+### Download Recommendation Template
+
+This demo uses
[predictionio-template-recommender](https://github.com/apache/predictionio-template-recommender).
+
+```
+$ cd templates
+$ git clone https://github.com/apache/predictionio-template-recommender.git
MyRecommendation
+$ cd MyRecommendation
+```
+
+### Register Application
+
+You need to register this application to PredictionIO:
+
+```
+$ pio-docker app new MyApp1
+[INFO] [App$] Initialized Event Store for this app ID: 1.
+[INFO] [Pio$] Created a new app:
+[INFO] [Pio$] Name: MyApp1
+[INFO] [Pio$] ID: 1
+[INFO] [Pio$] Access Key:
i-zc4EleEM577EJhx3CzQhZZ0NnjBKKdSbp3MiR5JDb2zdTKKzH9nF6KLqjlMnvl
+```
+
+Since an access key is required in subsequent steps, set it to ACCESS_KEY.
+
+```
+$ ACCESS_KEY=i-zc4EleEM577EJhx3CzQhZZ0NnjBKKdSbp3MiR5JDb2zdTKKzH9nF6KLqjlMnvl
+```
+
+`engine.json` contains an application name, so replace `INVALID_APP_NAME` with
`MyApp1`.
+
+```
+...
+"datasource": {
+ "params" : {
+ "appName": "MyApp1"
+ }
+},
+...
+```
+
+### Import Data
+
+To import training data to Event server for PredictionIO, this template
provides an import tool.
+The tool depends on PredictionIO Python SDK and install as below:
+
+```
+$ pip install predictionio
+```
+and then import data:
+```
+$ curl
https://raw.githubusercontent.com/apache/spark/master/data/mllib/sample_movielens_data.txt
--create-dirs -o data/sample_movielens_data.txt
+$ python data/import_eventserver.py --access_key $ACCESS_KEY
+```
+
+### Build Template
+
+This is Scala based template.
+So, you need to build this template by `pio` command.
+
+```
+$ pio-docker build --verbose
+```
+
+### Train and Create Model
+
+To train a recommendation model, run `train` sub-command:
+
+```
+$ pio-docker train
+```
+
+### Deploy Model
+
+If a recommendation model is created successfully, deploy it to Prediction
server for PredictionIO.
+
+```
+$ pio-docker deploy
+
+```
+You can check predictions as below:
+```
+$ curl -H "Content-Type: application/json" \
+-d '{ "user": "1", "num": 4 }' http://localhost:8000/queries.json
+```
+
+## Advanced Topics
+
+### Run with Elasticsearch
+
+For Elasticsearch, Meta and Event storage are available.
+To start PredictionIO with Elasticsearch,
+
+```
+docker-compose -f docker-compose.yml \
+ -f elasticsearch/docker-compose.base.yml \
+ -f elasticsearch/docker-compose.meta.yml \
+ -f elasticsearch/docker-compose.event.yml \
+ -f localfs/docker-compose.model.yml \
+ up
+```
+
+### Run with Spark Cluster
+
+Adding `docker-compose.spark.yml`, you can use Spark cluster on `pio train`.
+
+```
+docker-compose -f docker-compose.yml \
+ -f docker-compose.spark.yml \
+ -f elasticsearch/docker-compose.base.yml \
+ -f elasticsearch/docker-compose.meta.yml \
+ -f elasticsearch/docker-compose.event.yml \
+ -f localfs/docker-compose.model.yml \
+ up
+```
+
+To submit a training task to Spark Cluster, run `pio-deploy train` with
`--master` option:
+
+```
+pio-docker train -- --master spark://spark-master:7077
+```
+
+See `docker-compose.spark.yml` if changing settings for Spark Cluster.
+
+### Run Engine Server
+
+To deploy your engine and start an engine server, run Docker with
`docker-compose.deploy.yml`.
+
+```
+docker-compose -f docker-compose.yml \
+ -f pgsql/docker-compose.base.yml \
+ -f pgsql/docker-compose.meta.yml \
+ -f pgsql/docker-compose.event.yml \
+ -f pgsql/docker-compose.model.yml \
+ -f docker-compose.deploy.yml \
+ up
+```
+
+See `deploy/run.sh` and `docker-compose.deploy.yml` if changing a deployment.
+
+
+## Development
+
+### Build Docker Image
+
+```
+docker build -t predictionio/pio pio
+```
+
+### Push Docker Image
+
+```
+docker push predictionio/pio:latest
+docker tag predictionio/pio:latest predictionio/pio:$PIO\_VERSION
+docker push predictionio/pio:$PIO\_VERSION
+```
diff --git a/docker/bin/pio-docker b/docker/bin/pio-docker
new file mode 100755
index 000000000..8046060de
--- /dev/null
+++ b/docker/bin/pio-docker
@@ -0,0 +1,56 @@
+#!/usr/bin/env bash
+
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+BASE_WORK_DIR=/templates
+CURRENT_DIR=`pwd`
+
+get_container_id() {
+ if [ x"$PIO_CONTAINER_ID" != "x" ] ; then
+ echo $PIO_CONTAINER_ID
+ return
+ fi
+ for i in `docker ps -f "name=pio" -q` ; do
+ echo $i
+ return
+ done
+}
+
+get_current_dir() {
+ if [ x"$PIO_CURRENT_DIR" != "x" ] ; then
+ echo $PIO_CURRENT_DIR
+ return
+ fi
+ D=`echo $CURRENT_DIR | sed -e "s,.*$BASE_WORK_DIR,$BASE_WORK_DIR,"`
+ if [[ $D = $BASE_WORK_DIR* ]] ; then
+ echo $D
+ else
+ echo $BASE_WORK_DIR
+ fi
+}
+
+cid=`get_container_id`
+if [ x"$cid" = "x" ] ; then
+ echo "Docker Container is not found."
+ exit 1
+fi
+
+wdir=`get_current_dir`
+
+docker exec -w $wdir -it $cid pio $@
+
diff --git a/docker/deploy/run.sh b/docker/deploy/run.sh
new file mode 100644
index 000000000..f225a6220
--- /dev/null
+++ b/docker/deploy/run.sh
@@ -0,0 +1,21 @@
+#!/usr/bin/env bash
+
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+cd /templates/$PIO_TEMPLATE_NAME
+pio deploy
diff --git a/docker/docker-compose.deploy.yml b/docker/docker-compose.deploy.yml
new file mode 100644
index 000000000..145a96001
--- /dev/null
+++ b/docker/docker-compose.deploy.yml
@@ -0,0 +1,23 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+version: "3"
+services:
+ pio:
+ environment:
+ - "PIO_TEMPLATE_NAME=MyRecommendation"
+ - "PIO_RUN_FILE=/deploy/run.sh"
+ volumes:
+ - ./deploy:/deploy
diff --git a/docker/docker-compose.spark.yml b/docker/docker-compose.spark.yml
new file mode 100644
index 000000000..8f3c43e83
--- /dev/null
+++ b/docker/docker-compose.spark.yml
@@ -0,0 +1,34 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+version: "3"
+services:
+ spark-master:
+ image: bde2020/spark-master:2.2.2-hadoop2.7
+ container_name: spark-master
+ ports:
+ - "8080:8080"
+ - "7077:7077"
+ environment:
+ - INIT_DAEMON_STEP=setup_spark
+ spark-worker-1:
+ image: bde2020/spark-worker:2.2.2-hadoop2.7
+ container_name: spark-worker-1
+ depends_on:
+ - spark-master
+ ports:
+ - "8081:8081"
+ environment:
+ - "SPARK_MASTER=spark://spark-master:7077"
diff --git a/docker/docker-compose.yml b/docker/docker-compose.yml
new file mode 100644
index 000000000..e42c907a5
--- /dev/null
+++ b/docker/docker-compose.yml
@@ -0,0 +1,25 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+version: "3"
+services:
+ pio:
+ image: predictionio/pio:latest
+ ports:
+ - 7070:7070
+ - 8000:8000
+ volumes:
+ - ./templates:/templates
+ dns: 8.8.8.8
diff --git a/docker/elasticsearch/docker-compose.base.yml
b/docker/elasticsearch/docker-compose.base.yml
new file mode 100644
index 000000000..4784f4a6b
--- /dev/null
+++ b/docker/elasticsearch/docker-compose.base.yml
@@ -0,0 +1,41 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+version: "3"
+services:
+ elasticsearch:
+ image: docker.elastic.co/elasticsearch/elasticsearch:5.6.4
+ environment:
+ - xpack.graph.enabled=false
+ - xpack.ml.enabled=false
+ - xpack.monitoring.enabled=false
+ - xpack.security.enabled=false
+ - xpack.watcher.enabled=false
+ - cluster.name=predictionio
+ - bootstrap.memory_lock=true
+ - "ES_JAVA_OPTS=-Xms1g -Xmx1g"
+ ulimits:
+ memlock:
+ soft: -1
+ hard: -1
+ pio:
+ depends_on:
+ - elasticsearch
+ environment:
+ PIO_STORAGE_SOURCES_ELASTICSEARCH_TYPE: elasticsearch
+ PIO_STORAGE_SOURCES_ELASTICSEARCH_HOSTS: elasticsearch
+ PIO_STORAGE_SOURCES_ELASTICSEARCH_PORTS: 9200
+ PIO_STORAGE_SOURCES_ELASTICSEARCH_SCHEMES: http
+
diff --git a/docker/elasticsearch/docker-compose.event.yml
b/docker/elasticsearch/docker-compose.event.yml
new file mode 100644
index 000000000..5a77b6a4e
--- /dev/null
+++ b/docker/elasticsearch/docker-compose.event.yml
@@ -0,0 +1,22 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+version: "3"
+services:
+ pio:
+ environment:
+ PIO_STORAGE_REPOSITORIES_EVENTDATA_NAME: pio_event
+ PIO_STORAGE_REPOSITORIES_EVENTDATA_SOURCE: ELASTICSEARCH
+
diff --git a/docker/elasticsearch/docker-compose.meta.yml
b/docker/elasticsearch/docker-compose.meta.yml
new file mode 100644
index 000000000..0ce31ddf3
--- /dev/null
+++ b/docker/elasticsearch/docker-compose.meta.yml
@@ -0,0 +1,22 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+version: "3"
+services:
+ pio:
+ environment:
+ PIO_STORAGE_REPOSITORIES_METADATA_NAME: pio_meta
+ PIO_STORAGE_REPOSITORIES_METADATA_SOURCE: ELASTICSEARCH
+
diff --git a/docker/localfs/docker-compose.model.yml
b/docker/localfs/docker-compose.model.yml
new file mode 100644
index 000000000..f38ff53b0
--- /dev/null
+++ b/docker/localfs/docker-compose.model.yml
@@ -0,0 +1,27 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+version: "3"
+services:
+ pio:
+ environment:
+ PIO_STORAGE_REPOSITORIES_MODELDATA_NAME: pio_model
+ PIO_STORAGE_REPOSITORIES_MODELDATA_SOURCE: LOCALFS
+ PIO_FS_BASEDIR: /work/pio_store
+ PIO_FS_ENGINESDIR: /work/pio_store/engines
+ PIO_FS_TMPDIR: /work/pio_store/tmp
+ PIO_STORAGE_SOURCES_LOCALFS_TYPE: localfs
+ PIO_STORAGE_SOURCES_LOCALFS_PATH: /work/pio_store/models
+
diff --git a/docker/mysql/docker-compose.base.yml
b/docker/mysql/docker-compose.base.yml
new file mode 100644
index 000000000..fad53097c
--- /dev/null
+++ b/docker/mysql/docker-compose.base.yml
@@ -0,0 +1,34 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+version: "3"
+services:
+ mysql:
+ image: mysql:8
+ command: mysqld --character-set-server=utf8mb4
--collation-server=utf8mb4_unicode_ci
+ environment:
+ MYSQL_ROOT_PASSWORD: root
+ MYSQL_USER: pio
+ MYSQL_PASSWORD: pio
+ MYSQL_DATABASE: pio
+ pio:
+ depends_on:
+ - mysql
+ environment:
+ PIO_STORAGE_SOURCES_MYSQL_TYPE: jdbc
+ PIO_STORAGE_SOURCES_MYSQL_URL: "jdbc:mysql://mysql/pio"
+ PIO_STORAGE_SOURCES_MYSQL_USERNAME: pio
+ PIO_STORAGE_SOURCES_MYSQL_PASSWORD: pio
+
diff --git a/docker/mysql/docker-compose.event.yml
b/docker/mysql/docker-compose.event.yml
new file mode 100644
index 000000000..f5f4035b8
--- /dev/null
+++ b/docker/mysql/docker-compose.event.yml
@@ -0,0 +1,22 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+version: "3"
+services:
+ pio:
+ environment:
+ PIO_STORAGE_REPOSITORIES_EVENTDATA_NAME: pio_event
+ PIO_STORAGE_REPOSITORIES_EVENTDATA_SOURCE: MYSQL
+
diff --git a/docker/mysql/docker-compose.meta.yml
b/docker/mysql/docker-compose.meta.yml
new file mode 100644
index 000000000..f7a5ece75
--- /dev/null
+++ b/docker/mysql/docker-compose.meta.yml
@@ -0,0 +1,22 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+version: "3"
+services:
+ pio:
+ environment:
+ PIO_STORAGE_REPOSITORIES_METADATA_NAME: pio_meta
+ PIO_STORAGE_REPOSITORIES_METADATA_SOURCE: MYSQL
+
diff --git a/docker/mysql/docker-compose.model.yml
b/docker/mysql/docker-compose.model.yml
new file mode 100644
index 000000000..4a486846e
--- /dev/null
+++ b/docker/mysql/docker-compose.model.yml
@@ -0,0 +1,22 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+version: "3"
+services:
+ pio:
+ environment:
+ PIO_STORAGE_REPOSITORIES_MODELDATA_NAME: pio_model
+ PIO_STORAGE_REPOSITORIES_MODELDATA_SOURCE: MYSQL
+
diff --git a/docker/pgsql/docker-compose.base.yml
b/docker/pgsql/docker-compose.base.yml
new file mode 100644
index 000000000..297d6a850
--- /dev/null
+++ b/docker/pgsql/docker-compose.base.yml
@@ -0,0 +1,32 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+version: "3"
+services:
+ postgres:
+ image: postgres:9
+ environment:
+ POSTGRES_USER: pio
+ POSTGRES_PASSWORD: pio
+ POSTGRES_INITDB_ARGS: --encoding=UTF8
+ pio:
+ depends_on:
+ - postgres
+ environment:
+ PIO_STORAGE_SOURCES_PGSQL_TYPE: jdbc
+ PIO_STORAGE_SOURCES_PGSQL_URL: "jdbc:postgresql://postgres/pio"
+ PIO_STORAGE_SOURCES_PGSQL_USERNAME: pio
+ PIO_STORAGE_SOURCES_PGSQL_PASSWORD: pio
+
diff --git a/docker/pgsql/docker-compose.event.yml
b/docker/pgsql/docker-compose.event.yml
new file mode 100644
index 000000000..257914137
--- /dev/null
+++ b/docker/pgsql/docker-compose.event.yml
@@ -0,0 +1,22 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+version: "3"
+services:
+ pio:
+ environment:
+ PIO_STORAGE_REPOSITORIES_EVENTDATA_NAME: pio_event
+ PIO_STORAGE_REPOSITORIES_EVENTDATA_SOURCE: PGSQL
+
diff --git a/docker/pgsql/docker-compose.meta.yml
b/docker/pgsql/docker-compose.meta.yml
new file mode 100644
index 000000000..345bfbe87
--- /dev/null
+++ b/docker/pgsql/docker-compose.meta.yml
@@ -0,0 +1,22 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+version: "3"
+services:
+ pio:
+ environment:
+ PIO_STORAGE_REPOSITORIES_METADATA_NAME: pio_meta
+ PIO_STORAGE_REPOSITORIES_METADATA_SOURCE: PGSQL
+
diff --git a/docker/pgsql/docker-compose.model.yml
b/docker/pgsql/docker-compose.model.yml
new file mode 100644
index 000000000..329649f7e
--- /dev/null
+++ b/docker/pgsql/docker-compose.model.yml
@@ -0,0 +1,22 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+version: "3"
+services:
+ pio:
+ environment:
+ PIO_STORAGE_REPOSITORIES_MODELDATA_NAME: pio_model
+ PIO_STORAGE_REPOSITORIES_MODELDATA_SOURCE: PGSQL
+
diff --git a/docker/pio/Dockerfile b/docker/pio/Dockerfile
new file mode 100644
index 000000000..88930bd7e
--- /dev/null
+++ b/docker/pio/Dockerfile
@@ -0,0 +1,69 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+FROM openjdk:8
+
+ARG PIO_GIT_URL=https://github.com/apache/predictionio.git
+ARG PIO_TAG=v0.13.0
+ENV SCALA_VERSION=2.11.12
+ENV SPARK_VERSION=2.2.2
+ENV HADOOP_VERSION=2.7.7
+ENV ELASTICSEARCH_VERSION=5.5.3
+ENV PGSQL_VERSION=42.2.4
+ENV MYSQL_VERSION=8.0.12
+ENV PIO_HOME=/usr/share/predictionio
+
+RUN apt-get update && \
+ apt-get install -y dpkg-dev fakeroot && \
+ apt-get clean && \
+ rm -rf /var/lib/apt/lists/*
+
+WORKDIR /opt/src
+RUN git clone -b $PIO_TAG $PIO_GIT_URL
+WORKDIR /opt/src/predictionio
+RUN bash ./make-distribution.sh \
+ -Dscala.version=$SCALA_VERSION \
+ -Dspark.version=$SPARK_VERSION \
+ -Dhadoop.version=$HADOOP_VERSION \
+ -Delasticsearch.version=$ELASTICSEARCH_VERSION \
+ --with-deb && \
+ dpkg -i ./assembly/target/predictionio_*.deb && \
+ rm -rf /opt/src/predictionio/*
+
+
+RUN cp /etc/predictionio/pio-env.sh /etc/predictionio/pio-env.sh.orig && \
+ echo "#!/usr/bin/env bash" > /etc/predictionio/pio-env.sh
+RUN curl -o $PIO_HOME/lib/postgresql-$PGSQL_VERSION.jar \
+
http://central.maven.org/maven2/org/postgresql/postgresql/$PGSQL_VERSION/postgresql-$PGSQL_VERSION.jar
&& \
+ echo "POSTGRES_JDBC_DRIVER=$PIO_HOME/lib/postgresql-$PGSQL_VERSION.jar" >>
/etc/predictionio/pio-env.sh && \
+ echo
"MYSQL_JDBC_DRIVER=$PIO_HOME/lib/mysql-connector-java-$MYSQL_VERSION.jar" >>
/etc/predictionio/pio-env.sh
+
+WORKDIR /usr/share
+RUN curl -o /opt/src/spark-$SPARK_VERSION.tgz \
+
http://www-us.apache.org/dist/spark/spark-$SPARK_VERSION/spark-$SPARK_VERSION-bin-hadoop2.7.tgz
&& \
+ tar zxvf /opt/src/spark-$SPARK_VERSION.tgz && \
+ echo "SPARK_HOME="`pwd`/`ls -d spark*` >> /etc/predictionio/pio-env.sh && \
+ rm -rf /opt/src
+
+WORKDIR /templates
+ADD pio_run /usr/bin/pio_run
+
+EXPOSE 7070
+EXPOSE 8000
+
+CMD ["sh", "/usr/bin/pio_run"]
+
diff --git a/docker/pio/pio_run b/docker/pio/pio_run
new file mode 100644
index 000000000..83ac6cc11
--- /dev/null
+++ b/docker/pio/pio_run
@@ -0,0 +1,64 @@
+#!/usr/bin/env bash
+
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+. /etc/predictionio/pio-env.sh
+
+# check elasticsearch status
+if [ x"$PIO_STORAGE_SOURCES_ELASTICSEARCH_TYPE" != "x" ] ; then
+ RET=-1
+ COUNT=0
+ ES_HOST=`echo $PIO_STORAGE_SOURCES_ELASTICSEARCH_HOSTS | sed -e "s/,.*//"`
+ ES_PORT=`echo $PIO_STORAGE_SOURCES_ELASTICSEARCH_PORTS | sed -e "s/,.*//"`
+ # Wait for elasticsearch startup
+ while [ $RET != 0 -a $COUNT -lt 10 ] ; do
+ echo "Waiting for ${ES_HOST}..."
+ curl --connect-timeout 60 --retry 10 -s
"$ES_HOST:$ES_PORT/_cluster/health?wait_for_status=green&timeout=1m"
+ RET=$?
+ COUNT=`expr $COUNT + 1`
+ sleep 1
+ done
+fi
+
+# check mysql jar file
+if [ x"$PIO_STORAGE_SOURCES_MYSQL_TYPE" != "x" ] ; then
+ MYSQL_JAR_FILE=$PIO_HOME/lib/mysql-connector-java-$MYSQL_VERSION.jar
+ if [ ! -f $MYSQL_JAR_FILE ] ; then
+ curl -o $MYSQL_JAR_FILE
http://central.maven.org/maven2/mysql/mysql-connector-java/$MYSQL_VERSION/mysql-connector-java-$MYSQL_VERSION.jar
+ fi
+fi
+
+# Check PIO status
+RET=-1
+COUNT=0
+while [ $RET != 0 -a $COUNT -lt 10 ] ; do
+ echo "Waiting for PredictionIO..."
+ $PIO_HOME/bin/pio status
+ RET=$?
+ COUNT=`expr $COUNT + 1`
+ sleep 1
+done
+
+
+if [ x"$PIO_RUN_FILE" != "x" ] ; then
+ sh $PIO_RUN_FILE
+else
+ # Start PIO Event Server
+ $PIO_HOME/bin/pio eventserver
+fi
+
diff --git a/docker/templates/.keep b/docker/templates/.keep
new file mode 100644
index 000000000..ec2014340
--- /dev/null
+++ b/docker/templates/.keep
@@ -0,0 +1,14 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
> Maintain a working docker container
> -----------------------------------
>
> Key: PIO-86
> URL: https://issues.apache.org/jira/browse/PIO-86
> Project: PredictionIO
> Issue Type: Wish
> Reporter: Sara Asher
> Assignee: Shinsuke Sugaya
> Priority: Major
> Labels: needs-discussion
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
