This is an automated email from the ASF dual-hosted git repository.
jiwq pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/submarine.git
The following commit(s) were added to refs/heads/master by this push:
new db201cd SUBMARINE-530. [SDK] Submarine client e2e experiment test
db201cd is described below
commit db201cd06719cc7dfb994f5c7a071bde8c71d021
Author: Kevin Su <[email protected]>
AuthorDate: Mon Aug 17 12:09:47 2020 +0800
SUBMARINE-530. [SDK] Submarine client e2e experiment test
### What is this PR for?
Add an e2e test for the submarine experiment client API,
and separate unit test and integration test in GitHub action
### What type of PR is it?
[Improvement]
### Todos
* [ ] - Task
### What is the Jira issue?
https://issues.apache.org/jira/browse/SUBMARINE-530
### How should this be tested?
https://github.com/pingsutw/hadoop-submarine/runs/989243143
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
Author: Kevin Su <[email protected]>
Author: HUAN-PING SU <[email protected]>
Closes #379 from pingsutw/SUBMARINE-530 and squashes the following commits:
b40f76c [HUAN-PING SU] Update docs/submarine-sdk/pysubmarine/development.md
a30c501 [Kevin Su] SUBMARINE-530. [SDK] Submarine client e2e test
---
.github/workflows/python.yml | 57 ++++++++++++++++------
bin/common.sh | 2 +-
bin/submarine-daemon.sh | 4 +-
docs/submarine-sdk/pysubmarine/development.md | 11 ++++-
.../tests/experiment/test_experiment_client.py | 53 ++++++++++++++++++++
.../tests/store/test_sqlalchemy_store.py | 6 ++-
.../pysubmarine/tests/tracking/test_tracking.py | 3 ++
7 files changed, 114 insertions(+), 22 deletions(-)
diff --git a/.github/workflows/python.yml b/.github/workflows/python.yml
index edda543..636a726 100644
--- a/.github/workflows/python.yml
+++ b/.github/workflows/python.yml
@@ -4,7 +4,7 @@ name: python-sdk
on: [push, pull_request]
jobs:
- build:
+ unit:
runs-on: ubuntu-latest
timeout-minutes: 120
strategy:
@@ -16,19 +16,6 @@ jobs:
PYTHON: ${{ matrix.python }}
steps:
- uses: actions/checkout@v1
- - name: Verify MySQL connection from host
- run: |
- sudo apt update
- sudo apt-get install -y mysql-client
- sudo service mysql restart
- # default mysql account and password in github actions
- mysql -uroot -proot -e "SHOW DATABASES"
- - name: Setup submarine database
- run: |
- mysql -uroot -proot -e "create database submarine_test;"
- mysql -uroot -proot -e "CREATE USER 'submarine_test'@'%' IDENTIFIED
BY 'password_test';"
- mysql -uroot -proot -e "GRANT ALL PRIVILEGES ON *.* TO
'submarine_test'@'%';"
- mysql -uroot -proot -e "use submarine_test; source
./docs/database/submarine.sql; show tables;"
- name: Setup python environment
uses: actions/setup-python@v1
with:
@@ -45,6 +32,44 @@ jobs:
if: matrix.python-version == '3.6'
run: |
./submarine-sdk/pysubmarine/github-actions/lint.sh
- - name: Test with pytest
+ - name: Run unit test
+ run: |
+ pytest --cov=submarine -vs -m "not e2e"
+ integration:
+ runs-on: ubuntu-latest
+ timeout-minutes: 120
+ steps:
+ - uses: actions/checkout@v1
+ - uses: engineerd/[email protected]
+ with:
+ version: "v0.7.0"
+ image: kindest/node:v1.15.6
+ - name: Show K8s cluster information
+ run: |
+ kubectl cluster-info
+ kubectl version
+ kubectl get pods -n kube-system
+ echo "current-context:" $(kubectl config current-context)
+ echo "environment-kubeconfig:" ${KUBECONFIG}
+ - name: Start submarine
+ run: |
+ # Fix submarine-database start failed in kind.
https://github.com/kubernetes/minikube/issues/7906
+ sudo ln -s /etc/apparmor.d/usr.sbin.mysqld /etc/apparmor.d/disable/
+ sudo apparmor_parser -R /etc/apparmor.d/usr.sbin.mysqld
+ helm install --wait submarine ./helm-charts/submarine
+ kubectl get pods
+ kubectl port-forward svc/submarine-database 3306:3306 &
+ kubectl port-forward svc/submarine-server 8080:8080 &
+ - name: Setup python environment
+ uses: actions/setup-python@v1
+ with:
+ python-version: 3.6
+ - name: Install dependencies
+ run: |
+ pip install --upgrade pip
+ pip install --no-cache-dir ./submarine-sdk/pysubmarine/.
+ pip install -r
./submarine-sdk/pysubmarine/github-actions/test-requirements.txt
+ - name: Run integration test
run: |
- pytest --cov=submarine -vs
+ cd submarine-sdk/pysubmarine
+ pytest --cov=submarine -vs -m "e2e"
diff --git a/bin/common.sh b/bin/common.sh
index 0b89619..9f5f2ec 100755
--- a/bin/common.sh
+++ b/bin/common.sh
@@ -82,7 +82,7 @@ function download_mysql_jdbc_jar(){
else
echo 'We need a tool to transfer data from or to a server. Such as
wget/curl.'
echo 'Bye, bye!'
- exit -1
+ exit 1
fi
echo "Mysql jdbc jar is downloaded and put in the path of submarine/lib."
diff --git a/bin/submarine-daemon.sh b/bin/submarine-daemon.sh
index 42b059e..319f4b6 100755
--- a/bin/submarine-daemon.sh
+++ b/bin/submarine-daemon.sh
@@ -113,7 +113,7 @@ function start() {
pid=`found_submarine_server_pid`
if [[ ! -z "$pid" && "$pid" != 0 ]]; then
echo "${SUBMARINE_SERVER_NAME}:${pid} is already running"
- return 0;
+ return 1;
fi
check_jdbc_jar "${BIN}/../lib"
@@ -126,7 +126,7 @@ function start() {
pid=$!
if [[ ! -z "${pid}" ]]; then
echo "${SUBMARINE_SERVER_NAME} start"
- return 1;
+ return 0;
fi
}
diff --git a/docs/submarine-sdk/pysubmarine/development.md
b/docs/submarine-sdk/pysubmarine/development.md
index 6043d45..20cbb01 100644
--- a/docs/submarine-sdk/pysubmarine/development.md
+++ b/docs/submarine-sdk/pysubmarine/development.md
@@ -54,8 +54,17 @@ We are using [pytest](https://docs.pytest.org/en/latest/) to
develop our unit te
After building the project (see below) you can run its unit tests like so:
```bash
cd submarine-sdk/pysubmarine
-pytest --cov=submarine -vs
```
+- Run unit test
+```shell script
+pytest --cov=submarine -vs -m "not e2e"
+```
+- Run integration test
+```shell script
+pytest --cov=submarine -vs -m "e2e"
+```
+> Before run this command in local, you should make sure the submarine server
is running.
+
### Generate python SDK from swagger
We use
[swagger-codegen](https://swagger.io/docs/open-source-tools/swagger-codegen/)
to generate pysubmarine client API that used to communicate with submarine
server.
diff --git
a/submarine-sdk/pysubmarine/tests/experiment/test_experiment_client.py
b/submarine-sdk/pysubmarine/tests/experiment/test_experiment_client.py
new file mode 100644
index 0000000..d94e741
--- /dev/null
+++ b/submarine-sdk/pysubmarine/tests/experiment/test_experiment_client.py
@@ -0,0 +1,53 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import pytest
+
+import submarine
+from submarine.experiment.models.environment import Environment
+from submarine.experiment.models.experiment_meta import ExperimentMeta
+from submarine.experiment.models.experiment_spec import ExperimentSpec
+from submarine.experiment.models.experiment_task_spec import ExperimentTaskSpec
+
+
[email protected]
+def test_experiment_e2e():
+ submarine_client = submarine.ExperimentClient(host='http://localhost:8080')
+ environment =
Environment(image='gcr.io/kubeflow-ci/tf-dist-mnist-test:1.0')
+ experiment_meta = ExperimentMeta(
+ name='mnist-dist',
+ namespace='default',
+ framework='Tensorflow',
+ cmd='python /var/tf_dist_mnist/dist_mnist.py --train_steps=100',
+ env_vars={'ENV1': 'ENV1'})
+
+ worker_spec = ExperimentTaskSpec(resources='cpu=1,memory=1024M',
replicas=1)
+ ps_spec = ExperimentTaskSpec(resources='cpu=1,memory=1024M', replicas=1)
+
+ experiment_spec = ExperimentSpec(meta=experiment_meta,
+ environment=environment,
+ spec={
+ 'Ps': ps_spec,
+ 'Worker': worker_spec
+ })
+
+ experiment = submarine_client.create_experiment(
+ experiment_spec=experiment_spec)
+ id = experiment['experimentId']
+
+ submarine_client.get_experiment(id)
+ submarine_client.list_experiments()
+ submarine_client.get_log(id)
+ submarine_client.delete_experiment(id)
diff --git a/submarine-sdk/pysubmarine/tests/store/test_sqlalchemy_store.py
b/submarine-sdk/pysubmarine/tests/store/test_sqlalchemy_store.py
index 3be4e25..72c1c00 100644
--- a/submarine-sdk/pysubmarine/tests/store/test_sqlalchemy_store.py
+++ b/submarine-sdk/pysubmarine/tests/store/test_sqlalchemy_store.py
@@ -15,7 +15,8 @@
import time
import unittest
-from os import environ
+
+import pytest
import submarine
from submarine.entities import Metric, Param
@@ -26,6 +27,7 @@ from submarine.tracking import utils
JOB_ID = "application_123456789"
[email protected]
class TestSqlAlchemyStore(unittest.TestCase):
def setUp(self):
@@ -60,7 +62,7 @@ class TestSqlAlchemyStore(unittest.TestCase):
self.store.log_metric(JOB_ID, metric1)
self.store.log_metric(JOB_ID, metric2)
- # Validate params
+ # Validate metrics
with self.store.ManagedSessionMaker() as session:
metrics = session \
.query(SqlMetric) \
diff --git a/submarine-sdk/pysubmarine/tests/tracking/test_tracking.py
b/submarine-sdk/pysubmarine/tests/tracking/test_tracking.py
index 89e3b14..699fc4e 100644
--- a/submarine-sdk/pysubmarine/tests/tracking/test_tracking.py
+++ b/submarine-sdk/pysubmarine/tests/tracking/test_tracking.py
@@ -16,6 +16,8 @@
import unittest
from os import environ
+import pytest
+
import submarine
from submarine.store.database import models
from submarine.store.database.models import SqlMetric, SqlParam
@@ -24,6 +26,7 @@ from submarine.tracking import utils
JOB_ID = "application_123456789"
[email protected]
class TestTracking(unittest.TestCase):
def setUp(self):
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
