This is an automated email from the ASF dual-hosted git repository.
kaihsun pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/submarine.git
The following commit(s) were added to refs/heads/master by this push:
new 5ef3a42 SUBMARINE-894. Reduce the image size of submarine-operator
5ef3a42 is described below
commit 5ef3a42da48893d69ecabdb4892ddcfd81e7cee1
Author: Kai-Hsun Chen <[email protected]>
AuthorDate: Sun Jul 4 10:21:58 2021 +0800
SUBMARINE-894. Reduce the image size of submarine-operator
### What is this PR for?
The original submarine-operator docker image is 906 MB, and thus it will be
very time-consuming for users who want to pull the image from DockerHub. Hence,
a light submarine-operator image is needed. In addition, although the original
Docker image is large, it is suitable for development, so we will not remove
the Dockerfile.
In addition, the new docker image will replace the deprecated operator
image (submarine-cloud).
Reference:
(1) https://github.com/kubeflow/pytorch-operator/blob/master/Dockerfile
(2)
https://github.com/kubeflow/tf-operator/blob/master/build/images/tf_operator/Dockerfile
### What type of PR is it?
[Improvement]
### Todos
### What is the Jira issue?
https://issues.apache.org/jira/browse/SUBMARINE-894
### How should this be tested?
* Github action (deploy-image)
### Screenshots (if appropriate)
<img width="1440" alt="截圖 2021-07-03 下午9 46 29"
src="https://user-images.githubusercontent.com/20109646/124361792-7f447d00-dc63-11eb-9c18-51601b3d5771.png";>
<img width="916" alt="截圖 2021-07-03 下午9 47 10"
src="https://user-images.githubusercontent.com/20109646/124361797-87042180-dc63-11eb-9620-4619f73d7ff7.png";>
### Questions:
* Do the license files need updating? No
* Are there breaking changes for older versions? No
* Does this need new documentation? No
Author: Kai-Hsun Chen <[email protected]>
Signed-off-by: Kai-Hsun Chen <[email protected]>
Closes #638 from kevin85421/SUBMARINE-894 and squashes the following
commits:
237bec36 [Kai-Hsun Chen] Deprecate test-k8s and test-e2e in travis
497c559b [Kai-Hsun Chen] SUBMARINE-894. Reduce the image size of
submarine-operator
---
.travis.yml | 30 ----------------------
dev-support/docker-images/operator/Dockerfile | 19 ++++++++------
dev-support/docker-images/operator/build.sh | 14 ++--------
submarine-cloud-v2/Makefile | 3 ++-
submarine-cloud-v2/{Dockerfile => dev.Dockerfile} | 0
.../server/submitter/k8s/K8sSubmitter.java | 4 +--
6 files changed, 16 insertions(+), 54 deletions(-)
diff --git a/.travis.yml b/.travis.yml
index e82ff41..2903c11 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -74,36 +74,6 @@ before_install:
matrix:
include:
- - name: Test submarine test-k8s
- dist: xenial
- services: docker
- language: java
- jdk: openjdk8
- addons:
- chrome: stable
- firefox: latest
- # Because submarine on k8s uses port 80, it needs to be set
`-Durl=http://127.0.0.1`
- env:
- - PROFILE="-Phadoop-2.9"
- - BUILD_FLAG="clean package install -DskipTests" TEST_FLAG="verify
-DskipRat -am -Durl=http://127.0.0.1";
- - MODULES=""
- - TEST_MODULES="-pl
${EXCLUDE_SERVER},${EXCLUDE_COMMONS},${EXCLUDE_CLIENT},:submarine-test-k8s"
- - TEST_PROJECTS=""
-
- - name: Test submarine test-e2e
- language: java
- jdk: openjdk8
- dist: xenial
- addons:
- chrome: stable
- firefox: latest
- env:
- - PROFILE="-Phadoop-2.9"
- - BUILD_FLAG="clean package install -DskipTests"
- - TEST_FLAG="verify -DskipRat -am"
- - TEST_MODULES="-pl org.apache.submarine:submarine-test-e2e"
- - TEST_PROJECTS=""
-
- name: Test submarine commons-cluster
language: java
jdk: openjdk8
diff --git a/dev-support/docker-images/operator/Dockerfile
b/dev-support/docker-images/operator/Dockerfile
index d1a0080..bff01f6 100644
--- a/dev-support/docker-images/operator/Dockerfile
+++ b/dev-support/docker-images/operator/Dockerfile
@@ -13,15 +13,18 @@
# See the License for the specific language governing permissions and
# limitations under the License.
-FROM alpine:3.10
+FROM golang:1.16.2 AS build-image
MAINTAINER Apache Software Foundation <[email protected]>
-# If you are in China, enabling the following two lines of code can speed up
the build of the image, but it may cause failure in travis.
-# So when submitting the code, please make sure the following 2 lines of code
are in a commented state
-# RUN echo "http://mirrors.ustc.edu.cn/alpine/v3.10/main"; >
/etc/apk/repositories
-# RUN echo "http://mirrors.ustc.edu.cn/alpine/v3.10/community"; >>
/etc/apk/repositories
+ADD tmp/submarine-cloud-v2 /usr/src
-RUN apk add tzdata --no-cache
-ADD ./tmp/submarine-operator /usr/local/bin/submarine-operator
+WORKDIR /usr/src
-ENTRYPOINT [ "/usr/local/bin/submarine-operator" ]
+RUN GOOS=linux go build -o submarine-operator
+
+FROM gcr.io/distroless/base-debian10
+WORKDIR /usr/src
+COPY --from=build-image /usr/src/submarine-operator /usr/src/submarine-operator
+ADD tmp/charts/ /usr/src/charts
+
+CMD ["/usr/src/submarine-operator", "-incluster=true"]
diff --git a/dev-support/docker-images/operator/build.sh
b/dev-support/docker-images/operator/build.sh
index 92aad02..12c2123 100755
--- a/dev-support/docker-images/operator/build.sh
+++ b/dev-support/docker-images/operator/build.sh
@@ -27,20 +27,10 @@ fi
export CURRENT_PATH=$(cd "${PWD}">/dev/null; pwd)
export SUBMARINE_HOME=${CURRENT_PATH}/../../..
-if [ ! -d "${SUBMARINE_HOME}/submarine-cloud/bin" ]; then
- mkdir "${SUBMARINE_HOME}/submarine-cloud/bin"
-fi
-submarine_operator_exists=$(find -L "${SUBMARINE_HOME}/submarine-cloud/bin"
-name "submarine-operator")
-# Build source code if the package doesn't exist.
-if [[ -z "${submarine_operator_exists}" ]]; then
- cd "${SUBMARINE_HOME}/submarine-cloud"
- mvn package
-fi
-
mkdir -p "${CURRENT_PATH}/tmp"
-cp ${SUBMARINE_HOME}/submarine-cloud/bin/submarine-operator
"${CURRENT_PATH}/tmp"
+cp -r ${SUBMARINE_HOME}/submarine-cloud-v2/
"${CURRENT_PATH}/tmp/submarine-cloud-v2/"
+cp -r ${SUBMARINE_HOME}/helm-charts/submarine/charts/
"${CURRENT_PATH}/tmp/charts"
-# build image
cd ${CURRENT_PATH}
echo "Start building the ${SUBMARINE_IMAGE_NAME} docker image ..."
docker build -t ${SUBMARINE_IMAGE_NAME} .
diff --git a/submarine-cloud-v2/Makefile b/submarine-cloud-v2/Makefile
index 3509b5c..a729339 100644
--- a/submarine-cloud-v2/Makefile
+++ b/submarine-cloud-v2/Makefile
@@ -30,8 +30,9 @@ api:
.PHONY: image
image: charts
+ eval $(minikube docker-env)
GOOS=linux go build -o submarine-operator
- docker build -t submarine-operator .
+ docker build -t submarine-operator -f dev.Dockerfile .
go build -o submarine-operator
.PHONY: test-unit
diff --git a/submarine-cloud-v2/Dockerfile b/submarine-cloud-v2/dev.Dockerfile
similarity index 100%
rename from submarine-cloud-v2/Dockerfile
rename to submarine-cloud-v2/dev.Dockerfile
diff --git
a/submarine-server/server-submitter/submitter-k8s/src/main/java/org/apache/submarine/server/submitter/k8s/K8sSubmitter.java
b/submarine-server/server-submitter/submitter-k8s/src/main/java/org/apache/submarine/server/submitter/k8s/K8sSubmitter.java
index 446535b..396c872 100644
---
a/submarine-server/server-submitter/submitter-k8s/src/main/java/org/apache/submarine/server/submitter/k8s/K8sSubmitter.java
+++
b/submarine-server/server-submitter/submitter-k8s/src/main/java/org/apache/submarine/server/submitter/k8s/K8sSubmitter.java
@@ -407,9 +407,7 @@ public class K8sSubmitter implements Submitter {
namespace = System.getenv(ENV_NAMESPACE);
}
- try {
-
-
+ try {
NotebookCR notebookCR = NotebookSpecParser.parseNotebook(spec);
Object object = api.getNamespacedCustomObject(notebookCR.getGroup(),
notebookCR.getVersion(),
namespace,
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
![https://user-images.githubusercontent.com/20109646/124361792-7f447d00-dc63-11eb-9c18-51601b3d5771.png"](https://user-images.githubusercontent.com/20109646/124361792-7f447d00-dc63-11eb-9c18-51601b3d5771.png"){kind=link}
![https://user-images.githubusercontent.com/20109646/124361797-87042180-dc63-11eb-9620-4619f73d7ff7.png"](https://user-images.githubusercontent.com/20109646/124361797-87042180-dc63-11eb-9620-4619f73d7ff7.png"){kind=link}