[
https://issues.apache.org/jira/browse/SPARK-26742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16786543#comment-16786543
]
Stavros Kontopoulos edited comment on SPARK-26742 at 3/7/19 9:19 AM:
---------------------------------------------------------------------
This is one way to run things:
./dev/make-distribution.sh --name test --r --tgz -Psparkr -Phadoop-2.7
-Pkubernetes -Phive
tar -zxvf spark-2.4.0-SNAPSHOT-bin-test.tgz
cd spark-2.4.0-bin-test
./bin/docker-image-tool.sh -r $DOCKER_USERNAME -t $SPARK_K8S_IMAGE_TAG build
\#push either to local registry and start minikube with it or run $(minikube
docker-env) and use \#minikube docker daemon or push to an external registry
./bin/docker-image-tool.sh -r $DOCKER_USERNAME -t $SPARK_K8S_IMAGE_TAG push
cd ../resource-managers/kubernetes/integration-tests
kubectl create -f dev/spark-rbac.yaml
./dev/dev-run-integration-tests.sh --service-account spark-sa --namespace spark
--image-tag $SPARK_K8S_IMAGE_TAG --spark-tgz $TGZ_PATH --image-repo
$DOCKER_USERNAME
\# for 2.12
./dev/change-scala-version.sh 2.12
./dev/make-distribution.sh --name test --r --tgz -Psparkr -Pscala-2.12
-Phadoop-2.7 -Pkubernetes -Phive
was (Author: skonto):
This is one way to run things:
./dev/make-distribution.sh --name test --r --tgz -Psparkr -Phadoop-2.7
-Pkubernetes -Phive
tar -zxvf spark-2.4.0-SNAPSHOT-bin-test.tgz
cd spark-2.4.0-bin-test
./bin/docker-image-tool.sh -r $DOCKER_USERNAME -t $SPARK_K8S_IMAGE_TAG build
\#push either to local registry and start minikube with it or run $(minikube
docker-env) and use \#minikube docker daemon or push to an external registry
./bin/docker-image-tool.sh -r $DOCKER_USERNAME -t $SPARK_K8S_IMAGE_TAG push
cd ../resource-managers/kubernetes/integration-tests
kubectl create -f dev/spark-rbac.yaml
./dev/dev-run-integration-tests.sh --service-account spark-sa --namespace spark
--image-tag $SPARK_K8S_IMAGE_TAG --spark-tgz $TGZ_PATH --image-repo
$DOCKER_USERNAME
\# for 2.12
./dev/change-scala-version.sh 2.12
./dev/make-distribution.sh --name test-2.12 --r --tgz -Psparkr -Pscala-2.12
-Phadoop-2.7 -Pkubernetes -Phive
> Bump Kubernetes Client Version to 4.1.2
> ---------------------------------------
>
> Key: SPARK-26742
> URL: https://issues.apache.org/jira/browse/SPARK-26742
> Project: Spark
> Issue Type: Dependency upgrade
> Components: Kubernetes
> Affects Versions: 2.4.0, 3.0.0
> Reporter: Steve Davids
> Priority: Major
> Labels: easyfix
> Fix For: 3.0.0
>
>
> Spark 2.x is using Kubernetes Client 3.x which is pretty old, the master
> branch has 4.0, the client should be upgraded to 4.1.1 to have the broadest
> Kubernetes compatibility support for newer clusters:
> https://github.com/fabric8io/kubernetes-client#compatibility-matrix
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
