Bala Bharath Reddy Resapu created SPARK-26064:
-------------------------------------------------
Summary: Unable to fetch jar from remote repo while running
spark-submit on kubernetes
Key: SPARK-26064
URL: https://issues.apache.org/jira/browse/SPARK-26064
Project: Spark
Issue Type: Question
Components: Kubernetes
Affects Versions: 2.3.2
Reporter: Bala Bharath Reddy Resapu
I am trying to run spark on kubernetes with a docker image. My requirement is
to download the jar from the external repo while running spark-submit. I am
able to download the jar using wget in the container but it doesn't work when
inputting in the spark-submit command. I am not packaging the jar with docker
image. It works fine when I input the jar file inside the docker image.
./bin/spark-submit \
--master k8s://[https://ip:port|https://ipport/] \
--deploy-mode cluster \
--name test3 \
--class hello \
--conf spark.kubernetes.container.image.pullSecrets=abcd \
--conf spark.kubernetes.container.image=spark:h2.0 \
[https://devops.com/artifactory/local/testing/testing_2.11/h|https://bala.bharath.reddy.resapu%40ibm.com:akcp5bcbktykg2ti28sju4gtebsqwkg2mqkaf9w6g5rdbo3iwrwx7qb1m5dokgd54hdru2...@na.artifactory.swg-devops.com/artifactory/txo-cedp-garage-artifacts-sbt-local/testing/testing_2.11/arithmetic.jar]ello.jar
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
