[
https://issues.apache.org/jira/browse/SPARK-30619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17033374#comment-17033374
]
Abhishek Rao commented on SPARK-30619:
--------------------------------------
Hi [~hyukjin.kwon]
I just built container using spark-2.4.4-bin-without-hadoop.tgz. Here is the
spark-submit command that I used.
./spark-submit --deploy-mode cluster --class
org.apache.spark.examples.JavaWordCount --master k8s://https://<k8s-api-server>
--name spark-test --conf spark.kubernetes.container.image=<docker image built
using spark-2.4.4-bin-without-hadoop.tgz> --conf
spark.kubernetes.authenticate.driver.serviceAccountName=spark-sa --conf
spark.kubernetes.namespace=spark
local:///opt/spark/examples/jars/spark-examples_2.11-2.4.4.jar
file:///opt/spark/RELEASE
> org.slf4j.Logger and org.apache.commons.collections classes not built as part
> of hadoop-provided profile
> --------------------------------------------------------------------------------------------------------
>
> Key: SPARK-30619
> URL: https://issues.apache.org/jira/browse/SPARK-30619
> Project: Spark
> Issue Type: Bug
> Components: Build
> Affects Versions: 2.4.2, 2.4.4
> Environment: Spark on kubernetes
> Reporter: Abhishek Rao
> Priority: Major
>
> We're using spark-2.4.4-bin-without-hadoop.tgz and executing Java Word count
> (org.apache.spark.examples.JavaWordCount) example on local files.
> But we're seeing that it is expecting org.slf4j.Logger and
> org.apache.commons.collections classes to be available for executing this.
> We expected the binary to work as it is for local files. Is there anything
> which we're missing?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
