ScrapCodes edited a comment on pull request #33257:
URL: https://github.com/apache/spark/pull/33257#issuecomment-889078588
Interesting!
Can you run with `spark.logConf=true` ? and show the output?
EDIT: for this to work.
1. `cp conf/log4j.properties.template conf/log4j.properties`
2. configure DEBUG logging in `conf/log4j.properties`
```
./bin/spark-submit \
--master k8s://https://172.16.102.10:8443 \
--deploy-mode cluster \
--name java-queue-stream \
--class org.apache.spark.examples.streaming.JavaQueueStream \
--conf spark.executor.instances=3 \
--conf spark.logConf=true \
--conf spark.kubernetes.container.image=spark:testing \
--conf spark.kubernetes.authenticate.driver.serviceAccountName=spark \
local:///opt/spark/examples/jars/spark-examples_2.12-3.3.0-SNAPSHOT.jar
```
Another question: What happens when you run in deploy mode as client?
i.e. --deploy-mode client
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
