[GitHub] [spark] cutiechi commented on pull request #33257: [SPARK-36039][K8S] Fix executor pod hadoop conf mount

Wed, 28 Jul 2021 07:30:24 -0700 


cutiechi commented on pull request #33257:
URL: https://github.com/apache/spark/pull/33257#issuecomment-888356905


   I recompiled the code of the current branch and re-tested according to the 
following command:
   
   @dongjoon-hyun @ScrapCodes
   
   ```
   export HADOOP_CONF_DIR=`pwd`/conf
   
   ./bin/spark-submit  \
     --master k8s://https://172.16.102.10:8443 \
     --deploy-mode cluster \
     --name java-queue-stream \
     --class org.apache.spark.examples.streaming.JavaQueueStream \
     --conf spark.executor.instances=3  \
     --conf spark.kubernetes.container.image=spark:testing \
     --conf spark.kubernetes.authenticate.driver.serviceAccountName=spark \
     local:///opt/spark/examples/jars/spark-examples_2.12-3.3.0-SNAPSHOT.jar
   ```
   But the result is different from yours, all pods are running normally
   
   Pods:
   
   
![图片](https://user-images.githubusercontent.com/27606234/127339262-7d679ec4-ca1f-42f5-8c6d-429a02918d9e.png)
   
   One of the executor pod:
   
   
![图片](https://user-images.githubusercontent.com/27606234/127339496-85e4a74f-47e8-4b56-b540-0b014a1bab27.png)
   
   Hadoop properties ConfigMap of this Pod:
   
   
![图片](https://user-images.githubusercontent.com/27606234/127340003-a0ee914f-75cf-4762-8342-b229e87c35e6.png)
   
   Did you pull the latest code and recompile and test it? Please help me 
review again, thx


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]



---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to