yutianaiqingtian commented on issue #10938:
URL:
https://github.com/apache/dolphinscheduler/issues/10938#issuecomment-1213029754
I have the same issue when use the 3.0.0-release version. The log as brew.
```
[LOG-PATH]: /opt/dolphinscheduler/logs/20220812/6508614388448_7-10-14.log,
[HOST]: Host{address='my-do-worker-1.my-do-worker-headless:1234',
ip='my-do-worker-1.my-do-worker-headless', port=1234}
[INFO] 2022-08-12 19:12:36.086 +0800
[taskAppId=TASK-20220812-6508614388448_7-10-14] TaskLogLogger-class
org.apache.dolphinscheduler.plugin.task.spark.SparkTask:[69] - spark task
params
{"localParams":[],"rawScript":"","resourceList":[],"programType":"SCALA","mainClass":"org.apache.spark.examples.SparkPi","mainJar":{"id":2,"resourceName":"/spark-examples_2.12-3.2.1.jar","res":"spark-examples_2.12-3.2.1.jar"},"deployMode":"client","sparkVersion":"SPARK2","driverCores":1,"driverMemory":"512M","numExecutors":2,"executorMemory":"2G","executorCores":2}
[INFO] 2022-08-12 19:12:36.091 +0800
[taskAppId=TASK-20220812-6508614388448_7-10-14] TaskLogLogger-class
org.apache.dolphinscheduler.plugin.task.spark.SparkTask:[130] - spark task
command: ${SPARK_HOME2}/bin/spark-submit --master yarn --deploy-mode client
--class org.apache.spark.examples.SparkPi --driver-cores 1 --driver-memory 512M
--num-executors 2 --executor-cores 2 --executor-memory 2G --queue default
spark-examples_2.12-3.2.1.jar
[INFO] 2022-08-12 19:12:36.092 +0800
[taskAppId=TASK-20220812-6508614388448_7-10-14] TaskLogLogger-class
org.apache.dolphinscheduler.plugin.task.spark.SparkTask:[85] - tenantCode
user:telent01, task dir:10_14
[INFO] 2022-08-12 19:12:36.092 +0800
[taskAppId=TASK-20220812-6508614388448_7-10-14] TaskLogLogger-class
org.apache.dolphinscheduler.plugin.task.spark.SparkTask:[90] - create command
file:/tmp/dolphinscheduler/exec/process/6508601296608/6508614388448_7/10/14/10_14.command
[INFO] 2022-08-12 19:12:36.092 +0800
[taskAppId=TASK-20220812-6508614388448_7-10-14] TaskLogLogger-class
org.apache.dolphinscheduler.plugin.task.spark.SparkTask:[116] - command :
#!/bin/sh
BASEDIR=$(cd `dirname $0`; pwd)
cd $BASEDIR
source /opt/dolphinscheduler/conf/dolphinscheduler_env.sh
${SPARK_HOME2}/bin/spark-submit --master yarn --deploy-mode client --class
org.apache.spark.examples.SparkPi --driver-cores 1 --driver-memory 512M
--num-executors 2 --executor-cores 2 --executor-memory 2G --queue default
spark-examples_2.12-3.2.1.jar
[INFO] 2022-08-12 19:12:36.096 +0800
[taskAppId=TASK-20220812-6508614388448_7-10-14] TaskLogLogger-class
org.apache.dolphinscheduler.plugin.task.spark.SparkTask:[290] - task run
command: sudo -u telent01 sh
/tmp/dolphinscheduler/exec/process/6508601296608/6508614388448_7/10/14/10_14.command
[INFO] 2022-08-12 19:12:36.097 +0800
[taskAppId=TASK-20220812-6508614388448_7-10-14] TaskLogLogger-class
org.apache.dolphinscheduler.plugin.task.spark.SparkTask:[181] - process start,
process id is: 7776
[INFO] 2022-08-12 19:12:36.102 +0800
[taskAppId=TASK-20220812-6508614388448_7-10-14] TaskLogLogger-class
org.apache.dolphinscheduler.plugin.task.spark.SparkTask:[205] - process has
exited, execute
path:/tmp/dolphinscheduler/exec/process/6508601296608/6508614388448_7/10/14,
processId:7776 ,exitStatusCode:127 ,processWaitForStatus:true
,processExitValue:127
[INFO] 2022-08-12 19:12:37.098 +0800
[taskAppId=TASK-20220812-6508614388448_7-10-14] TaskLogLogger-class
org.apache.dolphinscheduler.plugin.task.spark.SparkTask:[63] - ->
/tmp/dolphinscheduler/exec/process/6508601296608/6508614388448_7/10/14/10_14.command:
4:
/tmp/dolphinscheduler/exec/process/6508601296608/6508614388448_7/10/14/10_14.command:
source: not found
/tmp/dolphinscheduler/exec/process/6508601296608/6508614388448_7/10/14/10_14.command:
5:
/tmp/dolphinscheduler/exec/process/6508601296608/6508614388448_7/10/14/10_14.command:
/bin/spark-submit: not found
[INFO] 2022-08-12 19:12:37.099 +0800
[taskAppId=TASK-20220812-6508614388448_7-10-14] TaskLogLogger-class
org.apache.dolphinscheduler.plugin.task.spark.SparkTask:[57] - FINALIZE_SESSION
```
**The default script compiler is sh, not the bash**. and the #!/bin/sh not
have the source command default. If you want to make the environment take
effect. you can overwrite the environment on the workflow.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
