Dear,
     ds版本1.3.3  
      
同样的sqoop任务都在01:07分调起,其中一部分返回成功状态,另外一部分显示一直运行中,查ds日志发现sqoop已经执行完毕,yran作业正常结束。
      查ds源库也存在app_link信息,然后我对比了两个作业(正常,异常)worker日志,信息如下:
------------------------------------------------------ 异常 
-------------------------------------------------------------
[INFO] 2021-12-01 01:06:59.949 
org.apache.dolphinscheduler.server.worker.runner.TaskExecuteThread:[95] - 
script path : /tmp/dolphinscheduler/exec/process/6/163/113409/533404
[INFO] 2021-12-01 01:06:59.950  - [taskAppId=TASK-163-113409-533404]:[82] - 
shell task params {"rawScript":"sh 
/data/scripts/ods/opt/sqoop_import/xxx_sqoop.sh 
${global_p_dt}","localParams":[],"resourceList":[]}
[INFO] 2021-12-01 01:06:59.951  - [taskAppId=TASK-163-113409-533404]:[156] - 
raw script : sh /data/scripts/ods/opt/sqoop_import/xxx_sqoop.sh 20211130
[INFO] 2021-12-01 01:06:59.952  - [taskAppId=TASK-163-113409-533404]:[157] - 
task execute path : /tmp/dolphinscheduler/exec/process/6/163/113409/533404
[INFO] 2021-12-01 01:06:59.952  - [taskAppId=TASK-163-113409-533404]:[87] - 
tenantCode user:xxx, task dir:163_113409_533404
[INFO] 2021-12-01 01:06:59.952  - [taskAppId=TASK-163-113409-533404]:[92] - 
create command 
file:/tmp/dolphinscheduler/exec/process/6/163/113409/533404/163_113409_533404.command
[INFO] 2021-12-01 01:06:59.953  - [taskAppId=TASK-163-113409-533404]:[111] - 
command : #!/bin/sh
BASEDIR=$(cd `dirname $0`; pwd)
cd $BASEDIR
source /opt/dolphinscheduler/conf/env/dolphinscheduler_env.sh
/tmp/dolphinscheduler/exec/process/6/163/113409/533404/163_113409_533404_node.sh
[INFO] 2021-12-01 01:06:59.957  - [taskAppId=TASK-163-113409-533404]:[312] - 
task run command:
sudo -u zj-developer sh 
/tmp/dolphinscheduler/exec/process/6/163/113409/533404/163_113409_533404.command
[INFO] 2021-12-01 01:06:59.958  - [taskAppId=TASK-163-113409-533404]:[193] - 
process start, process id is: 49423
[INFO] 2021-12-01 01:07:09.365 
org.apache.dolphinscheduler.server.worker.runner.TaskExecuteThread:[95] - 
script path : /tmp/dolphinscheduler/exec/process/6/163/113409/533406


--------------------------------------------------- 正常 
---------------------------------------------------------------
INFO] 2021-12-01 01:06:48.729  - [taskAppId=TASK-163-113409-533380]:[82] - 
shell task params {"rawScript":"sh 
/data/scripts/ods/opt/sqoop_import/xxx_sqoop.sh 
${global_p_dt}","localParams":[],"resourceList":[]}
[INFO] 2021-12-01 01:06:48.729  - [taskAppId=TASK-163-113409-533380]:[156] - 
raw script : sh /data/scripts/ods/opt/sqoop_import/xxxx_sqoop.sh 20211130
[INFO] 2021-12-01 01:06:48.765  - [taskAppId=TASK-163-113409-533380]:[157] - 
task execute path : /tmp/dolphinscheduler/exec/process/6/163/113409/533380
[INFO] 2021-12-01 01:06:48.766  - [taskAppId=TASK-163-113409-533380]:[87] - 
tenantCode user:xxx, task dir:163_113409_533380
[INFO] 2021-12-01 01:06:48.766  - [taskAppId=TASK-163-113409-533380]:[92] - 
create command 
file:/tmp/dolphinscheduler/exec/process/6/163/113409/533380/163_113409_533380.command
[INFO] 2021-12-01 01:06:48.766  - [taskAppId=TASK-163-113409-533380]:[111] - 
command : #!/bin/sh
BASEDIR=$(cd `dirname $0`; pwd)
cd $BASEDIR
source /opt/dolphinscheduler/conf/env/dolphinscheduler_env.sh
/tmp/dolphinscheduler/exec/process/6/163/113409/533380/163_113409_533380_node.sh
[INFO] 2021-12-01 01:06:48.770  - [taskAppId=TASK-163-113409-533380]:[312] - 
task run command:
sudo -u zj-developer sh 
/tmp/dolphinscheduler/exec/process/6/163/113409/533380/163_113409_533380.command
[INFO] 2021-12-01 01:06:48.772  - [taskAppId=TASK-124-113406-533248]:[419] - 
find app id: application_1635487561336_86389
[INFO] 2021-12-01 01:06:48.781  - [taskAppId=TASK-163-113409-533380]:[193] - 
process start, process id is: 55605
[INFO] 2021-12-01 01:06:48.800 
org.apache.dolphinscheduler.common.utils.HadoopUtils:[206] - application url : 
http://xxx:8088/ws/v1/cluster/apps/%s
[INFO] 2021-12-01 01:06:48.841 
org.apache.dolphinscheduler.common.utils.HadoopUtils:[206] - application url : 
http://xxxzj1-dipper40-cdh-hadoop01.cicc.com:8088/ws/v1/cluster/apps/%s
[INFO] 2021-12-01 01:06:48.852  - [taskAppId=TASK-163-113409-533307]:[379] - 
appId:application_1635487561336_86401, final state:SUCCESS
[INFO] 2021-12-01 01:06:48.852 
org.apache.dolphinscheduler.server.worker.runner.TaskExecuteThread:[137] - task 
instance id : 533307,task final status : SUCCESS
[INFO] 2021-12-01 01:06:48.853 
org.apache.dolphinscheduler.server.worker.runner.TaskExecuteThread:[95] - 
script path : /tmp/dolphinscheduler/exec/process/6/163/113409/533381



Reply via email to