Dear,
ds版本1.3.3
同样的sqoop任务都在01:07分调起,其中一部分返回成功状态,另外一部分显示一直运行中,查ds日志发现sqoop已经执行完毕,yran作业正常结束。
查ds源库也存在app_link信息,然后我对比了两个作业(正常,异常)worker日志,信息如下:
------------------------------------------------------ 异常
-------------------------------------------------------------
[INFO] 2021-12-01 01:06:59.949
org.apache.dolphinscheduler.server.worker.runner.TaskExecuteThread:[95] -
script path : /tmp/dolphinscheduler/exec/process/6/163/113409/533404
[INFO] 2021-12-01 01:06:59.950 - [taskAppId=TASK-163-113409-533404]:[82] -
shell task params {"rawScript":"sh
/data/scripts/ods/opt/sqoop_import/xxx_sqoop.sh
${global_p_dt}","localParams":[],"resourceList":[]}
[INFO] 2021-12-01 01:06:59.951 - [taskAppId=TASK-163-113409-533404]:[156] -
raw script : sh /data/scripts/ods/opt/sqoop_import/xxx_sqoop.sh 20211130
[INFO] 2021-12-01 01:06:59.952 - [taskAppId=TASK-163-113409-533404]:[157] -
task execute path : /tmp/dolphinscheduler/exec/process/6/163/113409/533404
[INFO] 2021-12-01 01:06:59.952 - [taskAppId=TASK-163-113409-533404]:[87] -
tenantCode user:xxx, task dir:163_113409_533404
[INFO] 2021-12-01 01:06:59.952 - [taskAppId=TASK-163-113409-533404]:[92] -
create command
file:/tmp/dolphinscheduler/exec/process/6/163/113409/533404/163_113409_533404.command
[INFO] 2021-12-01 01:06:59.953 - [taskAppId=TASK-163-113409-533404]:[111] -
command : #!/bin/sh
BASEDIR=$(cd `dirname $0`; pwd)
cd $BASEDIR
source /opt/dolphinscheduler/conf/env/dolphinscheduler_env.sh
/tmp/dolphinscheduler/exec/process/6/163/113409/533404/163_113409_533404_node.sh
[INFO] 2021-12-01 01:06:59.957 - [taskAppId=TASK-163-113409-533404]:[312] -
task run command:
sudo -u zj-developer sh
/tmp/dolphinscheduler/exec/process/6/163/113409/533404/163_113409_533404.command
[INFO] 2021-12-01 01:06:59.958 - [taskAppId=TASK-163-113409-533404]:[193] -
process start, process id is: 49423
[INFO] 2021-12-01 01:07:09.365
org.apache.dolphinscheduler.server.worker.runner.TaskExecuteThread:[95] -
script path : /tmp/dolphinscheduler/exec/process/6/163/113409/533406
--------------------------------------------------- 正常
---------------------------------------------------------------
INFO] 2021-12-01 01:06:48.729 - [taskAppId=TASK-163-113409-533380]:[82] -
shell task params {"rawScript":"sh
/data/scripts/ods/opt/sqoop_import/xxx_sqoop.sh
${global_p_dt}","localParams":[],"resourceList":[]}
[INFO] 2021-12-01 01:06:48.729 - [taskAppId=TASK-163-113409-533380]:[156] -
raw script : sh /data/scripts/ods/opt/sqoop_import/xxxx_sqoop.sh 20211130
[INFO] 2021-12-01 01:06:48.765 - [taskAppId=TASK-163-113409-533380]:[157] -
task execute path : /tmp/dolphinscheduler/exec/process/6/163/113409/533380
[INFO] 2021-12-01 01:06:48.766 - [taskAppId=TASK-163-113409-533380]:[87] -
tenantCode user:xxx, task dir:163_113409_533380
[INFO] 2021-12-01 01:06:48.766 - [taskAppId=TASK-163-113409-533380]:[92] -
create command
file:/tmp/dolphinscheduler/exec/process/6/163/113409/533380/163_113409_533380.command
[INFO] 2021-12-01 01:06:48.766 - [taskAppId=TASK-163-113409-533380]:[111] -
command : #!/bin/sh
BASEDIR=$(cd `dirname $0`; pwd)
cd $BASEDIR
source /opt/dolphinscheduler/conf/env/dolphinscheduler_env.sh
/tmp/dolphinscheduler/exec/process/6/163/113409/533380/163_113409_533380_node.sh
[INFO] 2021-12-01 01:06:48.770 - [taskAppId=TASK-163-113409-533380]:[312] -
task run command:
sudo -u zj-developer sh
/tmp/dolphinscheduler/exec/process/6/163/113409/533380/163_113409_533380.command
[INFO] 2021-12-01 01:06:48.772 - [taskAppId=TASK-124-113406-533248]:[419] -
find app id: application_1635487561336_86389
[INFO] 2021-12-01 01:06:48.781 - [taskAppId=TASK-163-113409-533380]:[193] -
process start, process id is: 55605
[INFO] 2021-12-01 01:06:48.800
org.apache.dolphinscheduler.common.utils.HadoopUtils:[206] - application url :
http://xxx:8088/ws/v1/cluster/apps/%s
[INFO] 2021-12-01 01:06:48.841
org.apache.dolphinscheduler.common.utils.HadoopUtils:[206] - application url :
http://xxxzj1-dipper40-cdh-hadoop01.cicc.com:8088/ws/v1/cluster/apps/%s
[INFO] 2021-12-01 01:06:48.852 - [taskAppId=TASK-163-113409-533307]:[379] -
appId:application_1635487561336_86401, final state:SUCCESS
[INFO] 2021-12-01 01:06:48.852
org.apache.dolphinscheduler.server.worker.runner.TaskExecuteThread:[137] - task
instance id : 533307,task final status : SUCCESS
[INFO] 2021-12-01 01:06:48.853
org.apache.dolphinscheduler.server.worker.runner.TaskExecuteThread:[95] -
script path : /tmp/dolphinscheduler/exec/process/6/163/113409/533381