15177592207 commented on issue #4035:
URL:
https://github.com/apache/incubator-dolphinscheduler/issues/4035#issuecomment-722962962
Basic environment:
spark 2.3.4
dolphinscheduler1.3.2
hdp3.1.0
Using dolphinscheduler1.3.2 to schedule spark tasks using cluster mode, the
task is successfully executed normally, but dolphinscheduler real task
execution fails.
status of dolphinscheduler
An icon ×-interpreted as failure
Status of spark log:
[INFO] 2020-11-06 01:13:40.138 - [taskAppId=TASK-4-4-10]:[121] - ->
20/11/06 01:13:40 INFO Client: Application report for
application_1604404915071_0018 (state: RUNNING)
[INFO] 2020-11-06 01:13:41.140 - [taskAppId=TASK-4-4-10]:[121] - ->
20/11/06 01:13:40 INFO Client:
client token: N/A
diagnostics: N/A
ApplicationMaster host: 172.13.1.95
ApplicationMaster RPC port: 0
queue: dolphinscheduler_queue_1
start time: 1604596414080
final status: UNDEFINED
tracking URL:
http://slaves02:8088/proxy/application_1604404915071_0018/
user: dolphinscheduler
20/11/06 01:13:41 INFO Client: Application report for
application_1604404915071_0018 (state: RUNNING)
[INFO] 2020-11-06 01:13:42.143 - [taskAppId=TASK-4-4-10]:[121] - ->
20/11/06 01:13:42 INFO Client: Application report for
application_1604404915071_0018 (state: RUNNING)
[INFO] 2020-11-06 01:13:43.145 - [taskAppId=TASK-4-4-10]:[121] - ->
20/11/06 01:13:43 INFO Client: Application report for
application_1604404915071_0018 (state: RUNNING)
[INFO] 2020-11-06 01:13:44.147 - [taskAppId=TASK-4-4-10]:[121] - ->
20/11/06 01:13:44 INFO Client: Application report for
application_1604404915071_0018 (state: RUNNING)
[INFO] 2020-11-06 01:13:45.149 - [taskAppId=TASK-4-4-10]:[121] - ->
20/11/06 01:13:45 INFO Client: Application report for
application_1604404915071_0018 (state: RUNNING)
[INFO] 2020-11-06 01:13:46.151 - [taskAppId=TASK-4-4-10]:[121] - ->
20/11/06 01:13:46 INFO Client: Application report for
application_1604404915071_0018 (state: RUNNING)
[INFO] 2020-11-06 01:13:47.153 - [taskAppId=TASK-4-4-10]:[121] - ->
20/11/06 01:13:47 INFO Client: Application report for
application_1604404915071_0018 (state: RUNNING)
[INFO] 2020-11-06 01:13:48.155 - [taskAppId=TASK-4-4-10]:[121] - ->
20/11/06 01:13:48 INFO Client: Application report for
application_1604404915071_0018 (state: RUNNING)
[INFO] 2020-11-06 01:13:49.159 - [taskAppId=TASK-4-4-10]:[121] - ->
20/11/06 01:13:49 INFO Client: Application report for
application_1604404915071_0018 (state: FINISHED)
[INFO] 2020-11-06 01:13:49.226 - [taskAppId=TASK-4-4-10]:[121] - ->
20/11/06 01:13:49 INFO Client:
client token: N/A
diagnostics: N/A
ApplicationMaster host: 172.13.1.95
ApplicationMaster RPC port: 0
queue: dolphinscheduler_queue_1
start time: 1604596414080
final status: SUCCEEDED
tracking URL:
http://slaves02:8088/proxy/application_1604404915071_0018/
user: dolphinscheduler
20/11/06 01:13:49 INFO ShutdownHookManager: Shutdown hook called
20/11/06 01:13:49 INFO ShutdownHookManager: Deleting directory
/tmp/spark-761e77c5-b590-48a5-a8ad-df5061db6c8b
20/11/06 01:13:49 INFO ShutdownHookManager: Deleting directory
/tmp/spark-320a73bb-f26d-4861-b4b1-127e28850f91
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
