Etti Gur created SPARK-20139:
--------------------------------
Summary: Spark UI reports partial success for completed stage
while log shows all tasks are finished
Key: SPARK-20139
URL: https://issues.apache.org/jira/browse/SPARK-20139
Project: Spark
Issue Type: Bug
Components: Web UI
Affects Versions: 2.1.0
Reporter: Etti Gur
Spark UI reports partial success for completed stage while log shows all tasks
are finished - i.e.:
We have a stage that is presented under completed stages on spark UI,
but the successful tasks are shown like so: (146372/524964) not as you'd expect
(524964/524964)
Looking at the application master log shows all tasks in that stage are
successful:
17/03/29 09:45:49 INFO TaskSetManager: Finished task 522973.0 in stage 0.0 (TID
522973) in 1163910 ms on ip-10-1-15-34.ec2.internal (executor 116)
(524963/524964)
17/03/29 09:45:49 INFO TaskSetManager: Finished task 12508.0 in stage 2.0 (TID
537472) in 241250 ms on ip-10-1-15-14.ec2.internal (executor 38) (20234/20262)
17/03/29 09:45:49 INFO TaskSetManager: Finished task 12465.0 in stage 2.0 (TID
537429) in 241994 ms on ip-10-1-15-106.ec2.internal (executor 133) (20235/20262)
17/03/29 09:45:49 INFO TaskSetManager: Finished task 15079.0 in stage 2.0 (TID
540043) in 202889 ms on ip-10-1-15-173.ec2.internal (executor 295) (20236/20262)
17/03/29 09:45:49 INFO TaskSetManager: Finished task 19828.0 in stage 2.0 (TID
544792) in 137845 ms on ip-10-1-15-147.ec2.internal (executor 43) (20237/20262)
17/03/29 09:45:50 INFO TaskSetManager: Finished task 19072.0 in stage 2.0 (TID
544036) in 147363 ms on ip-10-1-15-19.ec2.internal (executor 175) (20238/20262)
17/03/29 09:45:50 INFO TaskSetManager: Finished task 524146.0 in stage 0.0 (TID
524146) in 889950 ms on ip-10-1-15-72.ec2.internal (executor 74)
*(524964/524964)*
This looks like the stage is indeed completed with all its tasks but UI shows
like not all tasks really finished.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
