GitHub user SaintBacchus opened a pull request:
https://github.com/apache/spark/pull/3508
[YARN][SPARK-3293]Fix yarn's web show "SUCCEEDED" when the driver throw a
exception in yarn-client
spark has five exit situations:
1.system.exit(0) - succeeds
2.clean exit main -succeeds
3.sc.stop - succeeds
4.system.exit(1) - failed
5.exception - failed
But now yarn-client's ExecutorLauncher is not care about this and just
report Ok when driver is down.
So I catch this five situations in driver and send the relevant status to
ExecutorLauncher to reply the AM status.
In addition, ExecutorLauncher does not need retry if it report the failed
status because this driver has been ended before the sencond ExecutorLauncher
launched.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/SaintBacchus/spark yarn-client-exception
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/3508.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #3508
----
commit f7206d473b499fcbd2f03fea55a7bfb32f4bb516
Author: carlmartin <[email protected]>
Date: 2014-11-28T02:24:50Z
yarn-client web bug
commit 7f4040595ac2ec18ccc2a53ebf44df9d02bf0e81
Author: carlmartin <[email protected]>
Date: 2014-11-28T02:30:46Z
yarn-client web bug
commit d9d016a3d88b26e1ea6e0574505ad5cd2259c2cb
Author: carlmartin <[email protected]>
Date: 2014-11-28T03:02:32Z
yarn-client web bug
commit 7a8540391e221f13fa6dd761e33855fd93fe1177
Author: carlmartin <[email protected]>
Date: 2014-11-28T05:54:42Z
yarn-client web bug
commit d6d5986954f668aad848fb1b3dad34c8e7a50309
Author: carlmartin <[email protected]>
Date: 2014-11-28T06:07:17Z
yarn-client web bug
commit 070ebc91520b67f58d3611317530c009fa5adc6c
Author: carlmartin <[email protected]>
Date: 2014-11-28T06:44:14Z
yarn-client web bug
commit a1080c06bfc4f2d52b57ca23e4eb1b1e906fb81e
Author: carlmartin <[email protected]>
Date: 2014-11-28T06:45:55Z
yarn-client web bug
commit 88dd4c1e00f84ccbfed6b39b3285975a0647a330
Author: carlmartin <[email protected]>
Date: 2014-11-28T06:55:39Z
yarn-client web bug
commit 8033093aa2fcfee2db842a1efc8a5e56ec79f3f1
Author: carlmartin <[email protected]>
Date: 2014-11-28T08:16:14Z
yarn-client web bug
commit 6e473d443fc5e9a19a7867438bf5f0d2b463e7e4
Author: carlmartin <[email protected]>
Date: 2014-11-28T08:45:49Z
yarn-client web bug
commit f716e3b1bc4ce2f2dbd81e8c6f7887542a3dddd6
Author: carlmartin <[email protected]>
Date: 2014-11-28T08:51:38Z
yarn-client web bug
commit 9dbca023ee11cfe2ace73a866142f1b8e215fdb7
Author: carlmartin <[email protected]>
Date: 2014-11-28T09:04:14Z
yarn-client web bug
commit daa6495aa976c4eabf3acddacae7c362b19f73f1
Author: carlmartin <[email protected]>
Date: 2014-11-28T09:27:01Z
yarn-client web bug
commit 451166220795efa242dc339a1cc0a262dddfb8d9
Author: carlmartin <[email protected]>
Date: 2014-11-28T10:49:33Z
Merge branch 'master' of https://github.com/apache/spark into
yarn-client-exception
commit 411ab1a4ab8b4163f405d6083eedb26fd5284eb5
Author: carlmartin <[email protected]>
Date: 2014-11-28T11:11:04Z
yarn-client web bug
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
