GitHub user kayousterhout opened a pull request:
https://github.com/apache/spark/pull/16892
[SPARK-19560] Improve DAGScheduler tests.
This commit improves the tests that check the case when a
ShuffleMapTask completes successfully on an executor that has
failed. This commit improves the commenting around the existing
test for this, and adds some additional checks to make it more
clear what went wrong if the tests fail (the fact that these
tests are hard to understand came up in the context of @markhamstra's
proposed fix for #16620).
This commit also removes a test that I realized tested exactly
the same functionality.
@markhamstra, I verified that the new version of the test still fails (and
in a more helpful way) for your proposed change for #16620.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/kayousterhout/spark-1 SPARK-19560
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/16892.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #16892
----
commit 06b26d3738403da30bbf2c7ef2bfdb86039baa02
Author: Kay Ousterhout <[email protected]>
Date: 2017-02-11T06:04:11Z
[SPARK-19560] Improve DAGScheduler tests.
This commit improves the tests that check the case when a
ShuffleMapTask completes successfully on an executor that has
failed. This commit improves the commenting around the existing
test for this, and adds some additional checks to make it more
clear what went wrong if the tests fail (the fact that these
tests are hard to understand came up in the context of #16620).
This commit also removes a test that I realized tested exactly
the same functionality.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
