GitHub user squito opened a pull request:
https://github.com/apache/spark/pull/13848
[SPARK-16136][CORE] Fix flaky TaskManagerSuite
## What changes were proposed in this pull request?
TaskManagerSuite "Kill other task attempts when one attempt belonging to
the same task succeeds" was flaky. When checking whether a task is
speculatable, at least one millisecond must pass since the task was submitted.
Use a manual clock to avoid the problem.
I noticed these tests were leaving lots of threads lying around as well
(which prevented me from running the test repeatedly), so I fixed that too.
## How was this patch tested?
Ran the test 1k times on my laptop, passed every time (it failed about 20%
of the time before this).
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/squito/spark fix_flaky_taskmanagersuite
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/13848.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #13848
----
commit 6fb6c57f6b01c382e64498f9811626ba4bfab956
Author: Imran Rashid <[email protected]>
Date: 2016-06-22T15:07:16Z
need to stop FakeTaskScheduler to clean up threads
commit 34598a350852ae32a847f036d6e15772a7d96430
Author: Imran Rashid <[email protected]>
Date: 2016-06-22T14:58:49Z
fix flakiness with manual clock
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
