GitHub user squito opened a pull request:
https://github.com/apache/spark/pull/16270
[SPARK-18846][Scheduler] Fix flakiness in SchedulerIntegrationSuite
## What changes were proposed in this pull request?
There is a small race in SchedulerIntegrationSuite.
The test assumes that the taskscheduler thread
processing that last task will finish before the DAGScheduler processes
the task event and notifies the job waiter, but that is not 100%
guaranteed.
## How was this patch tested?
ran the test locally a bunch of times, never failed, though admittedly it
never failed locally for me either. However I am nearly 100% certain this is
what caused the failure of one jenkins build
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68694/consoleFull
(which is long gone now, sorry -- I fixed it as part of
https://github.com/apache/spark/pull/14079 initially)
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/squito/spark sched_integ_flakiness
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/16270.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #16270
----
commit 11b33c3bb65e62858ca13ab97c958332f101e60d
Author: Imran Rashid <[email protected]>
Date: 2016-12-13T21:25:06Z
[SPARK-18846][Scheduler] Fix flakiness in SchedulerIntegrationSuite
There is a small race in SchedulerIntegrationSuite.
The test assumes that the taskscheduler thread
processing that last task will finish before the DAGScheduler processes
the task event and notifies the job waiter, but that is not 100%
guaranteed.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
