GitHub user srowen opened a pull request:
https://github.com/apache/spark/pull/4180
SPARK-960 [CORE] [TEST] JobCancellationSuite "two jobs sharing the same
stage" is broken
This reenables and fixes this test, after addressing two issues:
- The Semaphore that was intended to be shared locally was being serialized
and copied; it's now a static member in the companion object as in other tests
- Later changes to Spark means that cancelling the first task will not
cancel the shared stage and therefore the second task should succeed
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/srowen/spark SPARK-960
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/4180.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #4180
----
commit 43da66fc6857f2199209e8a301d6b82c4afd3f5c
Author: Sean Owen <[email protected]>
Date: 2015-01-23T12:00:36Z
Fix 'two jobs sharing the same stage' test and reenable it: truly share a
Semaphore locally as intended, and update expectation of failure in
non-cancelled task
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
