GitHub user davies opened a pull request:
https://github.com/apache/spark/pull/13082
[SPARK-15300] Fix writer lock conflict when remove a block
## What changes were proposed in this pull request?
A writer lock could be acquired when 1) create a new block 2) remove a
block 3) evict a block to disk. 1) and 3) could happen in the same time within
the same task, all of them could happen in the same time outside a task. It's
OK that when someone try to grab the write block for a block, but the block is
acquired by another one that has the same task attempt id.
This PR remove the check.
## How was this patch tested?
Updated existing tests.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/davies/spark write_lock_conflict
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/13082.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #13082
----
commit 03aaf3723a1070d3e04be6ba925d85738bf51cf9
Author: Davies Liu <[email protected]>
Date: 2016-05-12T18:03:35Z
fix write lock conflict
commit c5916d050532a55cf1cb47bc6940f93f6ed76bf6
Author: Davies Liu <[email protected]>
Date: 2016-05-12T18:26:09Z
update tests
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
