Github user cloud-fan commented on the pull request:
https://github.com/apache/spark/pull/791#issuecomment-43733545
`ensureFreeSpace` has 2 jobs. 1) iterate entries and select blocks to be
dropped. 2) if to-be-dropped blocks can free enough space, mark them as
dropping and return them to the caller.
`ensureFreeSpace` is called within putLock, so each thread will see the
dropping flag modification(I will discuss flag resetting in exception handling
later) and thus get different to-be-dropped blocks. And block reading don't
need the dropping flag so no conflict there. Let's consider block removing and
exception handling(reset dropping flag)
Job 1 of `ensureFreeSpace`(selecting) and removing are both synchronized by
`entries`, so they must process by turn.
If a block is removed first, then everything is OK.
If a block is removed after Job 2 of `ensureFreeSpace`(marking) which is
also synchronized by `entries`(in my modification), then the block will be
dropped into disk and managed by diskStore, which I think is OK.
If a block is removed between selecting and marking, the marking will check
if entry is null, so it's OK, too.
About exception handling, flag resetting is also synchronized by `entries`,
so it won't process during selecting and marking.
If resetting happened before selecting, then selecting will be able to
select these blocks and re-drop them.
If resetting happened after selecting, which means the selected
to-be-dropped blocks won't include the resetted blocks, so there is no conflict.
Actually there are 3 place that write or read the dropping flag(selecting,
marking and resetting) and they are all synchronized by `entries`, so I think
we don't need to define the flag as volatile.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
