[
https://issues.apache.org/jira/browse/LUCENE-5925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Muir updated LUCENE-5925:
--------------------------------
Attachment: LUCENE-5925.patch
Updated patch. This also solves LUCENE-2585.
Today, segments.gen "helps" with the fact that directory listing is not point
in time (its a weakly consistent iterator and can reflect changes that happen
during iteration). But it cannot be totally relied upon due to timing (you can
get unlucky like LUCENE-2585).
Instead, in FindSegmentsFile, we simply detect that contents have changed
during the execution of listAll, by executing it again and doing a comparison.
This way we can detect "ConcurrentModificationException" and just continue the
loop.
> Use rename instead of segments_N fallback / segments.gen etc
> ------------------------------------------------------------
>
> Key: LUCENE-5925
> URL: https://issues.apache.org/jira/browse/LUCENE-5925
> Project: Lucene - Core
> Issue Type: Improvement
> Components: core/index, core/store
> Reporter: Robert Muir
> Labels: Java7
> Attachments: LUCENE-5925.patch, LUCENE-5925.patch
>
>
> Our commit logic is strange, we write corrupted commit points and only on the
> last phase of commit do we "correct them".
> This means the logic to get the latest commit is always scary and awkward,
> since it must deal with partial commits, and try to determine if it should
> fall back to segments_N-1 or actually relay an exception.
> This logic is incomplete/sheisty as we, e.g. i think we only fall back so far
> (at most one).
> If we somehow screw up in all this logic do the wrong thing, then we lose
> data (e.g. LUCENE-4870 wiped entire index because of TooManyOpenFiles).
> We now require java 7, i think we should expore instead writing
> {{pending_segments_N}} and then in finishCommit() doing an atomic rename to
> {{segments_N}}.
> We could then remove all the complex fallback logic completely, since we no
> longer have to deal with "ignoring partial commits", instead simply
> delivering any exception we get when trying to read the commit and sleep
> better at night.
> In java 7, we have the apis for this (ATOMIC_MOVE).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
