[
https://issues.apache.org/jira/browse/LUCENE-3237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13965182#comment-13965182
]
Simon Willnauer commented on LUCENE-3237:
-----------------------------------------
Hey mike, thanks for reopening this. I like the patch since it fixes multiple
issues.
* I like the fact that we get rid of the general unsynced files stuff in
Directory.
* given the last point we move it in the right place inside IW that is where
it should be
* the problem that the current patch has is that is holds on to the buffers in
BufferedIndexOutput. I think we need to work around this here are a couple of
ideas:
** introduce a SyncHandle class that we can pull from IndexOutput that
allows to close the IndexOutput but lets you fsync after the fact
** this handle can be refcounted internally and we just decrement the count
on IndexOutput#close() as well as on SyncHandle#close()
** we can just hold on to the SyncHandle until we need to sync in IW
** since this will basically close the underlying FD later we might want to
think about size-bounding the number of unsynced files and maybe let indexing
threads fsync them concurrently? maybe something we can do later.
** if we know we flush for commit we can already fsync directly which might
safe resources / time since it might be concurrent
just a couple of ideas....
> FSDirectory.fsync() may not work properly
> -----------------------------------------
>
> Key: LUCENE-3237
> URL: https://issues.apache.org/jira/browse/LUCENE-3237
> Project: Lucene - Core
> Issue Type: Bug
> Components: core/store
> Reporter: Shai Erera
> Attachments: LUCENE-3237.patch
>
>
> Spinoff from LUCENE-3230. FSDirectory.fsync() opens a new RAF, sync() its
> FileDescriptor and closes RAF. It is not clear that this syncs whatever was
> written to the file by other FileDescriptors. It would be better if we do
> this operation on the actual RAF/FileOS which wrote the data. We can add
> sync() to IndexOutput and FSIndexOutput will do that.
> Directory-wise, we should stop syncing on file names, and instead sync on the
> IOs that performed the write operations.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
