[
https://issues.apache.org/jira/browse/LUCENE-3218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13087712#comment-13087712
]
Robert Muir commented on LUCENE-3218:
-------------------------------------
This is definitely not a bug in the directory, and its a serious issue (i think
a blocker for release myself).
I'll try to explain the issue again a little better than I did on
https://issues.apache.org/jira/browse/LUCENE-3380?focusedCommentId=13086872&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13086872
This is just an example of the API problem, with FileSwitchDirectory.
In Lucene we have FileSwitchDirectory which is a Directory that lets you
"switch" between 2 different directory implementations based on file extension.
So conceptually it looks like this:
{code}
FileSwitchDirectory extends Directory {
Directory a;
Directory b;
Set extensions; // these are the file extensions that go to "a", all other
ones are handled by "b"
}
{code}
Imagine you configure this directory to put all "*.cfs" in "a", and everything
else in "b".
So when FileSwitchDirectory is asked where to put "1.cfs", it forwards the
request to "a".
But the "1.cfe" file is actually wrongly created in "a" also, causing
FileNotFoundExceptions later when the file is to be read, because its in the
wrong directory. This is because of how the compound file mechanism works now,
it calls a.createOutput("1.cfe") instead of
fileswitchdirectory.createOutput("1.cfe").
So this is a serious problem for any Directories that delegate responsibility
like this, not just the ones in Lucene.
> Make CFS appendable
> ---------------------
>
> Key: LUCENE-3218
> URL: https://issues.apache.org/jira/browse/LUCENE-3218
> Project: Lucene - Java
> Issue Type: Improvement
> Components: core/index
> Affects Versions: 4.0
> Reporter: Simon Willnauer
> Assignee: Simon Willnauer
> Fix For: 3.4, 4.0
>
> Attachments: LUCENE-3218.patch, LUCENE-3218.patch, LUCENE-3218.patch,
> LUCENE-3218.patch, LUCENE-3218_3x.patch, LUCENE-3218_test_fix.patch,
> LUCENE-3218_tests.patch
>
>
> Currently CFS is created once all files are written during a flush / merge.
> Once on disk the files are copied into the CFS format which is basically a
> unnecessary for some of the files. We can at any time write at least one file
> directly into the CFS which can save a reasonable amount of IO. For instance
> stored fields could be written directly during indexing and during a Codec
> Flush one of the written files can be appended directly. This optimization is
> a nice sideeffect for lucene indexing itself but more important for DocValues
> and LUCENE-3216 we could transparently pack per field files into a single
> file only for docvalues without changing any code once LUCENE-3216 is
> resolved.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
