Re: Does Lucene modifies existing file in an index

2014-10-27 Thread Chetan Mehrotra
To close the thread. With modified testcase [1] the Lucene index file do not get updated. Is this switch from cfs format to storing in separate files is automatic and done by Lucene after index reaches certain size. Or this done something specifically in Oak? This is done because as per

Re: Does Lucene modifies existing file in an index

2014-10-21 Thread Alex Parvulescu
The index indeed gets rebuilt. In IndexUpdate.collectIndexEditors() the provider does not return any editors and the following code is executed OAK-2203 On Tue, Oct 21, 2014 at 8:37 AM, Marcel Reutegger mreut...@adobe.com wrote: Hi, this is the output when I run it on my machine within

Does Lucene modifies existing file in an index

2014-10-20 Thread Chetan Mehrotra
While working on copy on read directory support (OAK-1724) and was checking how Lucene manages the index files. Following observation can be made with various test runs A - Small Index use Compound File format -- If index contain few entries then it seems it uses the compound

Re: Does Lucene modifies existing file in an index

2014-10-20 Thread Marcel Reutegger
Hi, this is very strange. in my experience .cfs files are written once and never modified. this write-once pattern is actually used for almost all files, except the segments.gen file you mentioned. E.g. see [0] by Mike McCandless when he talks about LUCENE-5574. is it possible the entire lucene

Re: Does Lucene modifies existing file in an index

2014-10-20 Thread Thomas Mueller
Hi, This blog post is interesting: they are using a physical switch (similar to a christmas light timer) to test a Lucene index doesn't get corrupt on power failure. It would be nice if we can do something similar with the Segment storage at some point. Regards, Thomas On 20/10/14 13:36,

Re: Does Lucene modifies existing file in an index

2014-10-20 Thread Chetan Mehrotra
Hi Marcel, in my experience .cfs files are written once and never modified I have checked in a testcase with [1] and if you run that you would see following output which indicate that same file is getting updated. _0.cfs - 621 _0.cfe - 194 segments.gen - 20 segments_1 -