Karel Tejnora <[EMAIL PROTECTED]> wrote on 03/05/2006 02:29:06 AM:> >.. > Before compound file is created the segments info are written but points > to non-existing coumpound file then new .tmp is created and renamed to .cfs > Between time when new segments was written and new cfs is creating Index > reader in different application whith no care about a lock can take > segment with empty cfs resulting in empty hits.
Hi Karel, I've also been bothered by Lucene crash-intolerance bugs, where killing Lucene in a specific (ill-)chosen moment leaves you with an unusable index. I filed a bug report in http://issues.apache.org/jira/browse/LUCENE-554 about another case where a crash can leave in an inconsistent state, but it likely that the problem you reported is even more serious, because it appears that the amount of time we are at risk is relatively high in the case you reported, unlike in my bug report which requires extraordinary bad luck to see. > I see solution in swap those two blocks. firstly create final cfs and > than write and delete old segments. > Has anybody seen a problem with that? Looking at the code, I see no reason why your suggested fix won't work, and I don't see any negative side-effects. But I'm new to Lucene so I hope someone with more experience can take a look. By the way, I wonder why the current code first creates a file called mergedName+".tmp", and only later renames it to ".cfs". What's the point of doing that, when directory.renameFile() is not atomic (see LUCENE-554), and in some cases it even resorts to copying (see FSDirectory.renameFile()'s comments about "some jvms".) And I also wonder why the existing code does the renameFile() with the commit lock held, and not outside it. > I can send patch but firstly I need to find svn client in gentoo :) and > it's to late here. > Can be smb so kind and give me link where I can find how to generate > patch in lucene/apache way? I'm sorry I can't really help you with that, I'm also new to Lucene and now working on my first patch for it... Good luck, Nadav. -- Nadav Har'El --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
