Floyd, I think Digy is probably correct, the index you posted earlier (documents.zip) appears corrupt.
I looked it over as I have been trying to collect the information needed to document a bug in Luke related to the inability to open indexes which have not been compacted after deleting documents. Inspecting your index there is 1 live document, and 2 deleted. The live one has a number of fields that do not look useful: Field: ???, Value: Field: ???, Value: Field: ????, Value: Field: ????, Value: Field: ????, Value: Field: ??????, Value: Field: ??, Value: Field: ??, Value: Field: ??, Value: If I run optimize to compact the index; Luke no longer displays an error, but it acts as if the index is entirely empty. Inspecting the compacted index, there are now 3 live documents. That the number of live documents changed as the result of optimize and that there are so many meaningless fields points to a corrupt index. -- Neal ________________________________________ From: Floyd Wu [mailto:[email protected]] Sent: Thursday, April 23, 2009 9:33 PM To: [email protected] Subject: Re: Luke-0.9.2 cannot open index files Hi DIGY Okay thanks, but I found another strange thing that the same system but different index files can be opened by Luke -0.9.2. What's the difference between two index file(one is sent before, the other as attachment of this mail) were fields count. You could compare these two by using Luke-0.8.1. Floyd 2009/4/24 Digy <[email protected]> Hi Floyd, Don't take the example seriously. It is just to show, how indexing threads can be blocked, and when the application exits you get an corrupted index. DIGY. -----Original Message----- From: Floyd Wu [mailto:[email protected]] Sent: Thursday, April 23, 2009 6:48 PM To: [email protected] Subject: Re: Luke-0.9.2 cannot open index files Hi Digy If the root of problem is lock(), I wonder why the same index files can be opened with previous version of Luke? i'm not very sure what happened with new version of Lucene.Net. But I will try to modified my code with no lock statement. FYI, in single user situation, the index files generated from my system could not be opened with new Luke also. 2009/4/23 Digy <[email protected]> > Hi Floyd, > > > > For ex, see the sample code below: If you lock IndexWriter the indexing > code > gets stuck. But If you don't use the lock and let the Lucene do it > internally, indexing goes fine. Maybe your index gets corrupted with such a > code? > > > > DIGY. > > > > > > using System; > > using System.Collections.Generic; > > using System.ComponentModel; > > using System.Data; > > using System.Drawing; > > using System.Text; > > using System.Windows.Forms; > > > > using System.IO; > > using Lucene.Net.Analysis.Unicode; > > using Lucene.Net.Index; > > using Lucene.Net.Search; > > using Lucene.Net.QueryParsers; > > using Lucene.Net.Documents; > > > > using System.Threading; > > > > > > > > > > namespace LuceneTest > > { > > public partial class IndexingSetsStuck : Form > > { > > public IndexingSetsStuck() > > { > > InitializeComponent(); > > } > > > > private void button1_Click(object sender, EventArgs e) > > { > > Thread t = new Thread(new ThreadStart(CreateIndex)); > > t.IsBackground = true; > > t.Start(); > > } > > > > private void timer1_Tick(object sender, EventArgs e) > > { > > this.Text = Count.ToString(); > > } > > > > > > string INDEX = @"TestIndex"; > > IndexWriter writer = null; > > object LockObject = new object(); > > int Count = 0; > > > > void Worker() > > { > > Field f = new Field("Line", "", Field.Store.YES, > Field.Index.TOKENIZED); > > Document d = new Document(); > > d.Add(f); > > > > string line = ""; > > for (int i = 0; i < 50000; i++) > > { > > line = i.ToString().PadLeft(12, '0') + " " + > i.ToString("x").PadLeft(10, '0'); > > f.SetValue(line); > > writer.AddDocument(d); > > > > lock (LockObject) Count++; > > } > > } > > > > void CreateIndex() > > { > > writer = new IndexWriter(INDEX, new > Lucene.Net.Analysis.WhitespaceAnalyzer(), true); > > > > /*****!!! ?????? !!!*****/ > > //lock (writer) //Indexing gets stuck when this line is > uncommented > > { > > Thread[] threads = new Thread[5]; > > for (int i = 0; i < threads.Length; i++) > > { > > threads[i] = new Thread(new ThreadStart(Worker)); > > threads[i].IsBackground = true; > > threads[i].Start(); > > } > > for (int i = 0; i < threads.Length; i++) > > { > > threads[i].Join(); > > } > > } > > writer.Close(true); > > > > MessageBox.Show("Indexing Completed"); > > } > > } > > } > > > > > > > > -----Original Message----- > From: Floyd Wu [mailto:[email protected]] > Sent: Thursday, April 23, 2009 1:26 PM > To: [email protected] > Subject: Re: Luke-0.9.2 cannot open index files > > > > Hi Digy, > > > > here is my code for add a document to lucene > > > > internal static void AddContent(Content content) > > { > > lock (writer) > > { > > writer.Open(); > > try > > { > > writer.AddContent(content); > > } > > finally > > { > > writer.Close(); > > } > > } > > } > > > > No magic, just like ordinary usage. > > > > Floyd > > > > > > > > 2009/4/23 Digy <[email protected]> > > > > > Hi Floyd, > > > > > > Is it possible to send your indexing code? > > > > > > DIGY > > > > > > -----Original Message----- > > > From: Floyd Wu [mailto:[email protected]] > > > Sent: Thursday, April 23, 2009 12:56 PM > > > To: [email protected] > > > Subject: Re: Luke-0.9.2 cannot open index files > > > > > > Hello Digy, > > > > > > Actually my index files that you owned is created by Lucene.Net-2.3.2 in > > > trunk this morning. > > > > > > Floyd > > > > > > > > > > > > 2009/4/23 Digy <[email protected]> > > > > > > > Hi Floyd, > > > > I am using Luke 0.9.1. Altought it has the same problem while opening > > > your > > > > index, it can open my indices created with Lucene.Net 2.3.2 smoothly. > > > > Can you try to create the same index with Lucene.Net 2.3.2 in trunk? > > > > > > > > DIGY > > > > > > > > -----Original Message----- > > > > From: Floyd Wu [mailto:[email protected]] > > > > Sent: Thursday, April 23, 2009 10:49 AM > > > > To: [email protected] > > > > Subject: Re: Luke-0.9.2 cannot open index files > > > > > > > > Hi Jeroen, > > > > First I'm sure that close the writer after doing my job. And if not > > > > properly > > > > closed writer is the main point, why Luke-0.8.1 could open? So I guess > > > > there > > > > must something wrong or changed between Lucene.Net or Luke. > > > > > > > > I've asked the same question to the author of Luke, and he( or she) > told > > > me > > > > that index files format mabe changed but not very sure. > > > > > > > > > > > > > > > > 2009/4/23 Jeroen Lauwers <[email protected]> > > > > > > > > > Hi, > > > > > > > > > > Just a thought: > > > > > Are you sure your IndexWriter was properly closed? > > > > > I have had the same symptoms when I forgot to call the Close() > method. > > > > > > > > > > Jeroen > > > > > > > > > > From: Floyd Wu [mailto:[email protected]] > > > > > Sent: donderdag 23 april 2009 8:24 > > > > > To: [email protected] > > > > > Subject: Luke-0.9.2 cannot open index files > > > > > > > > > > Hi does anyone have the some problem? > > > > > My index files is built by Lucene.Net 2.3.1 and it can't be opend by > > > java > > > > > tool "Luke-0.9.2". > > > > > I tried to open the same index files with Luke-0.8.1 and it works > fine. > > > > > > > > > > Attachment is my index files. > > > > > Every time I opened index files with Luke-0.9.2, Luke will show a > > > message > > > > > "Read past EOF". > > > > > > > > > > Please help on this. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
