I will do that the next time I see the same problem. Sorry I misunderstood the word "index" in your mail. I will definitely try your suggestion next time I got the truncated seg. Or better, if we modify the tool segread -fix to automatically delete that "index" file, that'll be great! I like your proposed fix to the MapFile to refuse openning a truncated file unless being forced to. I cannot wait to see your patch!
Thanks a lot! Jay -----Original Message----- From: Andrzej Bialecki [mailto:[EMAIL PROTECTED] Sent: Wednesday, April 13, 2005 11:29 AM To: [email protected] Subject: Re: MapFile.Reader bug (Re: Optimal segment size?) Jay Yu wrote: > Andrzej: > Thank you for your response to my comments. > The reason I said there may be bug in the fetcher is that in our case there > was no JVM crash or OOM Exception during the fetch and the fetch process was > successful by reading the log. > file. So I cannot tell what caused the truncation (Unexpected EOF exception > in the reader) > > The problem with the MapFile is not that the performance drops, instead, it > simply hangs on a deadlock ( I looked at the thread dump). I do not > understand why a segread would need a write lock on the seg. That's strange, indeed... Could you perhaps do a thread dump (^E or similar signal on Linux)? Are you sure you have killed the other process (fetcher) that was writing to this segment? > Your proposed manual step to fix the index may not work simply because > UnExpected EOF is in the fetch output (seq file), not the index. It will work, the "index" file I'm speaking here of, is not a Lucene index, it's one of the two files that constitute a MapFile ("data" and "index"). There is a pair of such files inside every segment's subdirectories, among others in fetcher_output. -- Best regards, Andrzej Bialecki ___. ___ ___ ___ _ _ __________________________________ [__ || __|__/|__||\/| Information Retrieval, Semantic Web ___|||__|| \| || | Embedded Unix, System Integration http://www.sigram.com Contact: info at sigram dot com
