Re: indexWriter.addIndexes, Disk space, and open files

2010-06-08 Thread Michael McCandless
On Mon, Jun 7, 2010 at 7:19 AM, Regan Heath regan.he...@bridgeheadsoftware.com wrote: That's pretty much exactly what I suspected was happening.  I've had the same problem myself on another occasion... out of interest is there any way to force the file closed without flushing? No,

Re: indexWriter.addIndexes, Disk space, and open files

2010-06-07 Thread Regan Heath
If you don't want to use the ImDisk software, a small flash drive will do just as well... Regan Heath wrote: Windows XP. The problem occurs on the local file system, but to replicate it more easily I am using http://www.ltr-data.se/opencode.html#ImDisk to mount a virtual 10mb disk on

Re: indexWriter.addIndexes, Disk space, and open files

2010-06-07 Thread Michael McCandless
This is a bug in how Lucene handles IOException while closing files. Look at SegmentMerger's sources, for 2.3.2: https://svn.apache.org/repos/asf/lucene/java/tags/lucene_2_3_2/src/java/org/apache/lucene/index/SegmentMerger.java Look at the finally clause in mergeTerms: } finally {

Re: indexWriter.addIndexes, Disk space, and open files

2010-06-07 Thread Regan Heath
That's pretty much exactly what I suspected was happening. I've had the same problem myself on another occasion... out of interest is there any way to force the file closed without flushing? From memory I tried everything I could think of at the time but couldn't manage it. Best I could do was

Re: indexWriter.addIndexes, Disk space, and open files

2010-06-07 Thread Michael McCandless
On Mon, Jun 7, 2010 at 6:18 AM, Regan Heath regan.he...@bridgeheadsoftware.com wrote: That's pretty much exactly what I suspected was happening.  I've had the same problem myself on another occasion... out of interest is there any way to force the file closed without flushing? No, IndexOutput

Re: indexWriter.addIndexes, Disk space, and open files

2010-06-07 Thread Regan Heath
That's pretty much exactly what I suspected was happening.  I've had the same problem myself on another occasion... out of interest is there any way to force the file closed without flushing? No, IndexOutput has no such method. We could consider adding one... That sounds useful in general.

Re: indexWriter.addIndexes, Disk space, and open files

2010-05-25 Thread Erick Erickson
What op system and what file system are you using? Is the file system local or networked? What does it take to remove the files. That is, can you do it manually after the program shuts down? Best Erick On Tue, May 25, 2010 at 5:42 AM, Regan Heath regan.he...@bridgeheadsoftware.com wrote:

Re: indexWriter.addIndexes, Disk space, and open files

2010-05-25 Thread Regan Heath
Windows XP. The problem occurs on the local file system, but to replicate it more easily I am using http://www.ltr-data.se/opencode.html#ImDisk to mount a virtual 10mb disk on F:\. It is formatted as an NTFS file system. The files can be removed normally (delete from explorer or command