[
https://issues.apache.org/jira/browse/LUCENE-1192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alex Falca updated LUCENE-1192:
-------------------------------
Description:
Hi,
I'm getting FileNotFoundException on adding RAMDirectory to an IndexWriter with
following stack trace:
{noformat}
java.io.FileNotFoundException: _8.fnm
at org.apache.lucene.store.RAMDirectory.openInput(RAMDirectory.java:234)
at org.apache.lucene.index.FieldInfos.<init>(FieldInfos.java:57)
at
org.apache.lucene.index.SegmentReader.initialize(SegmentReader.java:298)
at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:262)
at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:221)
at
org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:3099)
at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:2834)
at
org.apache.lucene.index.IndexWriter.copyExternalSegments(IndexWriter.java:2263)
at
org.apache.lucene.index.IndexWriter.addIndexesNoOptimize(IndexWriter.java:2238)
{noformat}
Same code works fine with Lucene 2.2.0. Basically I'm trying to index a huge
number of documents (~21mln) having following workflow:
1. First documents are added to and index which use a RAMDirectory as a storage
2. Once a given condition occurs RAM based index writer is flushed and his
directory is being added to another index writer, which use a FSDirectory as a
storage.
Sample code:
{code}
IndexWriter mainIndexWriter = new IndexWriter(indexDir, new JapaneseAnalyzer(),
true);
IndexWriter ramIndexWriter = new IndexWriter(new RAMDirectory(), new
JapaneseAnalyzer(), true);
mainIndexWriter.setMergeFactor(10);
mainIndexWriter.setUseCompoundFile(false);
ramIndexWriter.setUseCompoundFile(false);
// Here some code for adding documents to RAM index writer
...................................................................................................................
// method where I'm flushing from memory to disk and getting exception
ramIndexWriter.flush();
mainIndexWriter.addIndexesNoOptimize(new Directory[]
{ramIndexWriter.getDirectory()});
ramIndexWriter.close();
ramIndexWriter = new IndexWriter(new RAMDirectory(), new JapaneseAnalyzer(),
true);
ramIndexWriter.setUseCompoundFile(false);
System.gc();
{code}
I understand that I should be using new features of IndexWriter like flushing
by RAM Usage, but wanna see if there any bugs in my case. Possible I'm facing
issue described in LUCENE-1175, but need a confirmation on that.
was:
Hi,
I'm getting FileNotFoundException on adding RAMDirectory to an IndexWriter with
following stack trace:
{noformat}
java.io.FileNotFoundException: _8.fnm
at org.apache.lucene.store.RAMDirectory.openInput(RAMDirectory.java:234)
at org.apache.lucene.index.FieldInfos.<init>(FieldInfos.java:57)
at
org.apache.lucene.index.SegmentReader.initialize(SegmentReader.java:298)
at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:262)
at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:221)
at
org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:3099)
at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:2834)
at
org.apache.lucene.index.IndexWriter.copyExternalSegments(IndexWriter.java:2263)
at
org.apache.lucene.index.IndexWriter.addIndexesNoOptimize(IndexWriter.java:2238)
{noformat}
Same code works fine with Lucene 2.2.0. Basically I'm trying to index a huge
number of documents (~21mln) having following workflow:
1. First documents are added to and index which use a RAMDirectory as a storage
2. Once a given condition occurs RAM based index writer is flushed and his
directory is being added to another index writer, which use a FSDirectory as a
storage.
Sample code:
{code}
IndexWriter mainIndexWriter = new IndexWriter(indexDir, new JapaneseAnalyzer(),
true);
IndexWriter ramIndexWriter = new IndexWriter(new RAMDirectory(), new
JapaneseAnalyzer(), true);
mainIndexWriter.setMergeFactor(10);
mainIndexWriter.setUseCompoundFile(false);
ramIndexWriter.setUseCompoundFile(false);
// Here some code for adding documents to RAM index writer
...................................................................................................................
// method where I'm flushing from memory to disk and getting exception
ramIndexWriter.flush();
mainIndexWriter.addIndexesNoOptimize(new Directory[]
{ramIndexWriter.getDirectory()});
ramIndexWriter.close();
ramIndexWriter = new IndexWriter(new RAMDirectory(), new JapaneseAnalyzer(),
true);
ramIndexWriter.setUseCompoundFile(false);
System.gc();
{code}
I understand that I should be using new features of IndexWriter like flushing
by RAM Usage, but wanna see if there any bugs in my case. Possible I'm facing
issue described in LUCENE-1175, but need a confirmation on that.
If you need details on environment(i.e. Java version, settings, hardware
details etc.) - please let me know.
Environment:
|| Description || Value ||
| java -version | {noformat}java version "1.5.0_14"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_14-b03)
Java HotSpot(TM) 64-Bit Server VM (build 1.5.0_14-b03, mixed mode)
{noformat} |
| JVM settings | {noformat}-server -verbose:gc -Xloggc:gc.log
-XX:+PrintGCApplicationStoppedTime
-XX:NewRatio=2 -XX:ParallelGCThreads=4 -Xms6000m -Xmx6000m{noformat} |
| uname -a | {noformat}Linux pdbindexer2 2.6.22.14-72.fc6 #1
SMP Wed Nov 21 14:10:25 EST 2007 x86_64 x86_64 x86_64 GNU/Linux{noformat} |
| CPU | Dual Core AMD Opteron(tm) Processor 280 |
| Total available memory | 8G |
Updated details on environment
> FileNotFound exception on adding a RAMDirectory to an IndexWriter
> -----------------------------------------------------------------
>
> Key: LUCENE-1192
> URL: https://issues.apache.org/jira/browse/LUCENE-1192
> Project: Lucene - Java
> Issue Type: Bug
> Affects Versions: 2.3.1
> Environment: || Description || Value ||
> | java -version | {noformat}java version "1.5.0_14"
> Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_14-b03)
> Java HotSpot(TM) 64-Bit Server VM (build 1.5.0_14-b03, mixed mode)
> {noformat} |
> | JVM settings | {noformat}-server -verbose:gc -Xloggc:gc.log
> -XX:+PrintGCApplicationStoppedTime
> -XX:NewRatio=2 -XX:ParallelGCThreads=4 -Xms6000m -Xmx6000m{noformat} |
> | uname -a | {noformat}Linux pdbindexer2 2.6.22.14-72.fc6 #1
> SMP Wed Nov 21 14:10:25 EST 2007 x86_64 x86_64 x86_64 GNU/Linux{noformat} |
> | CPU | Dual Core AMD Opteron(tm) Processor 280 |
> | Total available memory | 8G |
> Reporter: Alex Falca
>
> Hi,
> I'm getting FileNotFoundException on adding RAMDirectory to an IndexWriter
> with following stack trace:
> {noformat}
> java.io.FileNotFoundException: _8.fnm
> at
> org.apache.lucene.store.RAMDirectory.openInput(RAMDirectory.java:234)
> at org.apache.lucene.index.FieldInfos.<init>(FieldInfos.java:57)
> at
> org.apache.lucene.index.SegmentReader.initialize(SegmentReader.java:298)
> at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:262)
> at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:221)
> at
> org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:3099)
> at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:2834)
> at
> org.apache.lucene.index.IndexWriter.copyExternalSegments(IndexWriter.java:2263)
> at
> org.apache.lucene.index.IndexWriter.addIndexesNoOptimize(IndexWriter.java:2238)
> {noformat}
> Same code works fine with Lucene 2.2.0. Basically I'm trying to index a huge
> number of documents (~21mln) having following workflow:
> 1. First documents are added to and index which use a RAMDirectory as a
> storage
> 2. Once a given condition occurs RAM based index writer is flushed and his
> directory is being added to another index writer, which use a FSDirectory as
> a storage.
> Sample code:
> {code}
> IndexWriter mainIndexWriter = new IndexWriter(indexDir, new
> JapaneseAnalyzer(), true);
> IndexWriter ramIndexWriter = new IndexWriter(new RAMDirectory(), new
> JapaneseAnalyzer(), true);
> mainIndexWriter.setMergeFactor(10);
> mainIndexWriter.setUseCompoundFile(false);
> ramIndexWriter.setUseCompoundFile(false);
> // Here some code for adding documents to RAM index writer
> ...................................................................................................................
> // method where I'm flushing from memory to disk and getting exception
> ramIndexWriter.flush();
> mainIndexWriter.addIndexesNoOptimize(new Directory[]
> {ramIndexWriter.getDirectory()});
> ramIndexWriter.close();
> ramIndexWriter = new IndexWriter(new RAMDirectory(), new JapaneseAnalyzer(),
> true);
> ramIndexWriter.setUseCompoundFile(false);
> System.gc();
> {code}
> I understand that I should be using new features of IndexWriter like flushing
> by RAM Usage, but wanna see if there any bugs in my case. Possible I'm facing
> issue described in LUCENE-1175, but need a confirmation on that.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
