[
https://issues.apache.org/jira/browse/LUCENE-2361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12853154#action_12853154
]
Shivender Devarakonda edited comment on LUCENE-2361 at 4/3/10 4:26 PM:
-----------------------------------------------------------------------
Thanks for helping us on this.
We create a Daemon Thread that does the index updating functionality.
1- Create a IndexDir like this:
fDirectoryName = new File(tracesDir, relativeIndexDir);
fIndexDir = FSDirectory.open(fDirectoryName,
NoLockFactory.getNoLockFactory());
fIndexDir.setReadChunkSize(8192);
2- The Index inserter thread does the following functionality, The new writer
is invoked for a batch size of 100.
IndexWriter writer = null;
try
{
writer = new IndexWriter(fIndexDir, this.getAnalyzer(),
!this.indexExists(), IndexWriter.MaxFieldLength.LIMITED);
writer.setMaxBufferedDocs(batchSize);
writer.addDocument(doc);
// while we're open and there's more in the queue, add more
numInserted += addMore(writer, luceneQueue, batchSize);
}
finally
{
Utils.close(writer);
insertionTime = MasterClock.currentTimeMillis() - startTime;
}
We do not set any RAMBufferSize in this case.
Please let me know if you need anything on this.
was (Author: shivenderd):
Thanks for helping us on this.
We create a Daemon Thread that does the index updating functionality.
1- Create a IndexDir like this:
fDirectoryName = new File(tracesDir, relativeIndexDir);
fIndexDir = FSDirectory.open(fDirectoryName,
NoLockFactory.getNoLockFactory());
fIndexDir.setReadChunkSize(8192);
2- The Index inserter thread does the following functionality, The new writer
is invoked for a batch size of 100.
IndexWriter writer = null;
try
{
writer = new IndexWriter(fIndexDir, this.getAnalyzer(),
!this.indexExists(), IndexWriter.MaxFieldLength.LIMITED);
writer.setMaxBufferedDocs(batchSize);
writer.addDocument(doc);
// while we're open and there's more in the queue, add more
numInserted += addMore(writer, luceneQueue, batchSize);
}
finally
{
Utils.close(writer);
insertionTime = MasterClock.currentTimeMillis() - startTime;
}
We do not set any RAMBufferSize in this case.
Please let me know if you need anything on this.
> OutOfMemoryException while Indexing
> -----------------------------------
>
> Key: LUCENE-2361
> URL: https://issues.apache.org/jira/browse/LUCENE-2361
> Project: Lucene - Java
> Issue Type: Bug
> Components: Index
> Affects Versions: 2.9.1
> Environment: Windows
> Reporter: Shivender Devarakonda
>
> Hi,
> We use lucene 2.9.1 version.
> We see the following OutOfMemory error in our environment, I think This is
> happening at a significant high load. Have you observed this anytime? Please
> let me know your thoughts on this.
> org.apache.lucene.index.MergePolicy$MergeException:
> java.lang.OutOfMemoryError: PermGen space
> at
> org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(ConcurrentMergeScheduler.java:351)
> at
> org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:315)
> Caused by: java.lang.OutOfMemoryError: PermGen space
> at java.lang.String.$$YJP$$intern(Native Method)
> at java.lang.String.intern(Unknown Source)
> at
> org.apache.lucene.util.SimpleStringInterner.intern(SimpleStringInterner.java:74)
> at org.apache.lucene.util.StringHelper.intern(StringHelper.java:36)
> at org.apache.lucene.index.FieldInfos.read(FieldInfos.java:356)
> at org.apache.lucene.index.FieldInfos.<init>(FieldInfos.java:71)
> at
> org.apache.lucene.index.SegmentReader$CoreReaders.<init>(SegmentReader.java:116)
> at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:638)
> at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:608)
> at
> org.apache.lucene.index.IndexWriter$ReaderPool.get(IndexWriter.java:686)
> at
> org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4979)
> at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:4614)
> at
> org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:235)
> at
> org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:291)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: java-dev-h...@lucene.apache.org
