Queries involving sorting can occupy a lot of memory. During
autowarming you need 2x peak memory usage. The only thing you can do
is increase your max heap size or be careful about cache autowarming
(possibly turning it off).
cheers,
-Mike
On 21-Jan-08, at 9:44 PM, Marcus Herou wrote:
Hi.
I get OOE with Solr 1.3 Autowarm seem to be the villain in
cojunction with FieldCache somehow.
JVM args: -Xmx512m -Xms512m -Xss128k
Index size is ~4 Million docs, where I index text and store
database primary keys.
du /srv/solr/feedItem/data/index/
1.7G /srv/solr/feedItem/data/index/
To ensure that the docs I index do not swell to much I only allow
5K per doc to over the wire i.e. I substring 0, 5000 on the field
"content"
I have removed "firstSearcher" and "newSearcher" since the queries
I used before killed performance on reindexing the whole index. I
will add them later again when I get into a delta update index state.
Stacktrace.
[06:25:53.122] [null] /update wt=xml&version=2.2 0 3165
[06:25:53.877] Error during auto-warming of
key:[EMAIL PROTECTED]:java.lang.OutOfMemo
ryError: Java heap space
[06:25:53.877] at org.apache.lucene.index.TermBuffer.toTerm
(TermBuffer.java:104)
[06:25:53.877] at org.apache.lucene.index.SegmentTermEnum.term
(SegmentTermEnum.java:159)
[06:25:53.877] at org.apache.lucene.index.SegmentMergeInfo.next
(SegmentMergeInfo.java:66)
[06:25:53.877] at org.apache.lucene.index.MultiTermEnum.next
(MultiReader.java:315)
[06:25:53.877] at org.apache.lucene.search.FieldCacheImpl
$10.createValue(FieldCacheImpl.java:388)
[06:25: 53.877] at org.apache.lucene.search.FieldCacheImpl
$Cache.get(FieldCacheImpl.java:72)
[06:25:53.877] at
org.apache.lucene.search.FieldCacheImpl.getStringIndex
(FieldCacheImpl.java:350)
[06:25:53.877] at
org.apache.lucene.search.FieldSortedHitQueue.comparatorString
(FieldSortedHitQueue.java:266)
[06:25:53.877] at org.apache.lucene.search.FieldSortedHitQueue
$1.createValue(FieldSortedHitQueue.java:182)
[06:25:53.877] at org.apache.lucene.search.FieldCacheImpl$Cache.get
(FieldCacheImpl.java :72)
[06:25:53.877] at
org.apache.lucene.search.FieldSortedHitQueue.getCachedComparator
(FieldSortedHitQueue.java:155)
[06:25:53.877] at
org.apache.lucene.search.FieldSortedHitQueue.<init>
(FieldSortedHitQueue.java :56)
[06:25:53.877] at
org.apache.solr.search.SolrIndexSearcher.getDocListNC
(SolrIndexSearcher.java:862)
[06:25:53.877] at
org.apache.solr.search.SolrIndexSearcher.getDocListC
(SolrIndexSearcher.java:808)
[06:25: 53.877] at org.apache.solr.search.SolrIndexSearcher.access
$000(SolrIndexSearcher.java:56)
[06:25:53.877] at org.apache.solr.search.SolrIndexSearcher
$2.regenerateItem(SolrIndexSearcher.java:254)
[06:25:53.877] at org.apache.solr.search.LRUCache.warm
(LRUCache.java:192)
[06:25:53.877] at org.apache.solr.search.SolrIndexSearcher.warm
(SolrIndexSearcher.java:1393)
[06:25:53.877] at org.apache.solr.core.SolrCore$2.call
(SolrCore.java :702)
[06:25:53.877] at java.util.concurrent.FutureTask$Sync.innerRun
(FutureTask.java:269)
[06:25:53.877] at java.util.concurrent.FutureTask.run
(FutureTask.java:123)
[06:25:53.877] at java.util.concurrent.ThreadPoolExecutor
$Worker.runTask (ThreadPoolExecutor.java:650)
[06:25:53.877] at java.util.concurrent.ThreadPoolExecutor
$Worker.run(ThreadPoolExecutor.java:675)
[06:25:53.877] at java.lang.Thread.run(Thread.java:595)
Help anyone?
Attaching schema.xml and solrconfig.xml
Kindly
//Marcus Herou
<schema.xml><solrconfig.xml>
