Re: OOE during indexing

2008-01-22 Thread Mike Klaas
Queries involving sorting can occupy a lot of memory. During autowarming you need 2x peak memory usage. The only thing you can do is increase your max heap size or be careful about cache autowarming (possibly turning it off). cheers, -Mike On 21-Jan-08, at 9:44 PM, Marcus Herou wrote:

Re: OOE during indexing

2008-01-22 Thread Marcus Herou
Thanks for your reply. I set autowarmcount = 0 for both LRUCache and the queryCache but still I got these errors on heavy reindexing (4M docs as fast as possible each doc 10K). I removed firstSearcher and newSearcher but I still got the same errors. The strange thing is that now when the server

Re: OOE during indexing

2008-01-22 Thread Chris Hostetter
: I get OOE with Solr 1.3 Autowarm seem to be the villain in cojunction with : FieldCache somehow. : JVM args: -Xmx512m -Xms512m -Xss128k : : Index size is ~4 Million docs, where I index text and store database primary it seems naive to me to only allow 512MB for an index of 4 million docs --

Re: OOE during indexing

2008-01-22 Thread Marcus Herou
Thanks! Yes I agree (to a certain level) on me being naive. Currently I'm only using one server for this but will go into distributed snapshot/pull mode soon. Then I can tune the slaves differently then the master I believe. The master for instance do not need autowarming nor caches if not

Re: OOE during indexing

2008-01-22 Thread Marcus Herou
Yep but we hire these god damn boxes and then my friend memory costs per month = not cheap in long term. Something like 50$ / month for 2G more. I might be an ultra geek when it comes to Linux and programming but I'm not an ultra-geek building servers from scratch. But I will straighten up and

OOE during indexing

2008-01-21 Thread Marcus Herou
Hi. I get OOE with Solr 1.3 Autowarm seem to be the villain in cojunction with FieldCache somehow. JVM args: -Xmx512m -Xms512m -Xss128k Index size is ~4 Million docs, where I index text and store database primary keys. du /srv/solr/feedItem/data/index/ 1.7G/srv/solr/feedItem/data/index/ To