Re: solr 3.5 taking long to index

2012-04-15 Thread Lance Norskog
Subject: Re: solr 3.5 taking long to index On 4/12/2012 8:42 PM, Rohit wrote: The machine has a total ram of around 46GB. My Biggest concern is Solr index time gradually increasing and then the commit stops because of timeouts, out commit rate is very high, but I am not able to find

Re: solr 3.5 taking long to index

2012-04-15 Thread Yonik Seeley
On Thu, Apr 12, 2012 at 10:42 PM, Rohit ro...@in-rev.com wrote: The machine has a total ram of around 46GB. My Biggest concern is Solr index time gradually increasing and then the commit stops because of timeouts, out commit rate is very high, but I am not able to find the root cause of the

RE: solr 3.5 taking long to index

2012-04-14 Thread Rohit
: Re: solr 3.5 taking long to index On 4/12/2012 8:42 PM, Rohit wrote: The machine has a total ram of around 46GB. My Biggest concern is Solr index time gradually increasing and then the commit stops because of timeouts, out commit rate is very high, but I am not able to find the root cause

RE: solr 3.5 taking long to index

2012-04-13 Thread Rohit
April 2012 11:01 To: solr-user@lucene.apache.org Subject: Re: solr 3.5 taking long to index On 4/12/2012 8:42 PM, Rohit wrote: The machine has a total ram of around 46GB. My Biggest concern is Solr index time gradually increasing and then the commit stops because of timeouts, out commit rate

Re: solr 3.5 taking long to index

2012-04-12 Thread Bernd Fehling
There were some changes in solrconfig.xml between solr3.1 and solr3.5. Always read CHANGES.txt when switching to a new version. Also helpful is comparing both versions of solrconfig.xml from the examples. Are you sure you need a MaxPermSize of 5g? Use jvisualvm to see what you really need. This

RE: solr 3.5 taking long to index

2012-04-12 Thread Rohit
To: solr-user@lucene.apache.org Subject: Re: solr 3.5 taking long to index There were some changes in solrconfig.xml between solr3.1 and solr3.5. Always read CHANGES.txt when switching to a new version. Also helpful is comparing both versions of solrconfig.xml from the examples. Are you sure you need

Re: solr 3.5 taking long to index

2012-04-12 Thread Shawn Heisey
On 4/12/2012 12:42 PM, Rohit wrote: Thanks for pointing these out, but I still have one concern, why is the Virtual Memory running in 300g+? Solr 3.5 uses MMapDirectoryFactory by default to read the index. This does an mmap on the files that make up your index, so their entire contents are

RE: solr 3.5 taking long to index

2012-04-12 Thread Rohit
://about.me/rohitg -Original Message- From: Shawn Heisey [mailto:s...@elyograg.org] Sent: 13 April 2012 05:15 To: solr-user@lucene.apache.org Subject: Re: solr 3.5 taking long to index On 4/12/2012 12:42 PM, Rohit wrote: Thanks for pointing these out, but I still have one concern, why

Re: solr 3.5 taking long to index

2012-04-12 Thread Shawn Heisey
On 4/12/2012 8:42 PM, Rohit wrote: The machine has a total ram of around 46GB. My Biggest concern is Solr index time gradually increasing and then the commit stops because of timeouts, out commit rate is very high, but I am not able to find the root cause of the issue. For good performance,

solr 3.5 taking long to index

2012-04-11 Thread Rohit
We recently migrated from solr3.1 to solr3.5, we have one master and one slave configured. The master has two cores, 1) Core1 - 44555972 documents 2) Core2 - 29419244 documents We commit every 5000 documents, but lately the commit is taking very long 15 minutes plus in some cases. What

Re: solr 3.5 taking long to index

2012-04-11 Thread Lance Norskog
It's telling you the problem. Try your solrconfig.xml against the one in 3.5/solr/example/solr/conf. You will what has changed in the suggested tools. On Wed, Apr 11, 2012 at 10:42 AM, Rohit ro...@in-rev.com wrote: We recently migrated from solr3.1 to solr3.5,  we have one master and one