CPU utilization is 90-100%, IO wait is 0.5-3%, The server is: CPU 2 x Xeon E5606 2.13GHz 8MB Cache 4.8GT/sec (8 total cores) Memory 32 GB RAM Disk Space 10 x 300GB 15k RPM Seagate SAS Raid Card(s)
OS scientific 6.3 This high CPU usage is only during indexing, since we have to recreate the whole index. Also we do not have a very high query volume. App is supposed to be for internal company usage only, so there are no thousands of users. But some of our background services use Solr to analyze some sorts of data. E.g. to perform keywords match and tag the content, etc. Current index size is 64GB, but it's only a part of the data. After complete re- indexing its size grow up to 160-180GB and always growing. But it is not optimized well, we're using EdgeNGramFilterFactory with minGramSize="1" where it is necessary and where it is not so think that size could be reduced at 30-50% after optimization. Best, Alex Friday 10 May 2013, you wrote: On 5/10/2013 2:00 AM, heaven wrote: > UPD: > Forget to confirm, we're using Solr 4.2.1 and will wait for Solr 4.3.1 or > for 4.4 as you advised. Solr 4.2.1 should be pretty stable. You mentioned on the previous message that your server load is below 16. This is very high. What is your CPU utilization, and most importantly, what is your iowait percentage? If you are looking at top, this is the "%wa" value. Some other things to check: - How big are your indexes? Your server is hosting six of them - three collections, each with two shards. Each instance of Solr has a max heap of 4GB. The OS and zookeeper probably use up another 1GB or so. If nothing else is running on the server, then that means you have about 23GB of free memory left for caching. With that much free memory, you could probably handle 40GB of index with good performance, unless your query volume is very very high. If there are other programs running on the server, then your free memory will go down. - What kind of RAID are you using? If it's RAID5 or RAID6, then sustained write performance (indexing) will not be very good. With standard hard drives, RAID10 is best. You could take the plunge and get SSD instead, if there's enough money in the budget for it. Comparing your server load with mine: My production Solr servers have 8 CPU cores and the load average is rarely above 1.5 even during busy times. Overall CPU utilization normally peaks at about 15 percent. Thanks, Shawn -------------------- *If you reply to this email, your message will be added to the discussion below:* http://lucene.472066.n3.nabble.com/ColrCloud-IOException-occured-when-talking-to-server-at-tp4061831p4062243.html[1] To unsubscribe from ColrCloud: IOException occured when talking to server at, click here[2]. NAML[3] -------- [1] http://lucene.472066.n3.nabble.com/ColrCloud-IOException-occured-when-talking-to-server-at-tp4061831p4062243.html [2] http://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=unsubscribe _by_code&node=4061831&code=YWhlYXZlbjg3QGdtYWlsLmNvbXw0MDYxODMxfDE 3MDI0ODI4OTY= [3] http://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=macro_view er&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.B asicNamespace-nabble.view.web.template.NabbleNamespace- nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21 nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml- send_instant_email%21nabble%3Aemail.naml -- View this message in context: http://lucene.472066.n3.nabble.com/ColrCloud-IOException-occured-when-talking-to-server-at-tp4061831p4062246.html Sent from the Solr - User mailing list archive at Nabble.com.
