We gave the machine 180G mem to see if it improves performance. However, after we increased the memory, Solr started using only 5% of the physical memory. It has always used 90-something%.
What could be causing solr to not grab all the physical memory (grabbing so little of the physical memory)? Rebecca Tang Applications Developer, UCSF CKM Industry Documents Digital Libraries E: rebecca.t...@ucsf.edu On 2/24/15 12:44 PM, "Shawn Heisey" <apa...@elyograg.org> wrote: >On 2/24/2015 1:09 PM, Tang, Rebecca wrote: >> Our solr index used to perform OK on our beta production box (anywhere >>between 0-3 seconds to complete any query), but today I noticed that the >>performance is very bad (queries take between 12 15 seconds). >> >> I haven't updated the solr index configuration >>(schema.xml/solrconfig.xml) lately. All that's changed is the data ‹ >>every month, I rebuild the solr index from scratch and deploy it to the >>box. We will eventually go to incremental builds. But for now, all >>indexes are built from scratch. >> >> Here are the stats: >> Solr index size 183G >> Documents in index 14364201 >> We just have single solr box >> It has 100G memory >> 500G Harddrive >> 16 cpus > >The bottom line on this problem, and I'm sure it's not something you're >going to want to hear: You don't have enough memory available to cache >your index. I'd plan on at least 192GB of RAM for an index this size, >and 256GB would be better. > >Depending on the exact index schema, the nature of your queries, and how >large your Java heap for Solr is, 100GB of RAM could be enough for good >performance on an index that size ... or it might be nowhere near >enough. I would imagine that one of two things is true here, possibly >both: 1) Your queries are very complex and involve accessing a very >large percentage of the index data. 2) Your Java heap is enormous, >leaving very little RAM for the OS to automatically cache the index. > >Adding more memory to the machine, if that's possible, might fix some of >the problems. You can find a discussion of the problem here: > >http://wiki.apache.org/solr/SolrPerformanceProblems > >If you have any questions after reading that wiki article, feel free to >ask them. > >Thanks, >Shawn >
