Memory leak for Multiple fields faceted searching
Hi, Here is my environment: 32bits windows server 2008 (4cpu and 12G RAM), 320million docs, index size 12G. When we use query=*:* and 10 fields for faceted searching, it works. However, 15 feilds or more faceted searching caused the following exception: java.lang.OutOfMemoryError: Java heap space at org.apache.lucene.search.FieldCacheImpl$StringIndexCache.createValue(FieldCacheImpl.java:704) Any suggestions are welcomed! THx
Re: Why the query performance is so different for queries?
Thanks for your reply. Our box is win server 2003 (32bits) and 6G RAM totally. Large heap (2G) may not be helpful for JVM in 32bits box. Therefore we set JAVA_OPTIONS to -Xms521m -Xmx1400m. Is my understanding right? Thanks. From: Lance Norskog goks...@gmail.com Reply-To: solr-user@lucene.apache.org To: solr-user@lucene.apache.org, newsam new...@zju.edu.cn Subject: Re: Why the query performance is so different for queries? Date: Wed, 29 Sep 2010 20:13:20 -0700 How much ram does the JVM have? Wildcard queries are slow. Starting with '*' are even slower. If you want all values try field:[* TO *]. This is a range query and lets you pick a range of values- this picks everything. The *:* is not a wildcard. It is a magic syntax for all documents and does not cause a search. 2010/9/28 newsam : Hi guys, I have posted a thread The search response time is too long. The SOLR searcher instance is deployed with Tomcat 5.5.21. . The index file is 8.2G. The doc num is 6110745. DELL Server has Intel(R) Xeon(TM) CPU (4 cores) 3.00GHZ and 6G RAM. In SOLR back-end, query=key:* costs almost 60s while query=*:* only needs 500ms. Another case is query=product_name_title:*, which costs 7s. I am confused about the query performance. Do you have any suggestions? btw, the cache setting is as follows: filterCache: 256, 256, 0 queryResultCache: 1024, 512, 128 documentCache: 16384, 4096, n/a Thanks. -- Lance Norskog goks...@gmail.com
Re: Re:The search response time is too loong
I guess you are correct. We used the default SOLR cache configuration. I will change the cache configuration. BTW, I want to deploy several shards from the existing 8G index file, such as 4G per shards. Is there any tool to generate two shards from one 8G index file? From: kenf_nc ken.fos...@realestate.com Reply-To: solr-user@lucene.apache.org To: solr-user@lucene.apache.org Subject: Re: Re:The search response time is too loong Date: Mon, 27 Sep 2010 05:37:25 -0700 (PDT) mem usage is over 400M, do you mean Tomcat mem size? If you don't give your cache sizes enough room to grow you will choke the performance. You should adjust your Tomcat settings to let the cache grow to at least 1GB or better would be 2GB. You may also want to look into http://wiki.apache.org/solr/SolrCaching warming the cache to make the first time call a little faster. For comparison, I also have about 8GB in my index but only 2.8 million documents. My search query times on a smaller box than you specify are 6533 milliseconds on an unwarmed (newly rebooted) instance. -- View this message in context: http://lucene.472066.n3.nabble.com/Re-The-search-response-time-is-too-loong-tp1587395p1588554.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Re:The search response time is too loong
Thx. I will let you know the latest status. From: Lance Norskog goks...@gmail.com Reply-To: solr-user@lucene.apache.org To: solr-user@lucene.apache.org, newsam new...@zju.edu.cn Subject: Re: Re:The search response time is too loong Date: Tue, 28 Sep 2010 13:34:53 -0700 Copy the index. Delete half of the documents. Optimize. Copy the index. Delete the other half of the documents. Optimize. 2010/9/28 newsam : I guess you are correct. We used the default SOLR cache configuration. I will change the cache configuration. BTW, I want to deploy several shards from the existing 8G index file, such as 4G per shards. Is there any tool to generate two shards from one 8G index file? From: kenf_nc Reply-To: solr-user@lucene.apache.org To: solr-user@lucene.apache.org Subject: Re: Re:The search response time is too loong Date: Mon, 27 Sep 2010 05:37:25 -0700 (PDT) mem usage is over 400M, do you mean Tomcat mem size? If you don't give your cache sizes enough room to grow you will choke the performance. You should adjust your Tomcat settings to let the cache grow to at least 1GB or better would be 2GB. You may also want to look into http://wiki.apache.org/solr/SolrCaching warming the cache to make the first time call a little faster. For comparison, I also have about 8GB in my index but only 2.8 million documents. My search query times on a smaller box than you specify are 6533 milliseconds on an unwarmed (newly rebooted) instance. -- View this message in context: http://lucene.472066.n3.nabble.com/Re-The-search-response-time-is-too-loong-tp1587395p1588554.html Sent from the Solr - User mailing list archive at Nabble.com. -- Lance Norskog goks...@gmail.com
Why the query performance is so different for queries?
Hi guys, I have posted a thread The search response time is too long. The SOLR searcher instance is deployed with Tomcat 5.5.21. . The index file is 8.2G. The doc num is 6110745. DELL Server has Intel(R) Xeon(TM) CPU (4 cores) 3.00GHZ and 6G RAM. In SOLR back-end, query=key:* costs almost 60s while query=*:* only needs 500ms. Another case is query=product_name_title:*, which costs 7s. I am confused about the query performance. Do you have any suggestions? btw, the cache setting is as follows: filterCache: 256, 256, 0 queryResultCache: 1024, 512, 128 documentCache: 16384, 4096, n/a Thanks.
Re:The search response time is too loong
We used SOLR 1.4. All queries were excuted in SOLR back-end. I guess that I/O operations consume the time too much. From: newsam new...@zju.edu.cn Reply-To: solr-user@lucene.apache.orgnewsam new...@zju.edu.cn To: solr-user@lucene.apache.org Subject: Re:The search response time is too loong Date: Mon, 27 Sep 2010 16:05:49 +0800 I have setup a SOLR searcher instance with Tomcat 5.5.21. However, the response time is too long. Here is my scenario: 1. The index file is 8.2G. The doc num is 6110745. 2. DELL Server: Intel(R) Xeon(TM) CPU (4 cores) 3.00GHZ, 6G Mem. I used Key:* to query all records by localhost:8080. The response time is 68703 milliseconds. The cpu load is 50% and mem useage is over 400M. Any comments are welcomed.
