Re: CPU Max Utilization
After doing some more testing, I've seen the performance decrease yet again. It happens after solr has been run for about 1/2 hour. I left my test running over the weekend and saw the CPU usage go down to a reasonable level at the end of the weekend. It is the same problem where the CPU has maximum usage. I attached a profiler to the solr instance and found that 99% of the CPU time is spent in the doFilter method of the SolrDispatchFilter class. Does anyone know why all of the CPU would be hogged on this particular method? I'm requesting by relevance without sorting. I'm requesting 500 results per query. There are no repititions in the query set. As for the fields. I'm using String and SortableInt fields. There are 3 string fields and 3 Sortable Int fields in my schema. One of the String Fields is multivalued. The fields are quite small. Since its 18 GB for a 100 million document index. Thanks, Brad 2009/11/6 ba ba solrm...@gmail.com After looking at the question about the sorting. It seems that the schema was using the SortableIntField class. When I did not return these fields in the queries, I got reasonable CPU usage. If I search only on one of these SortableIntFields, I get the bad query performance. I think the problem is the schema is using a Sortable field when I don't need a sortable field. Thanks for the help. -Brad 2009/11/5 Otis Gospodnetic otis_gospodne...@yahoo.com You may also want to share some sample queries, your fields definitions, and tell us how long a core remains 100% utilized. Otis -- Sematext is hiring -- http://sematext.com/about/jobs.html?mls Lucene, Solr, Nutch, Katta, Hadoop, HBase, UIMA, NLP, NER, IR - Original Message From: ba ba solrm...@gmail.com To: solr-user@lucene.apache.org Sent: Thu, November 5, 2009 9:20:13 PM Subject: CPU Max Utilization Greetings, I'm running a solr instance with 100 million documents in it. The index is 18 GB. The strange behavior I'm seeing is CPU utilization gets maxed out. I'm running on an 8 core machine with 32 GB or ram. Every concurrent query I run on it uses up one of the cores. So, if I am running 1 concurrent query I'm using up the cpu of one of the cores. If I have 8 concurrent queries I'm using up all of the cores. Is this normal to have such a high CPU utilization. If not, what am I doing wrong here. The only thing I have modified is the schema.xml file to correspond to the documents I want to store. Everything else is just using the default values for all the config files. Thanks.
Set MMap in Solr
Hi, I'm trying to set my default directory to MMap. I saw that this is done by specifying here A DirectoryProvider plugin can be configured in solrconfig.xml with the following XML: directoryProvider class=class.name !-- Parameters as required by the implementation -- /directoryProvider in solrconfig.xml. This did not work for me when I put in the MMapDirectory class name. I got this information from here http://issues.apache.org/jira/browse/SOLR-465?focusedCommentId=12715282#action_12715282 I'm using the latest nightly build. If anyone knows how to configure solr to use MMap, please let me know. I would greatly appreciate it. Thanks.
Re: Set MMap in Solr
Thanks for the help. -Brad Anderson 2009/11/5 Otis Gospodnetic otis_gospodne...@yahoo.com To use MMapDirectory, invoke Java with the System property org.apache.lucene.FSDirectory.class set to org.apache.lucene.store.MMapDirectory. This will cause FSDirectory.getDirectory(File,boolean) to return instances of this class. So, start your servlet container with -Dorg.apache.lucene.FSDirectory.class=org.apache.lucene.store.MMapDirectory Otis -- Sematext is hiring -- http://sematext.com/about/jobs.html?mls Lucene, Solr, Nutch, Katta, Hadoop, HBase, UIMA, NLP, NER, IR - Original Message From: ba ba solrm...@gmail.com To: solr-user@lucene.apache.org Sent: Thu, November 5, 2009 2:55:42 PM Subject: Set MMap in Solr Hi, I'm trying to set my default directory to MMap. I saw that this is done by specifying here A DirectoryProvider plugin can be configured in solrconfig.xml with the following XML: in solrconfig.xml. This did not work for me when I put in the MMapDirectory class name. I got this information from here http://issues.apache.org/jira/browse/SOLR-465?focusedCommentId=12715282#action_12715282 I'm using the latest nightly build. If anyone knows how to configure solr to use MMap, please let me know. I would greatly appreciate it. Thanks.
CPU Max Utilization
Greetings, I'm running a solr instance with 100 million documents in it. The index is 18 GB. The strange behavior I'm seeing is CPU utilization gets maxed out. I'm running on an 8 core machine with 32 GB or ram. Every concurrent query I run on it uses up one of the cores. So, if I am running 1 concurrent query I'm using up the cpu of one of the cores. If I have 8 concurrent queries I'm using up all of the cores. Is this normal to have such a high CPU utilization. If not, what am I doing wrong here. The only thing I have modified is the schema.xml file to correspond to the documents I want to store. Everything else is just using the default values for all the config files. Thanks.
