Re: SOLR Performance Tuning: Disable INFO Logging.
Hi
Can you quickly explain what you did to disable INFO-Level?
I am from a PHP background and am not so well versed in Tomcat or
Java. Is this a section in solrconfig.xml or did you have to edit
Solr Java source and recompile?
Thanks In Advance
Andrew
2009/12/20 Fuad Efendi f...@efendi.ca:
After researching how to configure default SOLR Tomcat logging, I finally
disabled INFO-level for SOLR.
And performance improved at least 7 times!!! ('at least 7' because I
restarted server 5 minutes ago; caches are not prepopulated yet)
Before that, I had 300-600 ms in HTTPD log files in average, and 4%-8% I/O
wait whenever top commands shows SOLR on top.
Now, I have 50ms-100ms in average (total response time logged by HTTPD).
P.S.
Of course, I am limited in RAM, and I use slow SATA... server is moderately
loaded, 5-10 requests per second.
P.P.S.
And suddenly synchronous I/O by Java/Tomcat Logger slows down performance
much higher than read-only I/O of Lucene.
Fuad Efendi
+1 416-993-2060
http://www.linkedin.com/in/liferay
Tokenizer Inc.
http://www.tokenizer.ca/
Data Mining, Vertical Search
RE: SOLR Performance Tuning: Disable INFO Logging.
Can you quickly explain what you did to disable INFO-Level?
I am from a PHP background and am not so well versed in Tomcat or
Java. Is this a section in solrconfig.xml or did you have to edit
Solr Java source and recompile?
1. Create a file called logging.properties with following content (I created it
in /home/tomcat/solr folder):
.level=INFO
handlers= java.util.logging.ConsoleHandler, java.util.logging.FileHandler
java.util.logging.FileHandler.formatter = java.util.logging.SimpleFormatter
java.util.logging.FileHandler.level = INFO
java.util.logging.ConsoleHandler.formatter = java.util.logging.SimpleFormatter
java.util.logging.ConsoleHandler.level = ALL
org.apache.solr.level=SEVERE
2. Modify file tomcat_installation/bin/catalina.sh to include following (as a
first line in script):
JAVA_OPTS=... ... ...
-Djava.util.logging.config.file=/home/tomcat/solr/logging.properties
(this line may include more parameters such as -Xmx8196m for memory,
-Dfile.encoding=UTF8 -Dsolr.solr.home=/home/tomcat/solr
-Dsolr.data.dir=/home/tomcat/solr for SOLR, etc.)
With these settings, SOLR (and Tomcat) will use standard Java 5/6 logging
capabilities. Log output will default to standard /logs folder of Tomcat.
You may find additional logging configuration settings by google for Java 5
Logging etc.
2009/12/20 Fuad Efendi f...@efendi.ca:
After researching how to configure default SOLR Tomcat logging, I
finally
disabled INFO-level for SOLR.
And performance improved at least 7 times!!! ('at least 7' because I
restarted server 5 minutes ago; caches are not prepopulated yet)
Before that, I had 300-600 ms in HTTPD log files in average, and 4%-8%
I/O
wait whenever top commands shows SOLR on top.
Now, I have 50ms-100ms in average (total response time logged by HTTPD).
P.S.
Of course, I am limited in RAM, and I use slow SATA... server is
moderately
loaded, 5-10 requests per second.
P.P.S.
And suddenly synchronous I/O by Java/Tomcat Logger slows down
performance
much higher than read-only I/O of Lucene.
Fuad Efendi
+1 416-993-2060
http://www.linkedin.com/in/liferay
Tokenizer Inc.
http://www.tokenizer.ca/
Data Mining, Vertical Search
Fuad Efendi
+1 416-993-2060
http://www.linkedin.com/in/liferay
Tokenizer Inc.
http://www.tokenizer.ca/
Data Mining, Vertical Search
SOLR Performance Tuning: Disable INFO Logging.
After researching how to configure default SOLR Tomcat logging, I finally
disabled INFO-level for SOLR.
And performance improved at least 7 times!!! ('at least 7' because I
restarted server 5 minutes ago; caches are not prepopulated yet)
Before that, I had 300-600 ms in HTTPD log files in average, and 4%-8% I/O
wait whenever top commands shows SOLR on top.
Now, I have 50ms-100ms in average (total response time logged by HTTPD).
P.S.
Of course, I am limited in RAM, and I use slow SATA... server is moderately
loaded, 5-10 requests per second.
P.P.S.
And suddenly synchronous I/O by Java/Tomcat Logger slows down performance
much higher than read-only I/O of Lucene.
Fuad Efendi
+1 416-993-2060
http://www.linkedin.com/in/liferay
Tokenizer Inc.
http://www.tokenizer.ca/
Data Mining, Vertical Search
RE: SOLR Performance Tuning: Disable INFO Logging.
We were talking about GC options a lot; don't forget to enclose following
into if (log.isInfoEnabled()):
...
final NamedListObject responseHeader = new SimpleOrderedMapObject();
rsp.add(responseHeader, responseHeader);
NamedList toLog = rsp.getToLog();
//toLog.add(core, getName());
toLog.add(webapp, req.getContext().get(webapp));
toLog.add(path, req.getContext().get(path));
toLog.add(params, { + req.getParamString() + });
handler.handleRequest(req,rsp);
setResponseHeaderValues(handler,req,rsp);
StringBuilder sb = new StringBuilder();
for (int i=0; itoLog.size(); i++) {
String name = toLog.getName(i);
Object val = toLog.getVal(i);
sb.append(name).append(=).append(val).append( );
}
log.info(logid + sb.toString());...
...
-Fuad
-Original Message-
From: Fuad Efendi [mailto:f...@efendi.ca]
Sent: December-20-09 2:54 PM
To: solr-user@lucene.apache.org
Subject: SOLR Performance Tuning: Disable INFO Logging.
After researching how to configure default SOLR Tomcat logging, I
finally
disabled INFO-level for SOLR.
And performance improved at least 7 times!!! ('at least 7' because I
restarted server 5 minutes ago; caches are not prepopulated yet)
Before that, I had 300-600 ms in HTTPD log files in average, and 4%-8% I/O
wait whenever top commands shows SOLR on top.
Now, I have 50ms-100ms in average (total response time logged by HTTPD).
P.S.
Of course, I am limited in RAM, and I use slow SATA... server is
moderately
loaded, 5-10 requests per second.
P.P.S.
And suddenly synchronous I/O by Java/Tomcat Logger slows down performance
much higher than read-only I/O of Lucene.
Fuad Efendi
+1 416-993-2060
http://www.linkedin.com/in/liferay
Tokenizer Inc.
http://www.tokenizer.ca/
Data Mining, Vertical Search
