Hi Fuad, The point at which the thread is blocking is the only synchronization on tracker, so another thread must be in the block. Is it possible that another thread is stalled during commit? Do you have autowarming enabled, and are these queries processor-intensive?
If it is in fact a deadlock situation, could you provide a full thread dump (kill -QUIT <pid>)? Thanks, -MIke On 1/29/07, Fuad Efendi <[EMAIL PROTECTED]> wrote:
I am doing some performance analysis, There are currently more than 550000 of documents in SOLR, and Tokenizer (web-crawler, http://www.tokenizer.org) adds about 2000 of new documents each hour. I was forced to stop crawler, but even after 20 minutes SOLR uses about 60% CPU (two Opteron 252 processors, SLES 10). I have autocommit set to 1000, and default merge to 1000. Didn't issue "optimize" yet, and I have 738 files in /solr/data/index folder. Usually "optimize" does not help (after I reached 400000 docs). I want to share some findings... Currently, database size can easily reach 2 millions of docs (by adding some URLs from USA); but I am forced to stop crawler. Max number of open files is set to 65000, SuSE 10 Ent. Server. 8192 "max open files" didn't help - in fact, this number is enough; but OS have some kind of delay (when it is overloaded), it shows 8000 open files when we have only 3000 (it will show correct number after some delay! It's not "truly" real-time number) Solr runs at Tomcat with 4Gb: -Xms4096M -Xmx4096M Ok. I issued "commit" (via HTTP XML), it took maybe 10 seconds... I have 17 file now in index folder, but SOLR still uses about 66% double-CPU, and there are no any incoming HTTP requests, and Robot stopped crawl. Size of SOLR index files on disk is 200Mb total, so I expect that 4Gb for dedicated Tomcat is more than enough. I see often such message at Admin screen: No deadlock found. Full Thread Dump: "http-10080-Processor50" Id=68 in BLOCKED on [EMAIL PROTECTED] total cpu time=72310.0000ms user time=71090.0000ms owned by http-10080-Processor9 Id=20 at org.apache.solr.update.DirectUpdateHandler2.checkCommit(DirectUpdateHandler2 .java:566) at org.apache.solr.update.DirectUpdateHandler2.addDoc(DirectUpdateHandler2.java :271) at org.apache.solr.core.SolrCore.update(SolrCore.java:716) at org.apache.solr.servlet.SolrUpdateServlet.doPost(SolrUpdateServlet.java:53) at javax.servlet.http.HttpServlet.service(HttpServlet.java:709) at javax.servlet.http.HttpServlet.service(HttpServlet.java:802) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Application FilterChain.java:252) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterCh ain.java:173) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.ja va:213) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.ja va:178) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:126 ) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:105 ) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java :107) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:148) at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:869) at org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processC onnection(Http11BaseProtocol.java:664) at org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.jav a:527) at org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWo rkerThread.java:80) at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.jav a:684) at java.lang.Thread.run(Thread.java:595) No deadlock, and BLOCKED. Usually after restart I have 3-5% CPU usage... SOLR-1.1 Thanks, Fuad P.S. It is still blocked: "http-10080-Processor50" Id=68 in BLOCKED on [EMAIL PROTECTED] total cpu time=72310.0000ms user time=71090.0000ms Why it shows same numbers after 5 minutes? I clicked F5 in Internet Explorer, and I don't expect HTTP Caching! cpu time=72310.0000ms user time=71090.0000ms
