We're using:
Solr Specification Version: 1.3.0.2009.01.23.10.46.02
Solr Implementation Version: 1.4-dev 737141M - root - 2009-01-23
10:46:02
Lucene Specification Version: 2.9-dev
Lucene Implementation Version: 2.9-dev 724059 - 2008-12-06 20:08:54
We'll see about getting up to trunk and firing off our load test and
seeing if we can get it to happen with that.
Thanks for your time!
Matthew Runo
Software Engineer, Zappos.com
mr...@zappos.com - 702-943-7833
On Feb 27, 2009, at 7:44 AM, Yonik Seeley wrote:
I'm using trunk, but I set a breakpoint on SegmentReader.isDeleted()
on an index with deletions, and I couldn't get it to be called.
numDocs : 26
maxDoc : 130
reader:SolrIndexReader
{this=1935e6f,r=readonlymultisegmentrea...@1935e6f,segments=5}
-Yonik
http://www.lucidimagination.com
On Thu, Feb 26, 2009 at 4:55 PM, Matthew Runo <mr...@zappos.com>
wrote:
I see a ReadOnlySegmentReader now - we're on an optimized index now
which
gets around the isDeleted() check.
(solr4, optimized)
searcherName : searc...@260f8e27 main
caching : true
numDocs : 139583
maxDoc : 139583
readerImpl : ReadOnlySegmentReader
readerDir :
org.apache.lucene.store.NIOFSDirectory@/opt/solr-data/zeta-main/index
indexVersion : 1233423823917
openedAt : Thu Feb 26 13:29:25 PST 2009
registeredAt : Thu Feb 26 13:29:42 PST 2009
warmupTime : 16910
(solr1, non optimized)
searcherName : searc...@36be11a1 main
caching : true
numDocs : 139561
maxDoc : 139591
readerImpl : ReadOnlyMultiSegmentReader
readerDir :
org.apache.lucene.store.NIOFSDirectory@/opt/solr-data/zeta-main/index
indexVersion : 1233423823924
openedAt : Thu Feb 26 13:48:16 PST 2009
registeredAt : Thu Feb 26 13:49:11 PST 2009
warmupTime : 54785
I did a thread dump against the optimized server just now, but
didn't find
anything blocked to check which reader was actually in use this time.
Thanks for your time!
Matthew Runo
Software Engineer, Zappos.com
mr...@zappos.com - 702-943-7833
On Feb 26, 2009, at 1:39 PM, Yonik Seeley wrote:
That's interesting.
We should be using read-only readers, which should not synchronize
on
the deleted docs check. But as your stack trace shows, you're using
SegmentReader and MultiSegmentReader.
Right now, if I look at the admin/statistics page at the searcher,
it
shows the following for the reader:
reader:SolrIndexReader
{this=42f352,r=readonlymultisegmentrea...@42f352,segments=6}
Hopefully the fact that it's a ReadOnlyMultiSegmentReader means that
it contains ReadOnlySegmentReader instances, which don't synchronize
on isDeleted.
What do you see?
-Yonik
On Thu, Feb 26, 2009 at 4:09 PM, Matthew Runo <mr...@zappos.com>
wrote:
Hello folks!
I was under the impression that this sync bottleneck was fixed in
recent
versions of Solr/Lucene, but we're seeing it with 1.4-dev right
now. When
we
load test a server with >100 threads (using jmeter), we see several
threads
all blocked at the same spot:
"http-8080-exec-505" - Thread t...@594
java.lang.Thread.State: BLOCKED on
org.apache.lucene.index.segmentrea...@2b6f5d18 owned by:
http-8080-exec-434
at
org
.apache.lucene.index.SegmentReader.isDeleted(SegmentReader.java:
737)
at
org
.apache
.lucene
.index.MultiSegmentReader.isDeleted(MultiSegmentReader.java:266)
at
org.apache.solr.search.function.FunctionQuery
$AllScorer.next(FunctionQuery.java:118)
at
org.apache.solr.search.function.FunctionQuery
$AllScorer.skipTo(FunctionQuery.java:137)
at
org
.apache
.lucene
.search
.BooleanScorer2$SingleMatchScorer.skipTo(BooleanScorer2.java:170)
at
org
.apache.lucene.search.ReqOptSumScorer.score(ReqOptSumScorer.java:
76)
at
org.apache.lucene.search.BooleanScorer2.score(BooleanScorer2.java:
357)
at
org.apache.lucene.search.BooleanScorer2.score(BooleanScorer2.java:
320)
at
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:
136)
at org.apache.lucene.search.Searcher.search(Searcher.java:
126)
at org.apache.lucene.search.Searcher.search(Searcher.java:
105)
at
org
.apache
.solr
.search
.SolrIndexSearcher.getDocListAndSetNC(SolrIndexSearcher.java:1231)
at
org
.apache
.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:
917)
at
org
.apache
.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:338)
at
org
.apache
.solr
.handler.component.QueryComponent.process(QueryComponent.java:164)
at
org
.apache
.solr
.handler
.component.SearchHandler.handleRequestBody(SearchHandler.java:171)
at
org
.apache
.solr
.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:
131)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1333)
at
org
.apache
.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:
303)
at
org
.apache
.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:
232)
at
org
.apache
.catalina
.core
.ApplicationFilterChain
.internalDoFilter(ApplicationFilterChain.java:235)
at
org
.apache
.catalina
.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:
206)
at
org
.apache
.catalina
.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
at
org
.apache
.catalina
.core.StandardContextValve.invoke(StandardContextValve.java:175)
at
org
.apache
.catalina.core.StandardHostValve.invoke(StandardHostValve.java:128)
at
org
.apache
.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
at
org
.apache
.catalina
.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
at
org
.apache
.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:286)
at
org
.apache
.coyote.http11.Http11NioProcessor.process(Http11NioProcessor.java:
879)
at
org.apache.coyote.http11.Http11NioProtocol
$Http11ConnectionHandler.process(Http11NioProtocol.java:719)
at
org.apache.tomcat.util.net.NioEndpoint
$SocketProcessor.run(NioEndpoint.java:2080)
at
java.util.concurrent.ThreadPoolExecutor
$Worker.runTask(ThreadPoolExecutor.java:885)
at
java.util.concurrent.ThreadPoolExecutor
$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:619)
Locked ownable synchronizers:
- locked
java.util.concurrent.locks.reentrantlock$nonfairs...@4d54c7be
I checked the Lucene SVN and it looks like that's still appearing
to be a
bottleneck.
http://svn.apache.org/viewvc/lucene/java/trunk/src/java/org/apache/lucene/index/SegmentReader.java?view=markup
Any tips?
Thanks for your time!
Matthew Runo
Software Engineer, Zappos.com
mr...@zappos.com - 702-943-7833
--
-Yonik
http://www.lucidimagination.com
