Re: Deadlock in lucene?
I know this isn't really the place for this, so please forgive me - but does this patch look reasonably safe to use to skip the isDeleted check inside of FunctionQuery? My reasoning behind this is that many people (us included) will be building the index on a separate server, and then using the replication scripts to publish the files out to several read-only servers. On those instances, deletedDocs would always be empty, since it's a read only instance - and so we can conveniently skip the Lucene code in question. This flag would also be good for other optimizations that can only be made when you assume the index is read-only. Solr seems to work with the flag set - any reasons why this will crash and/or kill my kitten? (please forgive my posting this here instead of in solr-dev!) Index: src/java/org/apache/solr/search/FunctionQParser.java === --- src/java/org/apache/solr/search/FunctionQParser.java (revision 687135) +++ src/java/org/apache/solr/search/FunctionQParser.java Tue Aug 19 11:08:45 PDT 2008 @@ -49,7 +49,7 @@ } ***/ -return new FunctionQuery(vs); +return new FunctionQuery(vs, req.getSchema().getSolrConfig().isReadOnly() ); } /** Index: src/java/org/apache/solr/search/function/FunctionQuery.java === --- src/java/org/apache/solr/search/function/FunctionQuery.java (revision 687135) +++ src/java/org/apache/solr/search/function/FunctionQuery.java Tue Aug 19 11:08:45 PDT 2008 @@ -31,12 +31,14 @@ */ public class FunctionQuery extends Query { ValueSource func; + Boolean readOnly; /** * @param func defines the function to be used for scoring */ - public FunctionQuery(ValueSource func) { + public FunctionQuery(ValueSource func, Boolean readOnly) { this.func=func; +this.readOnly=readOnly; } /** @return The associated ValueSource */ @@ -113,7 +115,7 @@ if (doc=maxDoc) { return false; } -if (reader.isDeleted(doc)) continue; +if (!readOnly reader.isDeleted(doc)) continue; // todo: maybe allow score() to throw a specific exception // and continue on to the next document if it is thrown... // that may be useful, but exceptions aren't really good Index: src/java/org/apache/solr/core/Config.java === --- src/java/org/apache/solr/core/Config.java (revision 687135) +++ src/java/org/apache/solr/core/Config.java Tue Aug 19 11:08:45 PDT 2008 @@ -45,6 +45,8 @@ private final String name; private final SolrResourceLoader loader; + private Boolean readOnly; + /** * @deprecated Use [EMAIL PROTECTED] #Config(SolrResourceLoader, String, InputStream, String)} instead. */ @@ -254,6 +256,19 @@ return val!=null ? Double.parseDouble(val) : def; } + /** + * Is the index set up to be readOnly? If so, this will cause the FunctionQuery stuff to not check + * for deleted documents. + * @return boolean readOnly + */ + public boolean isReadOnly() { + if( this.readOnly == null ){ + readOnly = getBool(/mainIndex/readOnly, false); + } + + return readOnly; + } + // The following functions were moved to ResourceLoader //- Index: example/solr/conf/solrconfig.xml === --- example/solr/conf/solrconfig.xml(revision 687135) +++ example/solr/conf/solrconfig.xmlTue Aug 19 11:13:13 PDT 2008 @@ -114,6 +114,12 @@ This is not needed if lock type is 'none' or 'single' -- unlockOnStartupfalse/unlockOnStartup + + !-- In the event that you are only using this index for reads, +you can enable this flag. This will skip some checks that +can cause performance issues when under high load + -- + readOnlyfalse/readOnly /mainIndex !-- Enables JMX if and only if an existing MBeanServer is found, use --- end patch --- On Aug 18, 2008, at 8:04 PM, Yonik Seeley wrote: It's not a deadlock (just a synchronization bottleneck) , but it is a known issue in Lucene and there has been some progress in improving the situation. -Yonik On Mon, Aug 18, 2008 at 10:55 PM, Matthew Runo [EMAIL PROTECTED] wrote: Hello folks! I was just wondering if anyone else has seen this issue under heavy load. We had some servers set to very high thread limits (12 core servers with 32 gigs of ram), and found several threads would end up in this state Name: http-8080-891 State: BLOCKED on [EMAIL PROTECTED] owned by: http-8080-191 Total blocked: 97,926 Total waited: 16 Stack trace: org.apache.lucene.index.SegmentReader.isDeleted(SegmentReader.java: 674) org.apache.solr.search.function.FunctionQuery
Re: Deadlock in lucene?
It doesn't matter that it's executed on the read-only server... it matters if any of the docs are marked as deleted. That's the condition that you probably want to check for. -Yonik On Tue, Aug 19, 2008 at 4:26 PM, Matthew Runo [EMAIL PROTECTED] wrote: I know this isn't really the place for this, so please forgive me - but does this patch look reasonably safe to use to skip the isDeleted check inside of FunctionQuery? My reasoning behind this is that many people (us included) will be building the index on a separate server, and then using the replication scripts to publish the files out to several read-only servers. On those instances, deletedDocs would always be empty, since it's a read only instance - and so we can conveniently skip the Lucene code in question. This flag would also be good for other optimizations that can only be made when you assume the index is read-only. Solr seems to work with the flag set - any reasons why this will crash and/or kill my kitten? (please forgive my posting this here instead of in solr-dev!) Index: src/java/org/apache/solr/search/FunctionQParser.java === --- src/java/org/apache/solr/search/FunctionQParser.java(revision 687135) +++ src/java/org/apache/solr/search/FunctionQParser.javaTue Aug 19 11:08:45 PDT 2008 @@ -49,7 +49,7 @@ } ***/ -return new FunctionQuery(vs); +return new FunctionQuery(vs, req.getSchema().getSolrConfig().isReadOnly() ); } /** Index: src/java/org/apache/solr/search/function/FunctionQuery.java === --- src/java/org/apache/solr/search/function/FunctionQuery.java (revision 687135) +++ src/java/org/apache/solr/search/function/FunctionQuery.java Tue Aug 19 11:08:45 PDT 2008 @@ -31,12 +31,14 @@ */ public class FunctionQuery extends Query { ValueSource func; + Boolean readOnly; /** * @param func defines the function to be used for scoring */ - public FunctionQuery(ValueSource func) { + public FunctionQuery(ValueSource func, Boolean readOnly) { this.func=func; +this.readOnly=readOnly; } /** @return The associated ValueSource */ @@ -113,7 +115,7 @@ if (doc=maxDoc) { return false; } -if (reader.isDeleted(doc)) continue; +if (!readOnly reader.isDeleted(doc)) continue; // todo: maybe allow score() to throw a specific exception // and continue on to the next document if it is thrown... // that may be useful, but exceptions aren't really good Index: src/java/org/apache/solr/core/Config.java === --- src/java/org/apache/solr/core/Config.java (revision 687135) +++ src/java/org/apache/solr/core/Config.java Tue Aug 19 11:08:45 PDT 2008 @@ -45,6 +45,8 @@ private final String name; private final SolrResourceLoader loader; + private Boolean readOnly; + /** * @deprecated Use [EMAIL PROTECTED] #Config(SolrResourceLoader, String, InputStream, String)} instead. */ @@ -254,6 +256,19 @@ return val!=null ? Double.parseDouble(val) : def; } + /** + * Is the index set up to be readOnly? If so, this will cause the FunctionQuery stuff to not check + * for deleted documents. + * @return boolean readOnly + */ + public boolean isReadOnly() { + if( this.readOnly == null ){ + readOnly = getBool(/mainIndex/readOnly, false); + } + + return readOnly; + } + // The following functions were moved to ResourceLoader //- Index: example/solr/conf/solrconfig.xml === --- example/solr/conf/solrconfig.xml(revision 687135) +++ example/solr/conf/solrconfig.xmlTue Aug 19 11:13:13 PDT 2008 @@ -114,6 +114,12 @@ This is not needed if lock type is 'none' or 'single' -- unlockOnStartupfalse/unlockOnStartup + + !-- In the event that you are only using this index for reads, +you can enable this flag. This will skip some checks that +can cause performance issues when under high load + -- + readOnlyfalse/readOnly /mainIndex !-- Enables JMX if and only if an existing MBeanServer is found, use --- end patch --- On Aug 18, 2008, at 8:04 PM, Yonik Seeley wrote: It's not a deadlock (just a synchronization bottleneck) , but it is a known issue in Lucene and there has been some progress in improving the situation. -Yonik On Mon, Aug 18, 2008 at 10:55 PM, Matthew Runo [EMAIL PROTECTED] wrote: Hello folks! I was just wondering if anyone else has seen this issue under heavy load. We had some servers set to very high thread limits (12 core servers with 32 gigs of ram), and found
Re: Deadlock in lucene?
FYI, I just slipped this optimization into trunk. -Yonik On Tue, Aug 19, 2008 at 4:37 PM, Yonik Seeley [EMAIL PROTECTED] wrote: It doesn't matter that it's executed on the read-only server... it matters if any of the docs are marked as deleted. That's the condition that you probably want to check for. -Yonik On Tue, Aug 19, 2008 at 4:26 PM, Matthew Runo [EMAIL PROTECTED] wrote: I know this isn't really the place for this, so please forgive me - but does this patch look reasonably safe to use to skip the isDeleted check inside of FunctionQuery? My reasoning behind this is that many people (us included) will be building the index on a separate server, and then using the replication scripts to publish the files out to several read-only servers. On those instances, deletedDocs would always be empty, since it's a read only instance - and so we can conveniently skip the Lucene code in question. This flag would also be good for other optimizations that can only be made when you assume the index is read-only. Solr seems to work with the flag set - any reasons why this will crash and/or kill my kitten? (please forgive my posting this here instead of in solr-dev!) Index: src/java/org/apache/solr/search/FunctionQParser.java === --- src/java/org/apache/solr/search/FunctionQParser.java(revision 687135) +++ src/java/org/apache/solr/search/FunctionQParser.javaTue Aug 19 11:08:45 PDT 2008 @@ -49,7 +49,7 @@ } ***/ -return new FunctionQuery(vs); +return new FunctionQuery(vs, req.getSchema().getSolrConfig().isReadOnly() ); } /** Index: src/java/org/apache/solr/search/function/FunctionQuery.java === --- src/java/org/apache/solr/search/function/FunctionQuery.java (revision 687135) +++ src/java/org/apache/solr/search/function/FunctionQuery.java Tue Aug 19 11:08:45 PDT 2008 @@ -31,12 +31,14 @@ */ public class FunctionQuery extends Query { ValueSource func; + Boolean readOnly; /** * @param func defines the function to be used for scoring */ - public FunctionQuery(ValueSource func) { + public FunctionQuery(ValueSource func, Boolean readOnly) { this.func=func; +this.readOnly=readOnly; } /** @return The associated ValueSource */ @@ -113,7 +115,7 @@ if (doc=maxDoc) { return false; } -if (reader.isDeleted(doc)) continue; +if (!readOnly reader.isDeleted(doc)) continue; // todo: maybe allow score() to throw a specific exception // and continue on to the next document if it is thrown... // that may be useful, but exceptions aren't really good Index: src/java/org/apache/solr/core/Config.java === --- src/java/org/apache/solr/core/Config.java (revision 687135) +++ src/java/org/apache/solr/core/Config.java Tue Aug 19 11:08:45 PDT 2008 @@ -45,6 +45,8 @@ private final String name; private final SolrResourceLoader loader; + private Boolean readOnly; + /** * @deprecated Use [EMAIL PROTECTED] #Config(SolrResourceLoader, String, InputStream, String)} instead. */ @@ -254,6 +256,19 @@ return val!=null ? Double.parseDouble(val) : def; } + /** + * Is the index set up to be readOnly? If so, this will cause the FunctionQuery stuff to not check + * for deleted documents. + * @return boolean readOnly + */ + public boolean isReadOnly() { + if( this.readOnly == null ){ + readOnly = getBool(/mainIndex/readOnly, false); + } + + return readOnly; + } + // The following functions were moved to ResourceLoader //- Index: example/solr/conf/solrconfig.xml === --- example/solr/conf/solrconfig.xml(revision 687135) +++ example/solr/conf/solrconfig.xmlTue Aug 19 11:13:13 PDT 2008 @@ -114,6 +114,12 @@ This is not needed if lock type is 'none' or 'single' -- unlockOnStartupfalse/unlockOnStartup + + !-- In the event that you are only using this index for reads, +you can enable this flag. This will skip some checks that +can cause performance issues when under high load + -- + readOnlyfalse/readOnly /mainIndex !-- Enables JMX if and only if an existing MBeanServer is found, use --- end patch --- On Aug 18, 2008, at 8:04 PM, Yonik Seeley wrote: It's not a deadlock (just a synchronization bottleneck) , but it is a known issue in Lucene and there has been some progress in improving the situation. -Yonik On Mon, Aug 18, 2008 at 10:55 PM, Matthew Runo [EMAIL PROTECTED] wrote: Hello folks! I was just wondering if anyone else has seen
Re: Deadlock in lucene?
I don't think it will help; for instance SegmentReader of Lucene: public synchronized Document document(int n, FieldSelector fieldSelector) Unsynchronized (in future) SOLR caching should help. -Fuad I know this isn't really the place for this, so please forgive me - but does this patch look reasonably safe to use to skip the isDeleted check inside of FunctionQuery? My reasoning behind this is that many people (us included) will be building the index on a separate server, and then using the replication scripts to publish the files out to several read-only servers. On those instances, deletedDocs would always be empty, since it's a read only instance - and so we can conveniently skip the Lucene code in question. This flag would also be good for other optimizations that can only be made when you assume the index is read-only. Solr seems to work with the flag set - any reasons why this will crash and/or kill my kitten? (please forgive my posting this here instead of in solr-dev!) Index: src/java/org/apache/solr/search/FunctionQParser.java ...
Re: Deadlock in lucene?
Matthew, just because an index is read-only on some server it doesn't mean it contains no deletes (no docs marked as deleted, but not yet removed from the index). So you still want to check isDeleted(doc) *unless* you are certain the index has no docs marked as deleted (this happens after optimization). Otis -- Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch - Original Message From: Matthew Runo [EMAIL PROTECTED] To: solr-user@lucene.apache.org Sent: Tuesday, August 19, 2008 4:26:59 PM Subject: Re: Deadlock in lucene? I know this isn't really the place for this, so please forgive me - but does this patch look reasonably safe to use to skip the isDeleted check inside of FunctionQuery? My reasoning behind this is that many people (us included) will be building the index on a separate server, and then using the replication scripts to publish the files out to several read-only servers. On those instances, deletedDocs would always be empty, since it's a read only instance - and so we can conveniently skip the Lucene code in question. This flag would also be good for other optimizations that can only be made when you assume the index is read-only. Solr seems to work with the flag set - any reasons why this will crash and/or kill my kitten? (please forgive my posting this here instead of in solr-dev!) Index: src/java/org/apache/solr/search/FunctionQParser.java === --- src/java/org/apache/solr/search/FunctionQParser.java(revision 687135) +++ src/java/org/apache/solr/search/FunctionQParser.javaTue Aug 19 11:08:45 PDT 2008 @@ -49,7 +49,7 @@ } ***/ -return new FunctionQuery(vs); +return new FunctionQuery(vs, req.getSchema().getSolrConfig().isReadOnly() ); } /** Index: src/java/org/apache/solr/search/function/FunctionQuery.java === --- src/java/org/apache/solr/search/function/FunctionQuery.java (revision 687135) +++ src/java/org/apache/solr/search/function/FunctionQuery.javaTue Aug 19 11:08:45 PDT 2008 @@ -31,12 +31,14 @@ */ public class FunctionQuery extends Query { ValueSource func; + Boolean readOnly; /** * @param func defines the function to be used for scoring */ - public FunctionQuery(ValueSource func) { + public FunctionQuery(ValueSource func, Boolean readOnly) { this.func=func; +this.readOnly=readOnly; } /** @return The associated ValueSource */ @@ -113,7 +115,7 @@ if (doc=maxDoc) { return false; } -if (reader.isDeleted(doc)) continue; +if (!readOnly reader.isDeleted(doc)) continue; // todo: maybe allow score() to throw a specific exception // and continue on to the next document if it is thrown... // that may be useful, but exceptions aren't really good Index: src/java/org/apache/solr/core/Config.java === --- src/java/org/apache/solr/core/Config.java(revision 687135) +++ src/java/org/apache/solr/core/Config.javaTue Aug 19 11:08:45 PDT 2008 @@ -45,6 +45,8 @@ private final String name; private final SolrResourceLoader loader; + private Boolean readOnly; + /** * @deprecated Use [EMAIL PROTECTED] #Config(SolrResourceLoader, String, InputStream, String)} instead. */ @@ -254,6 +256,19 @@ return val!=null ? Double.parseDouble(val) : def; } + /** + * Is the index set up to be readOnly? If so, this will cause the FunctionQuery stuff to not check + * for deleted documents. + * @return boolean readOnly + */ + public boolean isReadOnly() { + if( this.readOnly == null ){ + readOnly = getBool(/mainIndex/readOnly, false); + } + + return readOnly; + } + // The following functions were moved to ResourceLoader //- Index: example/solr/conf/solrconfig.xml === --- example/solr/conf/solrconfig.xml(revision 687135) +++ example/solr/conf/solrconfig.xmlTue Aug 19 11:13:13 PDT 2008 @@ -114,6 +114,12 @@ This is not needed if lock type is 'none' or 'single' -- false + + +false use --- end patch --- On Aug 18, 2008, at 8:04 PM, Yonik Seeley wrote: It's not a deadlock (just a synchronization bottleneck) , but it is a known issue in Lucene and there has been some progress in improving the situation. -Yonik On Mon, Aug 18, 2008 at 10:55 PM, Matthew Runo wrote: Hello folks! I was just wondering if anyone else has seen this issue under heavy load. We had some servers set to very
Re: Deadlock in lucene?
Ouch, that's certainly a problem! I'll have to think some more on this one. Thanks for your time! Matthew Runo Software Engineer, Zappos.com [EMAIL PROTECTED] - 702-943-7833 On Aug 19, 2008, at 1:42 PM, Otis Gospodnetic wrote: Matthew, just because an index is read-only on some server it doesn't mean it contains no deletes (no docs marked as deleted, but not yet removed from the index). So you still want to check isDeleted(doc) *unless* you are certain the index has no docs marked as deleted (this happens after optimization). Otis -- Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch - Original Message From: Matthew Runo [EMAIL PROTECTED] To: solr-user@lucene.apache.org Sent: Tuesday, August 19, 2008 4:26:59 PM Subject: Re: Deadlock in lucene? I know this isn't really the place for this, so please forgive me - but does this patch look reasonably safe to use to skip the isDeleted check inside of FunctionQuery? My reasoning behind this is that many people (us included) will be building the index on a separate server, and then using the replication scripts to publish the files out to several read-only servers. On those instances, deletedDocs would always be empty, since it's a read only instance - and so we can conveniently skip the Lucene code in question. This flag would also be good for other optimizations that can only be made when you assume the index is read-only. Solr seems to work with the flag set - any reasons why this will crash and/or kill my kitten? (please forgive my posting this here instead of in solr-dev!) Index: src/java/org/apache/solr/search/FunctionQParser.java === --- src/java/org/apache/solr/search/FunctionQParser.java(revision 687135) +++ src/java/org/apache/solr/search/FunctionQParser.javaTue Aug 19 11:08:45 PDT 2008 @@ -49,7 +49,7 @@ } ***/ -return new FunctionQuery(vs); +return new FunctionQuery(vs, req.getSchema().getSolrConfig().isReadOnly() ); } /** Index: src/java/org/apache/solr/search/function/FunctionQuery.java === --- src/java/org/apache/solr/search/function/FunctionQuery.java (revision 687135) +++ src/java/org/apache/solr/search/function/FunctionQuery.java Tue Aug 19 11:08:45 PDT 2008 @@ -31,12 +31,14 @@ */ public class FunctionQuery extends Query { ValueSource func; + Boolean readOnly; /** * @param func defines the function to be used for scoring */ - public FunctionQuery(ValueSource func) { + public FunctionQuery(ValueSource func, Boolean readOnly) { this.func=func; +this.readOnly=readOnly; } /** @return The associated ValueSource */ @@ -113,7 +115,7 @@ if (doc=maxDoc) { return false; } -if (reader.isDeleted(doc)) continue; +if (!readOnly reader.isDeleted(doc)) continue; // todo: maybe allow score() to throw a specific exception // and continue on to the next document if it is thrown... // that may be useful, but exceptions aren't really good Index: src/java/org/apache/solr/core/Config.java === --- src/java/org/apache/solr/core/Config.java(revision 687135) +++ src/java/org/apache/solr/core/Config.javaTue Aug 19 11:08:45 PDT 2008 @@ -45,6 +45,8 @@ private final String name; private final SolrResourceLoader loader; + private Boolean readOnly; + /** * @deprecated Use [EMAIL PROTECTED] #Config(SolrResourceLoader, String, InputStream, String)} instead. */ @@ -254,6 +256,19 @@ return val!=null ? Double.parseDouble(val) : def; } + /** + * Is the index set up to be readOnly? If so, this will cause the FunctionQuery stuff to not check + * for deleted documents. + * @return boolean readOnly + */ + public boolean isReadOnly() { + if( this.readOnly == null ){ + readOnly = getBool(/mainIndex/readOnly, false); + } + + return readOnly; + } + // The following functions were moved to ResourceLoader //- Index: example/solr/conf/solrconfig.xml === --- example/solr/conf/solrconfig.xml(revision 687135) +++ example/solr/conf/solrconfig.xmlTue Aug 19 11:13:13 PDT 2008 @@ -114,6 +114,12 @@ This is not needed if lock type is 'none' or 'single' -- false + + +false use --- end patch --- On Aug 18, 2008, at 8:04 PM, Yonik Seeley wrote: It's not a deadlock (just a synchronization bottleneck) , but it is a known issue in Lucene and there has been some progress in improving the situation. -Yonik On Mon, Aug 18, 2008 at 10:55 PM, Matthew Runo wrote: Hello folks! I was just wondering if anyone else has seen this issue under heavy load. We had
Deadlock in lucene?
Hello folks! I was just wondering if anyone else has seen this issue under heavy load. We had some servers set to very high thread limits (12 core servers with 32 gigs of ram), and found several threads would end up in this state Name: http-8080-891 State: BLOCKED on [EMAIL PROTECTED] owned by: http-8080-191 Total blocked: 97,926 Total waited: 16 Stack trace: org.apache.lucene.index.SegmentReader.isDeleted(SegmentReader.java:674) org.apache.solr.search.function.FunctionQuery $AllScorer.next(FunctionQuery.java:116) org .apache .lucene .util.ScorerDocQueue.topNextAndAdjustElsePop(ScorerDocQueue.java:116) org .apache .lucene .search .DisjunctionSumScorer.advanceAfterCurrent(DisjunctionSumScorer.java:175) org .apache .lucene.search.DisjunctionSumScorer.skipTo(DisjunctionSumScorer.java: 228) org.apache.lucene.search.ReqOptSumScorer.score(ReqOptSumScorer.java:76) org.apache.lucene.search.BooleanScorer2.score(BooleanScorer2.java:357) org.apache.lucene.search.BooleanScorer2.score(BooleanScorer2.java:320) org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:137) org.apache.lucene.search.Searcher.search(Searcher.java:126) org.apache.lucene.search.Searcher.search(Searcher.java:105) org .apache .solr .search.SolrIndexSearcher.getDocListAndSetNC(SolrIndexSearcher.java: 1148) org .apache .solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:834) org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java: 269) org .apache .solr.handler.component.QueryComponent.process(QueryComponent.java:160) org .apache .solr .handler.component.SearchHandler.handleRequestBody(SearchHandler.java: 169) org .apache .solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java: 128) org.apache.solr.core.SolrCore.execute(SolrCore.java:1143) org .apache .solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:338) org .apache .solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:272) org .apache .catalina .core .ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java: 235) org .apache .catalina .core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) org .apache .catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java: 233) org .apache .catalina.core.StandardContextValve.invoke(StandardContextValve.java: 175) org .apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java: 128) org .apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java: 102) org .apache .catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java: 286) org.apache.coyote.http11.Http11Processor.process(Http11Processor.java: 844) org.apache.coyote.http11.Http11Protocol $Http11ConnectionHandler.process(Http11Protocol.java:583) org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:447) java.lang.Thread.run(Thread.java:619) Thanks for your time! Matthew Runo Software Engineer, Zappos.com [EMAIL PROTECTED] - 702-943-7833
Re: Deadlock in lucene?
It's not a deadlock (just a synchronization bottleneck) , but it is a known issue in Lucene and there has been some progress in improving the situation. -Yonik On Mon, Aug 18, 2008 at 10:55 PM, Matthew Runo [EMAIL PROTECTED] wrote: Hello folks! I was just wondering if anyone else has seen this issue under heavy load. We had some servers set to very high thread limits (12 core servers with 32 gigs of ram), and found several threads would end up in this state Name: http-8080-891 State: BLOCKED on [EMAIL PROTECTED] owned by: http-8080-191 Total blocked: 97,926 Total waited: 16 Stack trace: org.apache.lucene.index.SegmentReader.isDeleted(SegmentReader.java:674) org.apache.solr.search.function.FunctionQuery$AllScorer.next(FunctionQuery.java:116) org.apache.lucene.util.ScorerDocQueue.topNextAndAdjustElsePop(ScorerDocQueue.java:116) org.apache.lucene.search.DisjunctionSumScorer.advanceAfterCurrent(DisjunctionSumScorer.java:175) org.apache.lucene.search.DisjunctionSumScorer.skipTo(DisjunctionSumScorer.java:228) org.apache.lucene.search.ReqOptSumScorer.score(ReqOptSumScorer.java:76) org.apache.lucene.search.BooleanScorer2.score(BooleanScorer2.java:357) org.apache.lucene.search.BooleanScorer2.score(BooleanScorer2.java:320) org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:137) org.apache.lucene.search.Searcher.search(Searcher.java:126) org.apache.lucene.search.Searcher.search(Searcher.java:105) org.apache.solr.search.SolrIndexSearcher.getDocListAndSetNC(SolrIndexSearcher.java:1148) org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:834) org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:269) org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:160) org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:169) org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:128) org.apache.solr.core.SolrCore.execute(SolrCore.java:1143) org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:338) org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:272) org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233) org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175) org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:128) org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:286) org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:844) org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:583) org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:447) java.lang.Thread.run(Thread.java:619) Thanks for your time! Matthew Runo Software Engineer, Zappos.com [EMAIL PROTECTED] - 702-943-7833