date:20190222

Re: Query of Death Lucene/Solr 7.6

2019-02-22 Thread Michael Gibney

Ah... I think there are two issues likely at play here. One is LUCENE-8531
, which reverts a bug
related to SpanNearQuery semantics, causing possible query paths to be
enumarated up front. Setting ps=0 (although perhaps not appropriate for
some use cases) should address problems related to this issue.

The other (likely affecting Gregg, for whom ps=0 did not help) is SOLR-12243
. Prior to 7.6,
SpanNearQuery (generated for relatively complex "graph" tokenized queries,
such as would be generated with WDGF, SynonymGraphFilter, etc.) were simply
getting dropped. This was surely a bug, in that pf did not contribute at
all to boosting such queries; but the silver lining was that performance
was great ;-)

Markus, Gregg, could send examples (parsed query toString()) of problematic
queries (and perhaps relevant analysis chain configs)?

Michael



On Fri, Feb 22, 2019 at 11:00 AM Gregg Donovan  wrote:

> FWIW: we have also seen serious Query of Death issues after our upgrade to
> Solr 7.6. Are there any open issues we can watch? Is Markus' findings
> around `pf` our best guess? We've seen these issues even with ps=0. We also
> use the WDF.
>
> On Fri, Feb 22, 2019 at 8:58 AM Markus Jelsma 
> wrote:
>
> > Hello Michael,
> >
> > Sorry it took so long to get back to this, too many things to do.
> >
> > Anyway, yes, we have WDF on our query-time analysers. I uploaded two log
> > files, both the same query of death with and without synonym filter
> enabled.
> >
> > https://mail.openindex.io/export/solr-8983-console.log 23 MB
> > https://mail.openindex.io/export/solr-8983-console-without-syns.log 1.9
> MB
> >
> > Without the synonym we still see a huge number of entries. Many different
> > parts of our analyser chain contribute to the expansion of queries, but
> pf
> > itself really turns the problem on or off.
> >
> > Since SOLR-12243 is new in 7.6, does anyone know that SOLR-12243 could
> > have this side-effect?
> >
> > Thanks,
> > Markus
> >
> >
> > -Original message-
> > > From:Michael Gibney 
> > > Sent: Friday 8th February 2019 17:19
> > > To: solr-user@lucene.apache.org
> > > Subject: Re: Query of Death Lucene/Solr 7.6
> > >
> > > Hi Markus,
> > > As of 7.6, LUCENE-8531 <
> > https://issues.apache.org/jira/browse/LUCENE-8531>
> > > reverted a graph/Spans-based phrase query implementation (introduced in
> > 6.5
> > > -- LUCENE-7699 ) to
> > an
> > > implementation that builds a separate phrase query for each possible
> > > enumerated path through the graph described by a parsed query.
> > > The potential for combinatoric explosion of the enumerated approach was
> > (as
> > > far as I can tell) one of the main motivations for introducing the
> > > Spans-based implementation. Some real-world use cases would be good to
> > > explore. Markus, could you send (as an attachment) the debug toString()
> > for
> > > the queries with/without synonyms enabled? I'm also guessing you may
> have
> > > WordDelimiterGraphFilter on the query analyzer?
> > > As an alternative to disabling pf, LUCENE-8531 only reverts to the
> > > enumerated approach for phrase queries where slop>0, so setting ps=0
> > would
> > > probably also help.
> > > Michael
> > >
> > > On Fri, Feb 8, 2019 at 5:57 AM Markus Jelsma <
> markus.jel...@openindex.io
> > >
> > > wrote:
> > >
> > > > Hello (apologies for cross-posting),
> > > >
> > > > While working on SOLR-12743, using 7.6 on two nodes and 7.2.1 on the
> > > > remaining four, we stumbled upon a situation where the 7.6 nodes
> > quickly
> > > > succumb when a 'Query-of-Death' is issued, 7.2.1 up to 7.5 are all
> > > > unaffected (tested and confirmed).
> > > >
> > > > Following Smiley's suggestion i used Eclipse MAT to find the problem
> in
> > > > the heap dump i obtained, this fantastic tool revealed within minutes
> > that
> > > > a query thread ate 65 % of all resources, in the class variables i
> > could
> > > > find the the query, and reproduce the problem.
> > > >
> > > > The problematic query is 'dubbele dijk/rijke dijkproject in het
> > dijktracé
> > > > eemshaven-delfzijl', on 7.6 this input produces a 40+ MB toString()
> > output
> > > > in edismax' newFieldQuery. If the node survives it takes 2+ seconds
> > for the
> > > > query to run (150 ms otherwise). If i disable all query time
> > > > SynonymGraphFilters it still takes a second and produces just a 9 MB
> > > > toString() for the query.
> > > >
> > > > I could not find anything like this in Jira. I did think of
> LUCENE-8479
> > > > and LUCENE-8531 but they were about graphs, this problem looked
> related
> > > > though.
> > > >
> > > > I think i tracked it further down to LUCENE-8589 or SOLR-12243. When
> i
> > > > leave Solr's edismax' pf parameter empty, everything runs fast. When
> > all
> > > > fields are configured for pf, the node dies.
> > > >
> > > > I am now unsure

Re: Solr Indexes not behaving as expected

2019-02-22 Thread Erick Erickson

You’ll need to look at the solr logs on the machines hosting the collection’s 
replicas.
There should be something more informative there.

Is the collection even available for on the target? Can you search anything 
there?

And what’s the reason the collection is being reloaded? If you have “schemaless”
mode enabled, updating the schema as docs are being indexed would cause
a collection reload, or if you’re updating the schema via the APIs. Are either 
of
those possible?

Best,
Erick

> On Feb 22, 2019, at 6:25 AM, mrityunjayap  wrote:
> 
> Hi 
> there is strange issue with our Solr Setup in production. We have a cdcr
> setup. Source machines are working fine. but target Solr is not allowing any
> operation on existing indexes
> 
> Solr Version: 6.5.2
> Source and Target are configured on 3 linux machines each with 2 shards and
> 2 replica
> 
> Below is the error log for reload operation. can you help on this? 
> Let me know if you need any more information from my side. 
> 
> 
> null:org.apache.solr.common.SolrException: reload the collection time
> out:180s
>   at
> org.apache.solr.handler.admin.CollectionsHandler.handleResponse(CollectionsHandler.java:303)
>   at
> org.apache.solr.handler.admin.CollectionsHandler.invokeAction(CollectionsHandler.java:235)
>   at
> org.apache.solr.handler.admin.CollectionsHandler.handleRequestBody(CollectionsHandler.java:213)
>   at
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:173)
>   at 
> org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:748)
>   at
> org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:729)
>   at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:510)
>   at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:347)
>   at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:298)
>   at
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1691)
>   at
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
>   at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
>   at
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
>   at
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)
>   at
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
>   at
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
>   at
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
>   at
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
>   at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
>   at
> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:213)
>   at
> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:119)
>   at
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
>   at
> org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:335)
>   at
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
>   at org.eclipse.jetty.server.Server.handle(Server.java:534)
>   at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
>   at
> org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
>   at
> org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
>   at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
>   at
> org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
>   at
> org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
>   at
> org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
>   at
> org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
>   at
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
>   at
> org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
>   at java.lang.Thread.run(Thread.java:748)
> 
> 
> 
> --
> Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

RE: Re: Suppress stack trace in error response

2019-02-22 Thread Markus Jelsma

Hello,

Solr's error responses respect the configured response writer settings, so you 
could probably remove the  element and the stuff it contains 
using XSLT. It is not too fancy, but it should work.

Regards,
Markus
 
-Original message-
> From:Branham, Jeremy (Experis) 
> Sent: Friday 22nd February 2019 16:53
> To: solr-user@lucene.apache.org
> Subject: Re:  Re: Suppress stack trace in error response
> 
> Thanks Edwin – You’re right, I could explain that a bit more.
> My security team has run a scan against the SOLR servers and identified a few 
> things they want suppressed, one being the stack trace in an error message.
> 
> For example –
> 
> 
> 500
> 1
> 
> `
> 
> 
> 
> For input string: "`"
> 
> java.lang.NumberFormatException: For input string: "`" at 
> java.lang.NumberFormatException.forInputString(NumberFormatException.java:65) 
> at …
> 
> 
> I’ve got a long-term solution involving middleware changes, but I’m not sure 
> there is a quick fix for this.
> 
>  
> Jeremy Branham
> jb...@allstate.com
> 
> On 2/21/19, 9:53 PM, "Zheng Lin Edwin Yeo"  wrote:
> 
> Hi,
> 
> There's too little information provided in your questions.
> You can explain more on the issue or the exception that you are facing.
> 
> Regards,
> Edwin
> 
> On Thu, 21 Feb 2019 at 23:45, Branham, Jeremy (Experis) 
> 
> wrote:
> 
> > When Solr throws an exception, like when a client sends a badly formed
> > query string, is there a way to suppress the stack trace in the error
> > response?
> >
> >
> >
> > Jeremy Branham
> > jb...@allstate.com
> > Allstate Insurance Company | UCV Technology Services | Information
> > Services Group
> >
> >
> 
> 
>

Re: Query of Death Lucene/Solr 7.6

2019-02-22 Thread Gregg Donovan

FWIW: we have also seen serious Query of Death issues after our upgrade to
Solr 7.6. Are there any open issues we can watch? Is Markus' findings
around `pf` our best guess? We've seen these issues even with ps=0. We also
use the WDF.

On Fri, Feb 22, 2019 at 8:58 AM Markus Jelsma 
wrote:

> Hello Michael,
>
> Sorry it took so long to get back to this, too many things to do.
>
> Anyway, yes, we have WDF on our query-time analysers. I uploaded two log
> files, both the same query of death with and without synonym filter enabled.
>
> https://mail.openindex.io/export/solr-8983-console.log 23 MB
> https://mail.openindex.io/export/solr-8983-console-without-syns.log 1.9 MB
>
> Without the synonym we still see a huge number of entries. Many different
> parts of our analyser chain contribute to the expansion of queries, but pf
> itself really turns the problem on or off.
>
> Since SOLR-12243 is new in 7.6, does anyone know that SOLR-12243 could
> have this side-effect?
>
> Thanks,
> Markus
>
>
> -Original message-
> > From:Michael Gibney 
> > Sent: Friday 8th February 2019 17:19
> > To: solr-user@lucene.apache.org
> > Subject: Re: Query of Death Lucene/Solr 7.6
> >
> > Hi Markus,
> > As of 7.6, LUCENE-8531 <
> https://issues.apache.org/jira/browse/LUCENE-8531>
> > reverted a graph/Spans-based phrase query implementation (introduced in
> 6.5
> > -- LUCENE-7699 ) to
> an
> > implementation that builds a separate phrase query for each possible
> > enumerated path through the graph described by a parsed query.
> > The potential for combinatoric explosion of the enumerated approach was
> (as
> > far as I can tell) one of the main motivations for introducing the
> > Spans-based implementation. Some real-world use cases would be good to
> > explore. Markus, could you send (as an attachment) the debug toString()
> for
> > the queries with/without synonyms enabled? I'm also guessing you may have
> > WordDelimiterGraphFilter on the query analyzer?
> > As an alternative to disabling pf, LUCENE-8531 only reverts to the
> > enumerated approach for phrase queries where slop>0, so setting ps=0
> would
> > probably also help.
> > Michael
> >
> > On Fri, Feb 8, 2019 at 5:57 AM Markus Jelsma  >
> > wrote:
> >
> > > Hello (apologies for cross-posting),
> > >
> > > While working on SOLR-12743, using 7.6 on two nodes and 7.2.1 on the
> > > remaining four, we stumbled upon a situation where the 7.6 nodes
> quickly
> > > succumb when a 'Query-of-Death' is issued, 7.2.1 up to 7.5 are all
> > > unaffected (tested and confirmed).
> > >
> > > Following Smiley's suggestion i used Eclipse MAT to find the problem in
> > > the heap dump i obtained, this fantastic tool revealed within minutes
> that
> > > a query thread ate 65 % of all resources, in the class variables i
> could
> > > find the the query, and reproduce the problem.
> > >
> > > The problematic query is 'dubbele dijk/rijke dijkproject in het
> dijktracé
> > > eemshaven-delfzijl', on 7.6 this input produces a 40+ MB toString()
> output
> > > in edismax' newFieldQuery. If the node survives it takes 2+ seconds
> for the
> > > query to run (150 ms otherwise). If i disable all query time
> > > SynonymGraphFilters it still takes a second and produces just a 9 MB
> > > toString() for the query.
> > >
> > > I could not find anything like this in Jira. I did think of LUCENE-8479
> > > and LUCENE-8531 but they were about graphs, this problem looked related
> > > though.
> > >
> > > I think i tracked it further down to LUCENE-8589 or SOLR-12243. When i
> > > leave Solr's edismax' pf parameter empty, everything runs fast. When
> all
> > > fields are configured for pf, the node dies.
> > >
> > > I am now unsure whether this is a Solr or a Lucene issue.
> > >
> > > Please let me know.
> > >
> > > Many thanks,
> > > Markus
> > >
> > > ps. in Solr i even got an 'Impossible Exception', my first!
> > >
> >
>

Re: Re: Suppress stack trace in error response

2019-02-22 Thread Branham, Jeremy (Experis)

Thanks Edwin – You’re right, I could explain that a bit more.
My security team has run a scan against the SOLR servers and identified a few 
things they want suppressed, one being the stack trace in an error message.

For example –

500
1

`

For input string: "`"

java.lang.NumberFormatException: For input string: "`" at 
java.lang.NumberFormatException.forInputString(NumberFormatException.java:65) 
at …

I’ve got a long-term solution involving middleware changes, but I’m not sure 
there is a quick fix for this.

Jeremy Branham
jb...@allstate.com

On 2/21/19, 9:53 PM, "Zheng Lin Edwin Yeo"  wrote:

Hi,

There's too little information provided in your questions.
You can explain more on the issue or the exception that you are facing.

Regards,
Edwin

On Thu, 21 Feb 2019 at 23:45, Branham, Jeremy (Experis) 
wrote:

> When Solr throws an exception, like when a client sends a badly formed
> query string, is there a way to suppress the stack trace in the error
> response?
>
>
>
> Jeremy Branham
> jb...@allstate.com
> Allstate Insurance Company | UCV Technology Services | Information
> Services Group
>
>

Re: Re: Re: Suppress stack trace in error response

2019-02-22 Thread Branham, Jeremy (Experis)

BTW – Congratulations on joining the PMC!

Jeremy Branham
jb...@allstate.com

On 2/22/19, 9:46 AM, "Branham, Jeremy (Experis)"  wrote:

Thanks Jason –
That’s what I was thinking too. It would require some development.

Jeremy Branham
jb...@allstate.com

On 2/22/19, 8:50 AM, "Jason Gerlowski"  wrote:

Hi Jeremy,

Unfortunately Solr doesn't offer anything like what you're looking
for, at least that I know of.  There's no sort of global "quiet" or
"suppressStack" option that you can pass on a request to _not_ get the
stacktrace information back.  There might be individual APIs which
offer something like this, but I've never run into them, so I doubt
it.

Best,

Jason

On Thu, Feb 21, 2019 at 10:53 PM Zheng Lin Edwin Yeo
 wrote:
>
> Hi,
>
> There's too little information provided in your questions.
> You can explain more on the issue or the exception that you are 
facing.
>
> Regards,
> Edwin
>
> On Thu, 21 Feb 2019 at 23:45, Branham, Jeremy (Experis) 

> wrote:
>
> > When Solr throws an exception, like when a client sends a badly 
formed
> > query string, is there a way to suppress the stack trace in the 
error
> > response?
> >
> >
> >
> > Jeremy Branham
> > jb...@allstate.com
> > Allstate Insurance Company | UCV Technology Services | Information
> > Services Group
> >
> >

Re: Re: Suppress stack trace in error response

2019-02-22 Thread Branham, Jeremy (Experis)

Thanks Jason –
That’s what I was thinking too. It would require some development.

Jeremy Branham
jb...@allstate.com

On 2/22/19, 8:50 AM, "Jason Gerlowski"  wrote:

Hi Jeremy,

Unfortunately Solr doesn't offer anything like what you're looking
for, at least that I know of.  There's no sort of global "quiet" or
"suppressStack" option that you can pass on a request to _not_ get the
stacktrace information back.  There might be individual APIs which
offer something like this, but I've never run into them, so I doubt
it.

Best,

Jason

On Thu, Feb 21, 2019 at 10:53 PM Zheng Lin Edwin Yeo
 wrote:
>
> Hi,
>
> There's too little information provided in your questions.
> You can explain more on the issue or the exception that you are facing.
>
> Regards,
> Edwin
>
> On Thu, 21 Feb 2019 at 23:45, Branham, Jeremy (Experis) 

> wrote:
>
> > When Solr throws an exception, like when a client sends a badly formed
> > query string, is there a way to suppress the stack trace in the error
> > response?
> >
> >
> >
> > Jeremy Branham
> > jb...@allstate.com
> > Allstate Insurance Company | UCV Technology Services | Information
> > Services Group
> >
> >

Re: Suppress stack trace in error response

2019-02-22 Thread Jason Gerlowski

Hi Jeremy,

Unfortunately Solr doesn't offer anything like what you're looking
for, at least that I know of.  There's no sort of global "quiet" or
"suppressStack" option that you can pass on a request to _not_ get the
stacktrace information back.  There might be individual APIs which
offer something like this, but I've never run into them, so I doubt
it.

Best,

Jason

On Thu, Feb 21, 2019 at 10:53 PM Zheng Lin Edwin Yeo
 wrote:
>
> Hi,
>
> There's too little information provided in your questions.
> You can explain more on the issue or the exception that you are facing.
>
> Regards,
> Edwin
>
> On Thu, 21 Feb 2019 at 23:45, Branham, Jeremy (Experis) 
> wrote:
>
> > When Solr throws an exception, like when a client sends a badly formed
> > query string, is there a way to suppress the stack trace in the error
> > response?
> >
> >
> >
> > Jeremy Branham
> > jb...@allstate.com
> > Allstate Insurance Company | UCV Technology Services | Information
> > Services Group
> >
> >

Solr Indexes not behaving as expected

2019-02-22 Thread mrityunjayap

Hi 
there is strange issue with our Solr Setup in production. We have a cdcr
setup. Source machines are working fine. but target Solr is not allowing any
operation on existing indexes

Solr Version: 6.5.2
Source and Target are configured on 3 linux machines each with 2 shards and
2 replica

Below is the error log for reload operation. can you help on this? 
Let me know if you need any more information from my side. 


null:org.apache.solr.common.SolrException: reload the collection time
out:180s
at
org.apache.solr.handler.admin.CollectionsHandler.handleResponse(CollectionsHandler.java:303)
at
org.apache.solr.handler.admin.CollectionsHandler.invokeAction(CollectionsHandler.java:235)
at
org.apache.solr.handler.admin.CollectionsHandler.handleRequestBody(CollectionsHandler.java:213)
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:173)
at 
org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:748)
at
org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:729)
at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:510)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:347)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:298)
at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1691)
at
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
at
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)
at
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
at
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
at
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
at
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at
org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:213)
at
org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:119)
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
at
org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:335)
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
at org.eclipse.jetty.server.Server.handle(Server.java:534)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
at
org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
at
org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
at
org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
at
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
at
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
at
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
at
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
at
org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
at java.lang.Thread.run(Thread.java:748)



--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

RE: Query of Death Lucene/Solr 7.6

2019-02-22 Thread Markus Jelsma

Hello Michael,

Sorry it took so long to get back to this, too many things to do.

Anyway, yes, we have WDF on our query-time analysers. I uploaded two log files, 
both the same query of death with and without synonym filter enabled.

https://mail.openindex.io/export/solr-8983-console.log 23 MB
https://mail.openindex.io/export/solr-8983-console-without-syns.log 1.9 MB

Without the synonym we still see a huge number of entries. Many different parts 
of our analyser chain contribute to the expansion of queries, but pf itself 
really turns the problem on or off.

Since SOLR-12243 is new in 7.6, does anyone know that SOLR-12243 could have 
this side-effect?

Thanks,
Markus


-Original message-
> From:Michael Gibney 
> Sent: Friday 8th February 2019 17:19
> To: solr-user@lucene.apache.org
> Subject: Re: Query of Death Lucene/Solr 7.6
> 
> Hi Markus,
> As of 7.6, LUCENE-8531 
> reverted a graph/Spans-based phrase query implementation (introduced in 6.5
> -- LUCENE-7699 ) to an
> implementation that builds a separate phrase query for each possible
> enumerated path through the graph described by a parsed query.
> The potential for combinatoric explosion of the enumerated approach was (as
> far as I can tell) one of the main motivations for introducing the
> Spans-based implementation. Some real-world use cases would be good to
> explore. Markus, could you send (as an attachment) the debug toString() for
> the queries with/without synonyms enabled? I'm also guessing you may have
> WordDelimiterGraphFilter on the query analyzer?
> As an alternative to disabling pf, LUCENE-8531 only reverts to the
> enumerated approach for phrase queries where slop>0, so setting ps=0 would
> probably also help.
> Michael
> 
> On Fri, Feb 8, 2019 at 5:57 AM Markus Jelsma 
> wrote:
> 
> > Hello (apologies for cross-posting),
> >
> > While working on SOLR-12743, using 7.6 on two nodes and 7.2.1 on the
> > remaining four, we stumbled upon a situation where the 7.6 nodes quickly
> > succumb when a 'Query-of-Death' is issued, 7.2.1 up to 7.5 are all
> > unaffected (tested and confirmed).
> >
> > Following Smiley's suggestion i used Eclipse MAT to find the problem in
> > the heap dump i obtained, this fantastic tool revealed within minutes that
> > a query thread ate 65 % of all resources, in the class variables i could
> > find the the query, and reproduce the problem.
> >
> > The problematic query is 'dubbele dijk/rijke dijkproject in het dijktracé
> > eemshaven-delfzijl', on 7.6 this input produces a 40+ MB toString() output
> > in edismax' newFieldQuery. If the node survives it takes 2+ seconds for the
> > query to run (150 ms otherwise). If i disable all query time
> > SynonymGraphFilters it still takes a second and produces just a 9 MB
> > toString() for the query.
> >
> > I could not find anything like this in Jira. I did think of LUCENE-8479
> > and LUCENE-8531 but they were about graphs, this problem looked related
> > though.
> >
> > I think i tracked it further down to LUCENE-8589 or SOLR-12243. When i
> > leave Solr's edismax' pf parameter empty, everything runs fast. When all
> > fields are configured for pf, the node dies.
> >
> > I am now unsure whether this is a Solr or a Lucene issue.
> >
> > Please let me know.
> >
> > Many thanks,
> > Markus
> >
> > ps. in Solr i even got an 'Impossible Exception', my first!
> >
>

Re: Query of Death Lucene/Solr 7.6

Re: Solr Indexes not behaving as expected

RE: Re: Suppress stack trace in error response

Re: Query of Death Lucene/Solr 7.6

Re: Re: Suppress stack trace in error response

Re: Re: Re: Suppress stack trace in error response

Re: Re: Suppress stack trace in error response

Re: Suppress stack trace in error response

Solr Indexes not behaving as expected

RE: Query of Death Lucene/Solr 7.6

10 matches

Site Navigation

Mail list logo

Footer information