[jira] [Commented] (LUCENE-6512) ToParentBlockJoinQuery fails with AIOOBE under certain circumstances
[
https://issues.apache.org/jira/browse/LUCENE-6512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14563690#comment-14563690
]
Mikhail Khludnev commented on LUCENE-6512:
--
bq. This was clearly a problem in the index, but due to Solr's buggy
implementation of parent/child documents (you have to set the parent flag in
contrast to Elasticsearch on your own - which is stupid!!!) this was not
detected at indexing time. We should open an issue in Solr to fix this bad
behaviour and make solr automatically add the parent field (it only adds a
_root_ field automatically, maybe it should also add a _parent_ field
automatically).
There is SOLR-5211, but I can't propose a viable way. Do you mean to add
{_parent_=true} and {_root_=PK} by default always? without any killswitch?
ToParentBlockJoinQuery fails with AIOOBE under certain circumstances
Key: LUCENE-6512
URL: https://issues.apache.org/jira/browse/LUCENE-6512
Project: Lucene - Core
Issue Type: Bug
Components: modules/join
Affects Versions: 4.10.4
Reporter: Uwe Schindler
Assignee: Uwe Schindler
Attachments: LUCENE-6512.patch
I had a customer using BlockJoin with Solr. He executed a block join query
and the following appeared in Solr's logs:
{noformat}
28 May 2015 17:19:20 ERROR (SolrException.java:131) -
java.lang.ArrayIndexOutOfBoundsException: -1
at org.apache.lucene.codecs.lucene40.BitVector.get(BitVector.java:149)
at
org.apache.lucene.search.join.ToParentBlockJoinQuery$BlockJoinScorer.nextDoc(ToParentBlockJoinQuery.java:293)
at
org.apache.lucene.search.Weight$DefaultBulkScorer.scoreAll(Weight.java:192)
at
org.apache.lucene.search.Weight$DefaultBulkScorer.score(Weight.java:163)
at org.apache.lucene.search.BulkScorer.score(BulkScorer.java:35)
at
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:621)
at
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:297)
at
org.apache.solr.search.SolrIndexSearcher.buildAndRunCollectorChain(SolrIndexSearcher.java:209)
at
org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:1619)
at
org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:1433)
at
org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:514)
at
org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:484)
at
org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:218)
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1976)
at
org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:777)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:418)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:207)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
at
org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:852)
at
org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588)
at
org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
at java.lang.Thread.run(Thread.java:745)
{noformat}
I debugged this stuff and found out when this happens:
The last block of documents was not followed by a parent. If one of the child
documents without a parent at the end of the index match the inner query,
scorer calls nextSetBit() to find next parent document. This returns -1.
There is an assert afterwards that checks for -1, but in production code,
this is of course never executed.
If the index has deletetions the false -1 is passed to acceptDocs and then
triggers the above problem.
We should change the assert to another IllegalStateException() which is used
to notify the
[jira] [Commented] (LUCENE-6512) ToParentBlockJoinQuery fails with AIOOBE under certain circumstances
[
https://issues.apache.org/jira/browse/LUCENE-6512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14563735#comment-14563735
]
Uwe Schindler commented on LUCENE-6512:
---
{{_root_=PK}} is already on by default. I would propose to do the same for a
new {{_parent_=true}} field. This would be consistent. Is there a killswitch
for {{_root_}}? I did not find one, because to me this field is useless -
unless you want to delete documents (by this you can delete the parent and all
childs will be deleted, too). But the {{_parent_}} field should be mandatory
(and automatic). Because block join queries would never work without them!
In any case the missing check that lead to this issue was only detected by this
:-) But I would hope that Solr fixes this issue (maybe like described above).
ToParentBlockJoinQuery fails with AIOOBE under certain circumstances
Key: LUCENE-6512
URL: https://issues.apache.org/jira/browse/LUCENE-6512
Project: Lucene - Core
Issue Type: Bug
Components: modules/join
Affects Versions: 4.10.4
Reporter: Uwe Schindler
Assignee: Uwe Schindler
Attachments: LUCENE-6512.patch
I had a customer using BlockJoin with Solr. He executed a block join query
and the following appeared in Solr's logs:
{noformat}
28 May 2015 17:19:20 ERROR (SolrException.java:131) -
java.lang.ArrayIndexOutOfBoundsException: -1
at org.apache.lucene.codecs.lucene40.BitVector.get(BitVector.java:149)
at
org.apache.lucene.search.join.ToParentBlockJoinQuery$BlockJoinScorer.nextDoc(ToParentBlockJoinQuery.java:293)
at
org.apache.lucene.search.Weight$DefaultBulkScorer.scoreAll(Weight.java:192)
at
org.apache.lucene.search.Weight$DefaultBulkScorer.score(Weight.java:163)
at org.apache.lucene.search.BulkScorer.score(BulkScorer.java:35)
at
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:621)
at
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:297)
at
org.apache.solr.search.SolrIndexSearcher.buildAndRunCollectorChain(SolrIndexSearcher.java:209)
at
org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:1619)
at
org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:1433)
at
org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:514)
at
org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:484)
at
org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:218)
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1976)
at
org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:777)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:418)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:207)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
at
org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:852)
at
org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588)
at
org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
at java.lang.Thread.run(Thread.java:745)
{noformat}
I debugged this stuff and found out when this happens:
The last block of documents was not followed by a parent. If one of the child
documents without a parent at the end of the index match the inner query,
scorer calls nextSetBit() to find next parent document. This returns -1.
There is an assert afterwards that checks for -1, but in production code,
this is of course never executed.
If the index has deletetions the false -1 is passed to acceptDocs and then
triggers the above problem.
We should change the assert to another IllegalStateException() which is used
to notify
[jira] [Commented] (LUCENE-6512) ToParentBlockJoinQuery fails with AIOOBE under certain circumstances
[
https://issues.apache.org/jira/browse/LUCENE-6512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14563692#comment-14563692
]
Mikhail Khludnev commented on LUCENE-6512:
--
bq. This was clearly a problem in the index, but due to Solr's buggy
implementation of parent/child documents (you have to set the parent flag in
contrast to Elasticsearch on your own - which is stupid!!!) this was not
detected at indexing time. We should open an issue in Solr to fix this bad
behaviour and make solr automatically add the parent field (it only adds a
_root_ field automatically, maybe it should also add a _parent_ field
automatically).
There is SOLR-5211, but I can't propose a viable way. Do you mean to add
{_parent_=true} and {_root_=PK} by default always? without any killswitch?
ToParentBlockJoinQuery fails with AIOOBE under certain circumstances
Key: LUCENE-6512
URL: https://issues.apache.org/jira/browse/LUCENE-6512
Project: Lucene - Core
Issue Type: Bug
Components: modules/join
Affects Versions: 4.10.4
Reporter: Uwe Schindler
Assignee: Uwe Schindler
Attachments: LUCENE-6512.patch
I had a customer using BlockJoin with Solr. He executed a block join query
and the following appeared in Solr's logs:
{noformat}
28 May 2015 17:19:20 ERROR (SolrException.java:131) -
java.lang.ArrayIndexOutOfBoundsException: -1
at org.apache.lucene.codecs.lucene40.BitVector.get(BitVector.java:149)
at
org.apache.lucene.search.join.ToParentBlockJoinQuery$BlockJoinScorer.nextDoc(ToParentBlockJoinQuery.java:293)
at
org.apache.lucene.search.Weight$DefaultBulkScorer.scoreAll(Weight.java:192)
at
org.apache.lucene.search.Weight$DefaultBulkScorer.score(Weight.java:163)
at org.apache.lucene.search.BulkScorer.score(BulkScorer.java:35)
at
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:621)
at
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:297)
at
org.apache.solr.search.SolrIndexSearcher.buildAndRunCollectorChain(SolrIndexSearcher.java:209)
at
org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:1619)
at
org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:1433)
at
org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:514)
at
org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:484)
at
org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:218)
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1976)
at
org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:777)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:418)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:207)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
at
org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:852)
at
org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588)
at
org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
at java.lang.Thread.run(Thread.java:745)
{noformat}
I debugged this stuff and found out when this happens:
The last block of documents was not followed by a parent. If one of the child
documents without a parent at the end of the index match the inner query,
scorer calls nextSetBit() to find next parent document. This returns -1.
There is an assert afterwards that checks for -1, but in production code,
this is of course never executed.
If the index has deletetions the false -1 is passed to acceptDocs and then
triggers the above problem.
We should change the assert to another IllegalStateException() which is used
to notify the
[jira] [Commented] (LUCENE-6512) ToParentBlockJoinQuery fails with AIOOBE under certain circumstances
[
https://issues.apache.org/jira/browse/LUCENE-6512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14563287#comment-14563287
]
Uwe Schindler commented on LUCENE-6512:
---
In trunk and 5.x the same happens, just that -1 is replaced by
DocIdSetIterator.NO_MORE_DOCS. But leads to the same problem.
ToParentBlockJoinQuery fails with AIOOBE under certain circumstances
Key: LUCENE-6512
URL: https://issues.apache.org/jira/browse/LUCENE-6512
Project: Lucene - Core
Issue Type: Bug
Components: modules/join
Affects Versions: 4.10.4
Reporter: Uwe Schindler
I had a customer using BlockJoin with Solr. He executed a block join query
and the following appeared in Solr's logs:
{noformat}
28 May 2015 17:19:20 ERROR (SolrException.java:131) -
java.lang.ArrayIndexOutOfBoundsException: -1
at org.apache.lucene.codecs.lucene40.BitVector.get(BitVector.java:149)
at
org.apache.lucene.search.join.ToParentBlockJoinQuery$BlockJoinScorer.nextDoc(ToParentBlockJoinQuery.java:293)
at
org.apache.lucene.search.Weight$DefaultBulkScorer.scoreAll(Weight.java:192)
at
org.apache.lucene.search.Weight$DefaultBulkScorer.score(Weight.java:163)
at org.apache.lucene.search.BulkScorer.score(BulkScorer.java:35)
at
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:621)
at
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:297)
at
org.apache.solr.search.SolrIndexSearcher.buildAndRunCollectorChain(SolrIndexSearcher.java:209)
at
org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:1619)
at
org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:1433)
at
org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:514)
at
org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:484)
at
org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:218)
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1976)
at
org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:777)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:418)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:207)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
at
org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:852)
at
org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588)
at
org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
at java.lang.Thread.run(Thread.java:745)
{noformat}
I debugged this stuff and found out when this happens:
The last block of documents was not followed by a parent. If one of the child
documents without a parent at the end of the index match the inner query,
scorer calls nextSetBit() to find next parent document. This returns -1.
There is an assert afterwards that checks for -1, but in production code,
this is of course never executed.
If the index has deletetions the false -1 is passed to acceptDocs and then
triggers the above problem.
We should change the assert to another IllegalStateException() which is used
to notify the user if the orthogonality is broken. By that the user gets the
information that his index is broken and contains child documents without a
parent at the very end of a segment.
I have seen this on 4.10.4. Maybe thats already fixed in 5.0, but I just open
this here for investigation. This was clearly a problem in the index, but due
to Solr's buggy implementation of parent/child documents (you have to set the
parent flag in contrast to Elasticsearch on your own - which is stupid!!!)
this was not detected at indexing time. We should open an issue in Solr to
[jira] [Commented] (LUCENE-6512) ToParentBlockJoinQuery fails with AIOOBE under certain circumstances
[
https://issues.apache.org/jira/browse/LUCENE-6512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14563574#comment-14563574
]
Adrien Grand commented on LUCENE-6512:
--
+1
ToParentBlockJoinQuery fails with AIOOBE under certain circumstances
Key: LUCENE-6512
URL: https://issues.apache.org/jira/browse/LUCENE-6512
Project: Lucene - Core
Issue Type: Bug
Components: modules/join
Affects Versions: 4.10.4
Reporter: Uwe Schindler
Assignee: Uwe Schindler
Attachments: LUCENE-6512.patch
I had a customer using BlockJoin with Solr. He executed a block join query
and the following appeared in Solr's logs:
{noformat}
28 May 2015 17:19:20 ERROR (SolrException.java:131) -
java.lang.ArrayIndexOutOfBoundsException: -1
at org.apache.lucene.codecs.lucene40.BitVector.get(BitVector.java:149)
at
org.apache.lucene.search.join.ToParentBlockJoinQuery$BlockJoinScorer.nextDoc(ToParentBlockJoinQuery.java:293)
at
org.apache.lucene.search.Weight$DefaultBulkScorer.scoreAll(Weight.java:192)
at
org.apache.lucene.search.Weight$DefaultBulkScorer.score(Weight.java:163)
at org.apache.lucene.search.BulkScorer.score(BulkScorer.java:35)
at
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:621)
at
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:297)
at
org.apache.solr.search.SolrIndexSearcher.buildAndRunCollectorChain(SolrIndexSearcher.java:209)
at
org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:1619)
at
org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:1433)
at
org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:514)
at
org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:484)
at
org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:218)
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1976)
at
org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:777)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:418)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:207)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
at
org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:852)
at
org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588)
at
org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
at java.lang.Thread.run(Thread.java:745)
{noformat}
I debugged this stuff and found out when this happens:
The last block of documents was not followed by a parent. If one of the child
documents without a parent at the end of the index match the inner query,
scorer calls nextSetBit() to find next parent document. This returns -1.
There is an assert afterwards that checks for -1, but in production code,
this is of course never executed.
If the index has deletetions the false -1 is passed to acceptDocs and then
triggers the above problem.
We should change the assert to another IllegalStateException() which is used
to notify the user if the orthogonality is broken. By that the user gets the
information that his index is broken and contains child documents without a
parent at the very end of a segment.
I have seen this on 4.10.4. Maybe thats already fixed in 5.0, but I just open
this here for investigation. This was clearly a problem in the index, but due
to Solr's buggy implementation of parent/child documents (you have to set the
parent flag in contrast to Elasticsearch on your own - which is stupid!!!)
this was not detected at indexing time. We should open an issue in Solr to
fix this bad behaviour and make solr
