[
https://issues.apache.org/jira/browse/SOLR-7606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14566051#comment-14566051
]
Uwe Schindler commented on SOLR-7606:
-------------------------------------
Hi Yonik,
I agree. The problem is currently Solr - this is why [~rcmuir] moved this issue
over to Solr:
- If you delete just the parent document (because the child documents are not
automatically removed), you exactly run into the issue described here. This was
exactly the issue at my customer, a code bug failed to delete the child. So the
very last parent in the segment was deleted, but the childs stayed alive ->
AIOOBE.
- Solr should make the "parent" marker field an internal field like "_root_"
(maybe named "_parent_"). If those 2 fields are internal, then stuff like
deleting childs would work automatically. Or if you forget to add the parent
marker.
I agree with [~mikemccand] that we should maybe add a separate API in Lucene's
Join module that allows indexing and deleting blocks and Lucene takes care of
also deleting Childs automatically and adding the parent "marker" during
indexing. If we have this, it is safe to use the queries without the checks,
but currently its very very trappy - unfortunately.
I would like to fix *this* current issue by a try...catch (AIOOBE). But we
should further improve the query and indexing. It is really too trappy,
especially for the Solr "end user". Elasticsearch is safe, because it's "nested
document" functionality is completely hidden to the end user. It is impossible
there to hit any of the traps as end user.
> ToParentBlockJoinQuery fails with AIOOBE under certain circumstances
> --------------------------------------------------------------------
>
> Key: SOLR-7606
> URL: https://issues.apache.org/jira/browse/SOLR-7606
> Project: Solr
> Issue Type: Bug
> Affects Versions: 4.10.4
> Reporter: Uwe Schindler
> Assignee: Uwe Schindler
> Attachments: LUCENE-6512.patch
>
>
> I had a customer using BlockJoin with Solr. He executed a block join query
> and the following appeared in Solr's logs:
> {noformat}
> 28 May 2015 17:19:20 ERROR (SolrException.java:131) -
> java.lang.ArrayIndexOutOfBoundsException: -1
> at org.apache.lucene.codecs.lucene40.BitVector.get(BitVector.java:149)
> at
> org.apache.lucene.search.join.ToParentBlockJoinQuery$BlockJoinScorer.nextDoc(ToParentBlockJoinQuery.java:293)
> at
> org.apache.lucene.search.Weight$DefaultBulkScorer.scoreAll(Weight.java:192)
> at
> org.apache.lucene.search.Weight$DefaultBulkScorer.score(Weight.java:163)
> at org.apache.lucene.search.BulkScorer.score(BulkScorer.java:35)
> at
> org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:621)
> at
> org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:297)
> at
> org.apache.solr.search.SolrIndexSearcher.buildAndRunCollectorChain(SolrIndexSearcher.java:209)
> at
> org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:1619)
> at
> org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:1433)
> at
> org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:514)
> at
> org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:484)
> at
> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:218)
> at
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
> at org.apache.solr.core.SolrCore.execute(SolrCore.java:1976)
> at
> org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:777)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:418)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:207)
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
> at
> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
> at
> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
> at
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
> at
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
> at
> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
> at
> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
> at
> org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:852)
> at
> org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588)
> at
> org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
> at java.lang.Thread.run(Thread.java:745)
> {noformat}
> I debugged this stuff and found out when this happens:
> The last block of documents was not followed by a parent. If one of the child
> documents without a parent at the end of the index match the inner query,
> scorer calls nextSetBit() to find next parent document. This returns -1.
> There is an assert afterwards that checks for -1, but in production code,
> this is of course never executed.
> If the index has deletetions the false -1 is passed to acceptDocs and then
> triggers the above problem.
> We should change the assert to another IllegalStateException() which is used
> to notify the user if the orthogonality is broken. By that the user gets the
> information that his index is broken and contains child documents without a
> parent at the very end of a segment.
> I have seen this on 4.10.4. Maybe thats already fixed in 5.0, but I just open
> this here for investigation. This was clearly a problem in the index, but due
> to Solr's buggy implementation of parent/child documents (you have to set the
> parent flag in contrast to Elasticsearch on your own - which is stupid!!!)
> this was not detected at indexing time. We should open an issue in Solr to
> fix this bad behaviour and make solr automatically add the parent field (it
> only adds a "_root_" field automatically, maybe it should also add a
> "_parent_" field automatically).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
