[jira] [Commented] (SOLR-7759) DebugComponent's explain should be implemented as a distributed query
[ https://issues.apache.org/jira/browse/SOLR-7759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16371167#comment-16371167 ] Alessandro Benedetti commented on SOLR-7759: Hi Markus, unfortunately I didn't keep the promise... It's on my to do list though... The initial patch I submitted is 1 year old ( 2017 and not 2018). I think it was for Apache Solr 6.4.0 ! > DebugComponent's explain should be implemented as a distributed query > - > > Key: SOLR-7759 > URL: https://issues.apache.org/jira/browse/SOLR-7759 > Project: Solr > Issue Type: Bug >Reporter: Varun Thacker >Priority: Major > Attachments: SOLR_7759.patch > > > Currently when we use debugQuery to see the explanation of the matched > documents, the query fired to get the statistics for the matched documents is > not a distributed query. > This is a problem when using distributed idf. The actual documents are being > scored using the global stats and not per shard stats , but the explain will > show us the score by taking into account the stats from the shard where the > document belongs to. > We should try to implement the explain query as a distributed request so that > the scores match the actual document scores. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7759) DebugComponent's explain should be implemented as a distributed query
[ https://issues.apache.org/jira/browse/SOLR-7759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16371153#comment-16371153 ] Markus Jelsma commented on SOLR-7759: - Alessandro, did you upload a patch three weeks ago? Against what version is that? > DebugComponent's explain should be implemented as a distributed query > - > > Key: SOLR-7759 > URL: https://issues.apache.org/jira/browse/SOLR-7759 > Project: Solr > Issue Type: Bug >Reporter: Varun Thacker >Priority: Major > Attachments: SOLR_7759.patch > > > Currently when we use debugQuery to see the explanation of the matched > documents, the query fired to get the statistics for the matched documents is > not a distributed query. > This is a problem when using distributed idf. The actual documents are being > scored using the global stats and not per shard stats , but the explain will > show us the score by taking into account the stats from the shard where the > document belongs to. > We should try to implement the explain query as a distributed request so that > the scores match the actual document scores. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7759) DebugComponent's explain should be implemented as a distributed query
[ https://issues.apache.org/jira/browse/SOLR-7759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16266713#comment-16266713 ] Alessandro Benedetti commented on SOLR-7759: Hi [~mkhludnev], thank you very much for your feedback. I will update the Jira with a patch including tests by the end of the week. Thanks > DebugComponent's explain should be implemented as a distributed query > - > > Key: SOLR-7759 > URL: https://issues.apache.org/jira/browse/SOLR-7759 > Project: Solr > Issue Type: Bug >Reporter: Varun Thacker > Attachments: SOLR_7759.patch > > > Currently when we use debugQuery to see the explanation of the matched > documents, the query fired to get the statistics for the matched documents is > not a distributed query. > This is a problem when using distributed idf. The actual documents are being > scored using the global stats and not per shard stats , but the explain will > show us the score by taking into account the stats from the shard where the > document belongs to. > We should try to implement the explain query as a distributed request so that > the scores match the actual document scores. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7759) DebugComponent's explain should be implemented as a distributed query
[ https://issues.apache.org/jira/browse/SOLR-7759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16258165#comment-16258165 ] Mikhail Khludnev commented on SOLR-7759: I can commit if there is a test. > DebugComponent's explain should be implemented as a distributed query > - > > Key: SOLR-7759 > URL: https://issues.apache.org/jira/browse/SOLR-7759 > Project: Solr > Issue Type: Bug >Reporter: Varun Thacker > Attachments: SOLR_7759.patch > > > Currently when we use debugQuery to see the explanation of the matched > documents, the query fired to get the statistics for the matched documents is > not a distributed query. > This is a problem when using distributed idf. The actual documents are being > scored using the global stats and not per shard stats , but the explain will > show us the score by taking into account the stats from the shard where the > document belongs to. > We should try to implement the explain query as a distributed request so that > the scores match the actual document scores. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7759) DebugComponent's explain should be implemented as a distributed query
[ https://issues.apache.org/jira/browse/SOLR-7759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16255340#comment-16255340 ] Alessandro Benedetti commented on SOLR-7759: Up, is anyone interested in fixing this problem ? I attached a tentative patch months ago, let me know if anyone is interested and we can work on it to fix it ! > DebugComponent's explain should be implemented as a distributed query > - > > Key: SOLR-7759 > URL: https://issues.apache.org/jira/browse/SOLR-7759 > Project: Solr > Issue Type: Bug >Reporter: Varun Thacker > Attachments: SOLR_7759.patch > > > Currently when we use debugQuery to see the explanation of the matched > documents, the query fired to get the statistics for the matched documents is > not a distributed query. > This is a problem when using distributed idf. The actual documents are being > scored using the global stats and not per shard stats , but the explain will > show us the score by taking into account the stats from the shard where the > document belongs to. > We should try to implement the explain query as a distributed request so that > the scores match the actual document scores. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7759) DebugComponent's explain should be implemented as a distributed query
[ https://issues.apache.org/jira/browse/SOLR-7759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16187761#comment-16187761 ] Alessandro Benedetti commented on SOLR-7759: Let me post here what I noticed when i was doing my investigations : 1) Real score / Debug score is not aligned When we operate a shard request with purpose '16388' ( GET_TOP_IDS,SET_TERM_STATS) we correctly pass the global collection stats and we calculate the real score. When we operate a shard request with purpose '320' ( GET_FIELDS,GET_DEBUG ) we don't pass the global collection stats so the debug score calculus and rendering is not the same as the real score. This can be really confusing and not easy to spot. This should help you reproducing the problem. Regards > DebugComponent's explain should be implemented as a distributed query > - > > Key: SOLR-7759 > URL: https://issues.apache.org/jira/browse/SOLR-7759 > Project: Solr > Issue Type: Bug >Reporter: Varun Thacker > Attachments: SOLR_7759.patch > > > Currently when we use debugQuery to see the explanation of the matched > documents, the query fired to get the statistics for the matched documents is > not a distributed query. > This is a problem when using distributed idf. The actual documents are being > scored using the global stats and not per shard stats , but the explain will > show us the score by taking into account the stats from the shard where the > document belongs to. > We should try to implement the explain query as a distributed request so that > the scores match the actual document scores. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7759) DebugComponent's explain should be implemented as a distributed query
[ https://issues.apache.org/jira/browse/SOLR-7759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15990793#comment-15990793 ] Markus Jelsma commented on SOLR-7759: - Hello Alessandro, i do not seem to be able to reproduce the problem in the first place. > DebugComponent's explain should be implemented as a distributed query > - > > Key: SOLR-7759 > URL: https://issues.apache.org/jira/browse/SOLR-7759 > Project: Solr > Issue Type: Bug >Reporter: Varun Thacker > Attachments: SOLR_7759.patch > > > Currently when we use debugQuery to see the explanation of the matched > documents, the query fired to get the statistics for the matched documents is > not a distributed query. > This is a problem when using distributed idf. The actual documents are being > scored using the global stats and not per shard stats , but the explain will > show us the score by taking into account the stats from the shard where the > document belongs to. > We should try to implement the explain query as a distributed request so that > the scores match the actual document scores. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7759) DebugComponent's explain should be implemented as a distributed query
[ https://issues.apache.org/jira/browse/SOLR-7759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15985609#comment-15985609 ] Alessandro Benedetti commented on SOLR-7759: UP, any feedback ? [~markus17] ? > DebugComponent's explain should be implemented as a distributed query > - > > Key: SOLR-7759 > URL: https://issues.apache.org/jira/browse/SOLR-7759 > Project: Solr > Issue Type: Bug >Reporter: Varun Thacker > Attachments: SOLR_7759.patch > > > Currently when we use debugQuery to see the explanation of the matched > documents, the query fired to get the statistics for the matched documents is > not a distributed query. > This is a problem when using distributed idf. The actual documents are being > scored using the global stats and not per shard stats , but the explain will > show us the score by taking into account the stats from the shard where the > document belongs to. > We should try to implement the explain query as a distributed request so that > the scores match the actual document scores. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7759) DebugComponent's explain should be implemented as a distributed query
[ https://issues.apache.org/jira/browse/SOLR-7759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15856066#comment-15856066 ] Alessandro Benedetti commented on SOLR-7759: [~markus17] any feedback on this ? [~anshumg] ? Anyone interested ? > DebugComponent's explain should be implemented as a distributed query > - > > Key: SOLR-7759 > URL: https://issues.apache.org/jira/browse/SOLR-7759 > Project: Solr > Issue Type: Bug >Reporter: Varun Thacker > Attachments: SOLR_7759.patch > > > Currently when we use debugQuery to see the explanation of the matched > documents, the query fired to get the statistics for the matched documents is > not a distributed query. > This is a problem when using distributed idf. The actual documents are being > scored using the global stats and not per shard stats , but the explain will > show us the score by taking into account the stats from the shard where the > document belongs to. > We should try to implement the explain query as a distributed request so that > the scores match the actual document scores. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7759) DebugComponent's explain should be implemented as a distributed query
[
https://issues.apache.org/jira/browse/SOLR-7759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15842774#comment-15842774
]
Alessandro Benedetti commented on SOLR-7759:
I didn't try it yet, but would this Naive approach work ?
org/apache/solr/handler/component/QueryComponent.java:318
if ((purpose & ShardRequest.PURPOSE_SET_TERM_STATS) != 0 || (purpose &
ShardRequest.PURPOSE_GET_DEBUG) != 0) {
// retrieve from request and update local cache
statsCache.receiveGlobalStats(req);
}
I will investigate further
> DebugComponent's explain should be implemented as a distributed query
> -
>
> Key: SOLR-7759
> URL: https://issues.apache.org/jira/browse/SOLR-7759
> Project: Solr
> Issue Type: Bug
>Reporter: Varun Thacker
>
> Currently when we use debugQuery to see the explanation of the matched
> documents, the query fired to get the statistics for the matched documents is
> not a distributed query.
> This is a problem when using distributed idf. The actual documents are being
> scored using the global stats and not per shard stats , but the explain will
> show us the score by taking into account the stats from the shard where the
> document belongs to.
> We should try to implement the explain query as a distributed request so that
> the scores match the actual document scores.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7759) DebugComponent's explain should be implemented as a distributed query
[ https://issues.apache.org/jira/browse/SOLR-7759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14900592#comment-14900592 ] Markus Jelsma commented on SOLR-7759: - Hello [~varunthacker], no unfortunately not. > DebugComponent's explain should be implemented as a distributed query > - > > Key: SOLR-7759 > URL: https://issues.apache.org/jira/browse/SOLR-7759 > Project: Solr > Issue Type: Bug >Reporter: Varun Thacker > > Currently when we use debugQuery to see the explanation of the matched > documents, the query fired to get the statistics for the matched documents is > not a distributed query. > This is a problem when using distributed idf. The actual documents are being > scored using the global stats and not per shard stats , but the explain will > show us the score by taking into account the stats from the shard where the > document belongs to. > We should try to implement the explain query as a distributed request so that > the scores match the actual document scores. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7759) DebugComponent's explain should be implemented as a distributed query
[ https://issues.apache.org/jira/browse/SOLR-7759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14877441#comment-14877441 ] Varun Thacker commented on SOLR-7759: - Hi [~markus17] , Are you working on a patch for this? > DebugComponent's explain should be implemented as a distributed query > - > > Key: SOLR-7759 > URL: https://issues.apache.org/jira/browse/SOLR-7759 > Project: Solr > Issue Type: Bug >Reporter: Varun Thacker > > Currently when we use debugQuery to see the explanation of the matched > documents, the query fired to get the statistics for the matched documents is > not a distributed query. > This is a problem when using distributed idf. The actual documents are being > scored using the global stats and not per shard stats , but the explain will > show us the score by taking into account the stats from the shard where the > document belongs to. > We should try to implement the explain query as a distributed request so that > the scores match the actual document scores. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
