Solr 7.3 cluster issue
Happy holidays folks, we have a production deployment usage Solr 7.3 in a three node cluster we have a number of collections setup, three shards with a replica factor of 2. The system has been fine, but we experienced issues with disk space one of the nodes. Node 0 starts but does not show any cores / replicas, the solr.log is full of these "o.a.s.c.ZkController org.apache.solr.common.SolrException: Replica core_node7 is not present in cluster state: null” Node 1 and Node 2 are OK, all data from all collections is accessible. Can I recreate node 0 as though it had failed completely ?, is it OK to remove the references to the replicas (missing) and recreate. Would you be able to provide me some guidance of the safest way to reintroduce node 0 given our situation. Many thanks Dave
Re: does copyFields increase indexe size ?
On 12/26/2019 1:21 PM, Nicolas Paris wrote: Below a part of the managed-schema. There is 1k section* fields. The second experience, I removed the copyField, droped the collection and re-indexed the whole. To mesure the index size, I went to solr-cloud and looked in the cloud part: 40GO per shard. I also look at the folder size. I made some tests and the _text_ field is indexed. Your schema says that the destination field is not stored and doesn't have docValues. So the only thing it has is indexed. All of the terms generated by index analysis will already be in the index from the source fields. So what will be added is just another set of pointers to each relevant term. That's not going to be very large. Probably only a few bytes for each term. So with this copyField, the index will get larger, but probably not significantly. Thanks, Shawn
Re: Boosting only top n results that match a criteria
Simply boosting on class A1 won't work since there may be many documents from that class, all getting equal boost. I want only top 5 docs of that class to get the boost. On Fri, 27 Dec 2019 at 22:42, Erick Erickson wrote: > Yes. Rerank essentially takes the top N results of one query and re-scores > them through another query. So just boost the secondary query. > > But you may not even have to do that. Just add a boost clause to a single > query and boost your class A1 quite high. See “boost” and/or “bq”. > > Best, > Erick > > > On Dec 27, 2019, at 10:57 AM, Nitin Arora > wrote: > > > > Hi Erick, I was not able to figure how exactly I will use > > RerankQParserPlugin to achieve the desired reranking. I see that I can > > rerank all the top RERANK_DOCS results - it is possible that they > contain a > > hundred results of class A1 or none. But the desired behaviour I want is > to > > pick (only) the top 5 results of class A1 from my potentially 100s of > > results. Then boost them to first page. > > Do you think this(or near this) behaviour is possible > > using RerankQParserPlugin? Please shed more light how. > > > > On Fri, 27 Dec 2019 at 19:48, Erick Erickson > > wrote: > > > >> Have you seen RerankQParserPlugin? > >> > >> Best, > >> Erick > >> > >>> On Dec 27, 2019, at 8:49 AM, Emir Arnautović < > >> emir.arnauto...@sematext.com> wrote: > >>> > >>> Hi Nitin, > >>> Can you simply filter and return top 5: > >>> > >>> ….=class:A1=5 > >>> > >>> Emir > >>> -- > >>> Monitoring - Log Management - Alerting - Anomaly Detection > >>> Solr & Elasticsearch Consulting Support Training - > http://sematext.com/ > >>> > >>> > >>> > On 27 Dec 2019, at 13:55, Nitin Arora wrote: > > Hello, I have a complex solr query with various boosts applied that > returns, say a few hundred results. Out of these hundreds of results I > >> want > to further boost, say the top 5 results that satisfy a particular > >> criteria > - e.g. class=A1. So I want the top 5 results from class A1 in my > >> existing > results set to come further higher, so that I can show them on the > first > page of my final results. How do I achieve this? > I am new to SOLR and this community so apologies if this is > >> trivial/repeat. > > Thanks, > Nitin > >>> > >> > >> > >
Re: Boosting only top n results that match a criteria
It sounds like this will work. Can we do it in a single query? Something like: OR ( AND class=A1 AND rows=5)^high_boost I am not good with SOLR syntax and need you to validate if such a query will serve the purpose. On Fri, 27 Dec 2019 at 22:10, Walter Underwood wrote: > You could use two queries. Do the first with rows=5, then for the second > use > an fq that filters out the IDs of those five. You’ll need to do the first > query again > to do the second and further page of results statelessly, but that should > still > be pretty fast. > > wunder > Walter Underwood > wun...@wunderwood.org > http://observer.wunderwood.org/ (my blog) > > > On Dec 27, 2019, at 9:57 AM, Nitin Arora wrote: > > > > Hi Erick, I was not able to figure how exactly I will use > > RerankQParserPlugin to achieve the desired reranking. I see that I can > > rerank all the top RERANK_DOCS results - it is possible that they > contain a > > hundred results of class A1 or none. But the desired behaviour I want is > to > > pick (only) the top 5 results of class A1 from my potentially 100s of > > results. Then boost them to first page. > > Do you think this(or near this) behaviour is possible > > using RerankQParserPlugin? Please shed more light how. > > > > On Fri, 27 Dec 2019 at 19:48, Erick Erickson > > wrote: > > > >> Have you seen RerankQParserPlugin? > >> > >> Best, > >> Erick > >> > >>> On Dec 27, 2019, at 8:49 AM, Emir Arnautović < > >> emir.arnauto...@sematext.com> wrote: > >>> > >>> Hi Nitin, > >>> Can you simply filter and return top 5: > >>> > >>> ….=class:A1=5 > >>> > >>> Emir > >>> -- > >>> Monitoring - Log Management - Alerting - Anomaly Detection > >>> Solr & Elasticsearch Consulting Support Training - > http://sematext.com/ > >>> > >>> > >>> > On 27 Dec 2019, at 13:55, Nitin Arora wrote: > > Hello, I have a complex solr query with various boosts applied that > returns, say a few hundred results. Out of these hundreds of results I > >> want > to further boost, say the top 5 results that satisfy a particular > >> criteria > - e.g. class=A1. So I want the top 5 results from class A1 in my > >> existing > results set to come further higher, so that I can show them on the > first > page of my final results. How do I achieve this? > I am new to SOLR and this community so apologies if this is > >> trivial/repeat. > > Thanks, > Nitin > >>> > >> > >> > >
Re: Boosting only top n results that match a criteria
Yes. Rerank essentially takes the top N results of one query and re-scores them through another query. So just boost the secondary query. But you may not even have to do that. Just add a boost clause to a single query and boost your class A1 quite high. See “boost” and/or “bq”. Best, Erick > On Dec 27, 2019, at 10:57 AM, Nitin Arora wrote: > > Hi Erick, I was not able to figure how exactly I will use > RerankQParserPlugin to achieve the desired reranking. I see that I can > rerank all the top RERANK_DOCS results - it is possible that they contain a > hundred results of class A1 or none. But the desired behaviour I want is to > pick (only) the top 5 results of class A1 from my potentially 100s of > results. Then boost them to first page. > Do you think this(or near this) behaviour is possible > using RerankQParserPlugin? Please shed more light how. > > On Fri, 27 Dec 2019 at 19:48, Erick Erickson > wrote: > >> Have you seen RerankQParserPlugin? >> >> Best, >> Erick >> >>> On Dec 27, 2019, at 8:49 AM, Emir Arnautović < >> emir.arnauto...@sematext.com> wrote: >>> >>> Hi Nitin, >>> Can you simply filter and return top 5: >>> >>> ….=class:A1=5 >>> >>> Emir >>> -- >>> Monitoring - Log Management - Alerting - Anomaly Detection >>> Solr & Elasticsearch Consulting Support Training - http://sematext.com/ >>> >>> >>> On 27 Dec 2019, at 13:55, Nitin Arora wrote: Hello, I have a complex solr query with various boosts applied that returns, say a few hundred results. Out of these hundreds of results I >> want to further boost, say the top 5 results that satisfy a particular >> criteria - e.g. class=A1. So I want the top 5 results from class A1 in my >> existing results set to come further higher, so that I can show them on the first page of my final results. How do I achieve this? I am new to SOLR and this community so apologies if this is >> trivial/repeat. Thanks, Nitin >>> >> >>
Re: Boosting only top n results that match a criteria
You could use two queries. Do the first with rows=5, then for the second use an fq that filters out the IDs of those five. You’ll need to do the first query again to do the second and further page of results statelessly, but that should still be pretty fast. wunder Walter Underwood wun...@wunderwood.org http://observer.wunderwood.org/ (my blog) > On Dec 27, 2019, at 9:57 AM, Nitin Arora wrote: > > Hi Erick, I was not able to figure how exactly I will use > RerankQParserPlugin to achieve the desired reranking. I see that I can > rerank all the top RERANK_DOCS results - it is possible that they contain a > hundred results of class A1 or none. But the desired behaviour I want is to > pick (only) the top 5 results of class A1 from my potentially 100s of > results. Then boost them to first page. > Do you think this(or near this) behaviour is possible > using RerankQParserPlugin? Please shed more light how. > > On Fri, 27 Dec 2019 at 19:48, Erick Erickson > wrote: > >> Have you seen RerankQParserPlugin? >> >> Best, >> Erick >> >>> On Dec 27, 2019, at 8:49 AM, Emir Arnautović < >> emir.arnauto...@sematext.com> wrote: >>> >>> Hi Nitin, >>> Can you simply filter and return top 5: >>> >>> ….=class:A1=5 >>> >>> Emir >>> -- >>> Monitoring - Log Management - Alerting - Anomaly Detection >>> Solr & Elasticsearch Consulting Support Training - http://sematext.com/ >>> >>> >>> On 27 Dec 2019, at 13:55, Nitin Arora wrote: Hello, I have a complex solr query with various boosts applied that returns, say a few hundred results. Out of these hundreds of results I >> want to further boost, say the top 5 results that satisfy a particular >> criteria - e.g. class=A1. So I want the top 5 results from class A1 in my >> existing results set to come further higher, so that I can show them on the first page of my final results. How do I achieve this? I am new to SOLR and this community so apologies if this is >> trivial/repeat. Thanks, Nitin >>> >> >>
Re: Boosting only top n results that match a criteria
Hi Erick, I was not able to figure how exactly I will use RerankQParserPlugin to achieve the desired reranking. I see that I can rerank all the top RERANK_DOCS results - it is possible that they contain a hundred results of class A1 or none. But the desired behaviour I want is to pick (only) the top 5 results of class A1 from my potentially 100s of results. Then boost them to first page. Do you think this(or near this) behaviour is possible using RerankQParserPlugin? Please shed more light how. On Fri, 27 Dec 2019 at 19:48, Erick Erickson wrote: > Have you seen RerankQParserPlugin? > > Best, > Erick > > > On Dec 27, 2019, at 8:49 AM, Emir Arnautović < > emir.arnauto...@sematext.com> wrote: > > > > Hi Nitin, > > Can you simply filter and return top 5: > > > > ….=class:A1=5 > > > > Emir > > -- > > Monitoring - Log Management - Alerting - Anomaly Detection > > Solr & Elasticsearch Consulting Support Training - http://sematext.com/ > > > > > > > >> On 27 Dec 2019, at 13:55, Nitin Arora wrote: > >> > >> Hello, I have a complex solr query with various boosts applied that > >> returns, say a few hundred results. Out of these hundreds of results I > want > >> to further boost, say the top 5 results that satisfy a particular > criteria > >> - e.g. class=A1. So I want the top 5 results from class A1 in my > existing > >> results set to come further higher, so that I can show them on the first > >> page of my final results. How do I achieve this? > >> I am new to SOLR and this community so apologies if this is > trivial/repeat. > >> > >> Thanks, > >> Nitin > > > >
Re: Boosting only top n results that match a criteria
Hi Emir, This will filter my results to only include top 5 results from class A1. I want the other results to stay as is while boosting only the top 5 results of class A1. On Fri, 27 Dec 2019 at 19:19, Emir Arnautović wrote: > Hi Nitin, > Can you simply filter and return top 5: > > ….=class:A1=5 > > Emir > -- > Monitoring - Log Management - Alerting - Anomaly Detection > Solr & Elasticsearch Consulting Support Training - http://sematext.com/ > > > > > On 27 Dec 2019, at 13:55, Nitin Arora wrote: > > > > Hello, I have a complex solr query with various boosts applied that > > returns, say a few hundred results. Out of these hundreds of results I > want > > to further boost, say the top 5 results that satisfy a particular > criteria > > - e.g. class=A1. So I want the top 5 results from class A1 in my existing > > results set to come further higher, so that I can show them on the first > > page of my final results. How do I achieve this? > > I am new to SOLR and this community so apologies if this is > trivial/repeat. > > > > Thanks, > > Nitin > >
Re: Exceptions in solr log
1> an incorrectly formed query has been sent to Solr. This is just a syntax error in the query. 2> Not sure about that one. 3> Older versions of Solr had a bogus error that would be generated by the admin UI, see: https://issues.apache.org/jira/browse/SOLR-8793 In future, it would help to see both the Solr version you use and more of the stack traces, particularly the “caused by” sections. Best, Erick > On Dec 27, 2019, at 12:20 AM, Akreeti Agarwal wrote: > > Hi All, > > Please help me with these exceptions and their workarounds: > > 1. org.apache.solr.common.SolrException: org.apache.solr.search.SyntaxError: > Cannot parse > 2. o.a.s.h.IndexFetcher No files to download for index generation: 1394327 > 3. o.a.s.h.a.LukeRequestHandler Error getting file length for [segments_b] > (this one is warning as discussed) > > I am getting these errors always in my solr logs, what can be the reason > behind them and how should I resolve it. > > > Thanks & Regards, > Akreeti Agarwal > ::DISCLAIMER:: > > The contents of this e-mail and any attachment(s) are confidential and > intended for the named recipient(s) only. E-mail transmission is not > guaranteed to be secure or error-free as information could be intercepted, > corrupted, lost, destroyed, arrive late or incomplete, or may contain viruses > in transmission. The e mail and its contents (with or without referred > errors) shall therefore not attach any liability on the originator or HCL or > its affiliates. Views or opinions, if any, presented in this email are solely > those of the author and may not necessarily reflect the views or opinions of > HCL or its affiliates. Any form of reproduction, dissemination, copying, > disclosure, modification, distribution and / or publication of this message > without the prior written consent of authorized representative of HCL is > strictly prohibited. If you have received this email in error please delete > it and notify the sender immediately. Before opening any email and/or > attachments, please check them for viruses and other defects. >
Re: Boosting only top n results that match a criteria
Have you seen RerankQParserPlugin? Best, Erick > On Dec 27, 2019, at 8:49 AM, Emir Arnautović > wrote: > > Hi Nitin, > Can you simply filter and return top 5: > > ….=class:A1=5 > > Emir > -- > Monitoring - Log Management - Alerting - Anomaly Detection > Solr & Elasticsearch Consulting Support Training - http://sematext.com/ > > > >> On 27 Dec 2019, at 13:55, Nitin Arora wrote: >> >> Hello, I have a complex solr query with various boosts applied that >> returns, say a few hundred results. Out of these hundreds of results I want >> to further boost, say the top 5 results that satisfy a particular criteria >> - e.g. class=A1. So I want the top 5 results from class A1 in my existing >> results set to come further higher, so that I can show them on the first >> page of my final results. How do I achieve this? >> I am new to SOLR and this community so apologies if this is trivial/repeat. >> >> Thanks, >> Nitin >
Re: Boosting only top n results that match a criteria
Hi Nitin, Can you simply filter and return top 5: ….=class:A1=5 Emir -- Monitoring - Log Management - Alerting - Anomaly Detection Solr & Elasticsearch Consulting Support Training - http://sematext.com/ > On 27 Dec 2019, at 13:55, Nitin Arora wrote: > > Hello, I have a complex solr query with various boosts applied that > returns, say a few hundred results. Out of these hundreds of results I want > to further boost, say the top 5 results that satisfy a particular criteria > - e.g. class=A1. So I want the top 5 results from class A1 in my existing > results set to come further higher, so that I can show them on the first > page of my final results. How do I achieve this? > I am new to SOLR and this community so apologies if this is trivial/repeat. > > Thanks, > Nitin
Boosting only top n results that match a criteria
Hello, I have a complex solr query with various boosts applied that returns, say a few hundred results. Out of these hundreds of results I want to further boost, say the top 5 results that satisfy a particular criteria - e.g. class=A1. So I want the top 5 results from class A1 in my existing results set to come further higher, so that I can show them on the first page of my final results. How do I achieve this? I am new to SOLR and this community so apologies if this is trivial/repeat. Thanks, Nitin
