Re: Potential Slow searching for unified highlighting on Solr 8.8.0/8.8.1
Hi,
Solr uses JIRA for issue tickets. You can find it here:
https://issues.apache.org/jira/browse/SOLR
I'd suggest filing a new bug issue in the SOLR project (note that
several other projects also use this JIRA installation). Here's an
example of an existing highlighter issue for reference:
https://issues.apache.org/jira/browse/SOLR-14019.
See also some brief documentation:
https://cwiki.apache.org/confluence/display/solr/HowToContribute#HowToContribute-JIRAtips(ourissue/bugtracker)
Regards,
Ere
Flowerday, Matthew J kirjoitti 1.3.2021 klo 14.58:
Hi Ere
Please to be of service!
No I have not filed a JIRA ticket. I am new to interacting with the Solr
Community and only beginning to 'find my legs'. I am not too sure what JIRA
is I am afraid!
Regards
Matthew
Matthew Flowerday | Consultant | ULEAF
Unisys | 01908 774830| matthew.flower...@unisys.com
Address Enigma | Wavendon Business Park | Wavendon | Milton Keynes | MK17
8LX
THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY
MATERIAL and is for use only by the intended recipient. If you received this
in error, please contact the sender and delete the e-mail and its
attachments from all devices.
-Original Message-
From: Ere Maijala
Sent: 01 March 2021 12:53
To: solr-user@lucene.apache.org
Subject: Re: Potential Slow searching for unified highlighting on Solr
8.8.0/8.8.1
EXTERNAL EMAIL - Be cautious of all links and attachments.
Hi,
Whoa, thanks for the heads-up! You may just have saved me from a whole lot
of trouble. Did you file a JIRA ticket already?
Thanks,
Ere
Flowerday, Matthew J kirjoitti 1.3.2021 klo 14.00:
Hi There
I just came across a situation where a unified highlighting search
under solr 8.8.0/8.8.1 can take over 20 mins to run and eventually times
out.
I resolved it by a config change – but it can catch you out. Hence
this email.
With solr 8.8.0 a new unified highlighting parameter
was implemented which if not set defaults to 0.5.
This attempts to improve the high lighting so that highlighted text
does not appear right at the left. This works well but if you have a
search result with numerous occurrences of the word in question within
the record performance goes right down!
2021-02-27 06:45:03.151 INFO (qtp762476028-20) [ x:uleaf]
o.a.s.c.S.Request [uleaf] webapp=/solr path=/select
params={hl.snippets=2=test=on=100=id,d
escription,specification,score=20=*=10&_=161440511913
4}
hits=57008 status=0 QTime=1414320
2021-02-27 06:45:03.245 INFO (qtp762476028-20) [ x:uleaf]
o.a.s.s.HttpSolrCall Unable to write response, client closed
connection or we are shutting down =>
org.eclipse.jetty.io.EofException
at
org.eclipse.jetty.io.ChannelEndPoint.flush(ChannelEndPoint.java:279)
org.eclipse.jetty.io.EofException: null
at
org.eclipse.jetty.io.ChannelEndPoint.flush(ChannelEndPoint.java:279)
~[jetty-io-9.4.34.v20201102.jar:9.4.34.v20201102]
at
org.eclipse.jetty.io.WriteFlusher.flush(WriteFlusher.java:422)
~[jetty-io-9.4.34.v20201102.jar:9.4.34.v20201102]
at
org.eclipse.jetty.io.WriteFlusher.completeWrite(WriteFlusher.java:378)
~[jetty-io-9.4.34.v20201102.jar:9.4.34.v20201102]
when I set =0.25 results came back much quicker
2021-02-27 14:59:57.189 INFO (qtp1291367132-24) [ x:holmes]
o.a.s.c.S.Request [holmes] webapp=/solr path=/select
params={hl.weightMatches=false=on=id,description,specification,s
core=1=0.25=100=2=test
axAnalyzedChars=100=*=unified=9&_=
1614430061690}
hits=136939 status=0 QTime=87024
And =0.1
2021-02-27 15:18:45.542 INFO (qtp1291367132-19) [ x:holmes]
o.a.s.c.S.Request [holmes] webapp=/solr path=/select
params={hl.weightMatches=false=on=id,description,specification,s
core=1=0.1=100=2=test
xAnalyzedChars=100=*=unified=9&_=1
614430061690}
hits=136939 status=0 QTime=69033
And =0.0
2021-02-27 15:20:38.194 INFO (qtp1291367132-24) [ x:holmes]
o.a.s.c.S.Request [holmes] webapp=/solr path=/select
params={hl.weightMatches=false=on=id,description,specification,s
core=1=0.0=100=2=test
xAnalyzedChars=100=*=unified=9&_=1
614430061690}
hits=136939 status=0 QTime=2841
I left our setting at 0.0 – this presumably how it was in 7.7.1 (fully
left aligned). I am not too sure as to how many time a word has to
occur in a record for performance to go right down – but if too many
it can have a BIG impact.
I also noticed that setting =9 did not break out of
the query until it finished. Perhaps because the query finished
quickly and what took the time was the highlighting. It might be an
idea to get to also cover any highlighting so that the
query does not run until the jetty timeout is hit. The machine 100%
one core for about
20 mins!.
Hope this helps.
Regards
Matthew
*Matthew Flowerday*| Consultant | ULEAF
Unisys | 01908 774830| matthew.flower...@unisys.com
<mailto:matthew.flower...@unisys.com>
Address Enigma | Wavendon Business Park |
Solr NRT Replicas Out of Sync
Hi, In our Solr 7.4 cluster, we have noticed that some replicas of some of our Collections are out of sync, the slave replica has more number of records than the leader. This is resulting in different number of records on subsequent queries on the same Collection. Commit is also not helping in this case. I'm able to replicate the issue using the steps given below: 1. Create a collection with 1 shard and 2 rf 2. Ingest 10k records in the collection 3. Turn down node with replica 2 4. Ingest 10k records in the collection 5. Turn down replica 1 6. Turn up replica 2, wait till it become leader 7. Ingest 20k records on replica 2 8. Turn down replica 2 9. Turn up replica 1, wait till it become leader or use FORCELEADER action of Collections API 10. Turn up replica 2 11. Now replica 2 has 30k records and replica 1 has 20k records and they never sync I tried the same steps with TLOG replicas and in that case both replicas had 20k records in the end and were in sync but 10k records were lost. Is there any way to sync the replicas? I am looking for a lightweight solution that doesn't require re-creating the index. Regards, Anshuman
Re: NPE in QueryComponent.mergeIds when using timeAllowed and sorting SOLR 8.7
Patch looks good to me. Since it's a bugfix it can be committed to 8_8 branch and released on the next bugfix release, though I don't think it should trigger one. In the meantime, if you can patch your environment and confirm that it fixes your problem, that's a good comment to leave in SOLR-14758. <https://issues.apache.org/jira/browse/SOLR-14758> On Mon, Mar 1, 2021 at 3:12 PM Phill Campbell wrote: > Anyone? > > > On Feb 24, 2021, at 7:47 AM, Phill Campbell > wrote: > > > > Last week I switched to Solr 8.7 from a “special” build of Solr 6.6 > > > > The system has a timeout set for querying. I am now seeing this bug. > > > > https://issues.apache.org/jira/browse/SOLR-14758 < > https://issues.apache.org/jira/browse/SOLR-14758> > > > > Max Query Time goes from 1.6 seconds to 20 seconds and affects the > entire system for about 2 minutes as reported in New Relic. > > > > null:java.lang.NullPointerException > > at > org.apache.solr.handler.component.QueryComponent.mergeIds(QueryComponent.java:935) > > at > org.apache.solr.handler.component.QueryComponent.handleRegularResponses(QueryComponent.java:626) > > at > org.apache.solr.handler.component.QueryComponent.handleResponses(QueryComponent.java:605) > > at > org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:486) > > at > org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:214) > > at org.apache.solr.core.SolrCore.execute(SolrCore.java:2627) > > > > > > Can this be fixed in a patch for Solr 8.8? I do not want to have to go > back to Solr 6 and reindex the system, that takes 2 days using 180 EMR > instances. > > > > Pease advise. Thank you. > >
Amazon Sponsor Product(Ads) Search team is looking for talents with expertise in Solr/Lucene at all levels
Hi folks, Are you interested in Amazon’s Advertising (that employs state of art solutions involving Search, Digital Advertising, AWS technologies etc) but wondering which team can best leverage your expertise in Solr/Lucene? Please continue to read: My team is Sponsored Search Delivery (SSD) where we directly use Solr to efficiently render relevant Ads for shoppers. We deliver billions of ad impressions and millions of clicks daily and are breaking fresh ground to create world-class products. Powered with Solr, our systems scan billions of records in milliseconds to return relevant ads. The architectural excellence is evident from the fact that despite we’ve such high-throughput-low-tps Tier-1 system, our team enjoys an exceptionally low operations burden. Our solutions employ latest ‘information retrieval(IR)’ algorithms and we try to leverage latest research in IR to ensure superior shopper experience. It’s a great mix of ‘engineering’ and ‘applied-science’ to render relevant ads within milliseconds. My team is making tremendous investment in Solr research, development and application this year and the future. For example, we plan to apply vector search in Solr (SOLR-12890) to extend power of current keyword-based search to improve ads recall. We are experimenting approximate nearest vector search (LUCENE-9004) to achieve accuracy at a reasonable cost. We also plan to leverage rich features of Solr to return ads based on 'themes' (viz. brand-similar, economical, 4-star above etc.). We also plan to share the experience and lessons to Solr/Lucene community and find ways to contribute back to the community. With a broad mandate to experiment and innovate, we are growing at an unprecedented rate together with Amazon Ads business. There is a seemingly endless range of new opportunities ahead of us. We’re looking for amazing minds at various positions in team for New York Location (Primary goal is to expand the team in NYC, However, Location preferences of Boulder, Seattle, Toronto: can also be considered on case by case basis). Prior experience in Solr/Lucene would be a great value add. SDM: www.amazon.jobs/jobs/1427445?no_int_redir=1 <https://www.amazon.jobs/jobs/1427445?no_int_redir=1> SDE2: www.amazon.jobs/jobs/1451662?no_int_redir=1 <https://www.amazon.jobs/jobs/1451662?no_int_redir=1> SDE3:www.amazon.jobs/jobs/1357591?no_int_redir=1 <https://www.amazon.jobs/jobs/1357591?no_int_redir=1> ASII: www.amazon.jobs/jobs/1427080?no_int_redir=1 <https://www.amazon.jobs/jobs/1427080?no_int_redir=1> ASIII: www.amazon.jobs/jobs/1379670?no_int_redir=1 <https://www.amazon.jobs/jobs/1379670?no_int_redir=1> TPM: www.amazon.jobs/jobs/1353298?no_int_redir=1 <https://www.amazon.jobs/jobs/1353298?no_int_redir=1> If you are interested, please reply this email directly or reach out to the hiring manager Vikas Dhaka (dhavi...@amazon.com <mailto:dhavi...@amazon.com>) Thanks! Best Pengcheng Xiong (Apache Hive PMC member and committer working at Amazon)
Solr backup no longer writes to a UNC path
Hi there, We've upgraded from Solr 7.7.1 to Solr 8.8.1 (running on Windows Operating System) and I've noticed that when running a Solr backup, it will no longer allow me to write to a UNC path. Is this something that has been purposely changed? I've noticed a new system property called SOLR_OPTS="%SOLR_OPTS% -Dsolr.allowPath=S:\" which I've enabled. Is there a way to point this to a remote server, rather than a different drive on the local server? Thanks, Daniel
How can I get dynamicField in Solr 6.1.0.
I am using Solr 6.1.0. We have 2 shards and each has one replica. My schema field is below in one collection <http://aka.ms/weboutlook> My data like FORM1065678510875540 2021-03-03T23:59:59Z false false £ No fdsfsfsfsf Yes 2021-03-03 yes 2021-03-01T06:55:29 12:00 PM no 12:00 PM -1 -1 I want to get only fields /myFields/FORM_CUSTOM_FIELDS/Bid_Opportunity/Start_Date , /myFields/FORM_CUSTOM_FIELDS/Bid_Opportunity/TenderEndDatePassed and /myFields/FORM_CUSTOM_FIELDS/Bid_Opportunity/Tender_Review_Time. How can I get only above fields? What need to pass in FL? Regards, Vishal
Query response time long for dynamicField in Solr 6.1.0
I am using Solr 6.1.0. We have 2 shards and each has one replica. My schema field is below in one collection When I execute below query It is taking more than 180 milliseconds every time. http://10.38.33.24:8983/solr/forms/select?q=project_id:(2117627+2102977+2109667+2102912+2113720+2102976+2102478+2114939+2101443+2123237+2078189+2086596+2079707+2079706+2079705+2079658+2088340+2088338+2113641+2117131+2117672+2120870+2079708+2113718+2096308+2125462+2117837+2115406+2123865+2081232+2080746+2081239+2082706+2098700+2103039+2098699+2082878+2082877+2079994+2113719+2107255+2103251+2100558+2112735+2100036+2100037+2115359+2099330+2112101+2115360+2112070+2125140+2103656+2090184+2090183+2088269+2088270+2115358+2113036+2096855+2098258+2097226+2097225+2113127+2102847+2081187+2082817+2085678+2085677+2100937+2116632+2117133+2121028+2102479+2080006+2117509+2091443+2094716+2109780+2109779+2102735+2102736+2102685+2101923+2103648+2102608+2102480+2103664+2079205+2075380+2079206+2091442+2088614+2088613+2079876+2079875+2082886+2088615+2079429+2079428+2117185+2082859+2082860+2125270+2081301+2117623+2112740+2086757+2086756+2101344+2086597+2086847+2102648+2113362+2109010+2100223+2079877+2082704+2109669+2103649+2100744+2101490+2117526+2117134+2124020+2124021+2123524+2127200+2125039+2103663)=updated+desc,id+desc=0=30==id,form_id,project_id,doctype,dc,form_type_id,status_id,originator_user_id,controller_user_id,form_num,originator_proxy_user_id,originator_user_type_id,controller_user_type_id,msg_id,msg_originator_id,msg_status_id,parent_msg_id,msg_type_id,msg_code,form_code,appType,instance_group_id,bim_model_id,is_draft,InvoiceColourCode,InvoiceCountAgainstOrder,msg_content,msg_content1,msg_content3,user_ref,form_type_name,form_group_name,observationId,locationId,pf_loc_folderId,hasFormAssociation,hasCommentAssociation,hasDocAssociation,hasBimViewAssociation,hasBimListAssociation,originator_org_id,form_closeby_date,form_creation_date,status_change_userId,status_update_date,lastmodified,is_public,title,*Start_Date,*Tender_End_Date,*Tender_End_Time,*Tender_Review_Date,*Tender_Review_Time,*TenderEndDatePassed,*Package_Description,*Budget,*Currency_Sign,*allowExternalVendor,*Enable_form_public_link,*Is_Tender_Public=off=true=http://10.38.33.24:8983/solr/forms,http://10.38.33.227:8983/solr/forms=true=form_id=msg_creation_date+desc=true When I execute below query It is taking less than 80 milliseconds every time. http://10.38.33.24:8983/solr/forms/select?q=project_id:(2117627+2102977+2109667+2102912+2113720+2102976+2102478+2114939+2101443+2123237+2078189+2086596+2079707+2079706+2079705+2079658+2088340+2088338+2113641+2117131+2117672+2120870+2079708+2113718+2096308+2125462+2117837+2115406+2123865+2081232+2080746+2081239+2082706+2098700+2103039+2098699+2082878+2082877+2079994+2113719+2107255+2103251+2100558+2112735+2100036+2100037+2115359+2099330+2112101+2115360+2112070+2125140+2103656+2090184+2090183+2088269+2088270+2115358+2113036+2096855+2098258+2097226+2097225+2113127+2102847+2081187+2082817+2085678+2085677+2100937+2116632+2117133+2121028+2102479+2080006+2117509+2091443+2094716+2109780+2109779+2102735+2102736+2102685+2101923+2103648+2102608+2102480+2103664+2079205+2075380+2079206+2091442+2088614+2088613+2079876+2079875+2082886+2088615+2079429+2079428+2117185+2082859+2082860+2125270+2081301+2117623+2112740+2086757+2086756+2101344+2086597+2086847+2102648+2113362+2109010+2100223+2079877+2082704+2109669+2103649+2100744+2101490+2117526+2117134+2124020+2124021+2123524+2127200+2125039+2103663)=updated+desc,id+desc=0=30==id,form_id,project_id,doctype,dc,form_type_id,status_id,originator_user_id,controller_user_id,form_num,originator_proxy_user_id,originator_user_type_id,controller_user_type_id,msg_id,msg_originator_id,msg_status_id,parent_msg_id,msg_type_id,msg_code,form_code,appType,instance_group_id,bim_model_id,is_draft,InvoiceColourCode,InvoiceCountAgainstOrder,msg_content,msg_content1,msg_content3,user_ref,form_type_name,form_group_name,observationId,locationId,pf_loc_folderId,hasFormAssociation,hasCommentAssociation,hasDocAssociation,hasBimViewAssociation,hasBimListAssociation,originator_org_id,form_closeby_date,form_creation_date,status_change_userId,status_update_date,lastmodified,is_public,title,/myFields/FORM_CUSTOM_FIELDS/Bid_Opportunity/Start_Date,/myFields/FORM_CUSTOM_FIELDS/Bid_Opportunity/Tender_End_Date,/myFields/FORM_CUSTOM_FIELDS/Bid_Opportunity/Tender_End_Time,/myFields/FORM_CUSTOM_FIELDS/Bid_Opportunity/Tender_Review_Date,/myFields/FORM_CUSTOM_FIELDS/Bid_Opportunity/Tender_Review_Time,/myFields/FORM_CUSTOM_FIELDS/Bid_Opportunity/TenderEndDatePassed,/myFields/FORM_CUSTOM_FIELDS/Bid_Opportunity/Package_Description,/myFields/FORM_CUSTOM_FIELDS/Bid_Opportunity/Budget,/myFields/FORM_CUSTOM_FIELDS/ORI_MSG_Custom_Fields/Currency_Sign,/myFields/FORM_CUSTOM_FIELDS/Bid_Opportunity/allowExternalVendor,/myFields/FORM_CUSTOM_FIELDS/ORI_MSG_Custom_Fields/Enable_form_public_link,/myFields/FORM_CUSTOM_FIELDS/Bid_Opportunity
Re: Location of Solr 9 Branch
Solr 9 is an unreleased major version, so it lives in *master*. Once the release process starts for Solr 9, it will live at *branch_9x*, and *master* will host Solr 10. On Tue, Mar 2, 2021 at 3:49 PM Phill Campbell wrote: > I have just begun investigating Solr source code. Where is the branch for > Solr 9? > > >
Location of Solr 9 Branch
I have just begun investigating Solr source code. Where is the branch for Solr 9?
Re: Partial update bug on solr 8.8.0
This looks like a bug that is already fixed but not yet released in 8.9 https://issues.apache.org/jira/plugins/servlet/mobile#issue/SOLR-13034 On Tue, Mar 2, 2021 at 6:27 AM Mohsen Saboorian wrote: > Any idea about this post? > https://stackoverflow.com/q/66335803/141438 > > Regards. >
Partial update bug on solr 8.8.0
Any idea about this post? https://stackoverflow.com/q/66335803/141438 Regards.
Re: Solr wiki page update
Vincent, I added you as editor, please try editing that page again. Jan > 11. feb. 2021 kl. 17:43 skrev Vincent Brehin : > > Hi community members, > I work for Adelean https://www.adelean.com/ , we are offering services > around everything Search related, and especially Solr consulting and > support. We are based in Paris and operate mainly in France. > Is it possible to list our company on the support page (Support - SOLR - > Apache Software Foundation > <https://cwiki.apache.org/confluence/display/SOLR/Support>) ? > Or give me the permission to edit it on confluence (my user: > vincent.brehin) ? > Thanks ! > Best Regards, > > Vincent
Re: Zookeeper 3.4.5 with Solr 8.8.0
On 3/1/2021 9:45 PM, Subhajit Das wrote: That is not possible at this time. Will it be ok, if remote the zookeeper dependencies (jars) from solr and replace it with 3.5.5 jars? Thanks in advance. Maybe. But I cannot say for sure. I know that when we upgraded to ZK 3.5, some fairly significant code changes in Solr were required. I did not see whether more changes were needed when we upgraded again. It would not surprise me to learn that a jar swap won't work. Upgrades are far more likely to work than downgrades. Thanks, Shawn
RE: Zookeeper 3.4.5 with Solr 8.8.0
Hi Shawn, That is not possible at this time. Will it be ok, if remote the zookeeper dependencies (jars) from solr and replace it with 3.5.5 jars? Thanks in advance. From: Shawn Heisey Sent: Monday, March 1, 2021 11:17:24 PM To: solr-user@lucene.apache.org ; u...@zookeeper.apache.org Subject: Re: Zookeeper 3.4.5 with Solr 8.8.0 On 3/1/2021 6:51 AM, Subhajit Das wrote: > I noticed, that Solr 8.8.0 uses Zookeeper 3.6.2 client, while Solr 6.3.0 uses > Zookeeper 3.4.6 client. Is this a client bug or mismatch issue? > If so, how to fix this? The ZK project guarantees that each minor version (X.Y.Z, where Y is the same) will work with the previous minor version or the next minor version. 3.4 and 3.6 are two minor versions apart, and thus compatibility cannot be guaranteed. See the "backward compatibility" matrix here: https://cwiki.apache.org/confluence/display/ZOOKEEPER/ReleaseManagement I think you'll need to upgrade your ZK server ensemble to fix it. Thanks, Shawn
Re: NPE in QueryComponent.mergeIds when using timeAllowed and sorting SOLR 8.7
Anyone? > On Feb 24, 2021, at 7:47 AM, Phill Campbell > wrote: > > Last week I switched to Solr 8.7 from a “special” build of Solr 6.6 > > The system has a timeout set for querying. I am now seeing this bug. > > https://issues.apache.org/jira/browse/SOLR-14758 > <https://issues.apache.org/jira/browse/SOLR-14758> > > Max Query Time goes from 1.6 seconds to 20 seconds and affects the entire > system for about 2 minutes as reported in New Relic. > > null:java.lang.NullPointerException > at > org.apache.solr.handler.component.QueryComponent.mergeIds(QueryComponent.java:935) > at > org.apache.solr.handler.component.QueryComponent.handleRegularResponses(QueryComponent.java:626) > at > org.apache.solr.handler.component.QueryComponent.handleResponses(QueryComponent.java:605) > at > org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:486) > at > org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:214) > at org.apache.solr.core.SolrCore.execute(SolrCore.java:2627) > > > Can this be fixed in a patch for Solr 8.8? I do not want to have to go back > to Solr 6 and reindex the system, that takes 2 days using 180 EMR instances. > > Pease advise. Thank you.
Re: Zookeeper 3.4.5 with Solr 8.8.0
On 3/1/2021 6:51 AM, Subhajit Das wrote: I noticed, that Solr 8.8.0 uses Zookeeper 3.6.2 client, while Solr 6.3.0 uses Zookeeper 3.4.6 client. Is this a client bug or mismatch issue? If so, how to fix this? The ZK project guarantees that each minor version (X.Y.Z, where Y is the same) will work with the previous minor version or the next minor version. 3.4 and 3.6 are two minor versions apart, and thus compatibility cannot be guaranteed. See the "backward compatibility" matrix here: https://cwiki.apache.org/confluence/display/ZOOKEEPER/ReleaseManagement I think you'll need to upgrade your ZK server ensemble to fix it. Thanks, Shawn
Zookeeper 3.4.5 with Solr 8.8.0
Hi There, I am setting up Solr 8.8.0 with Zookeeper 3.4.5. There seems to an issue. EndOfStream issue is coming, saying client must have closed the connection. If same tried with Solr 6.3.0, the issue dosen’t come. This comes with newer Solr only. I noticed, that Solr 8.8.0 uses Zookeeper 3.6.2 client, while Solr 6.3.0 uses Zookeeper 3.4.6 client. Is this a client bug or mismatch issue? If so, how to fix this? Thanks in advance.
RE: Potential Slow searching for unified highlighting on Solr 8.8.0/8.8.1
Hi Ere
Please to be of service!
No I have not filed a JIRA ticket. I am new to interacting with the Solr
Community and only beginning to 'find my legs'. I am not too sure what JIRA
is I am afraid!
Regards
Matthew
Matthew Flowerday | Consultant | ULEAF
Unisys | 01908 774830| matthew.flower...@unisys.com
Address Enigma | Wavendon Business Park | Wavendon | Milton Keynes | MK17
8LX
THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY
MATERIAL and is for use only by the intended recipient. If you received this
in error, please contact the sender and delete the e-mail and its
attachments from all devices.
-Original Message-
From: Ere Maijala
Sent: 01 March 2021 12:53
To: solr-user@lucene.apache.org
Subject: Re: Potential Slow searching for unified highlighting on Solr
8.8.0/8.8.1
EXTERNAL EMAIL - Be cautious of all links and attachments.
Hi,
Whoa, thanks for the heads-up! You may just have saved me from a whole lot
of trouble. Did you file a JIRA ticket already?
Thanks,
Ere
Flowerday, Matthew J kirjoitti 1.3.2021 klo 14.00:
> Hi There
>
> I just came across a situation where a unified highlighting search
> under solr 8.8.0/8.8.1 can take over 20 mins to run and eventually times
out.
> I resolved it by a config change but it can catch you out. Hence
> this email.
>
> With solr 8.8.0 a new unified highlighting parameter
> was implemented which if not set defaults to 0.5.
> This attempts to improve the high lighting so that highlighted text
> does not appear right at the left. This works well but if you have a
> search result with numerous occurrences of the word in question within
> the record performance goes right down!
>
> 2021-02-27 06:45:03.151 INFO (qtp762476028-20) [ x:uleaf]
> o.a.s.c.S.Request [uleaf] webapp=/solr path=/select
> params={hl.snippets=2=test=on=100=id,d
> escription,specification,score=20=*=10&_=161440511913
> 4}
> hits=57008 status=0 QTime=1414320
>
> 2021-02-27 06:45:03.245 INFO (qtp762476028-20) [ x:uleaf]
> o.a.s.s.HttpSolrCall Unable to write response, client closed
> connection or we are shutting down =>
> org.eclipse.jetty.io.EofException
>
>at
> org.eclipse.jetty.io.ChannelEndPoint.flush(ChannelEndPoint.java:279)
>
> org.eclipse.jetty.io.EofException: null
>
>at
> org.eclipse.jetty.io.ChannelEndPoint.flush(ChannelEndPoint.java:279)
> ~[jetty-io-9.4.34.v20201102.jar:9.4.34.v20201102]
>
>at
> org.eclipse.jetty.io.WriteFlusher.flush(WriteFlusher.java:422)
> ~[jetty-io-9.4.34.v20201102.jar:9.4.34.v20201102]
>
>at
> org.eclipse.jetty.io.WriteFlusher.completeWrite(WriteFlusher.java:378)
> ~[jetty-io-9.4.34.v20201102.jar:9.4.34.v20201102]
>
> when I set =0.25 results came back much quicker
>
> 2021-02-27 14:59:57.189 INFO (qtp1291367132-24) [ x:holmes]
> o.a.s.c.S.Request [holmes] webapp=/solr path=/select
> params={hl.weightMatches=false=on=id,description,specification,s
> core=1=0.25=100=2=test
> axAnalyzedChars=100=*=unified=9&_=
> 1614430061690}
> hits=136939 status=0 QTime=87024
>
> And =0.1
>
> 2021-02-27 15:18:45.542 INFO (qtp1291367132-19) [ x:holmes]
> o.a.s.c.S.Request [holmes] webapp=/solr path=/select
> params={hl.weightMatches=false=on=id,description,specification,s
> core=1=0.1=100=2=test
> xAnalyzedChars=100=*=unified=9&_=1
> 614430061690}
> hits=136939 status=0 QTime=69033
>
> And =0.0
>
> 2021-02-27 15:20:38.194 INFO (qtp1291367132-24) [ x:holmes]
> o.a.s.c.S.Request [holmes] webapp=/solr path=/select
> params={hl.weightMatches=false=on=id,description,specification,s
> core=1=0.0=100=2=test
> xAnalyzedChars=100=*=unified=9&_=1
> 614430061690}
> hits=136939 status=0 QTime=2841
>
> I left our setting at 0.0 this presumably how it was in 7.7.1 (fully
> left aligned). I am not too sure as to how many time a word has to
> occur in a record for performance to go right down but if too many
> it can have a BIG impact.
>
> I also noticed that setting =9 did not break out of
> the query until it finished. Perhaps because the query finished
> quickly and what took the time was the highlighting. It might be an
> idea to get to also cover any highlighting so that the
> query does not run until the jetty timeout is hit. The machine 100%
> one core for about
> 20 mins!.
>
> Hope this helps.
>
> Regards
>
> Matthew
>
> *Matthew Flowerday*| Consultant | ULEAF
>
> Unisys | 01908 774830| matthew.flower...@unisys.com
> <mailto:matthew.flower...@unisys.com>
>
> Address Enigma | Wavendon Business Park | Wavendon | Milton Keynes |
> MK17 8LX
>
> unisys_logo <http://www.
Re: Potential Slow searching for unified highlighting on Solr 8.8.0/8.8.1
Hi,
Whoa, thanks for the heads-up! You may just have saved me from a whole
lot of trouble. Did you file a JIRA ticket already?
Thanks,
Ere
Flowerday, Matthew J kirjoitti 1.3.2021 klo 14.00:
Hi There
I just came across a situation where a unified highlighting search under
solr 8.8.0/8.8.1 can take over 20 mins to run and eventually times out.
I resolved it by a config change – but it can catch you out. Hence this
email.
With solr 8.8.0 a new unified highlighting parameter
was implemented which if not set defaults to 0.5. This attempts to
improve the high lighting so that highlighted text does not appear right
at the left. This works well but if you have a search result with
numerous occurrences of the word in question within the record
performance goes right down!
2021-02-27 06:45:03.151 INFO (qtp762476028-20) [ x:uleaf]
o.a.s.c.S.Request [uleaf] webapp=/solr path=/select
params={hl.snippets=2=test=on=100=id,description,specification,score=20=*=10&_=1614405119134}
hits=57008 status=0 QTime=1414320
2021-02-27 06:45:03.245 INFO (qtp762476028-20) [ x:uleaf]
o.a.s.s.HttpSolrCall Unable to write response, client closed connection
or we are shutting down => org.eclipse.jetty.io.EofException
at
org.eclipse.jetty.io.ChannelEndPoint.flush(ChannelEndPoint.java:279)
org.eclipse.jetty.io.EofException: null
at
org.eclipse.jetty.io.ChannelEndPoint.flush(ChannelEndPoint.java:279)
~[jetty-io-9.4.34.v20201102.jar:9.4.34.v20201102]
at
org.eclipse.jetty.io.WriteFlusher.flush(WriteFlusher.java:422)
~[jetty-io-9.4.34.v20201102.jar:9.4.34.v20201102]
at
org.eclipse.jetty.io.WriteFlusher.completeWrite(WriteFlusher.java:378)
~[jetty-io-9.4.34.v20201102.jar:9.4.34.v20201102]
when I set =0.25 results came back much quicker
2021-02-27 14:59:57.189 INFO (qtp1291367132-24) [ x:holmes]
o.a.s.c.S.Request [holmes] webapp=/solr path=/select
params={hl.weightMatches=false=on=id,description,specification,score=1=0.25=100=2=test=100=*=unified=9&_=1614430061690}
hits=136939 status=0 QTime=87024
And =0.1
2021-02-27 15:18:45.542 INFO (qtp1291367132-19) [ x:holmes]
o.a.s.c.S.Request [holmes] webapp=/solr path=/select
params={hl.weightMatches=false=on=id,description,specification,score=1=0.1=100=2=test=100=*=unified=9&_=1614430061690}
hits=136939 status=0 QTime=69033
And =0.0
2021-02-27 15:20:38.194 INFO (qtp1291367132-24) [ x:holmes]
o.a.s.c.S.Request [holmes] webapp=/solr path=/select
params={hl.weightMatches=false=on=id,description,specification,score=1=0.0=100=2=test=100=*=unified=9&_=1614430061690}
hits=136939 status=0 QTime=2841
I left our setting at 0.0 – this presumably how it was in 7.7.1 (fully
left aligned). I am not too sure as to how many time a word has to
occur in a record for performance to go right down – but if too many it
can have a BIG impact.
I also noticed that setting =9 did not break out of the
query until it finished. Perhaps because the query finished quickly and
what took the time was the highlighting. It might be an idea to get
to also cover any highlighting so that the query does not
run until the jetty timeout is hit. The machine 100% one core for about
20 mins!.
Hope this helps.
Regards
Matthew
*Matthew Flowerday*| Consultant | ULEAF
Unisys | 01908 774830| matthew.flower...@unisys.com
<mailto:matthew.flower...@unisys.com>
Address Enigma | Wavendon Business Park | Wavendon | Milton Keynes |
MK17 8LX
unisys_logo <http://www.unisys.com/>
THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY
MATERIAL and is for use only by the intended recipient. If you received
this in error, please contact the sender and delete the e-mail and its
attachments from all devices.
Grey_LI <http://www.linkedin.com/company/unisys>Grey_TW
<http://twitter.com/unisyscorp>Grey_YT
<http://www.youtube.com/theunisyschannel>Grey_FB
<http://www.facebook.com/unisyscorp>Grey_Vimeo
<https://vimeo.com/unisys>Grey_UB <http://blogs.unisys.com/>
--
Ere Maijala
Kansalliskirjasto / The National Library of Finland
Potential Slow searching for unified highlighting on Solr 8.8.0/8.8.1
Hi There
I just came across a situation where a unified highlighting search under
solr 8.8.0/8.8.1 can take over 20 mins to run and eventually times out. I
resolved it by a config change - but it can catch you out. Hence this email.
With solr 8.8.0 a new unified highlighting parameter was
implemented which if not set defaults to 0.5. This attempts to improve the
high lighting so that highlighted text does not appear right at the left.
This works well but if you have a search result with numerous occurrences of
the word in question within the record performance goes right down!
2021-02-27 06:45:03.151 INFO (qtp762476028-20) [ x:uleaf]
o.a.s.c.S.Request [uleaf] webapp=/solr path=/select
params={hl.snippets=2=test=on=100=id,descrip
tion,specification,score=20=*=10&_=1614405119134}
hits=57008 status=0 QTime=1414320
2021-02-27 06:45:03.245 INFO (qtp762476028-20) [ x:uleaf]
o.a.s.s.HttpSolrCall Unable to write response, client closed connection or
we are shutting down => org.eclipse.jetty.io.EofException
at
org.eclipse.jetty.io.ChannelEndPoint.flush(ChannelEndPoint.java:279)
org.eclipse.jetty.io.EofException: null
at
org.eclipse.jetty.io.ChannelEndPoint.flush(ChannelEndPoint.java:279)
~[jetty-io-9.4.34.v20201102.jar:9.4.34.v20201102]
at
org.eclipse.jetty.io.WriteFlusher.flush(WriteFlusher.java:422)
~[jetty-io-9.4.34.v20201102.jar:9.4.34.v20201102]
at
org.eclipse.jetty.io.WriteFlusher.completeWrite(WriteFlusher.java:378)
~[jetty-io-9.4.34.v20201102.jar:9.4.34.v20201102]
when I set =0.25 results came back much quicker
2021-02-27 14:59:57.189 INFO (qtp1291367132-24) [ x:holmes]
o.a.s.c.S.Request [holmes] webapp=/solr path=/select
params={hl.weightMatches=false=on=id,description,specification,score
tart=1=0.25=100=2=test
ars=100=*=unified=9&_=1614430061690}
hits=136939 status=0 QTime=87024
And =0.1
2021-02-27 15:18:45.542 INFO (qtp1291367132-19) [ x:holmes]
o.a.s.c.S.Request [holmes] webapp=/solr path=/select
params={hl.weightMatches=false=on=id,description,specification,score
tart=1=0.1=100=2=test
rs=100=*=unified=9&_=1614430061690}
hits=136939 status=0 QTime=69033
And =0.0
2021-02-27 15:20:38.194 INFO (qtp1291367132-24) [ x:holmes]
o.a.s.c.S.Request [holmes] webapp=/solr path=/select
params={hl.weightMatches=false=on=id,description,specification,score
tart=1=0.0=100=2=test
rs=100=*=unified=9&_=1614430061690}
hits=136939 status=0 QTime=2841
I left our setting at 0.0 - this presumably how it was in 7.7.1 (fully left
aligned). I am not too sure as to how many time a word has to occur in a
record for performance to go right down - but if too many it can have a BIG
impact.
I also noticed that setting =9 did not break out of the
query until it finished. Perhaps because the query finished quickly and what
took the time was the highlighting. It might be an idea to get
to also cover any highlighting so that the query does not run until the
jetty timeout is hit. The machine 100% one core for about 20 mins!.
Hope this helps.
Regards
Matthew
Matthew Flowerday | Consultant | ULEAF
Unisys | 01908 774830| <mailto:matthew.flower...@unisys.com>
matthew.flower...@unisys.com
Address Enigma | Wavendon Business Park | Wavendon | Milton Keynes | MK17
8LX
<http://www.unisys.com/>
THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY
MATERIAL and is for use only by the intended recipient. If you received this
in error, please contact the sender and delete the e-mail and its
attachments from all devices.
<http://www.linkedin.com/company/unisys><http://twitter.com/unisyscorp>
<http://www.youtube.com/theunisyschannel>
<http://www.facebook.com/unisyscorp> <https://vimeo.com/unisys>
<http://blogs.unisys.com/>
smime.p7s
Description: S/MIME cryptographic signature
Re: Congratulations to the new Apache Solr PMC Chair, Jan Høydahl!
Congratulations Jan! Joel Bernstein http://joelsolr.blogspot.com/ On Mon, Feb 22, 2021 at 2:41 AM Danilo Tomasoni wrote: > Congratulations Jan! > > Danilo Tomasoni > > Fondazione The Microsoft Research - University of Trento Centre for > Computational and Systems Biology (COSBI) > Piazza Manifattura 1, 38068 Rovereto (TN), Italy > tomas...@cosbi.eu< > https://webmail.cosbi.eu/owa/redir.aspx?C=VNXi3_8-qSZTBi-FPvMwmwSB3IhCOjY8nuCBIfcNIs_5SgD-zNPWCA..=mailto%3acalabro%40cosbi.eu > > > http://www.cosbi.eu< > https://webmail.cosbi.eu/owa/redir.aspx?C=CkilyF54_imtLHzZqF1gCGvmYXjsnf4bzGynd8OXm__5SgD-zNPWCA..=http%3a%2f%2fwww.cosbi.eu%2f > > > > As for the European General Data Protection Regulation 2016/679 on the > protection of natural persons with regard to the processing of personal > data, we inform you that all the data we possess are object of treatment in > the respect of the normative provided for by the cited GDPR. > It is your right to be informed on which of your data are used and how; > you may ask for their correction, cancellation or you may oppose to their > use by written request sent by recorded delivery to The Microsoft Research > – University of Trento Centre for Computational and Systems Biology Scarl, > Piazza Manifattura 1, 38068 Rovereto (TN), Italy. > P Please don't print this e-mail unless you really need to > ____ > Da: Yonik Seeley > Inviato: domenica 21 febbraio 2021 05:51 > A: solr-user@lucene.apache.org > Cc: Lucene Dev > Oggetto: Re: Congratulations to the new Apache Solr PMC Chair, Jan Høydahl! > > [CAUTION: EXTERNAL SENDER] > [Please check correspondence between Sender Display Name and Sender Email > Address before clicking on any link or opening attachments] > > > Congrats Jan! Go Solr! > -Yonik > > > On Thu, Feb 18, 2021 at 1:56 PM Anshum Gupta > wrote: > > > Hi everyone, > > > > I’d like to inform everyone that the newly formed Apache Solr PMC > nominated > > and elected Jan Høydahl for the position of the Solr PMC Chair and Vice > > President. This decision was approved by the board in its February 2021 > > meeting. > > > > Congratulations Jan! > > > > -- > > Anshum Gupta > > >
Re: [ANNOUNCE] Apache Solr 8.8.1 released
Awesome! Thank you David and Tobias ;-) On Sat, Feb 27, 2021 at 12:21 PM David Smiley wrote: > > The corresponding docker image has been released as well: > https://hub.docker.com/_/solr > (credit to Tobias Kässmann for helping) > > ~ David Smiley > Apache Lucene/Solr Search Developer > http://www.linkedin.com/in/davidwsmiley > > > On Tue, Feb 23, 2021 at 10:39 AM Timothy Potter > wrote: > > > The Lucene PMC is pleased to announce the release of Apache Solr 8.8.1. > > > > > > Solr is the popular, blazing fast, open source NoSQL search platform from > > the Apache Lucene project. Its major features include powerful full-text > > search, hit highlighting, faceted search, dynamic clustering, database > > integration, rich document handling, and geospatial search. Solr is highly > > scalable, providing fault tolerant distributed search and indexing, and > > powers the search and navigation features of many of the world's largest > > internet sites. > > > > > > Solr 8.8.1 is available for immediate download at: > > > > > > <https://lucene.apache.org/solr/downloads.html> > > > > > > ### Solr 8.8.1 Release Highlights: > > > > > > Fix for a SolrJ backwards compatibility issue when upgrading the server to > > 8.8.0 without upgrading SolrJ to 8.8.0. > > > > > > Please refer to the Upgrade Notes in the Solr Ref Guide for information on > > upgrading from previous Solr versions: > > > > > > <https://lucene.apache.org/solr/guide/8_8/solr-upgrade-notes.html> > > > > > > Please read CHANGES.txt for a full list of bugfixes: > > > > > > <https://lucene.apache.org/solr/8_8_1/changes/Changes.html> > > > > > > Solr 8.8.1 also includes bugfixes in the corresponding Apache Lucene > > release: > > > > > > <https://lucene.apache.org/core/8_8_1/changes/Changes.html> > > > > > > > > Note: The Apache Software Foundation uses an extensive mirroring network > > for > > > > distributing releases. It is possible that the mirror you are using may not > > have > > > > replicated the release yet. If that is the case, please try another mirror. > > > > This also applies to Maven access. > > > > > >
Re: [ANNOUNCE] Apache Solr 8.8.1 released
The corresponding docker image has been released as well: https://hub.docker.com/_/solr (credit to Tobias Kässmann for helping) ~ David Smiley Apache Lucene/Solr Search Developer http://www.linkedin.com/in/davidwsmiley On Tue, Feb 23, 2021 at 10:39 AM Timothy Potter wrote: > The Lucene PMC is pleased to announce the release of Apache Solr 8.8.1. > > > Solr is the popular, blazing fast, open source NoSQL search platform from > the Apache Lucene project. Its major features include powerful full-text > search, hit highlighting, faceted search, dynamic clustering, database > integration, rich document handling, and geospatial search. Solr is highly > scalable, providing fault tolerant distributed search and indexing, and > powers the search and navigation features of many of the world's largest > internet sites. > > > Solr 8.8.1 is available for immediate download at: > > > <https://lucene.apache.org/solr/downloads.html> > > > ### Solr 8.8.1 Release Highlights: > > > Fix for a SolrJ backwards compatibility issue when upgrading the server to > 8.8.0 without upgrading SolrJ to 8.8.0. > > > Please refer to the Upgrade Notes in the Solr Ref Guide for information on > upgrading from previous Solr versions: > > > <https://lucene.apache.org/solr/guide/8_8/solr-upgrade-notes.html> > > > Please read CHANGES.txt for a full list of bugfixes: > > > <https://lucene.apache.org/solr/8_8_1/changes/Changes.html> > > > Solr 8.8.1 also includes bugfixes in the corresponding Apache Lucene > release: > > > <https://lucene.apache.org/core/8_8_1/changes/Changes.html> > > > > Note: The Apache Software Foundation uses an extensive mirroring network > for > > distributing releases. It is possible that the mirror you are using may not > have > > replicated the release yet. If that is the case, please try another mirror. > > This also applies to Maven access. > > >
What guarantees does solr have for keeping commit deadlines?
Hi! I'm working on building a NRT solr instance. The schema is designed so documents can be partially updated. Some documents will need to receive or lose filter tags in a multi-valued field. I have to be able to query already existing documents to add tags to them or remove tags from them. Obviously if I (soft-)commit after each document is added or removed the serializable consistency would guarantee that I can see all documents that I might want to change. However this is not desirable in terms of performance. I've come up with a potential solution: If I can track document updates that I make and only call commit before I would query documents that have been changed recently, the performance is not sacrificed and I will get the strict consistency where I need it. For this to work reliably the soft-auto-commit interval and the times specified through commit-within must strictly comply with the config and the requests. I have auto-commit interval of 60 seconds with open-searcher false and auto-soft-commit interval of 15 seconds. Documents will be submitted through the REST API where some of them will also have commit-within 2-3 seconds specified. My questions: - After a document indexing request has returned with success, what level of guarantee do I have that the document will be available after the configured soft-commit-interval? - After a document indexing request with commit-within has returned with success, what level of guarantee do I have that the document will be available after the requested commit timeout? - Alternatively if I could query solr when the last soft-commit was done I could ensure to call a soft-commit myself. Is there an API to see when or how long ago was the last (soft-)commit? I'm primarily interested in answers regards to solr in standalone mode. Thanks, Nandor
Re: Add plugins to Solr docker container
Hey Anil, If you want to execute anything before Solr starts, you can do the following: - mkdir initdb; echo "echo hi" > initdb/hi.sh docker run -v $PWD/initdb:/docker-entrypoint-initdb.d solr Using the above, you can have any script executed before Solr starts. Source: https://github.com/docker-solr/docker-solr/blob/master/8.8/scripts/run-initdb Hope this helps. Please feel free to ask any further questions. I am replying to this mailing list for the first time. If I am not following any convention, please let me know. Thank you, Prabhatika On Fri, Feb 26, 2021 at 11:32 AM anilkumar panditi < anilkumar.pand...@gmail.com> wrote: > Hi, > I am first time user of the apache Solr, and i have brought up the Solr as > docker container , and i am unable to install/enable someplugins > (authentication,authorization etc).. > Could you please help. how to add these plug ins to solr which is running > as docker container. > > Thanks > Anil >
Add plugins to Solr docker container
Hi, I am first time user of the apache Solr, and i have brought up the Solr as docker container , and i am unable to install/enable someplugins (authentication,authorization etc).. Could you please help. how to add these plug ins to solr which is running as docker container. Thanks Anil
RE: Handling Locales in Solr
Hi Markus, thank you a lot for your response, that helps us very much. We will try out the approach of separating the cores by topic only. Kind Regards, Florian -Original Message- From: Markus Jelsma Sent: Mittwoch, 24. Februar 2021 12:27 To: solr-user@lucene.apache.org Subject: Re: Handling Locales in Solr Hello, We put all our customers in the same core/collection because of this, it is not practical to manage hundreds of cores, including their small overhead. Although it can be advantageous when it comes to relevance tuning, no skewed statistics because of other customers. In your case, an unused core is probably slow because it is not in cached memory anymore, and/or it has to load from a slow drive. With regards to the locales, i would probably separate the cores by topic only, and have different languages share the same collection/core. Regards, Markus Op wo 24 feb. 2021 om 12:09 schreef Krönert Florian < florian.kroen...@orbis.de>: > Hi everyone, > > > > First up thanks for this group, I appreciate it very much for > exchanging opinions on how to use Solr. > > > > We built a Solr instance for one of our customers which is used for > searching data on his website. > > We need to search different data (kb articles, products and external > links) in different locales. > > > > For our logic it seemed best to separate solr Cores by topic and > locale, so we have cores like this: > > kbarticle_de-de > > kbarticle_en-us > > … > > products_de-de > > products_en-us > > … > > links_de-de > > links_en-us > > > > First we had only 3 locales, but it grew pretty fast to 16 locales, so > that we’re having 48 solr cores by now already. > > There would have been different approaches for realizing this of > course, so we’re wondering whether we are using Solr not in the optimal way? > > > > We found out that when a search on a locale that was not used for some > time is started, it takes >10 seconds in many cases to execute the search. > > > > We then find logs like this, where it seems as if Solr needs to start > a searcher first, which takes time: > > 2021-02-20 04:33:42.634 INFO (Thread-20674) [ ] > o.a.s.s.SolrIndexSearcher Opening [Searcher@775f8595[kbarticles_en-gb] > main] > > 2021-02-20 04:33:42.643 INFO (searcherExecutor-26-thread-1) [ ] > o.a.s.c.QuerySenderListener QuerySenderListener sending requests to > Searcher@775f8595[kbarticles_en-gb] > > … > > > > Is that an issue? It would be good to know whether our localization > approach causes issues with Solr and whether we should restructure our > core design. > > Any help would be very much appreciated. > > > > Kind Regards, > > > > *Florian Krönert* > Senior Software Developer > > <https://www.orbis.de/de.html> > > *ORBIS AG | *Planckstraße 10 | D-88677 Markdorf > > Phone: +49 7544 50398 21 | Mobile: +49 162 3065972 | E-Mail: > florian.kroen...@orbis.de > www.orbis.de > > > <https://www.orbis.de/de/sap-by-orbis.html> > > Registered Seat: Saarbrücken > Commercial Register Court: Amtsgericht Saarbrücken, HRB 12022 Board of > Management: Thomas Gard (Chairman), Michael Jung, Stefan Mailänder, > Frank Schmelzer Chairman of the Supervisory Board: Ulrich Holzer > > <https://www.facebook.com/ORBIS.AG> > <https://www.linkedin.com/company/orbis-ag/> > <https://www.xing.com/companies/orbisag><https://twitter.com/ORBIS_AG> > > <https://www.youtube.com/channel/UC5Fx5UsUy4FkzB0dfCWXwYw/featured> > > > > > <https://www.orbis.de/de/aktuelles/news/aktuelle-presse/news-detail/ar > ticle/inner-circle-award-orbis-zaehlt-erneut-zu-den-weltbesten-partner > n-fuer-microsoft-business-application.html> > > > > > <https://www.orbis.de/de/microsoft-by-orbis/beratungskompetenz/power-p > latform.html?wmc=Banner-Microsoft-Power-Platform> > > > <https://www.orbis.de/de/microsoft-by-orbis.html?wmc=Banner-Microsoft- > by-ORBIS> >
Re: Solr Cloud Autoscaling Basics
Any pointers here would be appreciated :) -- Sent from: https://lucene.472066.n3.nabble.com/Solr-User-f472068.html
Solr 7.6.0 - OOM Caused Down Replica. Cannot recover. Please advice
Hi everyone, We had an OOM event earlier this morning. This has caused one of our shards to lose all it's replicas and it's leader is still in a down state. We have restarted the Java process (solr) and it's still in a down state. Logs below: ``` Feb 25, 2021 @ 11:46:43.000 2021-02-25 00:46:43.268 WARN (updateExecutor-3-thread-1-processing-n:10.0.10.43:8983_solr x:search-collection-2018-10-30_shard2_5_replica_n1480 c:search-collection-2018-10-30 s:shard2_5 r:core_node1481) [c:search-collection-2018-10-30 s:shard2_5 r:core_node1481 x:search-collection-2018-10-30_shard2_5_replica_n1480] o.a.s.c.RecoveryStrategy Stopping recovery for core=[search-collection-2018-10-30_shard2_5_replica_n1480] coreNodeName=[core_node1481] ∎ Feb 25, 2021 @ 11:46:40.000 2021-02-25 00:46:40.759 WARN (zkCallback-7-thread-2) [c:search-collection-2018-10-30 s:shard2_5 r:core_node1481 x:search-collection-2018-10-30_shard2_5_replica_n1480] o.a.s.c.RecoveryStrategy Stopping recovery for core=[search-collection-2018-10-30_shard2_5_replica_n1480] coreNodeName=[core_node1481] ∎ Feb 25, 2021 @ 11:46:35.000 2021-02-25 00:46:35.761 WARN (zkCallback-7-thread-2) [c:search-collection-2018-10-30 s:shard2_5 r:core_node1481 x:search-collection-2018-10-30_shard2_5_replica_n1480] o.a.s.c.RecoveryStrategy Stopping recovery for core=[search-collection-2018-10-30_shard2_5_replica_n1480] coreNodeName=[core_node1481] ∎ Feb 25, 2021 @ 11:46:33.000 2021-02-25 00:46:33.270 WARN (updateExecutor-3-thread-2-processing-n:10.0.10.43:8983_solr x:search-collection-2018-10-30_shard2_5_replica_n1480 c:search-collection-2018-10-30 s:shard2_5 r:core_node1481) [c:search-collection-2018-10-30 s:shard2_5 r:core_node1481 x:search-collection-2018-10-30_shard2_5_replica_n1480] o.a.s.c.RecoveryStrategy Stopping recovery for core=[search-collection-2018-10-30_shard2_5_replica_n1480] coreNodeName=[core_node1481] ∎ Feb 25, 2021 @ 11:46:30.000 2021-02-25 00:46:30.759 WARN (zkCallback-7-thread-2) [c:search-collection-2018-10-30 s:shard2_5 r:core_node1481 x:search-collection-2018-10-30_shard2_5_replica_n1480] o.a.s.c.RecoveryStrategy Stopping recovery for core=[search-collection-2018-10-30_shard2_5_replica_n1480] coreNodeName=[core_node1481] ∎ Feb 25, 2021 @ 11:46:25.000 2021-02-25 00:46:25.761 WARN (zkCallback-7-thread-2) [c:search-collection-2018-10-30 s:shard2_5 r:core_node1481 x:search-collection-2018-10-30_shard2_5_replica_n1480] o.a.s.c.RecoveryStrategy Stopping recovery for core=[search-collection-2018-10-30_shard2_5_replica_n1480] coreNodeName=[core_node1481] ∎ Feb 25, 2021 @ 11:46:23.000 2021-02-25 00:46:23.279 WARN (updateExecutor-3-thread-1-processing-n:10.0.10.43:8983_solr x:search-collection-2018-10-30_shard2_5_replica_n1480 c:search-collection-2018-10-30 s:shard2_5 r:core_node1481) [c:search-collection-2018-10-30 s:shard2_5 r:core_node1481 x:search-collection-2018-10-30_shard2_5_replica_n1480] o.a.s.c.RecoveryStrategy Stopping recovery for core=[search-collection-2018-10-30_shard2_5_replica_n1480] coreNodeName=[core_node1481] ∎ ``` Questions: 1. Is there anything we can do to force this core to go live? 2. If the core is unrecoverable, is there a way to clear the core up such that we can reindex only that shard? Any other advice would be great too :) Ash -- ** ** <https://www.canva.com/>Empowering the world to design Share accurate information on COVID-19 and spread messages of support to your community. Here are some resources <https://about.canva.com/coronavirus-awareness-collection/?utm_medium=pr_source=news_campaign=covid19_templates> that can help. <https://twitter.com/canva> <https://facebook.com/canva> <https://au.linkedin.com/company/canva> <https://twitter.com/canva> <https://facebook.com/canva> <https://au.linkedin.com/company/canva> <https://instagram.com/canva>
NPE in QueryComponent.mergeIds when using timeAllowed and sorting SOLR 8.7
Last week I switched to Solr 8.7 from a “special” build of Solr 6.6 The system has a timeout set for querying. I am now seeing this bug. https://issues.apache.org/jira/browse/SOLR-14758 <https://issues.apache.org/jira/browse/SOLR-14758> Max Query Time goes from 1.6 seconds to 20 seconds and affects the entire system for about 2 minutes as reported in New Relic. null:java.lang.NullPointerException at org.apache.solr.handler.component.QueryComponent.mergeIds(QueryComponent.java:935) at org.apache.solr.handler.component.QueryComponent.handleRegularResponses(QueryComponent.java:626) at org.apache.solr.handler.component.QueryComponent.handleResponses(QueryComponent.java:605) at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:486) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:214) at org.apache.solr.core.SolrCore.execute(SolrCore.java:2627) Can this be fixed in a patch for Solr 8.8? I do not want to have to go back to Solr 6 and reindex the system, that takes 2 days using 180 EMR instances. Pease advise. Thank you.
NPE in QueryComponent.mergeIds when using timeAllowed and sorting SOLR 8.7
Last week I switched to Solr 8.7 from a “special” build of Solr 6.6 The system has a timeout set for querying. I am now seeing this bug. https://issues.apache.org/jira/browse/SOLR-14758 <https://issues.apache.org/jira/browse/SOLR-14758> Max Query Time goes from 1.6 seconds to 20 seconds and affects the entire system for about 2 minutes as reported in New Relic. null:java.lang.NullPointerException at org.apache.solr.handler.component.QueryComponent.mergeIds(QueryComponent.java:935) at org.apache.solr.handler.component.QueryComponent.handleRegularResponses(QueryComponent.java:626) at org.apache.solr.handler.component.QueryComponent.handleResponses(QueryComponent.java:605) at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:486) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:214) at org.apache.solr.core.SolrCore.execute(SolrCore.java:2627) Can this be fixed in a patch for Solr 8.8? I do not want to have to go back to Solr 6 and reindex the system, that takes 2 days using 180 EMR instances. Pease advise. Thank you.
Re: Handling Locales in Solr
Hello, We put all our customers in the same core/collection because of this, it is not practical to manage hundreds of cores, including their small overhead. Although it can be advantageous when it comes to relevance tuning, no skewed statistics because of other customers. In your case, an unused core is probably slow because it is not in cached memory anymore, and/or it has to load from a slow drive. With regards to the locales, i would probably separate the cores by topic only, and have different languages share the same collection/core. Regards, Markus Op wo 24 feb. 2021 om 12:09 schreef Krönert Florian < florian.kroen...@orbis.de>: > Hi everyone, > > > > First up thanks for this group, I appreciate it very much for exchanging > opinions on how to use Solr. > > > > We built a Solr instance for one of our customers which is used for > searching data on his website. > > We need to search different data (kb articles, products and external > links) in different locales. > > > > For our logic it seemed best to separate solr Cores by topic and locale, > so we have cores like this: > > kbarticle_de-de > > kbarticle_en-us > > … > > products_de-de > > products_en-us > > … > > links_de-de > > links_en-us > > > > First we had only 3 locales, but it grew pretty fast to 16 locales, so > that we’re having 48 solr cores by now already. > > There would have been different approaches for realizing this of course, > so we’re wondering whether we are using Solr not in the optimal way? > > > > We found out that when a search on a locale that was not used for some > time is started, it takes >10 seconds in many cases to execute the search. > > > > We then find logs like this, where it seems as if Solr needs to start a > searcher first, which takes time: > > 2021-02-20 04:33:42.634 INFO (Thread-20674) [ ] > o.a.s.s.SolrIndexSearcher Opening [Searcher@775f8595[kbarticles_en-gb] > main] > > 2021-02-20 04:33:42.643 INFO (searcherExecutor-26-thread-1) [ ] > o.a.s.c.QuerySenderListener QuerySenderListener sending requests to > Searcher@775f8595[kbarticles_en-gb] > > … > > > > Is that an issue? It would be good to know whether our localization > approach causes issues with Solr and whether we should restructure our core > design. > > Any help would be very much appreciated. > > > > Kind Regards, > > > > *Florian Krönert* > Senior Software Developer > > <https://www.orbis.de/de.html> > > *ORBIS AG | *Planckstraße 10 | D-88677 Markdorf > > Phone: +49 7544 50398 21 | Mobile: +49 162 3065972 | E-Mail: > florian.kroen...@orbis.de > www.orbis.de > > > <https://www.orbis.de/de/sap-by-orbis.html> > > Registered Seat: Saarbrücken > Commercial Register Court: Amtsgericht Saarbrücken, HRB 12022 > Board of Management: Thomas Gard (Chairman), Michael Jung, Stefan > Mailänder, Frank Schmelzer > Chairman of the Supervisory Board: Ulrich Holzer > > <https://www.facebook.com/ORBIS.AG> > <https://www.linkedin.com/company/orbis-ag/> > <https://www.xing.com/companies/orbisag><https://twitter.com/ORBIS_AG> > <https://www.youtube.com/channel/UC5Fx5UsUy4FkzB0dfCWXwYw/featured> > > > > > <https://www.orbis.de/de/aktuelles/news/aktuelle-presse/news-detail/article/inner-circle-award-orbis-zaehlt-erneut-zu-den-weltbesten-partnern-fuer-microsoft-business-application.html> > > > > > <https://www.orbis.de/de/microsoft-by-orbis/beratungskompetenz/power-platform.html?wmc=Banner-Microsoft-Power-Platform> > > > <https://www.orbis.de/de/microsoft-by-orbis.html?wmc=Banner-Microsoft-by-ORBIS> >
Handling Locales in Solr
Hi everyone, First up thanks for this group, I appreciate it very much for exchanging opinions on how to use Solr. We built a Solr instance for one of our customers which is used for searching data on his website. We need to search different data (kb articles, products and external links) in different locales. For our logic it seemed best to separate solr Cores by topic and locale, so we have cores like this: kbarticle_de-de kbarticle_en-us ... products_de-de products_en-us ... links_de-de links_en-us First we had only 3 locales, but it grew pretty fast to 16 locales, so that we're having 48 solr cores by now already. There would have been different approaches for realizing this of course, so we're wondering whether we are using Solr not in the optimal way? We found out that when a search on a locale that was not used for some time is started, it takes >10 seconds in many cases to execute the search. We then find logs like this, where it seems as if Solr needs to start a searcher first, which takes time: 2021-02-20 04:33:42.634 INFO (Thread-20674) [ ] o.a.s.s.SolrIndexSearcher Opening [Searcher@775f8595[kbarticles_en-gb] main] 2021-02-20 04:33:42.643 INFO (searcherExecutor-26-thread-1) [ ] o.a.s.c.QuerySenderListener QuerySenderListener sending requests to Searcher@775f8595[kbarticles_en-gb] ... Is that an issue? It would be good to know whether our localization approach causes issues with Solr and whether we should restructure our core design. Any help would be very much appreciated. Kind Regards, Florian Krönert Senior Software Developer [cid:image001.gif@01D70AA4.14778350]<https://www.orbis.de/de.html> ORBIS AG | Planckstraße 10 | D-88677 Markdorf Phone: +49 7544 50398 21 | Mobile: +49 162 3065972 | E-Mail: florian.kroen...@orbis.de<mailto:florian.kroen...@orbis.de> www.orbis.de<http://www.orbis.de/> [cid:image002.png@01D70AA4.14778350]<https://www.orbis.de/de/sap-by-orbis.html> [cid:image003.jpg@01D70AA4.14778350] Registered Seat: Saarbrücken Commercial Register Court: Amtsgericht Saarbrücken, HRB 12022 Board of Management: Thomas Gard (Chairman), Michael Jung, Stefan Mailänder, Frank Schmelzer Chairman of the Supervisory Board: Ulrich Holzer [cid:image004.png@01D70AA4.14778350]<https://www.facebook.com/ORBIS.AG> [cid:image005.png@01D70AA4.14778350] <https://www.linkedin.com/company/orbis-ag/> [cid:image006.png@01D70AA4.14778350] <https://www.xing.com/companies/orbisag> [cid:image007.png@01D70AA4.14778350] <https://twitter.com/ORBIS_AG> [cid:image008.png@01D70AA4.14778350] <https://www.youtube.com/channel/UC5Fx5UsUy4FkzB0dfCWXwYw/featured> [cid:image009.png@01D70AA4.14778350] [cid:image010.png@01D70AA4.14778350]<https://www.orbis.de/de/aktuelles/news/aktuelle-presse/news-detail/article/inner-circle-award-orbis-zaehlt-erneut-zu-den-weltbesten-partnern-fuer-microsoft-business-application.html> [cid:MicrosoftPowerPlatform_7054080f-b0bf-4e97-b9ab-55dae1373165.png]<https://www.orbis.de/de/microsoft-by-orbis/beratungskompetenz/power-platform.html?wmc=Banner-Microsoft-Power-Platform> [cid:Microsoftallgemein_e6028b0b-75cc-43da-9046-9fec28faad6d.png]<https://www.orbis.de/de/microsoft-by-orbis.html?wmc=Banner-Microsoft-by-ORBIS>
[IMPORTANT] Apache Solr TLP Update - Solr User email list migration
Hi Solr Users, As part of setting up Apache Solr as a Top Level Project, we’re migrating the existing solr-user@lucene.apache.org mailing list to us...@solr.apache.org. All existing subscriptions, and conversations will be migrated to the new list but if you have any mail client filters, please fix them accordingly. The migration has been requested and ASF Infra is working with the Lucene/Solr PMC for this[1]. We will update the list once the migration is completed. - Anshum Gupta On behalf of the Apache Solr PMC [1] - https://issues.apache.org/jira/browse/INFRA-21443
Solr Cloud Autoscaling Basics
Hi I have a master slave architecture setup currently. I'm evaluating SolrCloud. I've read through most of the documentation, but what I can't seem to find is the preferred way to autoscale the cluster. In the master slave architecture, we have a autoscaling policy (CPU based) configured on Spotinst which adds a node and it replicates the index from the master and starts serving traffic. This helps us bring the cost of the cluster down by downscaling the cluster at low traffic times. What would be right way to do this in Cloud mode? Will replication factor need to be changed again and again for this? Would I have to call the ADDREPLICA api after adding a node as suggested here? https://lucene.472066.n3.nabble.com/Autoscaling-in-8-2-td4451650.html#a4451659. It seems rather cumbersome compared to Master-Slave architecture How is SolrCloud usually run in production environments? With autoscaling turned off completely (apart from failure scenarios where a node crashes for ex)? -- Sent from: https://lucene.472066.n3.nabble.com/Solr-User-f472068.html
[ANNOUNCE] Apache Solr 8.8.1 released
The Lucene PMC is pleased to announce the release of Apache Solr 8.8.1. Solr is the popular, blazing fast, open source NoSQL search platform from the Apache Lucene project. Its major features include powerful full-text search, hit highlighting, faceted search, dynamic clustering, database integration, rich document handling, and geospatial search. Solr is highly scalable, providing fault tolerant distributed search and indexing, and powers the search and navigation features of many of the world's largest internet sites. Solr 8.8.1 is available for immediate download at: <https://lucene.apache.org/solr/downloads.html> ### Solr 8.8.1 Release Highlights: Fix for a SolrJ backwards compatibility issue when upgrading the server to 8.8.0 without upgrading SolrJ to 8.8.0. Please refer to the Upgrade Notes in the Solr Ref Guide for information on upgrading from previous Solr versions: <https://lucene.apache.org/solr/guide/8_8/solr-upgrade-notes.html> Please read CHANGES.txt for a full list of bugfixes: <https://lucene.apache.org/solr/8_8_1/changes/Changes.html> Solr 8.8.1 also includes bugfixes in the corresponding Apache Lucene release: <https://lucene.apache.org/core/8_8_1/changes/Changes.html> Note: The Apache Software Foundation uses an extensive mirroring network for distributing releases. It is possible that the mirror you are using may not have replicated the release yet. If that is the case, please try another mirror. This also applies to Maven access.
Query regarding integrating solr query functions into blockfacetjoin Query
Hi Team,
I was implementing block join faceting query in my project and was stuck in
integrating the existing functional queries in the block join faceting
query.
*The current query using 'select' handler is as follows* :-
https://localhost:8983/solr/master_Product_default/*select*?*yq*
=_query_:%22\{\!multiMaxScore\+tie%3D0.0\}\(\(bomCode_bc_string\:samsung\)\+OR\+\(description_text_en\:samsung\)\+OR\+\(belleaprice_cad_bc846_string\:samsung\^20.0\)\+OR\+\(name_text_en\:samsung\^50.0\)\+OR\+\(category_string_mv\:samsung\^20.0\)\)\+OR\+\(\(belleaprice_cad_bc846_string\:samsung\~\^10.0\)\)\+OR\+\(\(bomCode_bc_string\:\%22samsung\%22\^50.0\)\+OR\+\(code_string\:\%22samsung\%22\~1.0\^90.0\)\+OR\+\(vendorId_string\:\%22samsung\%22\^95.0\)\+OR\+\(description_text_en\:\%22samsung\%22\^50.0\)\+OR\+\(belleaprice_cad_bc846_string\:\%22samsung\%22\^40.0\)\+OR\+\(name_text_en\:\%22samsung\%22\^100.0\)\+OR\+\(category_string_mv\:\%22samsung\%22\^40.0\)\+OR\+\(upcCode_bc846_string\:\%22samsung\%22\^99.0\)\)%22&
*yab*
=sum(product(and(not(exists(omniOnlineStockStatus_boolean)),exists(inStoreStockStatus_bc846_bellea_boolean)),70.0),product(and(exists(omniOnlineStockStatus_boolean),exists(inStoreStockStatus_bc846_bellea_boolean)),80.0),product(and(exists(omniOnlineStockStatus_boolean),not(exists(inStoreStockStatus_bc846_bellea_boolean))),40.0),product(exists(omniInStoreStockStatus_bc_boolean),20.0))&*q={!boost}(+{!lucene
v=$yq} {!func v=$yab})*
=(omniAssortment_bc846_boolean:true+OR+omniAssortment_a002_boolean:true)=(srpPriceValue_bc846_string:[0.0+TO+*])=(omniVisible_20_bellea_bc_boolean:true)=(catalogId:%22belleaProductCatalog%22+AND+catalogVersion:%22Online%22)=score+desc,omniInStoreStockStatus_bc_boolean+asc,creationtime_sortable_date+desc,inStoreStockStatus_bc846_bellea_boolean+asc,omniOnlineStockStatus_boolean+asc=0=2=characteristics_string=inStoreStockStatus_bc846_bellea_boolean=memorySize_string_mv=color_en_string=belleaprice_cad_bc846_string=supplier_string=model_string_mv=omniOnlineStockStatus_boolean=category_string_mv=omniInStoreStockStatus_bc_boolean=stockAvailability_string=true=count=1=11=score,*=[child+parentFilter%3D%22itemtype_string:Product%22+childFilter%3D%22brands_stringignorecase_mv:BC+AND+regions_stringignorecase_mv:ON+AND+activationTypes_stringignorecase_mv:N+AND+channels_stringignorecase_mv:NR+AND+banners_stringignorecase_mv:\%22Walmart\%22+AND+(accountTypes_stringignorecase_mv:IR+OR+accountTypes_stringignorecase_mv:empty)%22+limit%3D1000]=true=samsung=en=true
In the above query, the *'yq'* and* 'yab'* functions are integrated in the
main query using expression below :-
*q={!boost}(+{!lucene v=$yq} {!func v=$yab}) *
I want to integrate the *'yq' and 'yab'* function queries in the *future
block join faceting query* mentioned below :-
https://localhost:8983/solr/master_Product_default/*blockJoinFacetRH*?
*q={!parent%20which=%22itemtype_string:Product%22}itemtype_string:TierPrice=json=true=true=contract_string=500*
=(omniAssortment_bc846_boolean:true+OR+omniAssortment_a002_boolean:true)=(srpPriceValue_bc846_string:[0.0+TO+*])=(omniVisible_20_bellea_bc_boolean:true)=(catalogId:%22belleaProductCatalog%22+AND+catalogVersion:%22Online%22)=score+desc,omniInStoreStockStatus_bc_boolean+asc,creationtime_sortable_date+desc,inStoreStockStatus_bc846_bellea_boolean+asc,omniOnlineStockStatus_boolean+asc=0=2000=characteristics_string=inStoreStockStatus_bc846_bellea_boolean=memorySize_string_mv=color_en_string=belleaprice_cad_bc846_string=supplier_string=model_string_mv=omniOnlineStockStatus_boolean=category_string_mv=omniInStoreStockStatus_bc_boolean=stockAvailability_string=true=count=1=11=score,*=[child+parentFilter%3D%22itemtype_string:Product%22+childFilter%3D%22brands_stringignorecase_mv:BC+AND+regions_stringignorecase_mv:ON+AND+activationTypes_stringignorecase_mv:N+AND+channels_stringignorecase_mv:NR+AND+banners_stringignorecase_mv:\%22Walmart\%22+AND+(accountTypes_stringignorecase_mv:IR+OR+accountTypes_stringignorecase_mv:empty)%22+limit%3D1000]=true=samsung=en=true
Can someone please suggest how can I add the expression '* {!boost}(+{!lucene
v=$yq} {!func v=$yab})*' functions in the block join facting query
-"*q={!parent%20which=%22itemtype_string:Product%22}
itemtype_string:TierPrice=json=true=true=contract_string=500*"
?
I shall be highly grateful if someone can suggest to me some insight.
Thanks & Regards,
Ravi Kumar
SAP Hybris Consultant
R: Congratulations to the new Apache Solr PMC Chair, Jan Høydahl!
Congratulations Jan! Danilo Tomasoni Fondazione The Microsoft Research - University of Trento Centre for Computational and Systems Biology (COSBI) Piazza Manifattura 1, 38068 Rovereto (TN), Italy tomas...@cosbi.eu<https://webmail.cosbi.eu/owa/redir.aspx?C=VNXi3_8-qSZTBi-FPvMwmwSB3IhCOjY8nuCBIfcNIs_5SgD-zNPWCA..=mailto%3acalabro%40cosbi.eu> http://www.cosbi.eu<https://webmail.cosbi.eu/owa/redir.aspx?C=CkilyF54_imtLHzZqF1gCGvmYXjsnf4bzGynd8OXm__5SgD-zNPWCA..=http%3a%2f%2fwww.cosbi.eu%2f> As for the European General Data Protection Regulation 2016/679 on the protection of natural persons with regard to the processing of personal data, we inform you that all the data we possess are object of treatment in the respect of the normative provided for by the cited GDPR. It is your right to be informed on which of your data are used and how; you may ask for their correction, cancellation or you may oppose to their use by written request sent by recorded delivery to The Microsoft Research – University of Trento Centre for Computational and Systems Biology Scarl, Piazza Manifattura 1, 38068 Rovereto (TN), Italy. P Please don't print this e-mail unless you really need to Da: Yonik Seeley Inviato: domenica 21 febbraio 2021 05:51 A: solr-user@lucene.apache.org Cc: Lucene Dev Oggetto: Re: Congratulations to the new Apache Solr PMC Chair, Jan Høydahl! [CAUTION: EXTERNAL SENDER] [Please check correspondence between Sender Display Name and Sender Email Address before clicking on any link or opening attachments] Congrats Jan! Go Solr! -Yonik On Thu, Feb 18, 2021 at 1:56 PM Anshum Gupta wrote: > Hi everyone, > > I’d like to inform everyone that the newly formed Apache Solr PMC nominated > and elected Jan Høydahl for the position of the Solr PMC Chair and Vice > President. This decision was approved by the board in its February 2021 > meeting. > > Congratulations Jan! > > -- > Anshum Gupta >
Re: HTML sample.html not indexing in Solr 8.8
On 2/21/2021 3:07 PM, cratervoid wrote: Thanks Shawn, I copied the solrconfig.xml file from the gettingstarted example on 7.7.3 installation to the 8.8.0 installation, restarted the server and it now works. Comparing the two files it looks like as you said this section was left out of the _default/solrconfig.xml file in version 8.8.0: true ignored_ _text_ So those trying out the tutorial will need to add this section to get it to work for sample.html. This line from that config also is involved: regex=".*\.jar" /> That loads the contrib jars needed for the ExtractingRequestHandler to work right. There are a LOT of jars there. Tika is a very heavyweight piece of software. Thanks, Shawn
Re: HTML sample.html not indexing in Solr 8.8
Thanks Shawn, I copied the solrconfig.xml file from the gettingstarted example on 7.7.3 installation to the 8.8.0 installation, restarted the server and it now works. Comparing the two files it looks like as you said this section was left out of the _default/solrconfig.xml file in version 8.8.0: true ignored_ _text_ So those trying out the tutorial will need to add this section to get it to work for sample.html. On Sat, Feb 20, 2021 at 4:21 PM Shawn Heisey wrote: > On 2/20/2021 3:58 PM, cratervoid wrote: > > SimplePostTool: WARNING: Solr returned an error #404 (Not Found) for url: > > > http://localhost:8983/solr/gettingstarted/update/extract?resource.name=C%3A%5Csolr-8.8.0%5Cexample%5Cexampledocs%5Csample.html=C%3A%5Csolr-8.8.0%5Cexample%5Cexampledocs%5Csample.html > > The problem here is that the solrconfig.xml in use by the index named > "gettingstarted" does not define a handler at /update/extract. > > Typically a handler defined at that URL path will utilize the extracting > request handler class. This handler uses Tika (another Apache project) > to extract usable data from rich text formats like PDF, HTML, etc. > > >startup="lazy" >class="solr.extraction.ExtractingRequestHandler" > > >true >ignored_ > _text_ > > > > Note that using this handler will require adding some contrib jars to Solr. > > Tika can become very unstable because it deals with undocumented file > formats, so we do not recommend using that handler in production. If > the functionality is important, Tika should be included in a program > that's separate from Solr, so that if it crashes, it does not take Solr > down with it. > > Thanks, > Shawn >
Re: HTML sample.html not indexing in Solr 8.8
Thanks Alex. I copied the solrconfig.xml over from 7.7.3 to the 8.8.0 conf folder and restarted the server. Now indexing works without erroring on sample.html. There is 1K difference between the 2 files so I'll diff them to see what was left out of the 8.8 version. On Sat, Feb 20, 2021 at 4:27 PM Alexandre Rafalovitch wrote: > Most likely issue is that your core configuration (solrconfig.xml) > does not have the request handler for that. The same config may have > had that in 7.x, but changed since. > > More details: > https://lucene.apache.org/solr/guide/8_8/uploading-data-with-solr-cell-using-apache-tika.html > > Regards, >Alex. > > On Sat, 20 Feb 2021 at 17:59, cratervoid wrote: > > > > I am trying out indexing the exampledocs in the examples folder with the > > SimplePostTool on windows 10 using solr 8.8. All the documents index > > except sample.html. For that file I get the errors below. I then > > downloaded solr 7.7.3 and indexed the exampledocs folder with no errors, > > including sample.html. > > ``` > > PS C:\solr-8.8.0> java -jar -Dc=gettingstarted -Dauto > > example\exampledocs\post.jar example\exampledocs\sample.html > > SimplePostTool version 5.0.0 > > Posting files to [base] url > > http://localhost:8983/solr/gettingstarted/update... > > Entering auto mode. File endings considered are > > > xml,json,jsonl,csv,pdf,doc,docx,ppt,pptx,xls,xlsx,odt,odp,ods,ott,otp,ots,rtf,htm,html,txt,log > > POSTing file sample.html (text/html) to [base]/extract > > SimplePostTool: WARNING: Solr returned an error #404 (Not Found) for url: > > > http://localhost:8983/solr/gettingstarted/update/extract?resource.name=C%3A%5Csolr-8.8.0%5Cexample%5Cexampledocs%5Csample.html=C%3A%5Csolr-8.8.0%5Cexample%5Cexampledocs%5Csample.html > > SimplePostTool: WARNING: Response: > > > > > > Error 404 Not Found > > > > HTTP ERROR 404 Not Found > > > > URI:/solr/gettingstarted/update/extract > > STATUS:404 > > MESSAGE:Not Found > > SERVLET:default > > > > > > > > > > SimplePostTool: WARNING: IOException while reading response: > > java.io.FileNotFoundException: > > > http://localhost:8983/solr/gettingstarted/update/extract?resource.name=C%3A%5Csolr-8.8.0%5Cexample%5Cexampledocs%5Csample.html=C%3A%5Csolr-8.8.0%5Cexample%5Cexampledocs%5Csample.html > > 1 files indexed. > > COMMITting Solr index changes to > > http://localhost:8983/solr/gettingstarted/update... > > Time spent: 0:00:00.086 > > ``` > > > > However the json and all other file types index with no problem. For > > example: > > ``` > > PS C:\solr-8.8.0> java -jar -Dc=gettingstarted -Dauto > > example\exampledocs\post.jar example\exampledocs\books.json > > SimplePostTool version 5.0.0 > > Posting files to [base] url > > http://localhost:8983/solr/gettingstarted/update... > > Entering auto mode. File endings considered are > > > xml,json,jsonl,csv,pdf,doc,docx,ppt,pptx,xls,xlsx,odt,odp,ods,ott,otp,ots,rtf,htm,html,txt,log > > POSTing file books.json (application/json) to [base]/json/docs > > 1 files indexed. > > COMMITting Solr index changes to > > http://localhost:8983/solr/gettingstarted/update... > > ``` > > Just following this tutorial:[ > > > https://lucene.apache.org/solr/guide/8_8/post-tool.html#post-tool-windows-support][1 > > ] > > > > [1]: > > > https://lucene.apache.org/solr/guide/8_8/post-tool.html#post-tool-windows-support >
Re: Congratulations to the new Apache Solr PMC Chair, Jan Høydahl!
Congrats Jan! Go Solr! -Yonik On Thu, Feb 18, 2021 at 1:56 PM Anshum Gupta wrote: > Hi everyone, > > I’d like to inform everyone that the newly formed Apache Solr PMC nominated > and elected Jan Høydahl for the position of the Solr PMC Chair and Vice > President. This decision was approved by the board in its February 2021 > meeting. > > Congratulations Jan! > > -- > Anshum Gupta >
Re: Congratulations to the new Apache Solr PMC Chair, Jan Høydahl!
Congratulations, Jan and thank you! Thanks, Anshum for all your work as chair. On Fri, Feb 19, 2021 at 12:26 AM Anshum Gupta wrote: > Hi everyone, > > I’d like to inform everyone that the newly formed Apache Solr PMC nominated > and elected Jan Høydahl for the position of the Solr PMC Chair and Vice > President. This decision was approved by the board in its February 2021 > meeting. > > Congratulations Jan! > > -- > Anshum Gupta > -- Regards, Shalin Shekhar Mangar.
Re: HTML sample.html not indexing in Solr 8.8
Most likely issue is that your core configuration (solrconfig.xml) does not have the request handler for that. The same config may have had that in 7.x, but changed since. More details: https://lucene.apache.org/solr/guide/8_8/uploading-data-with-solr-cell-using-apache-tika.html Regards, Alex. On Sat, 20 Feb 2021 at 17:59, cratervoid wrote: > > I am trying out indexing the exampledocs in the examples folder with the > SimplePostTool on windows 10 using solr 8.8. All the documents index > except sample.html. For that file I get the errors below. I then > downloaded solr 7.7.3 and indexed the exampledocs folder with no errors, > including sample.html. > ``` > PS C:\solr-8.8.0> java -jar -Dc=gettingstarted -Dauto > example\exampledocs\post.jar example\exampledocs\sample.html > SimplePostTool version 5.0.0 > Posting files to [base] url > http://localhost:8983/solr/gettingstarted/update... > Entering auto mode. File endings considered are > xml,json,jsonl,csv,pdf,doc,docx,ppt,pptx,xls,xlsx,odt,odp,ods,ott,otp,ots,rtf,htm,html,txt,log > POSTing file sample.html (text/html) to [base]/extract > SimplePostTool: WARNING: Solr returned an error #404 (Not Found) for url: > http://localhost:8983/solr/gettingstarted/update/extract?resource.name=C%3A%5Csolr-8.8.0%5Cexample%5Cexampledocs%5Csample.html=C%3A%5Csolr-8.8.0%5Cexample%5Cexampledocs%5Csample.html > SimplePostTool: WARNING: Response: > > > Error 404 Not Found > > HTTP ERROR 404 Not Found > > URI:/solr/gettingstarted/update/extract > STATUS:404 > MESSAGE:Not Found > SERVLET:default > > > > > SimplePostTool: WARNING: IOException while reading response: > java.io.FileNotFoundException: > http://localhost:8983/solr/gettingstarted/update/extract?resource.name=C%3A%5Csolr-8.8.0%5Cexample%5Cexampledocs%5Csample.html=C%3A%5Csolr-8.8.0%5Cexample%5Cexampledocs%5Csample.html > 1 files indexed. > COMMITting Solr index changes to > http://localhost:8983/solr/gettingstarted/update... > Time spent: 0:00:00.086 > ``` > > However the json and all other file types index with no problem. For > example: > ``` > PS C:\solr-8.8.0> java -jar -Dc=gettingstarted -Dauto > example\exampledocs\post.jar example\exampledocs\books.json > SimplePostTool version 5.0.0 > Posting files to [base] url > http://localhost:8983/solr/gettingstarted/update... > Entering auto mode. File endings considered are > xml,json,jsonl,csv,pdf,doc,docx,ppt,pptx,xls,xlsx,odt,odp,ods,ott,otp,ots,rtf,htm,html,txt,log > POSTing file books.json (application/json) to [base]/json/docs > 1 files indexed. > COMMITting Solr index changes to > http://localhost:8983/solr/gettingstarted/update... > ``` > Just following this tutorial:[ > https://lucene.apache.org/solr/guide/8_8/post-tool.html#post-tool-windows-support][1 > ] > > [1]: > https://lucene.apache.org/solr/guide/8_8/post-tool.html#post-tool-windows-support
Re: HTML sample.html not indexing in Solr 8.8
On 2/20/2021 3:58 PM, cratervoid wrote: SimplePostTool: WARNING: Solr returned an error #404 (Not Found) for url: http://localhost:8983/solr/gettingstarted/update/extract?resource.name=C%3A%5Csolr-8.8.0%5Cexample%5Cexampledocs%5Csample.html=C%3A%5Csolr-8.8.0%5Cexample%5Cexampledocs%5Csample.html The problem here is that the solrconfig.xml in use by the index named "gettingstarted" does not define a handler at /update/extract. Typically a handler defined at that URL path will utilize the extracting request handler class. This handler uses Tika (another Apache project) to extract usable data from rich text formats like PDF, HTML, etc. true ignored_ _text_ Note that using this handler will require adding some contrib jars to Solr. Tika can become very unstable because it deals with undocumented file formats, so we do not recommend using that handler in production. If the functionality is important, Tika should be included in a program that's separate from Solr, so that if it crashes, it does not take Solr down with it. Thanks, Shawn
HTML sample.html not indexing in Solr 8.8
I am trying out indexing the exampledocs in the examples folder with the SimplePostTool on windows 10 using solr 8.8. All the documents index except sample.html. For that file I get the errors below. I then downloaded solr 7.7.3 and indexed the exampledocs folder with no errors, including sample.html. ``` PS C:\solr-8.8.0> java -jar -Dc=gettingstarted -Dauto example\exampledocs\post.jar example\exampledocs\sample.html SimplePostTool version 5.0.0 Posting files to [base] url http://localhost:8983/solr/gettingstarted/update... Entering auto mode. File endings considered are xml,json,jsonl,csv,pdf,doc,docx,ppt,pptx,xls,xlsx,odt,odp,ods,ott,otp,ots,rtf,htm,html,txt,log POSTing file sample.html (text/html) to [base]/extract SimplePostTool: WARNING: Solr returned an error #404 (Not Found) for url: http://localhost:8983/solr/gettingstarted/update/extract?resource.name=C%3A%5Csolr-8.8.0%5Cexample%5Cexampledocs%5Csample.html=C%3A%5Csolr-8.8.0%5Cexample%5Cexampledocs%5Csample.html SimplePostTool: WARNING: Response: Error 404 Not Found HTTP ERROR 404 Not Found URI:/solr/gettingstarted/update/extract STATUS:404 MESSAGE:Not Found SERVLET:default SimplePostTool: WARNING: IOException while reading response: java.io.FileNotFoundException: http://localhost:8983/solr/gettingstarted/update/extract?resource.name=C%3A%5Csolr-8.8.0%5Cexample%5Cexampledocs%5Csample.html=C%3A%5Csolr-8.8.0%5Cexample%5Cexampledocs%5Csample.html 1 files indexed. COMMITting Solr index changes to http://localhost:8983/solr/gettingstarted/update... Time spent: 0:00:00.086 ``` However the json and all other file types index with no problem. For example: ``` PS C:\solr-8.8.0> java -jar -Dc=gettingstarted -Dauto example\exampledocs\post.jar example\exampledocs\books.json SimplePostTool version 5.0.0 Posting files to [base] url http://localhost:8983/solr/gettingstarted/update... Entering auto mode. File endings considered are xml,json,jsonl,csv,pdf,doc,docx,ppt,pptx,xls,xlsx,odt,odp,ods,ott,otp,ots,rtf,htm,html,txt,log POSTing file books.json (application/json) to [base]/json/docs 1 files indexed. COMMITting Solr index changes to http://localhost:8983/solr/gettingstarted/update... ``` Just following this tutorial:[ https://lucene.apache.org/solr/guide/8_8/post-tool.html#post-tool-windows-support][1 ] [1]: https://lucene.apache.org/solr/guide/8_8/post-tool.html#post-tool-windows-support
Re: Congratulations to the new Apache Solr PMC Chair, Jan Høydahl!
Congratulations Jan Regards, Lucky Sharma On Sat, 20 Feb 2021 at 8:07 PM, Karl Wright wrote: > Congratulations! > Karl > > On Sat, Feb 20, 2021 at 6:28 AM Uwe Schindler wrote: > >> Congrats Jan! >> >> >> >> Uwe >> >> >> >> - >> >> Uwe Schindler >> >> Achterdiek 19, D-28357 Bremen >> <https://www.google.com/maps/search/Achterdiek+19,+D-28357+Bremen?entry=gmail=g> >> >> https://www.thetaphi.de >> >> eMail: u...@thetaphi.de >> >> >> >> *From:* Anshum Gupta >> *Sent:* Thursday, February 18, 2021 7:55 PM >> *To:* Lucene Dev ; solr-user@lucene.apache.org >> *Subject:* Congratulations to the new Apache Solr PMC Chair, Jan Høydahl! >> >> >> >> Hi everyone, >> >> >> >> I’d like to inform everyone that the newly formed Apache Solr PMC >> nominated and elected Jan Høydahl for the position of the Solr PMC Chair >> and Vice President. This decision was approved by the board in its February >> 2021 meeting. >> >> >> >> Congratulations Jan! >> >> >> >> -- >> >> Anshum Gupta >> > -- Warm Regards, Lucky Sharma Contact No :+91 9821559918
Re: Congratulations to the new Apache Solr PMC Chair, Jan Høydahl!
Congratulations! Karl On Sat, Feb 20, 2021 at 6:28 AM Uwe Schindler wrote: > Congrats Jan! > > > > Uwe > > > > - > > Uwe Schindler > > Achterdiek 19, D-28357 Bremen > > https://www.thetaphi.de > > eMail: u...@thetaphi.de > > > > *From:* Anshum Gupta > *Sent:* Thursday, February 18, 2021 7:55 PM > *To:* Lucene Dev ; solr-user@lucene.apache.org > *Subject:* Congratulations to the new Apache Solr PMC Chair, Jan Høydahl! > > > > Hi everyone, > > > > I’d like to inform everyone that the newly formed Apache Solr PMC > nominated and elected Jan Høydahl for the position of the Solr PMC Chair > and Vice President. This decision was approved by the board in its February > 2021 meeting. > > > > Congratulations Jan! > > > > -- > > Anshum Gupta >
RE: Congratulations to the new Apache Solr PMC Chair, Jan Høydahl!
Congrats Jan! Uwe - Uwe Schindler Achterdiek 19, D-28357 Bremen <https://www.thetaphi.de> https://www.thetaphi.de eMail: u...@thetaphi.de From: Anshum Gupta Sent: Thursday, February 18, 2021 7:55 PM To: Lucene Dev ; solr-user@lucene.apache.org Subject: Congratulations to the new Apache Solr PMC Chair, Jan Høydahl! Hi everyone, I’d like to inform everyone that the newly formed Apache Solr PMC nominated and elected Jan Høydahl for the position of the Solr PMC Chair and Vice President. This decision was approved by the board in its February 2021 meeting. Congratulations Jan! -- Anshum Gupta
Re: Congratulations to the new Apache Solr PMC Chair, Jan Høydahl!
Congratulations Jan ! On Sat, 20 Feb 2021 at 5:30 AM, Jan Høydahl wrote: > Thanks everyone! > > It's an exciting season for Solr, and I look forward to serving the > project as chair. > Also thanks to Anshum for an excellent job last term for Lucene. > > Jan > > > 18. feb. 2021 kl. 19:55 skrev Anshum Gupta : > > > > Hi everyone, > > > > I’d like to inform everyone that the newly formed Apache Solr PMC > nominated > > and elected Jan Høydahl for the position of the Solr PMC Chair and Vice > > President. This decision was approved by the board in its February 2021 > > meeting. > > > > Congratulations Jan! > > > > -- > > Anshum Gupta > >
Re: Congratulations to the new Apache Solr PMC Chair, Jan Høydahl!
Thanks everyone! It's an exciting season for Solr, and I look forward to serving the project as chair. Also thanks to Anshum for an excellent job last term for Lucene. Jan > 18. feb. 2021 kl. 19:55 skrev Anshum Gupta : > > Hi everyone, > > I’d like to inform everyone that the newly formed Apache Solr PMC nominated > and elected Jan Høydahl for the position of the Solr PMC Chair and Vice > President. This decision was approved by the board in its February 2021 > meeting. > > Congratulations Jan! > > -- > Anshum Gupta
Re: Congratulations to the new Apache Solr PMC Chair, Jan Høydahl!
Congrats Jan and Best Wishes! Also many thanks to Anshum for all his efforts in the last term! Also a belated shout out to Cassandra and other chairs in the past for their much appreciated efforts for the community ! Regards Aroop > On Feb 18, 2021, at 10:55 AM, Anshum Gupta wrote: > > Hi everyone, > > I’d like to inform everyone that the newly formed Apache Solr PMC nominated > and elected Jan Høydahl for the position of the Solr PMC Chair and Vice > President. This decision was approved by the board in its February 2021 > meeting. > > Congratulations Jan! > > -- > Anshum Gupta
Re: Congratulations to the new Apache Solr PMC Chair, Jan Høydahl!
Congrats! On Fri, Feb 19, 2021 at 10:06 AM Divye wrote: > > Congratulations Jan! > > Regards, > Divye > > On Fri, 19 Feb, 2021, 00:26 Anshum Gupta, wrote: > > > Hi everyone, > > > > I’d like to inform everyone that the newly formed Apache Solr PMC nominated > > and elected Jan Høydahl for the position of the Solr PMC Chair and Vice > > President. This decision was approved by the board in its February 2021 > > meeting. > > > > Congratulations Jan! > > > > -- > > Anshum Gupta > >
Re: Elevation in dataDir in Solr Cloud
Thank you! I just filed the bug in Jira: https://issues.apache.org/jira/browse/SOLR-15170 About the workaround you mentioned, we ran a quick test on one server and it apparently worked, but we did not check it properly in a cluster (we decided that it is better not to go with this in production anyway). Just out of curiosity, even if we could load the elevate.xml file from the data folder in Cloud (or changing it directly in zk), does this make sense in terms of performance? We have frequent commits and a big elevate.xml file. We are considering your suggestion of using the elevation directly in the queries (I have seen work to improve this by removing the requirement of having an elevate.xml file at all). It seems to be straightforward to apply in some cases, but not so much when you need normalization. -- Disclaimer: This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error please notify the system manager. If you are not the named addressee you should not disseminate, distribute or copy this email. Please notify the sender immediately by email if you have received this email by mistake and delete this email from your system.
Re: Congratulations to the new Apache Solr PMC Chair, Jan Høydahl!
Congratulations Jan! Regards, Divye On Fri, 19 Feb, 2021, 00:26 Anshum Gupta, wrote: > Hi everyone, > > I’d like to inform everyone that the newly formed Apache Solr PMC nominated > and elected Jan Høydahl for the position of the Solr PMC Chair and Vice > President. This decision was approved by the board in its February 2021 > meeting. > > Congratulations Jan! > > -- > Anshum Gupta >
Re: Congratulations to the new Apache Solr PMC Chair, Jan Høydahl!
Congratulations :) On Fri, Feb 19, 2021 at 6:51 AM Juan Eduardo Hernandez < juaneduard...@gmail.com> wrote: > Congratulations Jan!! > > El vie, 19 feb 2021 a las 5:56, Atita Arora () > escribió: > > > Congratulations Jan! > > > > On Fri, Feb 19, 2021 at 9:41 AM Dawid Weiss > wrote: > > > > > Congratulations, Jan! > > > > > > On Thu, Feb 18, 2021 at 7:56 PM Anshum Gupta > > > wrote: > > > > > > > > Hi everyone, > > > > > > > > I’d like to inform everyone that the newly formed Apache Solr PMC > > > nominated and elected Jan Høydahl for the position of the Solr PMC > Chair > > > and Vice President. This decision was approved by the board in its > > February > > > 2021 meeting. > > > > > > > > Congratulations Jan! > > > > > > > > -- > > > > Anshum Gupta > > > > > > -- http://www.needhamsoftware.com (work) http://www.the111shift.com (play)
Re: Congratulations to the new Apache Solr PMC Chair, Jan Høydahl!
Congratulations Jan!! El vie, 19 feb 2021 a las 5:56, Atita Arora () escribió: > Congratulations Jan! > > On Fri, Feb 19, 2021 at 9:41 AM Dawid Weiss wrote: > > > Congratulations, Jan! > > > > On Thu, Feb 18, 2021 at 7:56 PM Anshum Gupta > > wrote: > > > > > > Hi everyone, > > > > > > I’d like to inform everyone that the newly formed Apache Solr PMC > > nominated and elected Jan Høydahl for the position of the Solr PMC Chair > > and Vice President. This decision was approved by the board in its > February > > 2021 meeting. > > > > > > Congratulations Jan! > > > > > > -- > > > Anshum Gupta > > >
Re: Congratulations to the new Apache Solr PMC Chair, Jan Høydahl!
Congratulations Jan! On Fri, Feb 19, 2021 at 9:41 AM Dawid Weiss wrote: > Congratulations, Jan! > > On Thu, Feb 18, 2021 at 7:56 PM Anshum Gupta > wrote: > > > > Hi everyone, > > > > I’d like to inform everyone that the newly formed Apache Solr PMC > nominated and elected Jan Høydahl for the position of the Solr PMC Chair > and Vice President. This decision was approved by the board in its February > 2021 meeting. > > > > Congratulations Jan! > > > > -- > > Anshum Gupta >
Re: Congratulations to the new Apache Solr PMC Chair, Jan Høydahl!
Congratulations, Jan! On Thu, Feb 18, 2021 at 7:56 PM Anshum Gupta wrote: > > Hi everyone, > > I’d like to inform everyone that the newly formed Apache Solr PMC nominated > and elected Jan Høydahl for the position of the Solr PMC Chair and Vice > President. This decision was approved by the board in its February 2021 > meeting. > > Congratulations Jan! > > -- > Anshum Gupta
Re: Congratulations to the new Apache Solr PMC Chair, Jan Høydahl!
Congratulations Jan! ~ David Smiley Apache Lucene/Solr Search Developer http://www.linkedin.com/in/davidwsmiley On Thu, Feb 18, 2021 at 1:56 PM Anshum Gupta wrote: > Hi everyone, > > I’d like to inform everyone that the newly formed Apache Solr PMC nominated > and elected Jan Høydahl for the position of the Solr PMC Chair and Vice > President. This decision was approved by the board in its February 2021 > meeting. > > Congratulations Jan! > > -- > Anshum Gupta >
Re: Congratulations to the new Apache Solr PMC Chair, Jan Høydahl!
Congratulations and thank you, Jan! It is so exciting that Solr is now a TLP! Mike McCandless http://blog.mikemccandless.com On Thu, Feb 18, 2021 at 1:56 PM Anshum Gupta wrote: > Hi everyone, > > I’d like to inform everyone that the newly formed Apache Solr PMC > nominated and elected Jan Høydahl for the position of the Solr PMC Chair > and Vice President. This decision was approved by the board in its February > 2021 meeting. > > Congratulations Jan! > > -- > Anshum Gupta >
Re: Congratulations to the new Apache Solr PMC Chair, Jan Høydahl!
Congratulations Jan! -- Steve > On Feb 18, 2021, at 1:55 PM, Anshum Gupta wrote: > > Hi everyone, > > I’d like to inform everyone that the newly formed Apache Solr PMC nominated > and elected Jan Høydahl for the position of the Solr PMC Chair and Vice > President. This decision was approved by the board in its February 2021 > meeting. > > Congratulations Jan! > > -- > Anshum Gupta
Re: Congratulations to the new Apache Solr PMC Chair, Jan Høydahl!
Yes, Congratulations and a big thank you Jan! On Thu, Feb 18, 2021 at 1:56 PM Anshum Gupta wrote: > > Hi everyone, > > I’d like to inform everyone that the newly formed Apache Solr PMC nominated > and elected Jan Høydahl for the position of the Solr PMC Chair and Vice > President. This decision was approved by the board in its February 2021 > meeting. > > Congratulations Jan! > > -- > Anshum Gupta
Congratulations to the new Apache Solr PMC Chair, Jan Høydahl!
Hi everyone, I’d like to inform everyone that the newly formed Apache Solr PMC nominated and elected Jan Høydahl for the position of the Solr PMC Chair and Vice President. This decision was approved by the board in its February 2021 meeting. Congratulations Jan! -- Anshum Gupta
[ANNOUNCE] Apache Solr TLP Created
Hi everyone, On behalf of the Apache Lucene PMC, and the newly formed Apache Solr PMC, I’d like to inform folks that the ASF board has approved the resolution to create the Solr TLP (Top Level Project). We are currently working on the next steps but would like to assure the community that they can continue to expect critical bug fixes for releases previously made under the Apache Lucene project. We will send another update as the mailing lists and website are set up for the Solr project. -Anshum On behalf of the Apache Lucene and Solr PMC
Re: Solr 8.0 query length limit
Thanks Alex and Shawn. Regards, Anuj On Thu, 18 Feb 2021 at 18:57, Shawn Heisey wrote: > On 2/18/2021 3:38 AM, Anuj Bhargava wrote: > > Solr 8.0 query length limit > > > > We are having an issue where queries are too big, we get no result. And > if > > we remove a few keywords we get the result. > > The best option is to convert the request to POST, as Thomas suggested. > With that, the query parameters could be up to 2 megabytes in size > with no config changes. > > The limit for this is enforced by Jetty -- the servlet container that > Solr ships with. If you cannot switch your requests to POST, then you > can find the following line in server/etc/jetty.xml, adjust it, and > restart Solr: > > name="solr.jetty.request.header.size" default="8192" /> > > A header limit of 8KB is found in nearly all web servers and related > software, like load balancers. > > Thanks, > Shawn >
Re: Cannot find Solr 7.4.1 release
On 2/18/2021 1:05 AM, Olivier Tavard wrote: I wanted to download Solr 7.4.1, but I cannot find the 7.4.1 release into http://archive.apache.org/dist/lucene/solr/ : there are Solr 7.4 and after directly 7.5. Of course I can build from source code, but this is frustrating because I can see that in the 7_4_branch there is a fix that I need (SOLR-12594) with the status fixed into 7.4.1 and 7.5 versions. Everythings seems to have been prepared to release the 7.4.1, but I cannot find it. Does this release exist ? That release does not exist. There was never any discussion about it on the dev list. 7.4.1 was added to Jira for tracking purposes, and the code change for that issue was saved to branch_7_4 just in case somebody felt a 7.4.1 release was required. That issue deals with a problem in metrics, which is outside of basic Solr functionality -- not critical enough to warrant a point release. The release process for 7.5.0 was underway about a month after that issue was committed. If 7.5.0 (or one of the many later releases) will not work for your needs, then you will need to compile branch_7_4 yourself. I have used custom-compiled versions before in production because we needed a bugfix that was not deemed severe enough for a new point release. You can create binary packages similar to what is available for download by running "ant package" in the solr directory of your code checkout. I think that build target only works on *NIX systems -- Windows is missing some of the required pieces. Thanks, Shawn
Re: Solr 8.0 query length limit
On 2/18/2021 3:38 AM, Anuj Bhargava wrote: Solr 8.0 query length limit We are having an issue where queries are too big, we get no result. And if we remove a few keywords we get the result. The best option is to convert the request to POST, as Thomas suggested. With that, the query parameters could be up to 2 megabytes in size with no config changes. The limit for this is enforced by Jetty -- the servlet container that Solr ships with. If you cannot switch your requests to POST, then you can find the following line in server/etc/jetty.xml, adjust it, and restart Solr: name="solr.jetty.request.header.size" default="8192" /> A header limit of 8KB is found in nearly all web servers and related software, like load balancers. Thanks, Shawn
Re: Solr 8.0 query length limit
Also, investigate if you have repeating conditions and push those into defaults in custom request handler endpoints (in solrconfig.xml). Also, Solr supports parameter substitutions, if you have repeated subconditions. Regards, Alex On Thu., Feb. 18, 2021, 7:08 a.m. Thomas Corthals, wrote: > You can send big queries as a POST request instead of a GET request. > > Op do 18 feb. 2021 om 11:38 schreef Anuj Bhargava : > > > Solr 8.0 query length limit > > > > We are having an issue where queries are too big, we get no result. And > if > > we remove a few keywords we get the result. > > > > Error we get - error 414 (Request-URI Too Long) > > > > > > Have made the following changes in jetty.xml, still the same error > > > > * > name="solr.jetty.output.buffer.size" default="32768" />* > > * > name="solr.jetty.output.aggregation.size" default="32768" />* > > * > name="solr.jetty.request.header.size" default="65536" />* > > * > name="solr.jetty.response.header.size" default="32768" />* > > * > name="solr.jetty.send.server.version" default="false" />* > > * > name="solr.jetty.send.date.header" default="false" />* > > * > name="solr.jetty.header.cache.size" default="1024" />* > > * > name="solr.jetty.delayDispatchUntilContent" default="false"/>* > > >
Re: Solr 8.0 query length limit
You can send big queries as a POST request instead of a GET request. Op do 18 feb. 2021 om 11:38 schreef Anuj Bhargava : > Solr 8.0 query length limit > > We are having an issue where queries are too big, we get no result. And if > we remove a few keywords we get the result. > > Error we get - error 414 (Request-URI Too Long) > > > Have made the following changes in jetty.xml, still the same error > > * name="solr.jetty.output.buffer.size" default="32768" />* > * name="solr.jetty.output.aggregation.size" default="32768" />* > * name="solr.jetty.request.header.size" default="65536" />* > * name="solr.jetty.response.header.size" default="32768" />* > * name="solr.jetty.send.server.version" default="false" />* > * name="solr.jetty.send.date.header" default="false" />* > * name="solr.jetty.header.cache.size" default="1024" />* > * name="solr.jetty.delayDispatchUntilContent" default="false"/>* >
Solr 8.0 query length limit
Solr 8.0 query length limit We are having an issue where queries are too big, we get no result. And if we remove a few keywords we get the result. Error we get - error 414 (Request-URI Too Long) Have made the following changes in jetty.xml, still the same error ** ** ** ** ** ** ** **
Cannot find Solr 7.4.1 release
Hi, I wanted to download Solr 7.4.1, but I cannot find the 7.4.1 release into http://archive.apache.org/dist/lucene/solr/ : there are Solr 7.4 and after directly 7.5. Of course I can build from source code, but this is frustrating because I can see that in the 7_4_branch there is a fix that I need (SOLR-12594) with the status fixed into 7.4.1 and 7.5 versions. Everythings seems to have been prepared to release the 7.4.1, but I cannot find it. Does this release exist ? Thank you, Olivier
Re: Elevation in dataDir in Solr Cloud
: Of course, here is the full stack trace (collection 'techproducts' with : just one core to make it easier): Ah yeah ... see -- this looks like a mistake introduced at some point... : Caused by: org.apache.solr.core.SolrResourceNotFoundException: Can't : find resource 'elevate.xml' in classpath or : '/configs/techproductsConfExp', cwd=/usr/share/solr-7.7.2/server : at org.apache.solr.cloud.ZkSolrResourceLoader.openResource(ZkSolrResourceLoader.java:130) : at org.apache.solr.core.SolrResourceLoader.openConfig(SolrResourceLoader.java:362) : at org.apache.solr.core.Config.(Config.java:120) : at org.apache.solr.core.Config.(Config.java:90) : at org.apache.solr.handler.component.QueryElevationComponent.loadElevationProvider(QueryElevationComponent.java:366) ...this bit of code is *expecting* to be able to init a Config object from the SolrResourceLoader, even thought this bit of code... : at org.apache.solr.handler.component.QueryElevationComponent.getElevationProvider(QueryElevationComponent.java:321) : at org.apache.solr.handler.component.QueryElevationComponent.loadElevationConfiguration(QueryElevationComponent.java:259) ...has already established that there is no "Config" file available from the resource loader, and we should be initializing an ElevationProvider that can raed from the data dir. 9and this code seems to be unchanged on branch_8x) Can you please file a jira pointing out that this doesn't work along with the full stack trace, and then add a comment copy/pasting my comments here that the code makes no sense? I'm not sure if/when someone who understands the code well enough will be able to help fix this (and write a test for it) ... was the experiment / work around i suggested viable? ... : > I don't know if it will work, but one thing you might want to experiment : > with is putting your elevate.xml back the configset in zk, and updating it : > on the fly in zk -- then see if it gets reloaded by each core the next : > time the index changes (NOTE that there will almost certainly need to be : > an index change for it to re-load, since I don't see any indication that : > it's watching for changes in zk) : > : > FWIW: the way most people seem to be using QEC these days is to have an : > empty elevate.xml file, and then have their application use some other : > key/val store, or more complex matching logic, to decide which documents : > to elevate, and then use the "elevateIds" param to pass that info to solr. -Hoss http://www.lucidworks.com/
Re: Down Replica is elected as Leader (solr v8.7.0)
> Are yours growing always, on all nodes, forever? Or is it one or two who ends up in a bad state? Randomly on some of the shards and some of the followers in the collection. Then whichever tlog was open on follower when it was the leader, that one doesn't stops growing. And that shard had active ingestion at a high rate. If we now add more shards, or do a cluster rebalance, the collection is unsusable and causes a production query and ingest outage. Very painful manual restoration twice a day. -- Sent from: https://lucene.472066.n3.nabble.com/Solr-User-f472068.html
Re: Down Replica is elected as Leader (solr v8.7.0)
I've run into this (or similar) issues in the past (solr6? I don't remember exactly) where tlogs get stuck either growing indefinitely and/or refusing to commit on restart. What I ended up doing was writing a monitor to check for the number of tlogs and alert if they got over some limit (100 or whatever) and then I could stay ahead of the issue by rebuilding individual nodes as-needed. Are yours growing always, on all nodes, forever? Or is it one or two who ends up in a bad state? On Tue, Feb 16, 2021 at 3:57 PM mmb1234 wrote: > > Looks like the problem is related to tlog rotation on the follower shard. > > We did the following for a specific shard. > > 0. start solr cloud > 1. solr-0 (leader), solr-1, solr-2 > 2. rebalance to make solr-1 as preferred leader > 3. solr-0, solr-1 (leader), solr-2 > > The tlog file on solr-0 kept on growing infinitely (100s of GBs) until we > shut the cluster and dropped all shards (manually). > > Only way to "restart" tlog rotation on solr-0 (follower) was to issue > /admin/cores/action=RELOAD=x atleast twice when the tlog size was > small (in MBs). > > Also if rebalance is is issued to select solr-0 as a leader, leader election > never completes. > > solr-0 output after step (3) above. > > solr-0 > 2140856 ./data2/mydata_0_e000-/tlog > 2140712 ./data2/mydata_0_e000-/tlog/tlog.021 > > solr-1 (leader) > 35268 ./data2/mydata_0_e000-ffff/tlog > 35264 ./data2/mydata_0_e000-/tlog/tlog.055 > > solr-2 > 35256 ./data2/mydata_0_e000-/tlog > 35252 ./data2/mydata_0_e000-/tlog/tlog.054 > > > > -- > Sent from: https://lucene.472066.n3.nabble.com/Solr-User-f472068.html
Re: Down Replica is elected as Leader (solr v8.7.0)
Looks like the problem is related to tlog rotation on the follower shard. We did the following for a specific shard. 0. start solr cloud 1. solr-0 (leader), solr-1, solr-2 2. rebalance to make solr-1 as preferred leader 3. solr-0, solr-1 (leader), solr-2 The tlog file on solr-0 kept on growing infinitely (100s of GBs) until we shut the cluster and dropped all shards (manually). Only way to "restart" tlog rotation on solr-0 (follower) was to issue /admin/cores/action=RELOAD=x atleast twice when the tlog size was small (in MBs). Also if rebalance is is issued to select solr-0 as a leader, leader election never completes. solr-0 output after step (3) above. solr-0 2140856 ./data2/mydata_0_e000-/tlog 2140712 ./data2/mydata_0_e000-/tlog/tlog.0000021 solr-1 (leader) 35268 ./data2/mydata_0_e000-/tlog 35264 ./data2/mydata_0_e000-/tlog/tlog.0000055 solr-2 35256 ./data2/mydata_0_e000-/tlog 35252 ./data2/mydata_0_e000-/tlog/tlog.054 -- Sent from: https://lucene.472066.n3.nabble.com/Solr-User-f472068.html
Re: Down Replica is elected as Leader (solr v8.7.0)
Looks like the problem is related to tlog rotation on the follower shard. We did the following for a specific shard. 0. start solr cloud 1. solr-0 (leader), solr-1, solr-2 2. rebalance to make solr-1 as preferred leader 3. solr-0, solr-1 (leader), solr-2 The tlog file on solr-0 kept on growing infinitely (100s of GBs) until we shut the cluster and dropped all shards (manually). Only way to "restart" tlog rotation on solr-0 (follower) was to issue /admin/cores/action=RELOAD=x atleast twice when the tlog size was small (in MBs). Also if rebalance is is issued to select solr-0 as a leader, leader election never completes. solr-0 output after step (3) above. solr-0 2140856 ./data2/mydata_0_e000-/tlog 2140712 ./data2/mydata_0_e000-/tlog/tlog.0000021 solr-1 (leader) 35268 ./data2/mydata_0_e000-/tlog 35264 ./data2/mydata_0_e000-/tlog/tlog.0000055 solr-2 35256 ./data2/mydata_0_e000-/tlog 35252 ./data2/mydata_0_e000-/tlog/tlog.054 -- Sent from: https://lucene.472066.n3.nabble.com/Solr-User-f472068.html
Re: Elevation in dataDir in Solr Cloud
) at org.apache.solr.core.SolrResourceLoader.inform(SolrResourceLoader.java:696) at org.apache.solr.core.SolrCore.(SolrCore.java:1000) ... 46 more Caused by: org.apache.solr.core.SolrResourceNotFoundException: Can't find resource 'elevate.xml' in classpath or '/configs/techproductsConfExp', cwd=/usr/share/solr-7.7.2/server at org.apache.solr.cloud.ZkSolrResourceLoader.openResource(ZkSolrResourceLoader.java:130) at org.apache.solr.core.SolrResourceLoader.openConfig(SolrResourceLoader.java:362) at org.apache.solr.core.Config.(Config.java:120) at org.apache.solr.core.Config.(Config.java:90) at org.apache.solr.handler.component.QueryElevationComponent.loadElevationProvider(QueryElevationComponent.java:366) at org.apache.solr.handler.component.QueryElevationComponent.getElevationProvider(QueryElevationComponent.java:321) at org.apache.solr.handler.component.QueryElevationComponent.loadElevationConfiguration(QueryElevationComponent.java:259) at org.apache.solr.handler.component.QueryElevationComponent.inform(QueryElevationComponent.java:158) On Fri, 12 Feb 2021 at 19:45, Chris Hostetter wrote: > > : I need to have the elevate.xml file updated frequently and I was > wondering > : if it is possible to put this file in the dataDir folder when using Solr > : Cloud. I know that this is possible in the standalone mode, and I haven't > : seen in the documentation [1] that it can not be done in Cloud. > : > : I am using Solr 7.7.2 and ZooKeeper. After creating the techproducts > : collection for the tests, I remove the elevate.xml file from the > : configuration and I put it in the dataDir folder of the cores. When I > : update the collection with that configuration, I get the following error: > : "Can't find resource 'elevate.xml' in classpath or > : '/configs/techproductsConfExp'". Is this expected or I am doing something > : wrong? > > Hmmm... can you share the full stack trace of that error? > > (I suspect at some point someone made a sloppy assumption in the QEC code > that no one would ever try to keep elevate.xml in the data dir in cloud > mode.) > > I don't know if it will work, but one thing you might want to experiment > with is putting your elevate.xml back the configset in zk, and updating it > on the fly in zk -- then see if it gets reloaded by each core the next > time the index changes (NOTE that there will almost certainly need to be > an index change for it to re-load, since I don't see any indication that > it's watching for changes in zk) > > FWIW: the way most people seem to be using QEC these days is to have an > empty elevate.xml file, and then have their application use some other > key/val store, or more complex matching logic, to decide which documents > to elevate, and then use the "elevateIds" param to pass that info to solr. > > > -Hoss > http://www.lucidworks.com/ > -- -- Disclaimer: This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error please notify the system manager. If you are not the named addressee you should not disseminate, distribute or copy this email. Please notify the sender immediately by email if you have received this email by mistake and delete this email from your system.
Re: Down Replica is elected as Leader (solr v8.7.0)
We found that for the shard that does not get a leader, the tlog replay did
not complete (we don't see "log replay finished", "creating leader
registration node", "I am the new leader" etc log messages) for hours.
Also not sure why the TLOG are 10's of GBs (anywhere from 30 to 40GB).
Collection's shards have 3x replicas, TLOG replication and 10sec hard
commit.
The situation is resulting in 2x per day outage. Current work around is to
stop ingestion, issue a collection rebalance and/or node restarts and repeat
until shards are online (a couple of hrs each day of manual recovery).
Any suggestions or workarounds?
Not sure if we're running into these issues:
https://issues.apache.org/jira/browse/SOLR-13486
https://issues.apache.org/jira/browse/SOLR-14679
Partial log output from both nodes (sorted by time asc):
myapp-data-solr-0
2021-02-12 19:36:05.965 INFO (zkCallback-14-thread-51) [c:mydata
s:0_8000-9fff r:core_node3 x:mydata_0_8000-9fff_replica_t1]
o.a.s.c.ShardLeaderElectionContext Replaying tlog before become new leader
myapp-data-solr-0
2021-02-12 19:36:05.966 WARN
(recoveryExecutor-96-thread-1-processing-n:myapp-data-solr-0.myapp-data-solr-headless:8983_solr
x:mydata_0_8000-9fff_replica_t1 c:mydata s:0_8000-9fff
r:core_node3) [c:mydata s:0_8000-9fff r:core_node3
x:mydata_0_8000-9fff_replica_t1] o.a.s.u.UpdateLog Starting log
replay
tlog{file=/opt/solr/volumes/data1/mydata_0_8000-9fff/tlog/tlog.0003525
refcount=2} active=false starting pos=0 inSortedOrder=true
myapp-data-solr-0
2021-02-12 22:13:03.084 INFO
(recoveryExecutor-96-thread-1-processing-n:myapp-data-solr-0.myapp-data-solr-headless:8983_solr
x:mydata_0_8000-9fff_replica_t1 c:mydata s:0_8000-9fff
r:core_node3) [c:mydata s:0_8000-9fff r:core_node3
x:mydata_0_8000-9fff_replica_t1] o.a.s.u.UpdateLog log replay status
tlog{file=/opt/solr/volumes/data1/mydata_0_8000-9fff/tlog/tlog.0003525
refcount=3} active=false starting pos=0 current pos=27101174167 current
size=33357447222 % read=81.0
myapp-data-solr-0
2021-02-12 22:13:06.602 ERROR (indexFetcher-4092-thread-1) [ ]
o.a.s.h.ReplicationHandler Index fetch failed
:org.apache.solr.common.SolrException: No registered leader was found after
waiting for 4000ms , collection: mydata slice: 0_8000-9fff saw
state=DocCollection(mydata//collections/mydata/state.json/750)={
"pullReplicas":"0", "replicationFactor":"0", "shards":{
"0_8000-9fff":{ "range":null, "state":"active", "replicas":{
"core_node3":{ "core":"mydata_0_8000-9fff_replica_t1",
"base_url":"http://myapp-data-solr-0.myapp-data-solr-headless:8983/solr;,
"node_name":"myapp-data-solr-0.myapp-data-solr-headless:8983_solr",
"state":"active", "type":"TLOG", "force_set_state":"false"}, "core_node5":{
"core":"mydata_0_8000-9fff_replica_t2",
"base_url":"http://myapp-data-solr-1.myapp-data-solr-headless:8983/solr;,
"node_name":"myapp-data-solr-1.myapp-data-solr-headless:8983_solr",
"state":"active", "type":"TLOG", "force_set_state":"false",
"property.preferredleader":"true"}, "core_node6":{
"core":"mydata_0_8000-9fff_replica_t4",
"base_url":"http://myapp-data-solr-2.myapp-data-solr-headless:8983/solr;,
"node_name":"myapp-data-solr-2.myapp-data-solr-headless:8983_solr",
"state":"down", "type":"TLOG", "force_set_state":"false"}}},
myapp-data-solr-0
2021-02-12 22:45:51.600 ERROR (indexFetcher-4092-thread-1) [ ]
o.a.s.h.ReplicationHandler Index fetch failed
:org.apache.solr.common.SolrException: No registered leader was found after
waiting for 4000ms , collection: mydata slice: 0_8000-9fff saw
state=DocCollection(mydata//collections/mydata/state.json/754)={
"pullReplicas":"0", "replicationFactor":"0", "shards":{
"0_8000-9fff":{ "range":null, "state":"active", "replicas":{
"core_node3":{ "core":"mydata_0_8000-9fff_replica_t1",
"base_url":"http://myapp-data-solr-0.myapp-data-solr-headless:8983/solr;,
"node_name":"myapp-data-solr-0.myapp-data-solr-headless:8983_solr",
"state":"active", "type":"TLOG", "force_set_state":"false"}, "core_node5":{
"core":"mydata_0_8000-9fff_replica_t2",
"
Re: CVE-2019-17558 on SOLR 6.1
(Resending to the list. Sorry, Rick.) FYI, my client was using 8.3.1, which should have mitigated the attack. But the server was suffering a sudden death of the Solr process, and the log showed it was being attacked using CVE-2019-17558. We blocked the external access of Solr API. Then this sudden death ended. So I tend to think just disabling the Velocity engine might not enough. Of course there is a possibility that this server was also getting a different kind of attack. We don't know. But in general, the Solr port should be closed from external access. TK On 2/12/21 10:17 AM, Rick Tham wrote: We are using Solr 6.1 and at the moment we can not upgrade due to application dependencies. We have mitigation steps in place to only trust specific machines within our DMZ. I am trying to figure out if the following is an additioanal valid mitigation step for CVE-2019-17558 on SOLR 6.1. None of our solrconfig.xml contains the lib references to the velocity jar files as follows: l It doesn't appear that you can add these jars references using the config API. Without these references, you are not able to flip the params.resource.loader.enabled to true using the config API. If you are not able to flip the flag and none of your cores have these lib references then is the risk present? Thanks in advance!
Re: Down Replica is elected as Leader (solr v8.7.0)
By tracing the output in the log files we see the following sequence. Overseer role list has POD-1, POD-2, POD-3 in that order POD-3 has 2 shard leaders. POD-3 restarts. A) Logs for the shard whose leader moves successfully from POD-3 to POD-1 On POD-1: o.a.s.c.ShardLeaderElectionContext Replaying tlog before become new leader On POD-1: o.a.s.u.UpdateLog Starting log replay On POD-1: o.a.s.u.UpdateLog Log replay finished. On POD-1: o.a.s.c.SolrCore Registered new searcher autowarm time: 0 ms On POD-1: o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node ... after winning as ... On POD-1: o.a.s.c.ShardLeaderElectionContext I am the new leader: B) Logs for the shard whose leader does not move from POD-3 to POD-1 On POD-1: o.a.s.c.ShardLeaderElectionContext Replaying tlog before become new leader On POD-1: o.a.s.u.UpdateLog Starting log replay... On POD-1: o.a.s.h.ReplicationHandler Index fetch failed :org.apache.solr.common.SolrException: No registered leader was found after waiting for 4000ms < POD-3 is back up at this time > On POD-3: o.a.s.h.RequestHandlerBase org.apache.solr.common.SolrException: No registered leader On was found after waiting for 4000ms It was odd to see no INFO, WARN or ERROR log message after "Starting log replay" on POD-1 for the shard which did not get its leader elected. -- Sent from: https://lucene.472066.n3.nabble.com/Solr-User-f472068.html
Re: CVE-2019-17558 on SOLR 6.1
Thanks Shawn. On Fri, Feb 12, 2021 at 7:43 PM Shawn Heisey wrote: > On 2/12/2021 11:17 AM, Rick Tham wrote: > > I am trying to figure out if the following is an additioanal valid > > mitigation step for CVE-2019-17558 on SOLR 6.1. None of our > solrconfig.xml > > contains the lib references to the velocity jar files as follows: > > > > > regex="..jar" /> > > l > regex="solr-velocity-\d..jar" /> > > > > It doesn't appear that you can add these jars references using the config > > API. Without these references, you are not able to flip the > > params.resource.loader.enabled to true using the config API. If you are > not > > able to flip the flag and none of your cores have these lib references > then > > is the risk present? > > In order to be vulnerable to that problem, all of the following things > must be true. If any of them is NOT true, then this vulnerability does > not apply: > > * The velocity jars must be loaded. A common way for this is the > configuration you mentioned, but there are other ways. Those other ways > require human intervention to move the actual files. > * Your config must *use* the jars, by containing a velocity config. > * The params resource loader must be enabled in the velocity config. > Note that the "velocity.params.resource.loader.enabled" flag only > applies if the velocity config in solrconfig.xml *references* that flag. > * Your Solr server must be reachable to unauthorized parties who would > exploit the vulnerability. > > I have no idea whether any of this config can be changed remotely. I > have never used the config API. But if your Solr server is not > reachable to bad guys, it won't matter. > > Simply controlling who can reach the Solr server is the easiest way to > avoid being vulnerable to anything. Although there are security > mechanisms available, Solr is not designed to be publicly reachable. It > should be heavily firewalled. > > The velocity response writer usually requires end users to have direct > access to the Solr server for it to be worth something. We STRONGLY > discourage leaving Solr exposed. > > Thanks, > Shawn >
Re: CVE-2019-17558 on SOLR 6.1
On 2/12/2021 11:17 AM, Rick Tham wrote: I am trying to figure out if the following is an additioanal valid mitigation step for CVE-2019-17558 on SOLR 6.1. None of our solrconfig.xml contains the lib references to the velocity jar files as follows: l It doesn't appear that you can add these jars references using the config API. Without these references, you are not able to flip the params.resource.loader.enabled to true using the config API. If you are not able to flip the flag and none of your cores have these lib references then is the risk present? In order to be vulnerable to that problem, all of the following things must be true. If any of them is NOT true, then this vulnerability does not apply: * The velocity jars must be loaded. A common way for this is the configuration you mentioned, but there are other ways. Those other ways require human intervention to move the actual files. * Your config must *use* the jars, by containing a velocity config. * The params resource loader must be enabled in the velocity config. Note that the "velocity.params.resource.loader.enabled" flag only applies if the velocity config in solrconfig.xml *references* that flag. * Your Solr server must be reachable to unauthorized parties who would exploit the vulnerability. I have no idea whether any of this config can be changed remotely. I have never used the config API. But if your Solr server is not reachable to bad guys, it won't matter. Simply controlling who can reach the Solr server is the easiest way to avoid being vulnerable to anything. Although there are security mechanisms available, Solr is not designed to be publicly reachable. It should be heavily firewalled. The velocity response writer usually requires end users to have direct access to the Solr server for it to be worth something. We STRONGLY discourage leaving Solr exposed. Thanks, Shawn
Re: Why Solr questions on stackoverflow get very few views and answers, if at all?
Many questions have responses as comments, but no actual answers. One frequent contributor doesn’t understand how StackOverflow works, so he’s posting answers as comments. He’s also doing conversations instead of crafting a useful, complete answer. I just answered a few. Mostly with “don’t use stop words” and “Solr is not a database”. wunder Walter Underwood wun...@wunderwood.org http://observer.wunderwood.org/ (my blog) > On Feb 12, 2021, at 3:03 AM, Charlie Hull > wrote: > > I've answered a few in my time, but my experience is that if you do so you > then get emailed a whole load more questions some of which aren't even > relevant to Solr! Also, quite a few of them are 'here is 3 pages of code > please debug it for me no I won't tell the actual error I got'. > > This is the best place to come, also there's the IRC channel, the new Slack > gateway to this at https://s.apache.org/solr-slack and in our own Relevance > Slack at http://opensourceconnections.com/slack there's a #solr channel (as > well as many others on search & relevance topics). > > Solr is 'hot' (but not as hot as Elasticsearch), and search is still a niche > business overall. > > HTH > > Cheers > > Charlie > > On 12/02/2021 10:37, ufuk yılmaz wrote: >> Is it because the main place for q is this mailing list, or somewhere else >> that I don’t know? >> >> Or Solr isn’t ‘hot’ as some other topics? >> >> Sent from Mail for Windows 10 >> >> > > -- > Charlie Hull - Managing Consultant at OpenSource Connections Limited > > Founding member of The Search Network <https://thesearchnetwork.com/> and > co-author of Searching the Enterprise > <https://opensourceconnections.com/about-us/books-resources/> > tel/fax: +44 (0)8700 118334 > mobile: +44 (0)7767 825828
Re: Elevation in dataDir in Solr Cloud
: I need to have the elevate.xml file updated frequently and I was wondering : if it is possible to put this file in the dataDir folder when using Solr : Cloud. I know that this is possible in the standalone mode, and I haven't : seen in the documentation [1] that it can not be done in Cloud. : : I am using Solr 7.7.2 and ZooKeeper. After creating the techproducts : collection for the tests, I remove the elevate.xml file from the : configuration and I put it in the dataDir folder of the cores. When I : update the collection with that configuration, I get the following error: : "Can't find resource 'elevate.xml' in classpath or : '/configs/techproductsConfExp'". Is this expected or I am doing something : wrong? Hmmm... can you share the full stack trace of that error? (I suspect at some point someone made a sloppy assumption in the QEC code that no one would ever try to keep elevate.xml in the data dir in cloud mode.) I don't know if it will work, but one thing you might want to experiment with is putting your elevate.xml back the configset in zk, and updating it on the fly in zk -- then see if it gets reloaded by each core the next time the index changes (NOTE that there will almost certainly need to be an index change for it to re-load, since I don't see any indication that it's watching for changes in zk) FWIW: the way most people seem to be using QEC these days is to have an empty elevate.xml file, and then have their application use some other key/val store, or more complex matching logic, to decide which documents to elevate, and then use the "elevateIds" param to pass that info to solr. -Hoss http://www.lucidworks.com/
CVE-2019-17558 on SOLR 6.1
We are using Solr 6.1 and at the moment we can not upgrade due to application dependencies. We have mitigation steps in place to only trust specific machines within our DMZ. I am trying to figure out if the following is an additioanal valid mitigation step for CVE-2019-17558 on SOLR 6.1. None of our solrconfig.xml contains the lib references to the velocity jar files as follows: l It doesn't appear that you can add these jars references using the config API. Without these references, you are not able to flip the params.resource.loader.enabled to true using the config API. If you are not able to flip the flag and none of your cores have these lib references then is the risk present? Thanks in advance!
Re: [ANNOUNCE] Apache Solr 8.8.0 released
Hi all, This release contains a critical bug, that should be fixed in 8.8.1 shortly. Please avoid upgrading to this release for the moment. https://twitter.com/ichattopadhyaya/status/1360163382171586562 Apologies for the inconvenience. Thanks, Ishan On Mon, Feb 1, 2021 at 6:01 PM Noble Paul wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA512 > > Solr is the popular, blazing fast, open source NoSQL search platform from > the Apache Lucene project. Its major features include powerful full-text > search, hit highlighting, faceted search and analytics, rich document > parsing, geospatial search, extensive REST APIs as well as parallel SQL. > Solr is enterprise grade, secure and highly scalable, providing fault > tolerant distributed search and indexing, and powers the search and > navigation features of many of the world's largest internet sites. > > The release is available for immediate download at: > > https://lucene.apache.org/solr/downloads.html > > Please read CHANGES.txt for a detailed list of changes: > > https://lucene.apache.org/solr/8_8_0/changes/Changes.html Solr 8.8.0 > > Release Highlights: > > Reducing overseer bottlenecks using per-replica states. More stability and > lesser load on large cluster that use this feature. > > Better restart and collection creation performance. > > Interleaving support in Learning To Rank > > A summary of important changes is published in the Solr Reference Guide at: > > https://lucene.apache.org/solr/guide/8_8/solr-upgrade-notes.html. > > For the most exhaustive list, see the full release notes at > https://lucene.apache.org/solr/8_8_0/changes/Changes.html > > or > > by viewing the CHANGES.txt file accompanying the distribution. Solr's > release notes usually don't include Lucene layer changes. Lucene's release > notes are at > > https://lucene.apache.org/core/8_8_0/changes/Changes.html > > - - > Noble Paul > -BEGIN PGP SIGNATURE- > Version: FlowCrypt Email Encryption 8.0.0 > Comment: Seamlessly send and receive encrypted email > > wsFzBAEBCgAGBQJgF/RnACEJEMOP9ew/z9s+FiEEz85fu5IMPHRc7uCEw4/1 > 7D/P2z6fzRAAm4AKbeIGWfPK+0nsrZCAPaDucGZYVL0lPQr3eF4jnmhi60dF > Sv9rD5Mq5ZSTTuJlpwoaxowxVp4M1tV1vmCdfBRkgoUD3dwS/snryr/AK69R > zdjjV/BABtcMNA7cMYIrkolGl37g4kI1alLfU36Uf/3M0NfUcw0keW1XuMOr > uV7AzXhZGw4eL4LJt7I7gXJs1kgE6/sPSmoKBVckKisrruiUSYmH9r/EhXXU > YB8cxd5tenMrchbjcOquC9X2JJjB++/LyJw3mFNIO5W3UpjqwtI8IGDo1Sxl > fM32FuAWVVDZsiBKXuRzsIO/iEPfgZFfTcoSJkD0Rt/Q6gJPZIuBmiUFaYfs > 9fzufNDuXdPKFEndSHfwdPMJwvk3XA5+xYzhkcQH+3FKOPmYXkvLolOC3j+r > ZtbgI421jDIahpVPbFtgUPB2dM3mw34B73wP5MIOHHxz22tVKe6PBOeihccK > mOr0r1tZHR+11aijYf+Nlhv3hpbpRoDbQ7pRkRyu53Od47p6itZAi60TFFIJ > bDw26wZRNRrEuYhriJUeM7ahvJNlcE6VaO0szUDL5g/x2Oa9jKMHPpsUF9pS > 9HbJWcnflxq0iU+sfdv7Eoxzv6zkXMTUsbpT2XjKcZZN5jd2rWV3JfiU6FiZ > jpqJBHzwGan9qKKswNKyDKhoa2jPdSYIbqQ= > =NbSI > -END PGP SIGNATURE- >
Re: SOLR upgrade
i generally will only upgrade every other release. since i started with 1.4, went to 3->5->7.X, and never EVER a .0 or an even .X release, On Fri, Feb 12, 2021 at 12:01 PM Ishan Chattopadhyaya < ichattopadhy...@gmail.com> wrote: > Just avoid 8.8.0 for the moment, until 8.8.1 is released. 8.7.x should be > fine. > > On Fri, Feb 12, 2021 at 10:28 PM Alessandro Benedetti < > a.benede...@sease.io> > wrote: > > > Hi, > > following up on Charlie's detailed response I would recommend carefully > > assess the code you are using to interact with Apache Solr (on top of the > > Solr changes themselves). > > Assuming you are using some sort of client, it's extremely important to > > fully understand both the syntax and semantic of each call. > > I saw a lot of "compiling ok" search-api migrations that were ok > > syntactically but doing a disaster from the semantic perspective (missing > > important parameters ect). > > > > In case you have plugins to maintain this would be even more complicated > > than just make them compile. > > > > Regards > > -- > > Alessandro Benedetti > > Apache Lucene/Solr Committer > > Director, R Software Engineer, Search Consultant > > www.sease.io > > > > > > On Tue, 9 Feb 2021 at 11:01, Charlie Hull < > ch...@opensourceconnections.com > > > > > wrote: > > > > > Hi Lulu, > > > > > > I'm afraid you're going to have to recognise that Solr 5.2.1 is very > > > out-of-date and the changes between this version and the current 8.x > > > releases are significant. A direct jump is I think the only sensible > > > option. > > > > > > Although you could take the current configuration and attempt to > upgrade > > > it to work with 8.x, I recommend that you should take the chance to > look > > > at your whole infrastructure (from data ingestion through to query > > > construction) and consider what needs upgrading/redesigning for both > > > performance and future-proofing. You shouldn't just attempt a > > > lift-and-shift of the current setup - some things just won't work and > > > some may lock you into future issues. If you're running at large scale > > > (I've talked to some people at the BL before and I know you have some > > > huge indexes there!) then a redesign may be necessary for scalability > > > reasons (cost and feasibility). You should also consider your skills > > > base and how the team can stay up to date with Solr changes and modern > > > search practice. > > > > > > Hope this helps - this is a common situation which I've seen many times > > > before, you're certainly not the oldest version of Solr running I've > > > seen recently either! > > > > > > best > > > > > > Charlie > > > > > > On 09/02/2021 01:14, Paul, Lulu wrote: > > > > Hi SOLR team, > > > > > > > > Please may I ask for advice regarding upgrading the SOLR version (our > > > project currently running on solr-5.2.1) to the latest version? > > > > What are the steps, breaking changes and potential issues ? Could > this > > > be done as an incremental version upgrade or a direct jump to the > newest > > > version? > > > > > > > > Much appreciate the advice, Thank you! > > > > > > > > Best Wishes > > > > Lulu > > > > > > > > > > > > > > > > > > ** > > > > Experience the British Library online at www.bl.uk<http://www.bl.uk/ > > > > > > The British Library's latest Annual Report and Accounts : > > > www.bl.uk/aboutus/annrep/index.html< > > > http://www.bl.uk/aboutus/annrep/index.html> > > > > Help the British Library conserve the world's knowledge. Adopt a > Book. > > > www.bl.uk/adoptabook<http://www.bl.uk/adoptabook> > > > > The Library's St Pancras site is WiFi - enabled > > > > > > > > > > * > > > > The information contained in this e-mail is confidential and may be > > > legally privileged. It is intended for the addressee(s) only. If you > are > > > not the intended recipient, please delete this e-mail and notify the > > > postmas...@bl.uk<mailto:pos
Re: SOLR upgrade
Just avoid 8.8.0 for the moment, until 8.8.1 is released. 8.7.x should be fine. On Fri, Feb 12, 2021 at 10:28 PM Alessandro Benedetti wrote: > Hi, > following up on Charlie's detailed response I would recommend carefully > assess the code you are using to interact with Apache Solr (on top of the > Solr changes themselves). > Assuming you are using some sort of client, it's extremely important to > fully understand both the syntax and semantic of each call. > I saw a lot of "compiling ok" search-api migrations that were ok > syntactically but doing a disaster from the semantic perspective (missing > important parameters ect). > > In case you have plugins to maintain this would be even more complicated > than just make them compile. > > Regards > ------ > Alessandro Benedetti > Apache Lucene/Solr Committer > Director, R Software Engineer, Search Consultant > www.sease.io > > > On Tue, 9 Feb 2021 at 11:01, Charlie Hull > > wrote: > > > Hi Lulu, > > > > I'm afraid you're going to have to recognise that Solr 5.2.1 is very > > out-of-date and the changes between this version and the current 8.x > > releases are significant. A direct jump is I think the only sensible > > option. > > > > Although you could take the current configuration and attempt to upgrade > > it to work with 8.x, I recommend that you should take the chance to look > > at your whole infrastructure (from data ingestion through to query > > construction) and consider what needs upgrading/redesigning for both > > performance and future-proofing. You shouldn't just attempt a > > lift-and-shift of the current setup - some things just won't work and > > some may lock you into future issues. If you're running at large scale > > (I've talked to some people at the BL before and I know you have some > > huge indexes there!) then a redesign may be necessary for scalability > > reasons (cost and feasibility). You should also consider your skills > > base and how the team can stay up to date with Solr changes and modern > > search practice. > > > > Hope this helps - this is a common situation which I've seen many times > > before, you're certainly not the oldest version of Solr running I've > > seen recently either! > > > > best > > > > Charlie > > > > On 09/02/2021 01:14, Paul, Lulu wrote: > > > Hi SOLR team, > > > > > > Please may I ask for advice regarding upgrading the SOLR version (our > > project currently running on solr-5.2.1) to the latest version? > > > What are the steps, breaking changes and potential issues ? Could this > > be done as an incremental version upgrade or a direct jump to the newest > > version? > > > > > > Much appreciate the advice, Thank you! > > > > > > Best Wishes > > > Lulu > > > > > > > > > > > > ** > > > Experience the British Library online at www.bl.uk<http://www.bl.uk/> > > > The British Library's latest Annual Report and Accounts : > > www.bl.uk/aboutus/annrep/index.html< > > http://www.bl.uk/aboutus/annrep/index.html> > > > Help the British Library conserve the world's knowledge. Adopt a Book. > > www.bl.uk/adoptabook<http://www.bl.uk/adoptabook> > > > The Library's St Pancras site is WiFi - enabled > > > > > > * > > > The information contained in this e-mail is confidential and may be > > legally privileged. It is intended for the addressee(s) only. If you are > > not the intended recipient, please delete this e-mail and notify the > > postmas...@bl.uk<mailto:postmas...@bl.uk> : The contents of this e-mail > > must not be disclosed or copied without the sender's consent. > > > The statements and opinions expressed in this message are those of the > > author and do not necessarily reflect those of the British Library. The > > British Library does not take any responsibility for the views of the > > author. > > > > > > * > > > Think before you print > > > > > > > -- > > Charlie Hull - Managing Consultant at OpenSource Connections Limited > > > > Founding member of The Search Network <https://thesearchnetwork.com/> > > and co-author of Searching the Enterprise > > <https://opensourceconnections.com/about-us/books-resources/> > > tel/fax: +44 (0)8700 118334 > > mobile: +44 (0)7767 825828 > > >
Re: SOLR upgrade
Hi, following up on Charlie's detailed response I would recommend carefully assess the code you are using to interact with Apache Solr (on top of the Solr changes themselves). Assuming you are using some sort of client, it's extremely important to fully understand both the syntax and semantic of each call. I saw a lot of "compiling ok" search-api migrations that were ok syntactically but doing a disaster from the semantic perspective (missing important parameters ect). In case you have plugins to maintain this would be even more complicated than just make them compile. Regards -- Alessandro Benedetti Apache Lucene/Solr Committer Director, R Software Engineer, Search Consultant www.sease.io On Tue, 9 Feb 2021 at 11:01, Charlie Hull wrote: > Hi Lulu, > > I'm afraid you're going to have to recognise that Solr 5.2.1 is very > out-of-date and the changes between this version and the current 8.x > releases are significant. A direct jump is I think the only sensible > option. > > Although you could take the current configuration and attempt to upgrade > it to work with 8.x, I recommend that you should take the chance to look > at your whole infrastructure (from data ingestion through to query > construction) and consider what needs upgrading/redesigning for both > performance and future-proofing. You shouldn't just attempt a > lift-and-shift of the current setup - some things just won't work and > some may lock you into future issues. If you're running at large scale > (I've talked to some people at the BL before and I know you have some > huge indexes there!) then a redesign may be necessary for scalability > reasons (cost and feasibility). You should also consider your skills > base and how the team can stay up to date with Solr changes and modern > search practice. > > Hope this helps - this is a common situation which I've seen many times > before, you're certainly not the oldest version of Solr running I've > seen recently either! > > best > > Charlie > > On 09/02/2021 01:14, Paul, Lulu wrote: > > Hi SOLR team, > > > > Please may I ask for advice regarding upgrading the SOLR version (our > project currently running on solr-5.2.1) to the latest version? > > What are the steps, breaking changes and potential issues ? Could this > be done as an incremental version upgrade or a direct jump to the newest > version? > > > > Much appreciate the advice, Thank you! > > > > Best Wishes > > Lulu > > > > > > > ** > > Experience the British Library online at www.bl.uk<http://www.bl.uk/> > > The British Library's latest Annual Report and Accounts : > www.bl.uk/aboutus/annrep/index.html< > http://www.bl.uk/aboutus/annrep/index.html> > > Help the British Library conserve the world's knowledge. Adopt a Book. > www.bl.uk/adoptabook<http://www.bl.uk/adoptabook> > > The Library's St Pancras site is WiFi - enabled > > > * > > The information contained in this e-mail is confidential and may be > legally privileged. It is intended for the addressee(s) only. If you are > not the intended recipient, please delete this e-mail and notify the > postmas...@bl.uk<mailto:postmas...@bl.uk> : The contents of this e-mail > must not be disclosed or copied without the sender's consent. > > The statements and opinions expressed in this message are those of the > author and do not necessarily reflect those of the British Library. The > British Library does not take any responsibility for the views of the > author. > > > * > > Think before you print > > > > -- > Charlie Hull - Managing Consultant at OpenSource Connections Limited > > Founding member of The Search Network <https://thesearchnetwork.com/> > and co-author of Searching the Enterprise > <https://opensourceconnections.com/about-us/books-resources/> > tel/fax: +44 (0)8700 118334 > mobile: +44 (0)7767 825828 >
Elevation in dataDir in Solr Cloud
Hi, I need to have the elevate.xml file updated frequently and I was wondering if it is possible to put this file in the dataDir folder when using Solr Cloud. I know that this is possible in the standalone mode, and I haven't seen in the documentation [1] that it can not be done in Cloud. I am using Solr 7.7.2 and ZooKeeper. After creating the techproducts collection for the tests, I remove the elevate.xml file from the configuration and I put it in the dataDir folder of the cores. When I update the collection with that configuration, I get the following error: "Can't find resource 'elevate.xml' in classpath or '/configs/techproductsConfExp'". Is this expected or I am doing something wrong? Thanks a lot for your help. [1] https://lucene.apache.org/solr/guide/7_7/the-query-elevation-component.html -- Disclaimer: This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error please notify the system manager. If you are not the named addressee you should not disseminate, distribute or copy this email. Please notify the sender immediately by email if you have received this email by mistake and delete this email from your system.
Re: Why Solr questions on stackoverflow get very few views and answers, if at all?
Re: Why Solr questions on stackoverflow get very few views and answers, if at all?
I answered quite a bunch a whole ago, as part of book writing process. I think a lot of them were missing core information like version of Solr. So they were not very timeless. The list allows a conversation and multiple perspectives, which is better than a one shot answer. Regards, Alex On Fri., Feb. 12, 2021, 5:37 a.m. ufuk yılmaz, wrote: > Is it because the main place for q is this mailing list, or somewhere > else that I don’t know? > > Or Solr isn’t ‘hot’ as some other topics? > > Sent from Mail for Windows 10 > >
Re: Why Solr questions on stackoverflow get very few views and answers, if at all?
I've answered a few in my time, but my experience is that if you do so you then get emailed a whole load more questions some of which aren't even relevant to Solr! Also, quite a few of them are 'here is 3 pages of code please debug it for me no I won't tell the actual error I got'. This is the best place to come, also there's the IRC channel, the new Slack gateway to this at https://s.apache.org/solr-slack and in our own Relevance Slack at http://opensourceconnections.com/slack there's a #solr channel (as well as many others on search & relevance topics). Solr is 'hot' (but not as hot as Elasticsearch), and search is still a niche business overall. HTH Cheers Charlie On 12/02/2021 10:37, ufuk yılmaz wrote: Is it because the main place for q is this mailing list, or somewhere else that I don’t know? Or Solr isn’t ‘hot’ as some other topics? Sent from Mail for Windows 10 -- Charlie Hull - Managing Consultant at OpenSource Connections Limited Founding member of The Search Network <https://thesearchnetwork.com/> and co-author of Searching the Enterprise <https://opensourceconnections.com/about-us/books-resources/> tel/fax: +44 (0)8700 118334 mobile: +44 (0)7767 825828
Why Solr questions on stackoverflow get very few views and answers, if at all?
Is it because the main place for q is this mailing list, or somewhere else that I don’t know? Or Solr isn’t ‘hot’ as some other topics? Sent from Mail for Windows 10
Re: Down Replica is elected as Leader (solr v8.7.0)
I haven’t delved into the exact reason for this, but what generally helps
to avoid this situation in a cluster is
i) During shutdown (in case you need to restart the cluster), let the
overseer node be the last one to shut down.
ii) While restarting, let the Overseer node be the first one to start
iii) Wait for 5-10 seconds between each subsequent node start
Hope this helps.
Best,
Rahul
On Thu, Feb 11, 2021 at 12:03 PM mmb1234 wrote:
> Hello,
>
> On reboot of one of the solr nodes in the cluster, we often see a
> collection's shards with
> 1. LEADER replica in DOWN state, and/or
> 2. shard with no LEADER
>
> Output from /solr/admin/collections?action=CLUSTERSTATUS is below.
>
> Even after 5 to 10 minutes, the collection often does not recover. Unclear
> why this is happening and what we can try to prevent or remedy it.
>
> ps: perReplicaState= true in solr v8.8.0 didn't work well because after a
> rebalance all replicas somehow get a "leader:true" status even though
> states.json looked ok.
>
> {
> "responseHeader": {
> "status": 0,
> "QTime": 2
> },
> "cluster": {
> "collections": {
> "datacore": {
> "pullReplicas": "0",
> "replicationFactor": "0",
> "shards": {
> "__": {
> "range": null,
> "state": "active",
> "replicas": {
> "core_node1": {
> "core": "datacore____replica_t187",
> "base_url": "http://solr-0.solr-headless:8983/solr;,
> "node_name": "solr-0.solr-headless:8983_solr",
> "state": "down",
> "type": "TLOG",
> "force_set_state": "false",
> "property.preferredleader": "true",
> "leader": "true"
> },
> "core_node2": {
> "core": "datacore___zzzz_replica_t188",
> "base_url": "http://solr-1.solr-headless:8983/solr;,
> "node_name": "solr-1.solr-headless:8983_solr",
> "state": "active",
> "type": "TLOG",
> "force_set_state": "false"
> },
> "core_node3": {
> "core": "datacore____replica_t189",
> "base_url": "http://solr-2.solr-headless:8983/solr;,
> "node_name": "solr-2.solr-headless:8983_solr",
> "state": "active",
> "type": "TLOG",
> "force_set_state": "false"
> }
> }
> },
> "__j": {
> "range": null,
> "state": "active",
> "replicas": {
> "core_node19": {
> "core": "datacore__hhhh_jjjjj_replica_t187",
> "base_url": "http://solr-0.solr-headless:8983/solr;,
> "node_name": "solr-0.solr-headless:8983_solr",
> "state": "down",
> "type": "TLOG",
> "force_set_state": "false",
> "property.preferredleader": "true"
> },
> "core_node20": {
> "core": "datacore_gggg_hhhh_j_replica_t188",
> "base_url": "http://solr-1.solr-headless:8983/solr;,
> "node_name": "solr-1.solr-headless:8983_solr",
> "state": "active",
> "type": "TLOG",
> "force_set_state": "false"
> },
> "core_node21": {
> "core": "datacore___j_replica_t189",
> "base_url": "http://solr-2.solr-headless:8983/solr;,
> "node_name": "solr-2.solr-headless:8983_solr",
> "state": "active",
> "type": "TLOG",
> "force_set_state": "false"
> }
> }
> },
> "__": {
> "range": null,
> "state": "active",
> "replicas": {
> "core_node4": {
> "core": "datacore____replica_t91",
> "base_url": "http://solr-0...
>
>
>
> --
> Sent from: https://lucene.472066.n3.nabble.com/Solr-User-f472068.html
>
Solr wiki page update
Hi community members, I work for Adelean https://www.adelean.com/ , we are offering services around everything Search related, and especially Solr consulting and support. We are based in Paris and operate mainly in France. Is it possible to list our company on the support page (Support - SOLR - Apache Software Foundation <https://cwiki.apache.org/confluence/display/SOLR/Support>) ? Or give me the permission to edit it on confluence (my user: vincent.brehin) ? Thanks ! Best Regards, Vincent
Re: Using multiple language stop words in Solr Core
Hell Abhay, Do not enable stopwords unless you absolutely know what you are doing. In general, it is a bad practice that somehow still lingers on. But to answer the question, you must have one field and fieldType for each language, so language specific filters go there. Also, using edismax and multi-language search using mm (minimum should match) with stopwords enabled spells trouble. Set up per language fieldTypes without stopwords. Regards, Markus Op do 11 feb. 2021 om 12:44 schreef Abhay Kumar < abhay.ku...@anjusoftware.com>: > Hello Team, > > > > Solr provides some data type out of box in managed schema for different > languages such as english, french, japanies etc. > > > > We are using common data type "text_general" for fields declaration and > using stopwards.txt for stopword filtering. > > > > autoGeneratePhraseQueries="true" positionIncrementGap="100" > multiValued="true"> > > > > > >ignoreCase="true"/> > > > >minGramSize="1"/> > > > > > > > >ignoreCase="true"/> > >ignoreCase="true" synonyms="synonyms.txt"/> > > > > > > > > > > While syncing data to Solr core we are importing different languages text > in the fields such as french, english, german etc. > > > > My query is shall we use all different language stopwords into same > "stopwards.txt" file or how solr use different language stopwords? > > > > > > > > *Warm Regards,* > > > > *Abhay Kumar* | Lead Developer > > 401/402, Pride Portal, Shivaji Housing Society, Off. S. B. Road | Shivaji > Nagar, Pune-411 016 > +91 20 2563 1011 | Mobile: +91 9096644108 > anjusoftware.com > > <https://anjusoftware.com/> > <https://www.linkedin.com/company/anju-software/> > <https://www.facebook.com/Anju-Software-1415613681916676/> > <https://twitter.com/AnjuSoftware> > > > > > > > > *Confidentiality Notice This email message, including > any attachments, is for the sole use of the intended recipient and may > contain confidential and privileged information. Any unauthorized view, > use, disclosure or distribution is prohibited. If you are not the intended > recipient, please contact the sender by reply email and destroy all copies > of the original message. Anju Software, Inc. 4500 S. Lakeshore Drive, Suite > 620, Tempe, AZ USA 85282.* >
Using multiple language stop words in Solr Core
Hello Team, Solr provides some data type out of box in managed schema for different languages such as english, french, japanies etc. We are using common data type "text_general" for fields declaration and using stopwards.txt for stopword filtering. While syncing data to Solr core we are importing different languages text in the fields such as french, english, german etc. My query is shall we use all different language stopwords into same "stopwards.txt" file or how solr use different language stopwords? Warm Regards, Abhay Kumar | Lead Developer 401/402, Pride Portal, Shivaji Housing Society, Off. S. B. Road | Shivaji Nagar, Pune-411 016 +91 20 2563 1011 | Mobile: +91 9096644108 anjusoftware.com<https://anjusoftware.com/> [cid:image001.png@01D70099.4ACD8C20]<https://anjusoftware.com/>[cid:image002.png@01D70099.4ACD8C20]<https://www.linkedin.com/company/anju-software/>[cid:image003.png@01D70099.4ACD8C20]<https://www.facebook.com/Anju-Software-1415613681916676/>[cid:image004.png@01D70099.4ACD8C20]<https://twitter.com/AnjuSoftware> Confidentiality Notice This email message, including any attachments, is for the sole use of the intended recipient and may contain confidential and privileged information. Any unauthorized view, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message. Anju Software, Inc. 4500 S. Lakeshore Drive, Suite 620, Tempe, AZ USA 85282.
Down Replica is elected as Leader (solr v8.7.0)
Hello,
On reboot of one of the solr nodes in the cluster, we often see a
collection's shards with
1. LEADER replica in DOWN state, and/or
2. shard with no LEADER
Output from /solr/admin/collections?action=CLUSTERSTATUS is below.
Even after 5 to 10 minutes, the collection often does not recover. Unclear
why this is happening and what we can try to prevent or remedy it.
ps: perReplicaState= true in solr v8.8.0 didn't work well because after a
rebalance all replicas somehow get a "leader:true" status even though
states.json looked ok.
{
"responseHeader": {
"status": 0,
"QTime": 2
},
"cluster": {
"collections": {
"datacore": {
"pullReplicas": "0",
"replicationFactor": "0",
"shards": {
"__": {
"range": null,
"state": "active",
"replicas": {
"core_node1": {
"core": "datacore____replica_t187",
"base_url": "http://solr-0.solr-headless:8983/solr;,
"node_name": "solr-0.solr-headless:8983_solr",
"state": "down",
"type": "TLOG",
"force_set_state": "false",
"property.preferredleader": "true",
"leader": "true"
},
"core_node2": {
"core": "datacore____replica_t188",
"base_url": "http://solr-1.solr-headless:8983/solr;,
"node_name": "solr-1.solr-headless:8983_solr",
"state": "active",
"type": "TLOG",
"force_set_state": "false"
},
"core_node3": {
"core": "datacore____replica_t189",
"base_url": "http://solr-2.solr-headless:8983/solr;,
"node_name": "solr-2.solr-headless:8983_solr",
"state": "active",
"type": "TLOG",
"force_set_state": "false"
}
}
},
"__j": {
"range": null,
"state": "active",
"replicas": {
"core_node19": {
"core": "datacore___j_replica_t187",
"base_url": "http://solr-0.solr-headless:8983/solr;,
"node_name": "solr-0.solr-headless:8983_solr",
"state": "down",
"type": "TLOG",
"force_set_state": "false",
"property.preferredleader": "true"
},
"core_node20": {
"core": "datacore___j_replica_t188",
"base_url": "http://solr-1.solr-headless:8983/solr;,
"node_name": "solr-1.solr-headless:8983_solr",
"state": "active",
"type": "TLOG",
"force_set_state": "false"
},
"core_node21": {
"core": "datacore___j_replica_t189",
"base_url": "http://solr-2.solr-headless:8983/solr;,
"node_name": "solr-2.solr-headless:8983_solr",
"state": "active",
"type": "TLOG",
"force_set_state": "false"
}
}
},
"__": {
"range": null,
"state": "active",
"replicas": {
"core_node4": {
"core": "datacore____replica_t91",
"base_url": "http://solr-0...
--
Sent from: https://lucene.472066.n3.nabble.com/Solr-User-f472068.html
