Solr TLOG Replicas going in recovery

2020-08-28 Thread amit3281
Hi, I am using Solr on EXT4 partition and have TLOG replicas in my collection. I am using 2 Solr nodes to utilize 2 disk (for getting IOPS) for same collection. My collection has 150 shards. Each shard size is ~9GB and 48Million docs per shard. My shards frequently goes into recovery with error

BasicAuth help

2020-08-28 Thread Vanalli, Ali A - DOT
Hello, Solr is running on windows machine and wondering if it possible to setup BasicAuth with the LDAP? Also, tried the example of Basic-Authentication that is published here but

Re: Exclude a folder/directory from indexing

2020-08-28 Thread Walter Underwood
For building a crawler, I’d start with Scrapy (https://scrapy.org ). It is a solid design and should be easy to use for crawling web pages, files, or an API. wunder Walter Underwood wun...@wunderwood.org http://observer.wunderwood.org/ (my blog) > On Aug 28, 2020, at 4:16

Re: Creating a phrase match feature in LTR

2020-08-28 Thread Dmitry Kan
Hi Krishan, What if you remove the query() wrapping? { "name": "phraseMatch", "class": "org.apache.solr.ltr.feature.SolrFeature", "params": { "q": "{!complexphrase inOrder=true}fieldName:${input}" }, "store": "_DEFAULT_" } or even: { "name": "phraseMatch", "class":

Re: Ranking issue when combining sorting and re-ranking on SolrCloud (multiple shards)

2020-08-28 Thread Dmitry Kan
Hi Jörg, Thanks for this link -- one of our search engineers started looking into this, because the issue with sorting in a federated setting concerns non-LTR based ranking as well. In particular, it becomes visible in cursor based pagination in collections that have shards with replicas. At any

Re: Ranking issue when combining sorting and re-ranking on SolrCloud (multiple shards)

2020-08-28 Thread Dmitry Kan
Hi Spyros, Thanks for sharing! This is certainly subject for a test, but I think that LTR plugin could be modified to rerank the documents on the merging node. For instance, if instead of solr cloud endpoint, you use a separate solr instance to route and aggregate the federated results, the

Re: Ranking issue when combining sorting and re-ranking on SolrCloud (multiple shards)

2020-08-28 Thread Spyros Kapnissis
Hi Dmitry, No, we were not able to solve the sorting/re-ranking issue. In the end we migrated the custom sorting formula to using the 'q' param instead of 'sort' to get back the results sorted by score as expected. That mostly solved our issues with inconsistent Solr scores. Maybe sorting and

Re: Online Solr password encryption tool for the Basic Authentication plugin

2020-08-28 Thread Clemente Biondo
Thank you Jan, your solution is also super easy, I was not aware of that, thanks for letting me know, it solves another use case for us. Yes, we use the REST API, but since we use solr as a docker image I feel unease to commit the initial password in the image. We came out with the following

Re: Ranking issue when combining sorting and re-ranking on SolrCloud (multiple shards)

2020-08-28 Thread Jörn Franke
Maybe this can help you? https://lucene.apache.org/solr/guide/7_5/distributed-requests.html#configuring-statscache-distributed-idf On Mon, May 11, 2020 at 9:24 AM Spyros Kapnissis wrote: > HI all, > > On our current master/slave setup (no cloud), we use a a custom sorting > function to get the

Re: Rerank for distributed requests

2020-08-28 Thread Dmitry Kan
Hi Arthur, I'm facing a similar issue with an LTR query over multiple collections in SolrCloud. The issue is that the documents returned and merged into a single page will have scores that don't look like sorted at all. For example (this is a single page of results): // collection1 -2.1818457

Re: Ranking issue when combining sorting and re-ranking on SolrCloud (multiple shards)

2020-08-28 Thread Dmitry Kan
Hi Spyros, Did you manage to solve this issue and if yes, can you please share your solution? On Mon, May 11, 2020 at 10:24 AM Spyros Kapnissis wrote: > HI all, > > On our current master/slave setup (no cloud), we use a a custom sorting > function to get the first pass results (using the sort

Re: Exclude a folder/directory from indexing

2020-08-28 Thread Joe Doupnik
    Some time ago I faced a roughly similar challenge. After many trials and tests I ended up creating my own programs to accomplish the tasks of fetching files, selecting which are allowed to be indexed, and feeding them into Solr (POST style). This work is open source, found on

Solr Schema API seems broken to me after 8.2.0

2020-08-28 Thread jeanc...@gmail.com
Hey everybody, First of all, I wanted to say that this is my first time writing here. I hope I don't do anything wrong. I went to create the "bug" ticket and saw it would be a good idea to first talk to some of you via IRC (didn't work for me or I did something wrong after 20 years of not using

How to configure "setExcludeCipherSuites" for org.eclipse.jetty.util.ssl.SslContextFactory$Client

2020-08-28 Thread Girish B C
Hello there: I am using the Solr 8.4.0. I am trying to figure out how I can set setExcludeCipherSuites to filter the unsecure Ciphers from the org.eclipse.jetty.util.ssl.SslContextFactory$Client. I am able to do the same with org.eclipse.jetty.util.ssl.SslContextFactory$Server by adding the

Re: Online Solr password encryption tool for the Basic Authentication plugin

2020-08-28 Thread Jan Høydahl
Cool, it’s even easier than my old Java tool: https://github.com/cominvent/solr-tools Also, I can recommend using the authenitcation REST API to add users instead of hardcoding. The API takes care of the encoding for you! Jan > 27. aug. 2020 kl.

Re: Apache Solr 8.6.0 with SSL

2020-08-28 Thread Jan Høydahl
Thanks for reporting back. I think perhaps we don’t claim to fully support FreeBSD. Feel free to submit a PullRequest though if you believe you have a working FreeBSD setup. Jan > 26. aug. 2020 kl. 14:23 skrev Patrik Peng : > > Followup regarding the bin/solr issue for anyone running Solr on

RE: Solr Logging In JSON Format

2020-08-28 Thread Gael Jourdan-Weil
Hi, Which version of Solr are you talking about? You should have a log4j.xml or log4j2.xml file in Solr resources that you can customize to your needs. At least that's the way we use to write JSON logs. Maybe there are other options that I don't know. Gaël, De : fidiv...@gmail.com Envoyé :