Re: Query in quotes cannot find results

2020-06-29 Thread Erick Erickson
Looks like you’re removing stopwords. Stopwords cause issues like this with the positions being off. It’s becoming more and more common to _NOT_ remove stopwords, is that an option? Best, Erick > On Jun 29, 2020, at 7:32 PM, Permakoff, Vadim > wrote: > > Hi Shawn, > Many thanks for the

RE: Query in quotes cannot find results

2020-06-29 Thread Permakoff, Vadim
Hi Shawn, Many thanks for the response, I checked the field and it is correct. Let's call it _text_ to make it easier. I believe the parsing is also correct, please see below: - Query without quotes (works): "querystring":"expand the methods", "parsedquery":"(PhraseQuery(_text_:\"blow

Re: Query in quotes cannot find results

2020-06-29 Thread Shawn Heisey
On 6/29/2020 3:34 PM, Permakoff, Vadim wrote: The basic query q=expand the methods <<< finds the document, the query (in quotes) q="expand the methods" <<< cannot find the document Am I doing something wrong, or is it known bug (I saw similar issues discussed in the past, but not for exact

Query in quotes cannot find results

2020-06-29 Thread Permakoff, Vadim
Hi, This might be known issue, but I cannot find a reference for this specific case - searching for exact query with synonyms and stopwords. I have a simple configuration for catch-all field:

Suggestion or recommendation for NRT

2020-06-29 Thread ramyogi
Hi, We are using SOLR 7.5.0 version, We are testing one collection for both Search and Index. Our collection created with below indexerconfig, We are using indexing process KAFKA connect plugin with every 5 min commit (cloud SOLRJ) as below https://github.com/jcustenborder/kafka-connect-solr

Re: Prefix + Suffix Wildcards in Searches

2020-06-29 Thread Mikhail Khludnev
Hello, Chris. I suppose index time analysis can yield these terms: "paid","ms-reply-unpaid","ms-reply-paid", and thus let you avoid these expensive wildcard queries. Here's why it's worth to avoid them https://www.slideshare.net/lucidworks/search-like-sql-mikhail-khludnev-epam On Mon, Jun 29,

Re: How to determine why solr stops running?

2020-06-29 Thread Erick Erickson
Really look at your cache size settings. This is to eliminate this scenario: - your cache sizes are very large - when you looked and the memory was 9G, you also had a lot of cache entries - there was a commit, which threw out the old cache and reduced your cache size This is frankly kind of

Re: How to determine why solr stops running?

2020-06-29 Thread Ryan W
On Mon, Jun 29, 2020 at 3:13 PM Erick Erickson wrote: > ps aux | grep solr > [solr@faspbsy0002 database-backups]$ ps aux | grep solr solr 72072 1.6 33.4 22847816 10966476 ? Sl 13:35 1:36 java -server -Xms16g -Xmx16g -XX:+UseG1GC -XX:+ParallelRefProcEnabled -XX:G1HeapRegionSize=8m

Re: How to determine why solr stops running?

2020-06-29 Thread Jörn Franke
Maybe you can identify in the logfiles some critical queries? What is the total size of the index? What client are you using on the web app side? Are you reusing clients or create one new for every query. > Am 29.06.2020 um 21:14 schrieb Ryan W : > > On Mon, Jun 29, 2020 at 1:49 PM David

Re: How to determine why solr stops running?

2020-06-29 Thread Ryan W
On Mon, Jun 29, 2020 at 1:49 PM David Hastings wrote: > little nit picky note here, use 31gb, never 32. Good to know. Just now I got this output from bin/solr status: "solr_home":"/opt/solr/server/solr", "version":"7.7.2 d4c30fc2856154f2c1fefc589eb7cd070a415b94 - janhoy - 2019-05-28

Re: How to determine why solr stops running?

2020-06-29 Thread Erick Erickson
ps aux | grep solr should show you all the parameters Solr is running with, as would the admin screen. You should see something like: -XX:OnOutOfMemoryError=your_solr_directory/bin/oom_solr.sh And there should be some logs laying around if that was the case similar to:

Re: How to determine why solr stops running?

2020-06-29 Thread David Hastings
little nit picky note here, use 31gb, never 32. On Mon, Jun 29, 2020 at 1:45 PM Ryan W wrote: > It figures it would happen again a couple hours after I suggested the issue > might be resolved. Just now, Solr stopped running. I cleared the cache in > my app a couple times around the time that

Re: How to determine why solr stops running?

2020-06-29 Thread Ryan W
It figures it would happen again a couple hours after I suggested the issue might be resolved. Just now, Solr stopped running. I cleared the cache in my app a couple times around the time that it happened, so perhaps that was somehow too taxing for the server. However, I've never allocated so

Re: How to determine why solr stops running?

2020-06-29 Thread Erick Erickson
The thing that’s unsettling about this is that assuming you were hitting OOMs, and were running the OOM-killer script, you _should_ have had very clear evidence that that was the cause. If you were not running the killer script, the apologies for not asking about that in the first place. Java’s

Re: Prefix + Suffix Wildcards in Searches

2020-06-29 Thread Erick Erickson
I was afraid of “totally arbitrary” OK, this field type is going to surprise the heck out of you. Whitespace tokenizer is really stupid. It’ll include punctuation for instance. Take a look at the admin UI/analysis page and pick your field and put some creative entries in and you’ll see what I

Re: How to determine why solr stops running?

2020-06-29 Thread David Hastings
sometimes just throwing money/ram/ssd at the problem is just the best answer. On Mon, Jun 29, 2020 at 11:38 AM Ryan W wrote: > Thanks everyone. Just to give an update on this issue, I bumped the RAM > available to Solr up to 16GB a couple weeks ago, and haven’t had any > problem since. > > > On

Re: Prefix + Suffix Wildcards in Searches

2020-06-29 Thread Chris Dempsey
First off, thanks for taking a look, Erick! I see you helping lots of folks out here and I've learned a lot from your answers. Much appreciated! > How regular are your patterns? Are they arbitrary? Good question. :) That's data that I should have included in the initial post but both the values

Re: How to determine why solr stops running?

2020-06-29 Thread Ryan W
Thanks everyone. Just to give an update on this issue, I bumped the RAM available to Solr up to 16GB a couple weeks ago, and haven’t had any problem since. On Tue, Jun 16, 2020 at 1:00 PM David Hastings wrote: > me personally, around 290gb. as much as we could shove into them > > On Tue, Jun

Re: [EXTERNAL] Getting rid of Master/Slave nomenclature in Solr

2020-06-29 Thread Bram Van Dam
On 28/06/2020 14:42, Erick Erickson wrote: > We need to draw a sharp distinction between standalone “going away” > in terms of our internal code and going away in terms of the user > experience. It'll be hard to make it completely transparant in terms of user experience. For instance, tere is

Re: Prefix + Suffix Wildcards in Searches

2020-06-29 Thread Erick Erickson
How regular are your patterns? Are they arbitrary? What I’m wondering is if you could shift your work the the indexing end, perhaps even in an auxiliary field. Could you, say, just index “paid”, “ms-reply-unpaid” etc? Then there are no wildcards at all. This akin to “concept search”. Otherwise

Re: [EXTERNAL] Getting rid of Master/Slave nomenclature in Solr

2020-06-29 Thread Mark H. Wood
Wandering off topic, but still apropos Solr. On Sun, Jun 28, 2020 at 12:14:56PM +0200, Ilan Ginzburg wrote: > I disagree Ishan. We shouldn't get rid of standalone mode. > I see three layers in Solr: > >1. Lucene (the actual search libraries) >2. The server infra ("standalone Solr"

Prefix + Suffix Wildcards in Searches

2020-06-29 Thread Chris Dempsey
Hello, all! I'm relatively new to Solr and Lucene (*using Solr 7.7.1*) but I'm looking into options for optimizing something like this: > fq=(tag:* -tag:*paid*) OR (tag:* -tag:*ms-reply-unpaid*) OR tag:*ms-reply-paid* It's probably not a surprise that we're seeing performance issues with

Announcing ApacheCon @Home 2020

2020-06-29 Thread Rich Bowen
Hi, Apache enthusiast! (You’re receiving this because you’re subscribed to one or more dev or user mailing lists for an Apache Software Foundation project.) The ApacheCon Planners and the Apache Software Foundation are pleased to announce that ApacheCon @Home will be held online, September

Re: [EXTERNAL] Getting rid of Master/Slave nomenclature in Solr

2020-06-29 Thread Jan Høydahl
Please start another thread to discuss removal of standalone mode, and stay on-topic in this one. > 28. jun. 2020 kl. 14:42 skrev Erick Erickson : > > We need to draw a sharp distinction between standalone “going away” > in terms of our internal code and going away in terms of the user >

Re: solrj - get metrics from all nodes

2020-06-29 Thread Jan Høydahl
The admin UI does this my requesting =,,… You will get a master response with each sub response as key:value pairs. The list of node_names can be found in live_nodes in CLUSTERSTATUS api. Jan > 27. jun. 2020 kl. 02:09 skrev ChienHuaWang : > > For people who is also looking for the solution -