Sort on docValue field is slow.

2019-05-20 Thread Ashwin Ramesh
Hello everybody, Hoping to get advice on a specific issue - We have a collection of 50M documents. We recently added a featuredAt field defined as such - This field is sparely populated such that only a small subset (3-5 thousand currently) have been tagged with that field. We have a business

Re: Sort on docValue field is slow.

2019-05-20 Thread Ashwin Ramesh
only difference. I am testing the raw URL call timing difference with and without the extra sort. Hope this helps, Regards, Ash On Mon, May 20, 2019 at 11:17 PM Shawn Heisey wrote: > On 5/20/2019 6:25 AM, Ashwin Ramesh wrote: > > Hoping to get advice on a specific issue - We have a coll

Are docValues useful for FilterQueries?

2019-07-08 Thread Ashwin Ramesh
Hi everybody, I can't find concrete evidence whether docValues are indeed useful for filter queries. One example of a field: This field will have a value between 0-1 The only usecase for this field is to filter on a range / subset of values. There will be no scoring / querying on this

Re: Is it possible to skip scoring completely?

2019-09-12 Thread Ashwin Ramesh
o these are just assumptions. > > Emir > -- > Monitoring - Log Management - Alerting - Anomaly Detection > Solr & Elasticsearch Consulting Support Training - http://sematext.com/ > > > > > On 12 Sep 2019, at 01:27, Ashwin Ramesh wrote: > > > > Hi everybody, >

Is it possible to skip scoring completely?

2019-09-11 Thread Ashwin Ramesh
Hi everybody, I was wondering if there is a way we can tell solr (7.3+) to run none of it's scoring logic. We would like to simply add a set of filter queries and order on a specific docValue field. e.g. "Give me all fq=color:red documents ORDER on popularityScore DESC" Thanks in advance, Ash

Re: Is it possible to skip scoring completely?

2019-09-12 Thread Ashwin Ramesh
Ah! Thanks so much! On Thu., 12 Sep. 2019, 11:56 pm Shawn Heisey, wrote: > On 9/12/2019 12:43 AM, Ashwin Ramesh wrote: > > I just tried a * query with filters with fl=id,score. I noticed that all > > scores were 1.0. Which I assume means no scoring was done. When I added a

Best field type for boosting all documents

2019-09-16 Thread Ashwin Ramesh
Hi everybody, We have a usecase where we want to push a popularity boost for each document in our collection. When a user searches for any term, we would like to arbitrarily add an additional boost by this value (which is different for each document). E.g. q=foo=def(popularityBoostField,1)

Re: Dealing with multi-word keywords and SOW=true

2019-09-30 Thread Ashwin Ramesh
am and adding > it explicitly? As keyword:”ice cream” > > Best, > Erick > > > On Sep 30, 2019, at 5:27 AM, Ashwin Ramesh wrote: > > > > Hi everybody, > > > > I am using the edismax parser and have noticed a very specific behaviour > > with how sow=tr

Dealing with multi-word keywords and SOW=true

2019-09-30 Thread Ashwin Ramesh
Hi everybody, I am using the edismax parser and have noticed a very specific behaviour with how sow=true (default) handles multiword keywords. We have a field called 'keywords', which uses the general KeywordTokenizerFactory. There are also other text fields like title and description. etc.

Re: Best Practises around relevance tuning per query

2020-02-26 Thread Ashwin Ramesh
uld describe the business > case in more detail without including the solution itself more people could > help. > > Eg it ie not clear why you have a scoring model and why this can address > business needs. > > > Am 18.02.2020 um 01:50 schrieb Ashwin Ramesh : > > >

Best Practises around relevance tuning per query

2020-02-17 Thread Ashwin Ramesh
Hi, We are in the process of applying a scoring model to our search results. In particular, we would like to add scores for documents per query and user context. For example, we want to have a score from 500 to 1 for the top 500 documents for the query “dog” for users who speak US English. We

Re: Best Practises around relevance tuning per query

2020-02-18 Thread Ashwin Ramesh
ping on this :) On Tue, Feb 18, 2020 at 11:50 AM Ashwin Ramesh wrote: > Hi, > > We are in the process of applying a scoring model to our search results. > In particular, we would like to add scores for documents per query and user > context. > > For example, we want to

Overseer & Backups - Questions

2020-03-10 Thread Ashwin Ramesh
Hi everybody, Quick Specs: - Solr 7.4 Solr Cloud - 30gb index on 8 shards Tlog/Pull We run daily backups on our 30gb index and noticed that the overseer does not process other jobs on it's task list while the backup is being taken. They remain on the pending list (in ZK). Is this expected? Also

LTR - FieldValueFeature Question

2020-04-24 Thread Ashwin Ramesh
Hi everybody, Do we need to have 'indexed=true' to be able to retrieve the value of a field via FieldValueFeature or is having docValue=true enough? Currently, we have some dynamic fields as [dynamicField=true, stored=false, indexed=false, docValue=true]. However when we noticing that the value

Solr 7.4 - LTR reranker not adhering by Elevate Plugin

2020-05-14 Thread Ashwin Ramesh
Hi everybody, We are running a query with both elevateIds=1,2,3 & a reranker phase using LTR plugin. We noticed that the results do not return in the expected order - per the elevateIds param. Example LTR rq param {!ltr.model=foo reRankDocs=250 efi.query=$q} When I used the standard reranker

Re: Overseer & Backups - Questions

2020-03-10 Thread Ashwin Ramesh
being flushed to segments though. Regards, Ash On Wed, Mar 11, 2020 at 3:18 AM Aroop Ganguly wrote: > May we know how you are invoking backups ? > > > On Mar 9, 2020, at 11:53 PM, Ashwin Ramesh > wrote: > > > > Hi everybody, > > > > Quick Specs: > >

Re: Overseer & Backups - Questions

2020-03-10 Thread Ashwin Ramesh
On Wed, Mar 11, 2020 at 9:21 AM Aroop Ganguly wrote: > Backups on hdfs ? > These should not be blocking if invoked asynchronously, are u doing them > async by passing the async flag? > > > On Mar 10, 2020, at 3:19 PM, Ashwin Ramesh > wrote: > > > > We use the

Re: Cannot add replica during backup

2020-08-11 Thread Ashwin Ramesh
tch to an ebs > disk for backups on a different project because efs couldn't keep up. > > > On Aug 10, 2020, at 9:43 PM, Ashwin Ramesh > wrote: > > > > Hey Aroop, the general process for our backup is: > > - Connect all machines to an EFS drive (AWS's NFS ser

Re: Backups in SolrCloud using snapshots of individual cores?

2020-08-10 Thread Ashwin Ramesh
I would love an answer to this too! On Fri, Aug 7, 2020 at 12:18 AM Bram Van Dam wrote: > Hey folks, > > Been reading up about the various ways of creating backups. The whole > "shared filesystem for Solrcloud backups"-thing is kind of a no-go in > our environment, so I've been looking for ways

Re: Cannot add replica during backup

2020-08-10 Thread Ashwin Ramesh
the > queue. > > It would also help to ensure your overseer is on a node with a role that > exempts it from any Solr index responsibilities. > > > > On Aug 10, 2020, at 6:43 PM, Ashwin Ramesh > wrote: > > > > Hey Aroop, the general process for our backup is

Re: Cannot add replica during backup

2020-08-10 Thread Ashwin Ramesh
t place and have > autoAddReplicas=true for indemnity? > > > On Aug 10, 2020, at 6:32 PM, Ashwin Ramesh > wrote: > > > > Hi everybody, > > > > We are using solr 7.6 (SolrCloud). We notices that when the backup is > > running, we cannot add any rep

Cannot add replica during backup

2020-08-10 Thread Ashwin Ramesh
Hi everybody, We are using solr 7.6 (SolrCloud). We notices that when the backup is running, we cannot add any replicas to the collection. By the looks of it, the job to add the replica is put into the Overseer queue, but it is not being processed. Is this expected? And are there any workarounds?

Solr 7.6.0 - OOM Caused Down Replica. Cannot recover. Please advice

2021-02-24 Thread Ashwin Ramesh
Hi everyone, We had an OOM event earlier this morning. This has caused one of our shards to lose all it's replicas and it's leader is still in a down state. We have restarted the Java process (solr) and it's still in a down state. Logs below: ``` Feb 25, 2021 @ 11:46:43.000 2021-02-25