Re: Overriding Sort and boosting some docs to the top

t a dozen of documents for a given query > and it works just fine. > > I also do not expect the function query variant to be more performant, but > i am not sure. If it were, would it be measurable? > > Regards, > Markus > > Op wo 24 feb. 2021 om 12:15 schreef Mark Robinson

Re: Overriding Sort and boosting some docs to the top

, Feb 24, 2021 at 4:41 PM Markus Jelsma wrote: > Hello, > > You are probably looking for the elevator component, check it out: > https://lucene.apache.org/solr/guide/8_8/the-query-elevation-component.html > > Regards, > Markus > > Op wo 24 feb. 2021 om 11:59 schreef

Overriding Sort and boosting some docs to the top

Hi, I wanted to sort and then boost some docs to the top and these docs should be my first set in the results and the following ones appearing according to my sort criteria. I understand that sort overrides bq hence bq may not be used in this case - I brought my boost into sort using "query()"

Re: Avoiding single digit and single charcater ONLY query by putting them in stopwords list

gt;> On Tue, Oct 27, 2020 at 11:45 AM Konstantinos Koukouvis < > >> konstantinos.koukou...@mecenat.com> wrote: > >> > >> Oh hi Mark! > >> > >> Why would you wanna do such a thing in the solr end. Imho it would be > much > >> more clean and e

Re: Avoiding single digit and single charcater ONLY query by putting them in stopwords list

ld you wanna do such a thing in the solr end. Imho it would be much > more clean and easy to do it on the client side > > Regards, > Konstantinos > > > > On 27 Oct 2020, at 16:42, Mark Robinson wrote: > > > > Hello, > > > > I want to block queries having

Avoiding single digit and single charcater ONLY query by putting them in stopwords list

Hello, I want to block queries having only a digit like "1" or "2" ,... or just a letter like "a" or "b" ... Is it a good idea to block them ... ie just single digits 0 - 9 and a - z by putting them as a stop word? The problem with this I can anticipate is a query like "1 inch screw" can

ElevateIds - should I remove those that might be filtered off in the underlying query

Hi, Suppose I have say 50 ElevateIds and I have a way to identify those that would get filtered out in the query by predefined fqs. So they would in reality never be even in the results and hence never be elevated. Is there any advantage if I avoid passing them in the elevateIds at the time of

Re: "timeAllowed" param with "numFound" having a count value but doc list is empty

from our own server since we don't want to serve incomplete > results ever. But you could attempt to return whatever results you do > receive, perhaps with a warning message for your client indicating what > happened. > > > On Wed, 16 Sep 2020 at 02:05, Mark Robinson > wrote: &

Re: "timeAllowed" param with "numFound" having a count value but doc list is empty

jira/browse/SOLR-9882 > > - Bram > > On 16/09/2020 03:04, Mark Robinson wrote: > > Thanks Dominique! > > So is this parameter generally recommended or not. I wanted to try with a > > value of 10s. We are not using it now. > > My goal is to prevent a query from runnin

Re: "timeAllowed" param with "numFound" having a count value but doc list is empty

at 5:38 PM Dominique Bejean wrote: > Hi, > > 1. Yes, your analysis is correct > > 2. Yes, it can occurs too with very slow query. > > Regards > > Dominique > > Le mar. 15 sept. 2020 à 15:14, Mark Robinson a > écrit : > > > Hi, > > > > When

"timeAllowed" param with "numFound" having a count value but doc list is empty

Hi, When in a sample query I used "timeAllowed" as low as 10mS, I got value for "numFound" as say 2000, but no docs were returned. But when I increased the value for timeAllowed to be in seconds, never got this scenario. I have 2 qns:- 1. Why does numFound have a value like say 2000 or even 6000

Re: What is the Best way to block certain types of queries/ query patterns in Solr?

r > http://www.linkedin.com/in/davidwsmiley > > > On Mon, Aug 31, 2020 at 10:57 AM Mark Robinson > wrote: > > > Hi, > > I had come across a mail (Oct, 2019 one) which suggested the best way is > to > > handle it before it reaches Solr. I was curious whether:- &g

What is the Best way to block certain types of queries/ query patterns in Solr?

Hi, I had come across a mail (Oct, 2019 one) which suggested the best way is to handle it before it reaches Solr. I was curious whether:- 1. Jetty query filter can be used (came across something like that,, need to check) 2. Any new features in Solr itself (like in a request handler...or

Re: HttpShardHandlerFactory

uests": > > https://lucene.apache.org/solr/guide/8_1/distributed-requests.html#configuring-the-shardhandlerfactory > > Michael > > On Fri, Aug 16, 2019 at 9:40 AM Shawn Heisey wrote: > > > On 8/16/2019 3:51 AM, Mark Robinson wrote: > > > I am trying to understan

Re: HttpShardHandlerFactory

Hello Shawn, Thank you so much for the detailed response. It was so helpful! Thanks! Mark. On Fri, Aug 16, 2019 at 9:40 AM Shawn Heisey wrote: > On 8/16/2019 3:51 AM, Mark Robinson wrote: > > I am trying to understand the socket time out and connection

HttpShardHandlerFactory

Hello, I am trying to understand the socket time out and connection time out in the HttpShardHandlerFactory:- 10 20 1.Could some one please help me understand the effect of using such low values of 10 ms and 20ms as given above inside my /select

Re: Solr restricting time-consuming/heavy processing queries

ning > faceting using timeAllowed, but there are other ways to optimize faceting, > such as using jsonFacet, threaded execution etc. > > -- > Jan Høydahl, search solution architect > Cominvent AS - www.cominvent.com > > 12. aug. 2019 kl. 23:10 skrev Mark Robinson : > > Hi

Re: Solr restricting time-consuming/heavy processing queries

ndle normal expected load > instead of trying to give users timeout exceptions. What kind of query > times do you experience that are above 1s and are these not important > enough to invest extra HW? Trying to understand the real reason behind your > questions. > > Jan Høyd

Re: Solr restricting time-consuming/heavy processing queries

curious to know different ways if any n SOLR to restrict/ limit a time consuming query from processing for a long time. Thanks! Mark On Fri, Aug 9, 2019 at 2:15 PM Mark Robinson wrote: > > Hello, > I have the following questions please:- > > In solrconfig.xml I created a new "/se

Solr restricting time-consuming/heavy processing queries

Hello, I have the following questions please:- In solrconfig.xml I created a new "/selecttimeout" handler copying "/select" handler and added the following to my new "/selecttimeout":- 10 20 1. Does the above mean that if I dont get a request once in 10ms on the socket handling the

Re: UIMA-SOLR integration

repos/asf?p=lucene-solr.git;a=blob;f=solr/contrib/ > analysis-extras/src/java/org/apache/solr/update/processor/ > OpenNLPExtractNamedEntitiesUpdateProcessorFactory.java;hb= > refs/heads/branch_7_3#l60> > > -- > Steve > www.lucidworks.com > > > On Mar 29, 2018, a

Fwd: UIMA-SOLR integration

Hi All, Is it still advisable to pursue UIMA or can some one pls advise something else to check on related to SOLR and NLP? Thanks! Mark -- Forwarded message -- From: Mark Robinson <mark123lea...@gmail.com> Date: Wed, Mar 28, 2018 at 2:21 PM Subject: UIMA-SOLR integ

UIMA-SOLR integration

Hi, I was trying to integrate UIMA into SOLR following the solr docs and many other hints on the net. While trying to get a VALID_ALCHEMYAPI_KEY I contacted IBM support and got the following advice:- "As announced a year a go the Alchemy Service was scheduled and shutdown on March 7th, 2018, and

Re: Reading a prameter set in browser in request handler

; > Newsletter and resources for Solr beginners and intermediates: > http://www.solr-start.com/ > > > On 2 October 2016 at 23:19, Mark Robinson <mark123lea...@gmail.com> wrote: > > Thanks Alex for the reply. > > > > Yes. in this context I want to determine the wea

Re: Reading a prameter set in browser in request handler

> business terms. E.g. 'weather' term needs to feed into something else > in query I am guessing. > > Regards, > Alex. > > Newsletter and resources for Solr beginners and intermediates: > http://www.solr-start.com/ > > > On 2 October 2016 at 22:29, Mark Robinson &

Re: Reading a prameter set in browser in request handler

country weather_${country} ...but it is not working. Even tried:- weather_$country I am trying to set my weather field based on the *country *passed. Any suggestionsis is highly appreciated. Thanks! On Sun, Oct 2, 2016 at 11:29 AM, Mark Robinson &

Reading a prameter set in browser in request handler

Hi, I pass a parameter *=USA* from the browser as part of my query. (I have fields *weather_USA*, *weather_UK* etc... in my document). How do I retrieve the parameter "country" in my requesthandler. I tried in my requesthandler:- storeId weather_${country} ...but

Re: Load a java class on start up

gt; On 2 Jul 2016 11:53, "Mark Robinson" <mark123lea...@gmail.com> wrote: > > > Thanks much Andrea esp. for the suggestion of SolrCoreAware! > > > > > > > > Best, > > Mark. > > > > On Thu, Jun 30, 2016 at 10:23 AM, Andrea Gazzari

Re: Load a java class on start up

://lucene.apache.org/solr/6_1_0/solr-core/org/apache/solr/handler/component/SearchComponent.html > [2] https://wiki.apache.org/solr/SolrPlugins#SolrCoreAware > > > On 30/06/16 16:00, Mark Robinson wrote: > >> Hi, >> >> I have a java OBJECT which I need to load once. >>

Load a java class on start up

Hi, I have a java OBJECT which I need to load once. I have written a java custom component, which I have added in "last-components" in solrconfig.xml, from which I want to access the above mentioned OBJECT when each search request comes in. Is there a way I can load a java object on server/

Accessing response docs in process method

Hi, I would like to check the response for the *authors *data that comes in my multiValued *authors* field and do some activity related to it before the output is send back. I know to access the facets and investigate it. Could some one pls advise (the apis/ methods etc) on how I can get

Re: Add a new field dynamically to each of the result docs and sort on it

Thanks Charlie! I will check this and try it out. Best, Mark. On Wed, Jun 1, 2016 at 7:00 AM, Charlie Hull <char...@flax.co.uk> wrote: > On 01/06/2016 11:56, Mark Robinson wrote: > >> Just to complete my prev use case, in case no direct way is possible in >>

Re: Sorting documents in one core based on a field in another core

3.html > ? > > On Tue, May 31, 2016 at 5:41 PM, Mark Robinson <mark123lea...@gmail.com> > wrote: > > > Hi, > > > > I have a requirement to sort records in one core/ collection based on a > > field in > > another core/collection. > > > >

Re: Add a new field dynamically to each of the result docs and sort on it

Thanks for the reply Hoss! Let me do a quick explanation to the sort by tagValue. (Actually I quickly added this part in a different mail when I found I missed it in this mail..) That is where the dynamic input parameter comes in. The input will specify for which local outlet (outlet id passed)

Re: Add a new field dynamically to each of the result docs and sort on it

ny initial sort was applied and can we re-sort at this very late stage using some java sorting in the custom component. Thanks! Mark. On Wed, Jun 1, 2016 at 6:44 AM, Mark Robinson <mark123lea...@gmail.com> wrote: > Thanks much Eric and Hoss! > > Let me try to detail. >

Re: Add a new field dynamically to each of the result docs and sort on it

Thanks much Eric and Hoss! Let me try to detail. We have our "product" core with a couple of million docs. We have a couple of thousand outlets where the products get sold. Each product can have a different *tagValue* in each outlet. Our "product_tag" core (around 2M times 2000 records), captures

Re: Add a new field dynamically to each of the result docs and sort on it

mal > sort rules apply and you can specify sortMisingFirst/Last > to tell Solr where other docs without that field shold go. > > Normal sort rules are '=field1 asc,field2 desc' etc. > > Best, > Erick > > On Tue, May 31, 2016 at 7:53 AM, Mark Robinson <mark123lea...@gmail

Re: Sorting documents in one core based on a field in another core

sn't work like that, which is why it's referred > to as "pseudo join". There's no way that I know of > to do what you want here. > > I'd strongly recommend you flatten your data at index time. > > Best, > Erick > > On Tue, May 31, 2016 at 7:41 AM, Mark Robi

Add a new field dynamically to each of the result docs and sort on it

Hi, My core does not have a field say *fieldnew*. *Case 1:-* But in my results I would like to have *fieldnew *also and my results should be sorted on only this new field. *Case 2:-* Just adding one more case further. Suppose I have other fields also in the sort criteria and *fieldnew *is one

Sorting documents in one core based on a field in another core

Hi, I have a requirement to sort records in one core/ collection based on a field in another core/collection. Could some one please advise how it can be done in SOLR. I have used !join to restrict documents in one core based on field values in another core. Is there some way to sort like that?

Re: Atomic updates and "stored"

> be what you really need. > > Best, > Erick > > On Mon, May 23, 2016 at 6:13 AM, Mark Robinson <mark123lea...@gmail.com> > wrote: > > Hi, > > > > I have some 150 fields in my schema out of which about 100 are dynamic > > fields which I am not storing

Atomic updates and "stored"

Hi, I have some 150 fields in my schema out of which about 100 are dynamic fields which I am not storing (stored="false"). In case I need to do an atomic update to one or two fields which belong to the stored list of fields, do I need to change my dynamic fields (100 or so now not "stored") to

Re: SOLR edismax and mm request parameter

> > You can also play around with changing q.op parameter to 'AND' > > > > > > > > On 4 May 2016 at 11:40, Mark Robinson <mark123lea...@gmail.com> wrote: > > > >> Thanks much Ahmet! > >> > >> I will try that out. > >> &g

Re: SOLR edismax and mm request parameter

qf='field2' mm='100%' v=$qq} > OR > _query_:{!dismax qf='field3' mm='100%' v=$qq} > > > > https://cwiki.apache.org/confluence/display/solr/Local+Parameters+in+Queries > > Ahmet > > > > On Wednesday, May 4, 2016 4:59 AM, Mark Robinson <mark123lea...@gmail.com> &g

Re: SOLR edismax and mm request parameter

Hi, On further checking cld identify that *blue *is indeed appearing in one of the qf fields.My bad! Cld someone pls help me with the 2nd question. Thanks! Mark. On Tue, May 3, 2016 at 8:03 PM, Mark Robinson <mark123lea...@gmail.com> wrote: > Hi, > > I made a typo err in the p

Re: SOLR edismax and mm request parameter

ne or more of the qf fields? Cld some one pls help. Thanks! Mark On Tue, May 3, 2016 at 6:28 PM, Mark Robinson <mark123lea...@gmail.com> wrote: > Hi, > > 1. > My understanding of the mm parameter related to edismax is that, > if mm=100%, only if ALL my query terms appear across an

Re: Phrases and edismax

> > Erick > > On Mon, May 2, 2016 at 8:05 PM, Mark Robinson <mark123lea...@gmail.com> > wrote: > > Hi Eric, > > > > I have raised a JIRA:- *KYLIN-1644* with the problem mentioned. > > > > Thanks! > > Mark. > > > > On Sun, May 1

SOLR edismax and mm request parameter

Hi, 1. My understanding of the mm parameter related to edismax is that, if mm=100%, only if ALL my query terms appear across any of the qf fields will I get back documents ... ie all the terms *need not be present in one single field* .. they just need to be present across any of the fields in

Re: Phrases and edismax

Hi Eric, I have raised a JIRA:- *KYLIN-1644* with the problem mentioned. Thanks! Mark. On Sun, May 1, 2016 at 5:25 PM, Mark Robinson <mark123lea...@gmail.com> wrote: > Thanks much Eric for checking in detail. > Yes I found the first term being left out in pf. > Because of

Re: Phrases and edismax

> "+(name:erick name:men name:truck) (features:"men truck")", > > Confusingly, the terms (with "erick" left out, strike 1) > goes against the pf field even though it's fully qualified against the > name field. Not entirely sure whether this is intended o

Re: Decide on facets from results

for another > expression. You can even do this across collections. > > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=62693238 > > This is slated for Solr 6.1 > > > > Joel Bernstein > http://joelsolr.blogspot.com/ > > On Fri, Apr 29, 2016 at 10:38 AM,

Phrases and edismax

Hi, q=productType:(two piece bathtub white) =edismax=productType^20.0=productType^15.0 In the debug section this is what I see:- (+(productType:two productType:piec productType:bathtub productType:white) DisjunctionMaxQuery((productType:"piec bathtub white"^20.0)))/no_coord My question is

Re: Decide on facets from results

then include the > facets that you need in the second pass. > > > > On Apr 27, 2016, at 1:21 PM, Mark Robinson <mark123lea...@gmail.com> > wrote: > > > > Thanks Eric! > > So that will mean another call will be definitely required to SOLR with > the > > face

Re: Decide on facets from results

dworks.com <http://www.lucidworks.com/> > > > > > On Apr 27, 2016, at 12:13 PM, Mark Robinson <mark123lea...@gmail.com> > wrote: > > > > Hi, > > > > If I don't have my facet list at query time, from the results can I > select > > some fi

Decide on facets from results

Hi, If I don't have my facet list at query time, from the results can I select some fields and by any means create a facet on them? ie after I get the results I want to identify some fields as facets and send back facets for them in the response. A kind of very dynamic faceting based on the

Re: Indexing 700 docs per second

Thank you all for your very valuable suggestions. I will try out the options shared once our set up is ready and probably get back on my experience once it is done. Thanks! Mark. On Wed, Apr 20, 2016 at 9:54 AM, Bram Van Dam wrote: > > I have a requirement to index

Indexing 700 docs per second

Hi, I have a requirement to index (mainly updation) 700 docs per second. Suppose I have a 128GB RAM, 32 CPU machine, with each doc size around 260 byes (6 fields out of which only 2 will undergo updation at the above rate). This collection has around 122Million docs and that count is pretty much

Re: Not seeing the tokenized values when using solr.PathHierarchyTokenizerFactory

a verbatim copy of the original input. > > Best, > Erick > > On Mon, Apr 18, 2016 at 12:51 PM, Mark Robinson <mark123lea...@gmail.com> > wrote: > > Hi, > > > > I was using the solr.PathHierarchyTokenizerFactory for a field say > fieldB. > > An input data li

Not seeing the tokenized values when using solr.PathHierarchyTokenizerFactory

Hi, I was using the solr.PathHierarchyTokenizerFactory for a field say fieldB. An input data like A/B/C when I check using the ANALYSIS facility in the admin UI, is tokenized as A, A/B, A/B/C in fieldB. A/B/C in my system is a "string" value in a fieldA which is both indexed=stored=true. I

Which query to prefer

Hi, I have a 125 million doc index1. I identified 25 values for fieldA in the index. Each value can appear multiple times (1). There is another fieldB in the same index. I identified 6 values for this fieldB. I want only those records in (1) which contain any of these values in fieldB. Query:-

Querying through SolrJ taking lot of time

Hi, I am running the following query on an index that has around 123 million records, using SolrJ.. Each record has only 5 fields. String *qry*="( fieldA:(value1 OR value2 OR value24) AND fieldB:(value1 OR value2 OR value3 OR value4 OR value5) ) (...basically a simple AND of 2 ORs) When I

Re: Capture facets and its values in output and modify them

to do is get the facet object from the solr response and > then > > work in it. > > I have done something similar with highlighting and will put that code > > here in some time so you know how to fetch the facet values. > > > > On Sat, 27 Feb 2016, 13:42 Mark Robinso

Capture facets and its values in output and modify them

Hi, I have a requirement to capture facet fields in the output and append an additional data to each of the facet values before the final output (along with the results as well as the facets and values) is send back so that a middle layer can use this additional value added. I read that a custom

Re: Retrieving 1000 records at a time

Thanks Shawn! Best, Mark. On Wed, Feb 17, 2016 at 7:48 PM, Shawn Heisey <apa...@elyograg.org> wrote: > On 2/17/2016 3:49 PM, Mark Robinson wrote: > > I have around 121 fields out of which 12 of them are indexed and almost > all > > 121 are stored. > > Average si

Re: Retrieving 1000 records at a time

Thanks Joel and Chris! I have around 121 fields out of which 12 of them are indexed and almost all 121 are stored. Average size of a doc is 10KB. I was checking for start=0, rows=1000. We were querying a Solr instance which was on another server and I think network lag might have come into the

Retrieving 1000 records at a time

Hi, I have a requirement where I need to retrieve 1 to 15000 records at a time from SOLR. With 20 or 100 records everything happens in milliseconds. When it goes to 1000, 1 it is taking more time... like even 30 seconds. Will Solr be able to return 1 records at a time in less than

Re: Solr architecture

er and impact upon your ability to succeed. > > I'd suggest that what you are undertaking can certainly be done, but is > a substantial project. > > Upayavira > > On Wed, Feb 10, 2016, at 09:48 PM, Mark Robinson wrote: > > Thanks everyone for your suggestions. > > Ba

Re: Solr architecture

at it, as Jack says probably in the order of >>> hundreds of machines... >>> >>> On 9 February 2016 at 19:00, Upayavira <u...@odoko.co.uk> wrote: >>> >>> Bear in mind that Lucene is optimised towards high read lower write. >>>> That is, i

Re: Solr architecture

, > Emir > > -- > Monitoring * Alerting * Anomaly Detection * Centralized Log Management > Solr & Elasticsearch Support * http://sematext.com/ > > > > On 10.02.2016 03:22, Mark Robinson wrote: > >> Thanks for your replies and suggestions! >> >> Why I st

Re: Solr architecture

c). For more accurate sizing you may > >> also > >> > want to index say 10 million documents which may give you idea how > much > >> is > >> > your index size and then use that for extrapolation to come up with > >> memory > >> > req

Re: Solr architecture

ime to make reading > > efficient. It sounds like you are going to be doing far more writing > > than reading, and I wonder whether you are necessarily choosing the > > right tool for the job. > > > > How would you later use this data, and what advantage is there to

Solr architecture

Hi, We have a requirement where we would need to index around 2 Billion docs in a day. The queries against this indexed data set can be around 80K queries per second during peak time and during non peak hours around 12K queries per second. Can Solr realize this huge volumes. If so, assuming we

Dynamically Adding query parameters in my custom Request Handler class

Hi, When I initially fire a query against my Solr instance using SOLRJ I pass only, say q=*:*=(myfield:vaue1). I have written a custom RequestHandler, which is what I call in my SolrJ query. Inside this custom request handler can I add more query params like say the facets etc.. so that

Re: Dynamically Adding query parameters in my custom Request Handler class

Jan 9, 2016 at 12:08 PM, Ahmet Arslan <iori...@yahoo.com.invalid> > wrote: > > > Hi Mark, > > > > Yes this is possible. Better, you can use a custom SearchComponent for > > this task too. > > You retrieve solr parameters, wrap it into ModifiableSolrParams. Add

Re: Dynamically Adding query parameters in my custom Request Handler class

t? > > > — > Erik Hatcher, Senior Solutions Architect > http://www.lucidworks.com <http://www.lucidworks.com/> > > > > > On Jan 9, 2016, at 3:19 PM, Mark Robinson <mark123lea...@gmail.com> > wrote: > > > > Hi, > > > > Ahme