Re: Custom sort function
My exact requirement is, I will have a new field for a set of documents
for sorting. So the number of fields is growing like 10k
fields(custom_sort1,custom_sort2.,etc), for 10k groups, each group grouping
a set of documents. Since the documents can reside in more than one group I
cannot use a single field.
So I tried to use a single field for each document in the format as
"custom_field:group1=1$group2=3". During query time I will filter all the
group1 documents and parse this field and return the respective sort
number.
My solr version is *7.1.0*.
*This is my custom Class:*
public class CustomSortValueSource extends ValueSource{
protected final boolean isasc;
protected final String s1;
protected final ValueSource source;
public CustomSort(ValueSource sources, String s1, String order ) {
if (sources == null || s1 ==null || order == null) {
throw new SolrException(SolrException.ErrorCode.BAD_REQUEST,
"One or more inputs missing ");
}
this.s1 = s1;
this.source = sources;
this.isasc = order.equals("asc") ? true : false;
}
@Override
public FunctionValues getValues(Map context, LeafReaderContext reader)
throws IOException {
final FunctionValues vals = source.getValues(context, reader);
return new FunctionValues() {
public float floatVal(int doc) {
float toRet = isasc ? Float.MAX_VALUE : Float.MIN_VALUE;
/*
My custom logic in reading a string array and returning a value out
of some condition checks
*/
}
}
}
}
There is nothing happening in the constructor other than assigning.
Thanks,
Sripradeep P
On Wed, Nov 27, 2019 at 2:30 PM Jörn Franke wrote:
> Maybe can you do it as part of the loading to calculate the score?
> Which Solr version are you using?
> Are you doing some heavily lifting into the constructor or your filter?
>
> Am 27.11.2019 um 09:34 schrieb Sripra deep :
>
>
> Hi Jörn Franke,
>
> I modified the custom function to just return a constant value as 1.0
> for all the docs and ran the load again, the latency is worst like more
> than 20sec. The filter I am using will fetch 15k documents (so this
> function is called 15k times). And if I don't call this function in my
> query then latency is less than 10ms.
>
> So it looks like some fundamental issue with this approach and I believe
> its been used in many business scenarios. Looking out for what's going
> wrong.
>
> Thanks,
> Sripradeep P
>
>
> On Wed, Nov 27, 2019 at 1:00 PM Jörn Franke wrote:
>
>> And have you tried how fast it is if you don’t do anything in this
>> method?
>>
>> > Am 27.11.2019 um 07:52 schrieb Sripra deep > >:
>> >
>> > Hi Team,
>> > I wrote a custom sort function that will read the field value and parse
>> > and returns a float value that will be used for sorting. this field is
>> > indexed, stored and docvalues enabled. my latency increases drastically
>> > from 10ms when the filter loads 50 documents and 200ms when it loads 250
>> > documents and 1sec when it loads 1000 documents.
>> >
>> > This is my function:
>> >
>> > @Override
>> > public FunctionValues getValues(Map context, LeafReaderContext reader)
>> > throws IOException {
>> > final FunctionValues vals = source.getValues(context, reader);
>> > return new FunctionValues() {
>> >public float floatVal(int doc) {
>> >float toRet = isasc ? Float.MAX_VALUE : Float.MIN_VALUE;
>> >/*
>> >My custom logic in reading a string array and returning a value
>> out
>> > of some condition checks
>> >*/
>> >}
>> >}
>> > }
>> >
>> > Can you suggest me some suggestions on this ?
>> >
>> > Thanks,
>> > Sripradeep P
>>
>
Re: Custom sort function
Maybe can you do it as part of the loading to calculate the score?
Which Solr version are you using?
Are you doing some heavily lifting into the constructor or your filter?
> Am 27.11.2019 um 09:34 schrieb Sripra deep :
>
>
> Hi Jörn Franke,
>
> I modified the custom function to just return a constant value as 1.0 for
> all the docs and ran the load again, the latency is worst like more than
> 20sec. The filter I am using will fetch 15k documents (so this function is
> called 15k times). And if I don't call this function in my query then latency
> is less than 10ms.
>
> So it looks like some fundamental issue with this approach and I believe its
> been used in many business scenarios. Looking out for what's going wrong.
>
> Thanks,
> Sripradeep P
>
>
>> On Wed, Nov 27, 2019 at 1:00 PM Jörn Franke wrote:
>> And have you tried how fast it is if you don’t do anything in this method?
>>
>> > Am 27.11.2019 um 07:52 schrieb Sripra deep :
>> >
>> > Hi Team,
>> > I wrote a custom sort function that will read the field value and parse
>> > and returns a float value that will be used for sorting. this field is
>> > indexed, stored and docvalues enabled. my latency increases drastically
>> > from 10ms when the filter loads 50 documents and 200ms when it loads 250
>> > documents and 1sec when it loads 1000 documents.
>> >
>> > This is my function:
>> >
>> > @Override
>> > public FunctionValues getValues(Map context, LeafReaderContext reader)
>> > throws IOException {
>> > final FunctionValues vals = source.getValues(context, reader);
>> > return new FunctionValues() {
>> >public float floatVal(int doc) {
>> >float toRet = isasc ? Float.MAX_VALUE : Float.MIN_VALUE;
>> >/*
>> >My custom logic in reading a string array and returning a value out
>> > of some condition checks
>> >*/
>> >}
>> >}
>> > }
>> >
>> > Can you suggest me some suggestions on this ?
>> >
>> > Thanks,
>> > Sripradeep P
Re: Custom sort function
Hi Jörn Franke,
I modified the custom function to just return a constant value as 1.0 for
all the docs and ran the load again, the latency is worst like more than
20sec. The filter I am using will fetch 15k documents (so this function is
called 15k times). And if I don't call this function in my query then
latency is less than 10ms.
So it looks like some fundamental issue with this approach and I believe
its been used in many business scenarios. Looking out for what's going
wrong.
Thanks,
Sripradeep P
On Wed, Nov 27, 2019 at 1:00 PM Jörn Franke wrote:
> And have you tried how fast it is if you don’t do anything in this method?
>
> > Am 27.11.2019 um 07:52 schrieb Sripra deep >:
> >
> > Hi Team,
> > I wrote a custom sort function that will read the field value and parse
> > and returns a float value that will be used for sorting. this field is
> > indexed, stored and docvalues enabled. my latency increases drastically
> > from 10ms when the filter loads 50 documents and 200ms when it loads 250
> > documents and 1sec when it loads 1000 documents.
> >
> > This is my function:
> >
> > @Override
> > public FunctionValues getValues(Map context, LeafReaderContext reader)
> > throws IOException {
> > final FunctionValues vals = source.getValues(context, reader);
> > return new FunctionValues() {
> >public float floatVal(int doc) {
> >float toRet = isasc ? Float.MAX_VALUE : Float.MIN_VALUE;
> >/*
> >My custom logic in reading a string array and returning a value
> out
> > of some condition checks
> >*/
> >}
> >}
> > }
> >
> > Can you suggest me some suggestions on this ?
> >
> > Thanks,
> > Sripradeep P
>
Re: Custom sort function
And have you tried how fast it is if you don’t do anything in this method?
> Am 27.11.2019 um 07:52 schrieb Sripra deep :
>
> Hi Team,
> I wrote a custom sort function that will read the field value and parse
> and returns a float value that will be used for sorting. this field is
> indexed, stored and docvalues enabled. my latency increases drastically
> from 10ms when the filter loads 50 documents and 200ms when it loads 250
> documents and 1sec when it loads 1000 documents.
>
> This is my function:
>
> @Override
> public FunctionValues getValues(Map context, LeafReaderContext reader)
> throws IOException {
> final FunctionValues vals = source.getValues(context, reader);
> return new FunctionValues() {
>public float floatVal(int doc) {
>float toRet = isasc ? Float.MAX_VALUE : Float.MIN_VALUE;
>/*
>My custom logic in reading a string array and returning a value out
> of some condition checks
>*/
>}
>}
> }
>
> Can you suggest me some suggestions on this ?
>
> Thanks,
> Sripradeep P
Re: Custom sort function
What methods do you use for your condition checks? Regexes ? Then you could for
instance precompile the regexes (saves a lot of time). Any other method? I
don’t ask about the exact condition check but only the methods you use within
those checks.
> Am 27.11.2019 um 07:52 schrieb Sripra deep :
>
> Hi Team,
> I wrote a custom sort function that will read the field value and parse
> and returns a float value that will be used for sorting. this field is
> indexed, stored and docvalues enabled. my latency increases drastically
> from 10ms when the filter loads 50 documents and 200ms when it loads 250
> documents and 1sec when it loads 1000 documents.
>
> This is my function:
>
> @Override
> public FunctionValues getValues(Map context, LeafReaderContext reader)
> throws IOException {
> final FunctionValues vals = source.getValues(context, reader);
> return new FunctionValues() {
>public float floatVal(int doc) {
>float toRet = isasc ? Float.MAX_VALUE : Float.MIN_VALUE;
>/*
>My custom logic in reading a string array and returning a value out
> of some condition checks
>*/
>}
>}
> }
>
> Can you suggest me some suggestions on this ?
>
> Thanks,
> Sripradeep P
Re: Custom Sort option to apply at SOLR index
Thank you Erick, it worked. -- Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
Re: Custom Sort option to apply at SOLR index
I guess I don't see the problem, just store it as a string and sort on the field. # sorts before numbers which sort before characters. Or I'm reading the ASCII chart wrong. Best, Erick On Wed, Aug 2, 2017 at 6:55 AM, padmanabhan wrote: > Hello Solr Geeks, > > Am newbie to SOLR. I have a requirement as given below, Could any one please > provide some insights on how to go about on this. > > "Ascending by name" (#, 0 - 9, A - Z) > > "Descending by name" (Z - A, 9 - 0, #) > > Sample name value can be > > ABCD5678 > 1234ABCD > #2345ABCD > #1234ABCD > 5678ABCD > #2345ACBD > 5678EFGH > #2345DBCA > ABCD1234 > 1234#ABCD > > *Expected Ascending order* > > #2345ABCD > #2345ACBD > #2345DBCA > 1234#ABCD > 1234ABCD > 5678ABCD > ABCD1234 > ABCD5678 > > *Expected Descending order* > > ABCD5678 > ABCD1234 > 5678ABCD > 1234ABCD > 1234#ABCD > #2345DBCA > #2345ACBD > #2345ABCD > > Thanks & Regards, > Paddy > > > > -- > View this message in context: > http://lucene.472066.n3.nabble.com/Custom-Sort-option-to-apply-at-SOLR-index-tp4348787.html > Sent from the Solr - User mailing list archive at Nabble.com.
Re: Custom Sort(0.2*relervanceScore + 0.8*numberic_field_value)
Thank you very much!
Then could you tell me how to implement
relervance_score*numberic_field/(relervance_score + numberic_field) ? I think
it's better to sort by harmmean
-- Original --
From: "Jack Krupansky";
Date: Tue, Aug 20, 2013 10:47 AM
To: "solr-user";
Subject: Re: Custom Sort(0.2*relervanceScore + 0.8*numberic_field_value)
Edismax applies the multiplicative boost ("boost") after applying the
additive boost functions ("bf").
I think (0.2*relervance score + 0.8* specified_numberic_field) should be
equivalent to:
0.2*(relevance score + (0.8/0.2)* specified_numeric_field)
or
0.2*(relevance score + 4.0* specified_numeric_field)
Unless the actual numeric range of the scores matters, just use:
relevance score + 4.0* specified_numeric_field
or
bf=mul(4.0, specified_numeric_field)
-- Jack Krupansky
-Original Message-
From: 刘健
Sent: Monday, August 19, 2013 9:50 PM
To: solr-user
Subject: Custom Sort(0.2*relervanceScore + 0.8*numberic_field_value)
Hello:
I want to get final search result sorted by (0.2*relervance score +
0.8* specified_numberic_field) .
I have known that if I use “bf”in edismax (e.g. bf=field(value)), I can get
a result sorted by(relervance sore + field(value)) ,but I don`t know how to
Implement the result sorted by (0.2*relervance score +
0.8*specified_numberic_field) .
Thanks!
Leo
Re: Custom Sort(0.2*relervanceScore + 0.8*numberic_field_value)
Edismax applies the multiplicative boost ("boost") after applying the
additive boost functions ("bf").
I think (0.2*relervance score + 0.8* specified_numberic_field) should be
equivalent to:
0.2*(relevance score + (0.8/0.2)* specified_numeric_field)
or
0.2*(relevance score + 4.0* specified_numeric_field)
Unless the actual numeric range of the scores matters, just use:
relevance score + 4.0* specified_numeric_field
or
bf=mul(4.0, specified_numeric_field)
-- Jack Krupansky
-Original Message-
From: 刘健
Sent: Monday, August 19, 2013 9:50 PM
To: solr-user
Subject: Custom Sort(0.2*relervanceScore + 0.8*numberic_field_value)
Hello:
I want to get final search result sorted by (0.2*relervance score +
0.8* specified_numberic_field) .
I have known that if I use “bf”in edismax (e.g. bf=field(value)), I can get
a result sorted by(relervance sore + field(value)) ,but I don`t know how to
Implement the result sorted by (0.2*relervance score +
0.8*specified_numberic_field) .
Thanks!
Leo
Re: Custom sort
okobloko wrote: > > It could be that you should be providing an implementation of > "SortComparatorSource" > I have missed the earlier part of this thread, I assume you're trying to > implement some form of custom search? > > B > Yes, exactly. What I'm trying to do is sort the results of an otherwise normal query, based on a scoring method that does some calculations using some data cached from database queries. I have a subclass of FieldType inspired by RandomSortField. Like RandomSortField, it has inner classes that extend SortComparatorSource (as you mention), Comparator, SortField, and ValueSource. The ValueSource stuff is actually what's confusing me right now. I wish I had an example other than RandomSortField. Any advice? Alex -- View this message in context: http://www.nabble.com/Custom-sort-tp23722921p24434489.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Custom sort
It could be that you should be providing an implementation of "SortComparatorSource" I have missed the earlier part of this thread, I assume you're trying to implement some form of custom search? B dontthinktwice wrote: Marc Sturlese wrote: I have been able to create my custom field. The problem is that I have laoded in the solr core a couple of HashMaps from a DB with values that will influence in the sort. My problem is that I don't know how to let my custom sort have access to this HashMaps. I am a bit confused now. I think that would be easy to reach my goal using: CustomSortComponent extends SearchComponent implements SolrCoreAware This way, I would load the HashMaps in the "inform" method and would create de custom sort using the HashMaps in the "preprare" method. Don't know how to do that with the CustomField (similar to the RandomField)... any advice? Marc, did you get this working somehow? I'm looking at doing something similar, and before I make a custom sort field (like RandomSortField) I would be delighted to know that I can give it access to a the data structure it will need to calculate the sort...
Re: Custom sort
Marc Sturlese wrote: > > I have been able to create my custom field. The problem is that I have > laoded in the solr core a couple of HashMaps > from a DB with values that will influence in the sort. My problem is that > I don't know how to let my custom sort have access to this HashMaps. > I am a bit confused now. I think that would be easy to reach my goal > using: > > CustomSortComponent extends SearchComponent implements SolrCoreAware > > This way, I would load the HashMaps in the "inform" method and would > create de custom sort using the HashMaps in the "preprare" method. > > Don't know how to do that with the CustomField (similar to the > RandomField)... any advice? > > Marc, did you get this working somehow? I'm looking at doing something similar, and before I make a custom sort field (like RandomSortField) I would be delighted to know that I can give it access to a the data structure it will need to calculate the sort... -- View this message in context: http://www.nabble.com/Custom-sort-tp23722921p24432604.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Custom sort
I have been able to create my custom field. The problem is that I have laoded in the solr core a couple of HashMaps from a DB with values that will influence in the sort. My problem is that I don't know how to let my custom sort have access to this HashMaps. I am a bit confused now. I think that would be easy to reach my goal using: CustomSortComponent extends SearchComponent implements SolrCoreAware This way, I would load the HashMaps in the "inform" method and would create de custom sort using the HashMaps in the "preprare" method. Don't know how to do that with the CustomField (similar to the RandomField)... any advice? Yonik Seeley-2 wrote: > > The custom Field type similar to RandomSortField is definitely the way > to go over a custom component since it's much less invasive - then > everything else like distributed search should work. > > But see SOLR-... I'm in the process of converting Solr to use the > new FieldComparatorSource. > > -Yonik > http://www.lucidimagination.com > > > > On Tue, May 26, 2009 at 9:25 AM, Marc Sturlese > wrote: >> >> Hey there, >> >> I want to implement a custom sort coding my own SortComparatorSource. The >> sorter would sort the results doing some calculations of some document >> attributes witch I have loaded previously from a DB at the solr core >> init. I >> mean, I have a preloaded hash with some attributes per doc id and I want >> to >> use them to implement the sorting. >> I have considered two options: >> >> First would be to implement a custom field similar to the solr >> RandomSortField and when search requests are done I would tell solr to >> sort >> using this custom field. >> >> The other option would be to plug a component (wich would extend from >> SearchComponent) and do the custom sort in the prepare method. >> >> Any advice witch is the most optimal way to do that? >> >> Thanks in advance. >> -- >> View this message in context: >> http://www.nabble.com/Custom-sort-tp23722921p23722921.html >> Sent from the Solr - User mailing list archive at Nabble.com. >> >> > > -- View this message in context: http://www.nabble.com/Custom-sort-tp23722921p23783579.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Custom sort
The custom Field type similar to RandomSortField is definitely the way to go over a custom component since it's much less invasive - then everything else like distributed search should work. But see SOLR-... I'm in the process of converting Solr to use the new FieldComparatorSource. -Yonik http://www.lucidimagination.com On Tue, May 26, 2009 at 9:25 AM, Marc Sturlese wrote: > > Hey there, > > I want to implement a custom sort coding my own SortComparatorSource. The > sorter would sort the results doing some calculations of some document > attributes witch I have loaded previously from a DB at the solr core init. I > mean, I have a preloaded hash with some attributes per doc id and I want to > use them to implement the sorting. > I have considered two options: > > First would be to implement a custom field similar to the solr > RandomSortField and when search requests are done I would tell solr to sort > using this custom field. > > The other option would be to plug a component (wich would extend from > SearchComponent) and do the custom sort in the prepare method. > > Any advice witch is the most optimal way to do that? > > Thanks in advance. > -- > View this message in context: > http://www.nabble.com/Custom-sort-tp23722921p23722921.html > Sent from the Solr - User mailing list archive at Nabble.com. > >
Re: Custom sort based on arbitrary order
: custom order that is fairly simple: there is a list of venues and some of : them are more relevant than others (there is no logic, it's arbitrary, it's : not an alphabetic order), it'd be something like this: : : Orange venue = 1 : Red venu = 2 : Blue venue = 3 : : So results where venue is "orange" should go first, then "red" and finally : "blue". : Could you advice on the easiest way to have this example working? use your rules to add values to all the docs at index time ... then sort on that value (ie: for each doc you actually index the value of 1, 2, or 3 in a field no one ever looks at, but you sort on it.) -Hoss
Re: Custom sort (score + custom value)
Todd: Yes, I looked into these arguments before I found the problem I
described in the first email.
Yonik: It's exactly what I was looking for.
George
On Mon, Nov 3, 2008 at 7:10 PM, Yonik Seeley <[EMAIL PROTECTED]> wrote:
> On Mon, Nov 3, 2008 at 12:37 PM, George <[EMAIL PROTECTED]> wrote:
> > Ok Yonik, thank you.
> >
> > I've tried to execute the following query: "{!boost b=log(myrank)
> > defType=dismax}q" and it works great.
> >
> > Do you know if I can do the same (combine a DisjunctionMaxQuery with a
> > BoostedQuery) in solrconfig.xml?
>
> Do you mean set it as a default for a handler in solrconfig.xml? That
> should work.
> You could set a default of q={!boost b=log(myrank) defType=dismax v=$uq}
> then all the client would have to pass in is uq (the user query)
>
> -Yonik
>
Re: Custom sort (score + custom value)
On Mon, Nov 3, 2008 at 12:37 PM, George <[EMAIL PROTECTED]> wrote:
> Ok Yonik, thank you.
>
> I've tried to execute the following query: "{!boost b=log(myrank)
> defType=dismax}q" and it works great.
>
> Do you know if I can do the same (combine a DisjunctionMaxQuery with a
> BoostedQuery) in solrconfig.xml?
Do you mean set it as a default for a handler in solrconfig.xml? That
should work.
You could set a default of q={!boost b=log(myrank) defType=dismax v=$uq}
then all the client would have to pass in is uq (the user query)
-Yonik
RE: Custom sort (score + custom value)
Have you looked into the "bf" and "bq" arguments on the
DisMaxRequestHandler?
http://wiki.apache.org/solr/DisMaxRequestHandler?highlight=(dismax)#head
-6862070cf279d9a09bdab971309135c7aea22fb3
-Todd
-Original Message-
From: George [mailto:[EMAIL PROTECTED]
Sent: Monday, November 03, 2008 9:38 AM
To: solr-user@lucene.apache.org
Subject: Re: Custom sort (score + custom value)
Ok Yonik, thank you.
I've tried to execute the following query: "{!boost b=log(myrank)
defType=dismax}q" and it works great.
Do you know if I can do the same (combine a DisjunctionMaxQuery with a
BoostedQuery) in solrconfig.xml?
George
On Sun, Nov 2, 2008 at 3:01 PM, Yonik Seeley <[EMAIL PROTECTED]> wrote:
> On Sun, Nov 2, 2008 at 5:09 AM, George <[EMAIL PROTECTED]> wrote:
> > I want to implement a custom sort in Solr based on a combination of
> > relevance (Solr gives me it yet => score) and a custom value I've
> calculated
> > previously for each document. I see two options:
> >
> > 1. Use a function query (I'm using a DisMaxRequestHandler).
> > 2. Create a component that set SortSpec with a sort that has a
custom
> > ComparatorSource (similar to QueryElevationComponent).
> >
> > The first option has the problem: While the relevance value changes
for
> > every query, my custom value is constant for each doc.
>
> Yes, that can be an issue when adding unrelated scores.
> Multiplying them might give you better results:
>
>
http://lucene.apache.org/solr/api/org/apache/solr/search/BoostQParserPlu
gin.html
>
> -Yonik
>
Re: Custom sort (score + custom value)
Ok Yonik, thank you.
I've tried to execute the following query: "{!boost b=log(myrank)
defType=dismax}q" and it works great.
Do you know if I can do the same (combine a DisjunctionMaxQuery with a
BoostedQuery) in solrconfig.xml?
George
On Sun, Nov 2, 2008 at 3:01 PM, Yonik Seeley <[EMAIL PROTECTED]> wrote:
> On Sun, Nov 2, 2008 at 5:09 AM, George <[EMAIL PROTECTED]> wrote:
> > I want to implement a custom sort in Solr based on a combination of
> > relevance (Solr gives me it yet => score) and a custom value I've
> calculated
> > previously for each document. I see two options:
> >
> > 1. Use a function query (I'm using a DisMaxRequestHandler).
> > 2. Create a component that set SortSpec with a sort that has a custom
> > ComparatorSource (similar to QueryElevationComponent).
> >
> > The first option has the problem: While the relevance value changes for
> > every query, my custom value is constant for each doc.
>
> Yes, that can be an issue when adding unrelated scores.
> Multiplying them might give you better results:
>
> http://lucene.apache.org/solr/api/org/apache/solr/search/BoostQParserPlugin.html
>
> -Yonik
>
Re: Custom sort (score + custom value)
On Sun, Nov 2, 2008 at 5:09 AM, George <[EMAIL PROTECTED]> wrote: > I want to implement a custom sort in Solr based on a combination of > relevance (Solr gives me it yet => score) and a custom value I've calculated > previously for each document. I see two options: > > 1. Use a function query (I'm using a DisMaxRequestHandler). > 2. Create a component that set SortSpec with a sort that has a custom > ComparatorSource (similar to QueryElevationComponent). > > The first option has the problem: While the relevance value changes for > every query, my custom value is constant for each doc. Yes, that can be an issue when adding unrelated scores. Multiplying them might give you better results: http://lucene.apache.org/solr/api/org/apache/solr/search/BoostQParserPlugin.html -Yonik
