Re: Solr admin Segments page legend

[Asher Shih]

unsubscribe

On Thu, May 17, 2018 at 8:17 PM, Yasufumi Mizoguchi
 wrote:
> Hi,
>
> I found some information about the pink bar from mail archive.
> I think this should be written in ref. guide.
>
>> I think that pink segments are those segments
>> which the system thinks are most likely to be chosen for automatic
>> merging, according to whatever merge policy you have active.  Most
>> likely the merge policy is TieredMergePolicy.
>
> For details, check the following:
> http://lucene.472066.n3.nabble.com/what-s-the-pink-segment-on-solr-UI-meaning-td4378940.html
>
> Thanks,
> Yasufumi
>
>
> 2018年5月18日(金) 11:55 Nawab Zada Asad Iqbal :
>
>> Hi,
>>
>> Solr has a nice segments visualization at [core_name]/segments , but I am
>> wondering what the colors mean?
>>
>> Gray color is probably deleted documents., But I couldn't guess the
>> significance of pink color:
>>
>>
>>
>> Thanks
>> Nawab
>>

Re: Solr admin Segments page legend

[Yasufumi Mizoguchi]

Hi,

I found some information about the pink bar from mail archive.
I think this should be written in ref. guide.

> I think that pink segments are those segments
> which the system thinks are most likely to be chosen for automatic
> merging, according to whatever merge policy you have active.  Most
> likely the merge policy is TieredMergePolicy.

For details, check the following:
http://lucene.472066.n3.nabble.com/what-s-the-pink-segment-on-solr-UI-meaning-td4378940.html

Thanks,
Yasufumi


2018年5月18日(金) 11:55 Nawab Zada Asad Iqbal :

> Hi,
>
> Solr has a nice segments visualization at [core_name]/segments , but I am
> wondering what the colors mean?
>
> Gray color is probably deleted documents., But I couldn't guess the
> significance of pink color:
>
>
>
> Thanks
> Nawab
>

Solr admin Segments page legend

[Nawab Zada Asad Iqbal]

Hi,

Solr has a nice segments visualization at [core_name]/segments , but I am
wondering what the colors mean?

Gray color is probably deleted documents., But I couldn't guess the
significance of pink color:



Thanks
Nawab

RE: Solr 7.3, FunctionScoreQuery no longer displays debug output

[Markus Jelsma]

Thanks Yonik,

That is the suspect issue i stumbled upon when reading through the CHANGES.txt. 
Can you, or someone, please verify this? I need to know this before i can file 
a bug.

There is a definitive difference in 7.2 and 7.3's respective outputs, i triple 
checked the debug output. But on one hand i can't believe that issue was 
committed with this flaw. And although we have a lot of custom code, we have 
nothing that should interfere this much with the debug prints, or that should 
be obvious in the change log.

Please verify and let me open a ticket, or we'll change the discussion into 
what has changed in Solr/Lucene so much, for us to get back on track.

Many thanks,
Markus
 
 
-Original message-
> From:Yonik Seeley 
> Sent: Friday 18th May 2018 0:04
> To: solr-user@lucene.apache.org
> Subject: Re: Solr 7.3, FunctionScoreQuery no longer displays debug output
> 
> If this used to work, I wonder if it's something to do with changes to boost:
> https://issues.apache.org/jira/browse/LUCENE-8099
> 
> -Yonik
> 
> 
> On Thu, May 17, 2018 at 5:48 PM, Markus Jelsma
>  wrote:
> > Hello,
> >
> > Sorry to disturb. Is there anyone here able to reproduce and verify this 
> > issue?
> >
> > Many thanks,
> > Markus
> >
> >
> >
> > -Original message-
> >> From:Markus Jelsma 
> >> Sent: Wednesday 9th May 2018 18:25
> >> To: solr-user 
> >> Subject: Solr 7.3, FunctionScoreQuery no longer displays debug output
> >>
> >> Hi,
> >>
> >> Is this a known problem? For example, the following query:
> >> q=australia=true=if(exists(query($bqlang)),2,1)=lang:en=edismax=content_en
> >>  content_ro
> >>
> >> returns the following toString for 7.2.1:
> >> boost(+(Synonym(content_en:australia content_en:australia) | 
> >> Synonym(content_ro:austral 
> >> content_ro:australia)),if(exists(query(lang:en,def=0.0)),const(2),const(1)))
> >>
> >> 7.3:
> >> FunctionScoreQuery(+(Synonym(content_en:australia content_en:australia) | 
> >> Synonym(content_ro:austral content_ro:australia)), scored by 
> >> boost(if(exists(query(lang:en,def=0.0)),const(2),const(1
> >>
> >> and the following debug output for 7.2.1:
> >>
> >> 11.226025 = boost((Synonym(content_en:australia content_en:australia) | 
> >> Synonym(content_ro:austral 
> >> content_ro:australia)),if(exists(query(lang:en,def=0.0)),const(2),const(1))),
> >>  product of:
> >>   11.226025 = max of:
> >> 11.226025 = weight(Synonym(content_ro:austral content_ro:australia) in 
> >> 6761) [SchemaSimilarity], result of:
> >>   11.226025 = score(doc=6761,freq=18.0 = termFreq=18.0
> >> ), product of:
> >> 5.442921 = idf(docFreq=193, docCount=44720)
> >> 2.0625 = tfNorm, computed as (freq * (k1 + 1)) / (freq + k1) from:
> >>   18.0 = termFreq=18.0
> >>   1.2 = parameter k1
> >>   0.0 = parameter b (norms omitted for field)
> >>   1.0 = if(exists(query(lang:en,def=0.0)=0.0),const(2),const(1))
> >>
> >> but for 7.3 i get only:
> >>
> >> 11.226025 = product of:
> >>   1.0 = boost
> >>   11.226025 = boost(if(exists(query(lang:en,def=0.0)),const(2),const(1)))
> >>
> >> The scores are still the same, but the debug output is useless. Removing 
> >> the boost fixes the problem of debug output immediately.
> >>
> >> Thanks,
> >> Markus
> >>
> >>
> 

Re: Solr 7.3, FunctionScoreQuery no longer displays debug output

[Yonik Seeley]

If this used to work, I wonder if it's something to do with changes to boost:
https://issues.apache.org/jira/browse/LUCENE-8099

-Yonik


On Thu, May 17, 2018 at 5:48 PM, Markus Jelsma
 wrote:
> Hello,
>
> Sorry to disturb. Is there anyone here able to reproduce and verify this 
> issue?
>
> Many thanks,
> Markus
>
>
>
> -Original message-
>> From:Markus Jelsma 
>> Sent: Wednesday 9th May 2018 18:25
>> To: solr-user 
>> Subject: Solr 7.3, FunctionScoreQuery no longer displays debug output
>>
>> Hi,
>>
>> Is this a known problem? For example, the following query:
>> q=australia=true=if(exists(query($bqlang)),2,1)=lang:en=edismax=content_en
>>  content_ro
>>
>> returns the following toString for 7.2.1:
>> boost(+(Synonym(content_en:australia content_en:australia) | 
>> Synonym(content_ro:austral 
>> content_ro:australia)),if(exists(query(lang:en,def=0.0)),const(2),const(1)))
>>
>> 7.3:
>> FunctionScoreQuery(+(Synonym(content_en:australia content_en:australia) | 
>> Synonym(content_ro:austral content_ro:australia)), scored by 
>> boost(if(exists(query(lang:en,def=0.0)),const(2),const(1
>>
>> and the following debug output for 7.2.1:
>>
>> 11.226025 = boost((Synonym(content_en:australia content_en:australia) | 
>> Synonym(content_ro:austral 
>> content_ro:australia)),if(exists(query(lang:en,def=0.0)),const(2),const(1))),
>>  product of:
>>   11.226025 = max of:
>> 11.226025 = weight(Synonym(content_ro:austral content_ro:australia) in 
>> 6761) [SchemaSimilarity], result of:
>>   11.226025 = score(doc=6761,freq=18.0 = termFreq=18.0
>> ), product of:
>> 5.442921 = idf(docFreq=193, docCount=44720)
>> 2.0625 = tfNorm, computed as (freq * (k1 + 1)) / (freq + k1) from:
>>   18.0 = termFreq=18.0
>>   1.2 = parameter k1
>>   0.0 = parameter b (norms omitted for field)
>>   1.0 = if(exists(query(lang:en,def=0.0)=0.0),const(2),const(1))
>>
>> but for 7.3 i get only:
>>
>> 11.226025 = product of:
>>   1.0 = boost
>>   11.226025 = boost(if(exists(query(lang:en,def=0.0)),const(2),const(1)))
>>
>> The scores are still the same, but the debug output is useless. Removing the 
>> boost fixes the problem of debug output immediately.
>>
>> Thanks,
>> Markus
>>
>>

RE: Multiple languages, boosting and, stemming and KeywordRepeat

[Markus Jelsma]

Hello,

And sorry to disturb again. Does anyone of you have any meaningful opinion on 
this peculiar matter? The RemoveDuplicates filter exists for a reason, but with 
query-time KeywordRepeat filter it causes trouble in some cases. Is it normal 
for the clauses to be absent in the debug output, but the boost doubled in 
value?

I like this behaviour, but is it a side effect that is considered a bug in 
later versions? And where is the documentation in this. I cannot find anything 
in the Lucene or Solr Javadocs, or the reference manual.

Many thanks, again,
Markus

 
 
-Original message-
> From:Markus Jelsma 
> Sent: Wednesday 9th May 2018 17:39
> To: solr-user 
> Subject: Multiple languages, boosting and, stemming and KeywordRepeat
> 
> Hello,
> 
> First, apologies for the weird subject line.
> 
> We index many languages and search over all those languages at once, but 
> boost the language of the user's preference. To differentiate between stemmed 
> tokens and unstemmed tokens we use KeywordRepeat and RemoveDuplicates, this 
> works very well.
> 
> However, we just stumbled over the following example, q=australia is not 
> stemmed in English, but its suffix is removed by the Romanian stemmer, 
> causing the Romanian results to be returned on top of English results, 
> despite language boosting.
> 
> This is because the Romanian part of the query consists of the stemmed and 
> unstemmed version of the word, but the English part of the query is just one 
> clause per field (title, content etc). Thus the Romanian results score 
> roughtly twice that of English results.
> 
> Now, this is of course really obvious, but the 'solution' is not. To work 
> around the problem i removed the RemoveDuplicates filter so i get two clauses 
> for English as well, really ugly but it works. What i don't understand is the 
> debug output, it doesn't list two identical clauses, instead, it doubled the 
> boost on the field, so instead of:
> 
> 27.048403 = PayloadSpanQuery, product of:
>   27.048403 = weight(title_en:australia in 15850) [SchemaSimilarity], 
> result of:
> 27.048403 = score(doc=15850,freq=4.0 = phraseFreq=4.0
> ), product of:
>   7.4 = boost
>   3.084852 = idf(docFreq=14539, docCount=317894)
>   1.1848832 = tfNorm, computed as (freq * (k1 + 1)) / (freq + k1 * (1 
> - b + b * fieldLength / avgFieldLength)) from:
> 4.0 = phraseFreq=4.0
> 0.3 = parameter k1
> 0.5 = parameter b
> 15.08689 = avgFieldLength
> 24.0 = fieldLength
>   1.0 = AveragePayloadFunction.docScore()
> 
> I now get 
> 
> 54.096806 = PayloadSpanQuery, product of:
>   54.096806 = weight(title_en:australia in 15850) [SchemaSimilarity], 
> result of:
> 54.096806 = score(doc=15850,freq=4.0 = phraseFreq=4.0
> ), product of:
>   14.8 = boost
>   3.084852 = idf(docFreq=14539, docCount=317894)
>   1.1848832 = tfNorm, computed as (freq * (k1 + 1)) / (freq + k1 * (1 
> - b + b * fieldLength / avgFieldLength)) from:
> 4.0 = phraseFreq=4.0
> 0.3 = parameter k1
> 0.5 = parameter b
> 15.08689 = avgFieldLength
> 24.0 = fieldLength
>   1.0 = AveragePayloadFunction.docScore()
> 
> So instead of expecting two clauses in the debug, i get one but with a 
> doubled boost.
> 
> The question is, is this supposed to be like this?
> 
> Also, are there any real solutions to this problem? Removing the 
> RemoveDuplicats filter looks really silly.
> 
> Many thanks!
> Markus
> 

RE: Solr 7.3, FunctionScoreQuery no longer displays debug output

[Markus Jelsma]

Hello,

Sorry to disturb. Is there anyone here able to reproduce and verify this issue?

Many thanks,
Markus 

 
 
-Original message-
> From:Markus Jelsma 
> Sent: Wednesday 9th May 2018 18:25
> To: solr-user 
> Subject: Solr 7.3, FunctionScoreQuery no longer displays debug output
> 
> Hi,
> 
> Is this a known problem? For example, the following query:
> q=australia=true=if(exists(query($bqlang)),2,1)=lang:en=edismax=content_en
>  content_ro
> 
> returns the following toString for 7.2.1:
> boost(+(Synonym(content_en:australia content_en:australia) | 
> Synonym(content_ro:austral 
> content_ro:australia)),if(exists(query(lang:en,def=0.0)),const(2),const(1)))
> 
> 7.3:
> FunctionScoreQuery(+(Synonym(content_en:australia content_en:australia) | 
> Synonym(content_ro:austral content_ro:australia)), scored by 
> boost(if(exists(query(lang:en,def=0.0)),const(2),const(1
> 
> and the following debug output for 7.2.1:
> 
> 11.226025 = boost((Synonym(content_en:australia content_en:australia) | 
> Synonym(content_ro:austral 
> content_ro:australia)),if(exists(query(lang:en,def=0.0)),const(2),const(1))), 
> product of:
>   11.226025 = max of:
> 11.226025 = weight(Synonym(content_ro:austral content_ro:australia) in 
> 6761) [SchemaSimilarity], result of:
>   11.226025 = score(doc=6761,freq=18.0 = termFreq=18.0
> ), product of:
> 5.442921 = idf(docFreq=193, docCount=44720)
> 2.0625 = tfNorm, computed as (freq * (k1 + 1)) / (freq + k1) from:
>   18.0 = termFreq=18.0
>   1.2 = parameter k1
>   0.0 = parameter b (norms omitted for field)
>   1.0 = if(exists(query(lang:en,def=0.0)=0.0),const(2),const(1))
> 
> but for 7.3 i get only:
> 
> 11.226025 = product of:
>   1.0 = boost
>   11.226025 = boost(if(exists(query(lang:en,def=0.0)),const(2),const(1)))
> 
> The scores are still the same, but the debug output is useless. Removing the 
> boost fixes the problem of debug output immediately.
> 
> Thanks,
> Markus
> 
> 

Re: Date Query Confusion

[Tim Casey]

A simple date range query does not really represent how people query over
time and dates.  If you want any form of date queries, above a single
range, then a special field allowing tokenized query will be the only way
to find documents.

A query for 'ever tuesday in november of 2017' would have to be written as
an or clause over a set of date ranges.  A tokenized date field would just
have to query for "+nov +tues +2017".  How you choose to tokenize a date
into a field will determine the types of queries you can run over the data.

Another part of this is query for a date range, when the source material
has date ranges built into it is kinda odd.  But it occurs.  If you query
from noon-1p does that include meeting notes which started at 1130a, but
went for an hour?  You have to choose what to do.

tim

On Thu, May 17, 2018 at 6:11 AM, Terry Steichen  wrote:

> To me, one of the more frustrating things I've encountered in Solr is
> working with date fields.  Supposedly, according to the documentation,
> this is straightforward.  But in my experience, it is anything but
> that.  In particular, I've found that the abbreviated forms of date
> queries, don't work as described.
>
> If I create a query like creation_date: [2016-10-01 To 2016-11-01], it
> will produce a set of documents produced in the month of November 2016.
> That's the good news.
>
> But, the abbreviated date queries (described in Solr documentation
> )
> don't work.  Tried creation_date: 2016-11.  That's supposed to match
> documents with any November 2016 date.  But actually produces:
> |"Invalid Date String:'2016-11'|
>
> ||And Solr doesn't seem to let me sort on a date field.  Tried
> creation_date asc  Produced: |"can not sort on multivalued field:
> creation_date"|
>
> In the AdminUI, if you go to the schema option for my collection, and
> examine creation_date it show it to be:
> org.apache.solr.schema.TrieDateField  (This was automatically chosen by
> the managed-schema)
>
> In that same AdminUI display, if I click "Load Term Info" I get a list
> of dates, but when I click on one, it transforms it into a different
> query form: {!term f=creation_date}2016-10-26T07:59:09.824Z  But this
> query still produces 0 hits (even though the listing says it should
> produce dozens of hits).
>
> I imagine that I'm missing something basic here.  But I have no idea
> what.  Any thoughts would be MOST welcome.
>
> PS: I'm using Solr 6.6.0.
>

Re: Regarding LTR feature

[Alessandro Benedetti]

"FQ_filter were 365 but below in the 
debugging part the docfreq used in the payload_score calculation was 
3360" 

If you are talking about the doc frequency of a term, obviously this is
corpus based ( necessary for the TF /IDF calculations) so it wil not be
affected by the filter queries.
The payload score part may be different.

Anyway, you mentioned that you assign the weights, in that case the learning
to rank plugin may be not necessary at all.

Regards




-
---
Alessandro Benedetti
Search Consultant, R Software Engineer, Director
Sease Ltd. - www.sease.io
--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Re: How to implement Solr auto suggester and spell checker simultaneously on a single search box

[Alessandro Benedetti]

Hi Sonal,
if you want to go with a plain Solr suggester, what about the :
FuzzyLookupFactory ?

1) it does support fuzzy matching ( spellcheck)
2) it does support auto complete

If you want the context filtering as well, unfortunately the FST based Solr
suggesters don't support this feature.

I would recommend in that case to build your own autocompletion service
defining a dedicated Lucene index ( to make it simple you could define an ad
hoc Solr collection).

Then, at query time, when a query doesn't return results you may want to
execute a fuzzy query ( to bring the spellcheck functionality or just run
the spellcheck response collation from the main query)

Cheers



-
---
Alessandro Benedetti
Search Consultant, R Software Engineer, Director
Sease Ltd. - www.sease.io
--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Re: Date Query Confusion

[Alessandro Benedetti]

Hi Terry,
let me go in order :

/"Tried creation_date: 2016-11.  That's supposed to match 
documents with any November 2016 date.  But actually produces:  
|"Invalid Date String:'2016-11'| "/

Is "*DateRangeField*" the field type for your field : "creation_date" ? [1]
You mentioned : org.apache.solr.schema.TrieDateField, this is not going to
work, you need the specific field type I mentioned to use that date range
syntax.

/"||And Solr doesn't seem to let me sort on a date field.  Tried 
creation_date asc  Produced: |"can not sort on multivalued field: 
creation_date"| "/

Is your "creation_date" single valued ?
If it is single valued semantically, make sure it is defined as single
valued in the schema.
Solr doesn't support sorting on multi valued fields.
You schemaless conf may have assigned the multi valued attribute to that
field.

>From the Wiki[2] :
"Solr can sort query responses according to document scores or the value of
any field with a single value that is either indexed or uses DocValues (that
is, any field whose attributes in the Schema include multiValued="false" and
either docValues="true" or indexed="true" – if the field does not have
DocValues enabled, the indexed terms are used to build them on the fly at
runtime), provided that:"

Hope this helps,

Regards



[1]
https://lucene.apache.org/solr/guide/6_6/working-with-dates.html#WorkingwithDates-DateRangeFormatting
[2]
https://lucene.apache.org/solr/guide/6_6/common-query-parameters.html#CommonQueryParameters-ThesortParameter



-
---
Alessandro Benedetti
Search Consultant, R Software Engineer, Director
Sease Ltd. - www.sease.io
--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Re: Used debug log level on the interface

[Shawn Heisey]

On 5/17/2018 3:03 AM, msaunier wrote:

On solrCloud interface, I don't have with solr4j the info and debug level on
the console. In < level > I have add my URP with INFO param and DEBUG param
but never of the two work. I have just WARN and ERROR log on the interface.


The admin UI won't show anything lower than WARN.  I know there's a way 
to make it show lower levels, but a server that's got any traffic logs a 
LOT at the INFO level, so much so that the auto-refresh in the admin UI 
would make it very difficult to actually look at the logs.


You will find lower levels like INFO and DEBUG in the actual logfile -- 
solr.log.


Thanks,
Shawn

Re: Date Query Confusion

[Erick Erickson]

Yeah, dates are "special".

Those abbreviated dates are for DateRangeField, which is a distinct
type from "TrieDate" in your schema.

bq. And Solr doesn't seem to let me sort on a date field

It's not a date field that's the problem, it's the "multiValued" part.
When you specify in your schema that the field is multiValued, it
means you can have more than one date in the doc. So how should it be
sorted? Newest first? Oldest first? Whatever you choose is wrong.
Again it's a schema change, set multiValued="false". You _might_ be
able to solve both problems by sorting via a function query (warning,
haven't tried this lately with date fields but "it should work"), see:
https://lucidworks.com/2015/09/10/minmax-on-multivalued-field/ The
problem there is it must be a docValues="true" field.

This is why we strongly recommend against using "schemaless" mode in
production, schemaless makes the best decision it can, but pretty soon
you want run into issues like these when your intended use isn't
supported.

bq.  {!term f=creation_date}2016-10-26T07:59:09.824Z

Well, "it works on my machine", I admit I had a Solr 6.1 version lying
around and used the techproducts example where the date field is
defined as:
type="date"indexed="true"  stored="true"
and "date" is:


The "tdate" field should work identically.

The critical bits here I believe are docValues=true and
multiValued=false by default.

So I'd start by trying the techproducts example "bin/solr start -e
techproducts" which will create docs as I did, and see if you have the
same problem, then use a similar field definition for your real
system.

And if you do change the schema, you need to blow away the entire
index "rm -rf core/data" or create a new collection if using SolrCloud
and re-index.

Best,
Erick

On Thu, May 17, 2018 at 9:11 AM, Terry Steichen  wrote:
> To me, one of the more frustrating things I've encountered in Solr is
> working with date fields.  Supposedly, according to the documentation,
> this is straightforward.  But in my experience, it is anything but
> that.  In particular, I've found that the abbreviated forms of date
> queries, don't work as described.
>
> If I create a query like creation_date: [2016-10-01 To 2016-11-01], it
> will produce a set of documents produced in the month of November 2016.
> That's the good news.
>
> But, the abbreviated date queries (described in Solr documentation
> )
> don't work.  Tried creation_date: 2016-11.  That's supposed to match
> documents with any November 2016 date.  But actually produces:
> |"Invalid Date String:'2016-11'|
>
> ||And Solr doesn't seem to let me sort on a date field.  Tried
> creation_date asc  Produced: |"can not sort on multivalued field:
> creation_date"|
>
> In the AdminUI, if you go to the schema option for my collection, and
> examine creation_date it show it to be:
> org.apache.solr.schema.TrieDateField  (This was automatically chosen by
> the managed-schema)
>
> In that same AdminUI display, if I click "Load Term Info" I get a list
> of dates, but when I click on one, it transforms it into a different
> query form: {!term f=creation_date}2016-10-26T07:59:09.824Z  But this
> query still produces 0 hits (even though the listing says it should
> produce dozens of hits).
>
> I imagine that I'm missing something basic here.  But I have no idea
> what.  Any thoughts would be MOST welcome.
>
> PS: I'm using Solr 6.6.0.

Re: Default Searches not working after migrating from Solr 4.7 to 7.3

[THADC]

Ok this is fixed. the _text_<\df> was not the issue. Certain copyFields
had to be changed from dest="text" to dest="_text_".

this issue is closed. thank you



--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Re: Default Searches not working after migrating from Solr 4.7 to 7.3

[Shawn Heisey]

On 5/17/2018 7:23 AM, THADC wrote:

, however for 7.3, "defaultSearchField" apparently no longer a valid type. I
switched to "df". Also, "text" is no longer default data type, but rather
"_text_". So, I replaced above with:

_text_

, but still default search not working properly. By the way, when I say "not
working properly"  I am testing this in solr admin by entering various
strings like "car" or "title" or "service" and nothing is returned. However,
when I enter "text:car" or "text:title" or "text:service" data is returned.
I need the default (i.e., not specifier left of colon) to work.


The df field should be set to whatever you have left of the colon in the 
searches that work.  It looks like that is text without the underscores, 
so that's what you should set df to.


The schema from the default config for 7.3 may populate the field name 
with the underscores, but apparently the schema that you're actually 
using isn't populating that field.


Thanks,
Shawn

Default Searches not working after migrating from Solr 4.7 to 7.3

[THADC]

Hello,

Migrating our system from solr 4.7 to 7.3. Simple default searches are not
working. For our simply searches, we had the following in our schema.xml for
4.7:

 text

, however for 7.3, "defaultSearchField" apparently no longer a valid type. I
switched to "df". Also, "text" is no longer default data type, but rather
"_text_". So, I replaced above with:

_text_

, but still default search not working properly. By the way, when I say "not
working properly"  I am testing this in solr admin by entering various
strings like "car" or "title" or "service" and nothing is returned. However,
when I enter "text:car" or "text:title" or "text:service" data is returned.
I need the default (i.e., not specifier left of colon) to work.

Grateful for any help. thank you





--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Re: Sorting on pseudo field(The one which is added during doctransformer)

[Mikhail Khludnev]

Here is the reference I've found so far.

On Thu, May 17, 2018 at 12:26 PM, prateek.agar...@bigbasket.com <
prateek.agar...@bigbasket.com> wrote:

>
> Hi Mikhail,
>
> > You can either sort by function that needs to turn the logic into value
> > source parser.
>
> But like my requirement for this was to add a field dynamically from cache
> or external source to the returned documents from the solr and perform
> sorting in the solr itself if required otherwise use the score to sort.
> So how would you advise to go about this??
>
> And how to go about your way "to turn the logic into value source parser"
> like how to do this for this case??
>
>
> > If you need to toss just result page, check rerank.
>
> I don't want to use it to rank the relevancy of results.
>
> Thanks for the response.
>
>
>
> Regards,
> Prateek
>



-- 
Sincerely yours
Mikhail Khludnev

Date Query Confusion

[Terry Steichen]

To me, one of the more frustrating things I've encountered in Solr is
working with date fields.  Supposedly, according to the documentation,
this is straightforward.  But in my experience, it is anything but
that.  In particular, I've found that the abbreviated forms of date
queries, don't work as described.

If I create a query like creation_date: [2016-10-01 To 2016-11-01], it
will produce a set of documents produced in the month of November 2016. 
That's the good news.

But, the abbreviated date queries (described in Solr documentation
)
don't work.  Tried creation_date: 2016-11.  That's supposed to match
documents with any November 2016 date.  But actually produces: 
|"Invalid Date String:'2016-11'|

||And Solr doesn't seem to let me sort on a date field.  Tried
creation_date asc  Produced: |"can not sort on multivalued field:
creation_date"|

In the AdminUI, if you go to the schema option for my collection, and
examine creation_date it show it to be:
org.apache.solr.schema.TrieDateField  (This was automatically chosen by
the managed-schema)

In that same AdminUI display, if I click "Load Term Info" I get a list
of dates, but when I click on one, it transforms it into a different
query form: {!term f=creation_date}2016-10-26T07:59:09.824Z  But this
query still produces 0 hits (even though the listing says it should
produce dozens of hits).

I imagine that I'm missing something basic here.  But I have no idea
what.  Any thoughts would be MOST welcome.

PS: I'm using Solr 6.6.0.

Re: Add UUID on the schema

[Erick Erickson]

I strongly advise downloading the full Solr reference guide, searching
there first often gets you answers quickly, in this case

At the top of every on-line version, there's an "other formats" link
that has an "archived PDFs" link that will take you to a page that
allows you to download the version specific to your release way back
to about 4.10, here it is for convenience:
https://archive.apache.org/dist/lucene/solr/ref-guide/

If you search for "UUID" you'll get a note about how SolrCloud needs
UUIDUpdateProcessorFactory to work correctly in SolrCloud mode.

Best,
Erick

On Thu, May 17, 2018 at 5:11 AM, @Nandan@
 wrote:
> Here is some basic question based on that answers vary.
> 1) Are you going to use UUID column as unique column.
> 2) what do you with that column.
>
> Process :-
> Column is :-
>
> 
>
>For Add this :-
>
>  default="NEW" multiValued="false"/>
>
> And then need to add in uniqueid section.
>
> id
>
>
>
> On Thu, May 17, 2018 at 5:00 PM, msaunier  wrote:
>
>> Hello,
>>
>>
>>
>> I use SolrCloud 7.0 and I don't have find how to add an UID field on the
>> schema. I have add UUIDField type and add a field but it do not work.
>>
>>
>>
>>
>>
>>   {
>>
>> "name": "uid",
>>
>> "class": "solr.UUIDField",
>>
>> "indexed": true,
>>
>> "stored": true
>>
>>   },
>>
>>
>>
>> Thanks for your help.
>>
>>
>>
>>
>>
>>

Re: Question regarding TLS version for solr

[Christopher Schultz]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

Shawn,

On 5/17/18 4:23 AM, Shawn Heisey wrote:
> On 5/17/2018 1:53 AM, Anchal Sharma2 wrote:
>> We are using solr version 5.3.0 and  have been  trying to enable 
>> security on our solr .We followed steps mentioned on site 
>> -https://lucene.apache.org/solr/guide/6_6/enabling-ssl.html .But
>> by default it picks ,TLS version  1.0,which is causing an issue
>> as our application uses TLSv 1.2.We tried using online resources
>> ,but could not find anything regarding TLS enablement for solr .
>> 
>> It will be a huge help if anyone can provide some suggestions as
>> to how we can enable TLS v 1.2 for solr.
> 
> The choice of ciphers and encryption protocols is mostly made by
> Java. The servlet container might influence it as well. The only
> servlet container that is supported since Solr 5.0 is the Jetty
> that is bundled in the Solr download.
> 
> TLS 1.2 was added in Java 7, and it became default in Java 8. If
> you can install the latest version of Java 8 and make sure that it
> has the policy files for unlimited crypto strength installed,
> support for TLS 1.2 might happen automatically.

There is no "default" TLS version for either the client or the server:
the two endpoints always negotiate the highest mutual version they
both support. The key agreement, authentication, and cipher suites are
the items that are negotiated during the handshake.

> Solr 5.3.0 is running a fairly old version of Jetty -- 9.2.11. 
> Information for 9.2.x versions is hard to find, so although I think
> it probably CAN do TLS 1.2 if the Java version supports it, I can't
> be absolutely sure.  You'll need to upgrade Solr to get an upgraded
> Jetty.

I would be shocked if Jetty ships with its own crypto libraries; it
should be using JSSE.

Anchal,

Java 1.7 or later is an absolute requirement if you want to use
TLSv1.2 (and you SHOULD want to use it).

I have recently spent a lot of time getting Solr 7.3.0 running with
TLS mutual-authentication, but I haven't worked with the 5.3.x line. I
can tell you have I've done things for my version, but they may need
some adjustments for yours.

- -chris
-BEGIN PGP SIGNATURE-
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQIzBAEBCAAdFiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAlr9fKYACgkQHPApP6U8
pFh8lRAAmmvBMUSk35keW0OG0/SHpUy/ExJK69JGIKGwi96ddbz2yH8MG+OjjE3G
GNq/o5+EMT7tP/nW6XuPQou5UQvA2nlA9jsskox3A+CqOH7e6cbSxfxIkTqf9YDl
Kxr4J6mYjvTIjJAqLXGF+ghJfswS6RjZezDgo1PdSUox+gUOvmY61tlSjuYTaAYw
vH1i1DRzb8PkkR4ULePF48Y4r5+ZYz/4ZwSvnJTTkyl97KCw93rZ/kI5v9p3cCHK
Ycuwi/ZirO/VNf/9ruAOtgET3aojNfuNCX/A+vrSbJfiY7mXo05lYKN+eT80elQr
X8OKQaqHP6haF2aNPHrqXGtY2YoiGrdyaGtrXkUHFDfXgQeOmlk/eSVWemcSsatk
eEHSWW9NALMaalRAM7NuXQtgqq1badJhKysiJwSqFgcdgVKcSt8SsQ/09qTPjaNE
Ce1/EHdR6j1hM0Bnv5Hzf85cZjM7PfLmh7P8fnUD5d8eSbBpeWYVBDsS+fXp8WWv
FO5axbnSYIScOIz33i0UZyxpJgcsAkABLGghL6WWQSkfBf4ANgdTumS7K9Pn7Thz
Uq+lD9QPEPWJ91Fc0gnCWtDAEIRjOyLLbYzgI4ebV5qo41GO1WDDHfQZEcqA0Vod
+K8oAMD8nnwU+TprTFkjlQwbDnW1q1efTD6IrpEL5H7h6Xw2cgg=
=RpO6
-END PGP SIGNATURE-

Re: Sorting on pseudo field(The one which is added during doctransformer)

[prateek . agarwal]

Hi Mikhail,

> You can either sort by function that needs to turn the logic into value
> source parser.

But like my requirement for this was to add a field dynamically from cache or 
external source to the returned documents from the solr and perform sorting in 
the solr itself if required otherwise use the score to sort.
So how would you advise to go about this??

And how to go about your way "to turn the logic into value source parser" like 
how to do this for this case??


> If you need to toss just result page, check rerank.

I don't want to use it to rank the relevancy of results.

Thanks for the response.



Regards,
Prateek

Re: Add UUID on the schema

[@Nandan@]

Here is some basic question based on that answers vary.
1) Are you going to use UUID column as unique column.
2) what do you with that column.

Process :-
Column is :-



   For Add this :-



And then need to add in uniqueid section.

id



On Thu, May 17, 2018 at 5:00 PM, msaunier  wrote:

> Hello,
>
>
>
> I use SolrCloud 7.0 and I don't have find how to add an UID field on the
> schema. I have add UUIDField type and add a field but it do not work.
>
>
>
>
>
>   {
>
> "name": "uid",
>
> "class": "solr.UUIDField",
>
> "indexed": true,
>
> "stored": true
>
>   },
>
>
>
> Thanks for your help.
>
>
>
>
>
>

Used debug log level on the interface

[msaunier]

Hello,

 

On solrCloud interface, I don't have with solr4j the info and debug level on
the console. In < level > I have add my URP with INFO param and DEBUG param
but never of the two work. I have just WARN and ERROR log on the interface. 

 

Have an idea ?

 

Thanks you,

Add UUID on the schema

[msaunier]

Hello,

 

I use SolrCloud 7.0 and I don't have find how to add an UID field on the
schema. I have add UUIDField type and add a field but it do not work.  

 

 

  {

"name": "uid",

"class": "solr.UUIDField",

"indexed": true,

"stored": true

  },

 

Thanks for your help.

 

 

Re: Question regarding TLS version for solr

[Shawn Heisey]

On 5/17/2018 1:53 AM, Anchal Sharma2 wrote:

We are using solr version 5.3.0 and  have been  trying to enable security on 
our solr .We followed steps mentioned on site 
-https://lucene.apache.org/solr/guide/6_6/enabling-ssl.html .But by default it 
picks ,TLS version  1.0,which is causing an issue as our application uses TLSv 
1.2.We tried using online resources ,but could not find anything regarding TLS 
enablement for solr .

It will be a huge help if anyone can provide some suggestions as to how we can 
enable TLS v 1.2 for solr.


The choice of ciphers and encryption protocols is mostly made by Java.  
The servlet container might influence it as well. The only servlet 
container that is supported since Solr 5.0 is the Jetty that is bundled 
in the Solr download.


TLS 1.2 was added in Java 7, and it became default in Java 8.  If you 
can install the latest version of Java 8 and make sure that it has the 
policy files for unlimited crypto strength installed, support for TLS 
1.2 might happen automatically.


Solr 5.3.0 is running a fairly old version of Jetty -- 9.2.11.  
Information for 9.2.x versions is hard to find, so although I think it 
probably CAN do TLS 1.2 if the Java version supports it, I can't be 
absolutely sure.  You'll need to upgrade Solr to get an upgraded Jetty.


Thanks,
Shawn

Question regarding TLS version for solr

[Anchal Sharma2]

 Hi All,

We are using solr version 5.3.0 and  have been  trying to enable security on 
our solr .We followed steps mentioned on site 
-https://lucene.apache.org/solr/guide/6_6/enabling-ssl.html .But by default it 
picks ,TLS version  1.0,which is causing an issue as our application uses TLSv 
1.2.We tried using online resources ,but could not find anything regarding TLS 
enablement for solr .

It will be a huge help if anyone can provide some suggestions as to how we can 
enable TLS v 1.2 for solr.


Thanks & Regards,
-
Anchal Sharma

Re: Sorting on pseudo field(The one which is added during doctransformer)

[Mikhail Khludnev]

Prateek,
It's too late to sort on transformer result.
You can either sort by function that needs to turn the logic into value
source parser.
If you need to toss just result page, check rerank.

On Thu, May 17, 2018 at 10:03 AM, prateek.agar...@bigbasket.com <
prateek.agar...@bigbasket.com> wrote:

> Hi all,
>
> I wanted to ask is it possible to sort on the field which is added during
> DocTransformer.
> I'm referring to something like this: (https://mariofebbraio.files.
> wordpress.com/2014/10/doctransformer.png) here the price is added using
> doctransformer so is it possible to sort on this field if I add that field
> dynamically to the every returned document. If Yes, Can you please guide me
> like how to go about it.
>
> Can anyone also tell me like at what stage sorting is performed after
> doctransformer or after the requestHandler??
>
> Sorry if you find these queries silly but I'm really confused where all
> this happens.
>
> Thanks in advance.
>
>
> Regards,
> Prateek
>



-- 
Sincerely yours
Mikhail Khludnev

Sorting on pseudo field(The one which is added during doctransformer)

[prateek . agarwal]

Hi all,

I wanted to ask is it possible to sort on the field which is added during 
DocTransformer. 
I'm referring to something like this: 
(https://mariofebbraio.files.wordpress.com/2014/10/doctransformer.png) here the 
price is added using doctransformer so is it possible to sort on this field if 
I add that field dynamically to the every returned document. If Yes, Can you 
please guide me like how to go about it.

Can anyone also tell me like at what stage sorting is performed after 
doctransformer or after the requestHandler??

Sorry if you find these queries silly but I'm really confused where all this 
happens.

Thanks in advance.


Regards,
Prateek