Re: Reg:- Create Solr Core Using Command Line

[@Nandan@]

Hi ,
I created core name as "films". Now I am trying to insert my csv file by
below step:-
C:\solr>curl "http://localhost:8983/solr/films/update?commit=true;
--data-binary @example/books_data.csv -H 'Content-type:application/csv'
Got Below result.
{
  "responseHeader":{
"status":0,
"QTime":279}}

But in Solr Admin UI, can't able to see any data.
please tell me where am i wrong ?
Thanks


On Tue, Feb 6, 2018 at 1:42 PM, Shawn Heisey  wrote:

> On 2/5/2018 10:39 PM, Shawn Heisey wrote:
>
>> In order for this solr script command to work, the argument to the -d
>> option (which you have as C:\solr) would have to be a config directory,
>> containing a minimum of solrconfig.xml and the schema.
>>
> Replying to myself because I made an error here.
>
> The directory provided with -d needs to contain a "conf" subdirectory,
> which in turn must contain the files that I mentioned.
>
> Thanks,
> Shawn
>

Re: [SolrCloud] No config data found after SolrCloud server restart

[A.Guptar]

Hello,

I have removed the -cloud from the startup script and restarted the server.
Issue still persists.

Attached is the solr.log file.

solr.zip   

Thanks for the help.



--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Re[2]: SynonymGraphFilterFactory with WordDelimiterGraphFilterFactory usage

[Александр Шестак]

Hi, thank you for your explanation.
I have one more question related to this topic.
I have changed my schema in next way (replaced SynonymGraphFilterFactory with 
SynonymFilterFactory):

   
      
      
      
   
   
      
      
      
      
   

Now I have another strange issue.
If I have configured synonyms in next way 
b=>b,boron
2=>ii,2
Then for query "my_field:b2" parsedQuery looks so "my_field:b2 
Synonym(my_field:2 my_field:ii)"
But when I changed synonyms to 
b,boron
ii,2
Then for query "my_field:b2" parsedQuery looks so "my_field:b2 my_field:\"b 2\" 
my_field:\"b ii\" my_field:\"boron 2\" my_field:\"boron ii\")"
The second query is correct (it uses synonyms for two parts after word split). 
May be somebody can explain why synonym behavior depends on kind of synonym 
mappings?
And generally is it correct to use SynonymFilterFactory after 
WordDelimiterGraphFilterFactory? We can't use two graph filters together but in 
another way I am forced to use deprecated SynonymFilterFactory?


>Понедельник,  5 февраля 2018, 19:27 +03:00 от Steve Rowe < sar...@gmail.com >:
>
>Hi Александр,
>
>> On Feb 5, 2018, at 11:19 AM, Shawn Heisey <  apa...@elyograg.org > wrote:
>> 
>> There should be no problem with using them together.
>
>I believe Shawn is wrong.
>
>From <  
>http://lucene.apache.org/core/7_2_0/analyzers-common/org/apache/lucene/analysis/synonym/SynonymGraphFilter.html
> >:
>
>> NOTE: this cannot consume an incoming graph; results will be undefined.
>
>Unfortunately, the ref guide entry for Synonym Graph Filter <  
>https://lucene.apache.org/solr/guide/7_2/filter-descriptions.html#synonym-graph-filter
> > doesn’t include a warning about this, but it should, like the warning on 
>Word Delimiter Graph Filter <  
>https://lucene.apache.org/solr/guide/7_2/filter-descriptions.html#word-delimiter-graph-filter
> >:
>
>> Note: although this filter produces correct token graphs, it cannot consume 
>> an input token graph correctly.
>
>(I’ve just committed a change to the ref guide source to add this also on the 
>Synonym Graph Filter and Managed Synonym Graph Filter entries, to be included 
>in the ref guide for Solr 7.3.)
>
>In short, the combination of the two filters is not supported, because WDGF 
>produces a token graph, which SGF cannot correctly interpret.
>
>Other filters also have this issue, see e.g. <  
>https://issues.apache.org/jira/browse/LUCENE-3475 > for ShingleFilter; this 
>issue has gotten some attention recently, and hopefully it will inspire fixes 
>elsewhere.
>
>Patches welcome!
>
>--
>Steve
> www.lucidworks.com
>
>
>> On Feb 5, 2018, at 11:19 AM, Shawn Heisey <  apa...@elyograg.org > wrote:
>> 
>> On 2/5/2018 3:55 AM, Александр Шестак wrote:
>>> 
>>> Hi, I have misunderstanding about usage of SynonymGraphFilterFactory
>>> and  WordDelimiterGraphFilterFactory. Can they be used together?
>>> 
>> 
>> There should be no problem with using them together.  But it is always
>> possible that the behavior will surprise you, while working 100% as
>> designed.
>> 
>>> I have solr type configured in next way
>>> 
>>> >> autoGeneratePhraseQueries="true">
>>>   
>>> 
>>> >> generateWordParts="1" generateNumberParts="1"
>>> splitOnNumerics="1"
>>> catenateWords="1" catenateNumbers="1" catenateAll="0"
>>> preserveOriginal="1" protected="protwords_en.txt"/>
>>> 
>>>   
>>>   
>>> 
>>> >> generateWordParts="1" generateNumberParts="1"
>>> splitOnNumerics="1"
>>> catenateWords="0" catenateNumbers="0" catenateAll="0"
>>> preserveOriginal="1" protected="protwords_en.txt"/>
>>> 
>>> >> synonyms="synonyms_en.txt" ignoreCase="true" expand="true"/>
>>>   
>>> 
>>> 
>>> So on query time it uses SynonymGraphFilterFactory after
>>> WordDelimiterGraphFilterFactory.
>>> Synonyms are configured in next way:
>>> b=>b,boron
>>> 2=>ii,2
>>> 
>>> Query in solr analysis tool looks so. It is shown that terms after SGF
>>> have positions 3 and 4. Is it correct? I thought that they should had
>>> 1 and 2 positions.
>>> 
>> 
>> What matters is the *relative* positions.  The exact position number
>> doesn't matter much.  Something new that the Graph implementations use
>> is the position length.  That feature is necessary for multi-term
>> synonyms to function correctly in phrase queries.
>> 
>> In your analysis screenshot, WDGF creates three tokens.  The two tokens
>> created by splitting the input are at positions 1 and 2, which I think
>> is 100% as expected.  It also sets the positionLength of the first term
>> to 2, probably because it has split that term into 2 additional terms.
>> 
>> Then the SGF takes those last two terms and expands them.  Each of the
>> synonyms is at the same position as the original term, and the relative
>> positions of the two synonym pairs have not changed -- the second one is
>> still one higher than the first.  I think the reason that SGF moves the
>> positions two higher is because the positionLength 

Re: SynonymGraphFilterFactory with WordDelimiterGraphFilterFactory usage

[AlexandrShestak]

Hi, thank you for your explanation.

I have one more question related to this topic.

I have changed my schema in next way (replaced SynonymGraphFilterFactory
with SynonymFilterFactory):

 
  



  
  




  


Now I have another strange issue.
If I have configured synonyms in next way 
b=>b,boron
2=>ii,2

Then for query "my_field:b2" parsedQuery looks so "my_field:b2
Synonym(my_field:2 my_field:ii)"

But when I changed synonyms to 
b,boron
ii,2

Then for query "my_field:b2" parsedQuery looks so "my_field:b2 my_field:\"b
2\" my_field:\"b ii\" my_field:\"boron 2\" my_field:\"boron ii\")"

The second query is correct (it uses synonyms for two parts after word
split). 
May be somebody can explain why synonym behavior depends on kind of synonym
mappings?

And generally is it correct to use SynonymFilterFactory after
WordDelimiterGraphFilterFactory? We can't use two graph filters together but
in another way I am forced to use deprecated SynonymFilterFactory?



--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Re: Reg:- Create Solr Core Using Command Line

[Shawn Heisey]

On 2/5/2018 10:39 PM, Shawn Heisey wrote:
In order for 
this solr script command to work, the argument to the -d option (which 
you have as C:\solr) would have to be a config directory, containing a 
minimum of solrconfig.xml and the schema.

Replying to myself because I made an error here.

The directory provided with -d needs to contain a "conf" subdirectory, 
which in turn must contain the files that I mentioned.


Thanks,
Shawn

Re: Reg:- Create Solr Core Using Command Line

[Shawn Heisey]

On 2/5/2018 7:45 PM, @Nandan@ wrote:

Now I want to create solr core to start my basic search using Solr UI.
Steps which I Follow :-
1) Go to bin directory and start solr
C:\solr\bin>solr start -p 8983
2) books_data.csv is in C:\solr location
3) Now I try to create solr core.
C:\solr\bin>solr create_core -c books_data -d C:\solr


I think that your usage of the -d option is incorrect.  In order for 
this solr script command to work, the argument to the -d option (which 
you have as C:\solr) would have to be a config directory, containing a 
minimum of solrconfig.xml and the schema.  The schema file will 
typically be named either managed-schema (with no extension) or 
schema.xml.  The exact name will depend on what solrconfig.xml contains. 
 If either of those configs reference other files, those files usually 
must also be in that directory.


Alternatively, the argument to the -d option can be the name of one of 
the configset directories in server/solr/configsets.  If -d is not 
included on the commandline, the source of the config will be 
server/solr/configsets/_default.


Once the config source is found, the create_core action will make the 
directory for the new core, make the conf directory inside that, copy 
the conf directory from your -d source, then make an HTTP call to the 
CoreAdmin API to help Solr actually find the core and load it.


Calling the CoreAdmin API yourself when you haven't provided a 
configuration won't work.  See the large warning box with the title 
"CREATE must be able to find a configuration!" at this URL:


https://lucene.apache.org/solr/guide/7_2/coreadmin-api.html

Thanks,
Shawn

Re: Reg:- Create Solr Core Using Command Line

[@Nandan@]

Hi Abhishek,
In my conf sub directory, I put only solrconfig.xml file as per error I was
getting.
I didn't put any other file .

Please check .

Thanks

On Tue, Feb 6, 2018 at 12:57 PM, abhi Abhishek  wrote:

> Hello,
> I followed the steps outlined in your mail. i was able to get a running
> core up fine. only thing I can think of in your case is the config
> directory having all the required files for the SOLR Core to get
> initialized. can you check if you have all the SOLR config files in the
> config directory ( i.e, schema.xml, solrconfig.xml, also various supporting
> files referred in the schema.xml) you specified on the command line?
>
> can you share the conf directory, if you can?
>
> Cheers!
> Abhishek
>
> On Tue, Feb 6, 2018 at 9:30 AM, @Nandan@ 
> wrote:
>
> > Hi Sadiki,
> > I checked Sample Techproduct Conf folder. Inside that folder, there are
> > numerous files. So Again my question will be how those files came.
> > I want to create core from Scratch and want to check and create each and
> > every config files from my sides. Then only I can able to understand what
> > and which files needs in different solr search function.
> > I hope you can understand my query.
> >
> > Thanks
> >
> > On Tue, Feb 6, 2018 at 11:48 AM, Sadiki Latty  wrote:
> >
> > > If I'm not mistaken the command requires that the books_data folder
> > > already exists with a conf folder inside and the various required files
> > > (solrconfig.xml, solr.xml,etc). To get an idea of what you should have
> in
> > > your conf folder you can check out the included configsets
> > > (sample_techproducts_configs for example). These configsets have the
> > > required files and you can copy and modify to accommodate your own
> > needs. I
> > > am not 100% sure where to find them on a windows installation but I
> > believe
> > > it would be C:\solr\server\configsets\  or another subfolder of the
> > server
> > > folder.
> > >
> > > -Original Message-
> > > From: @Nandan@ [mailto:nandanpriyadarshi...@gmail.com]
> > > Sent: Monday, February 5, 2018 9:46 PM
> > > To: solr-user@lucene.apache.org
> > > Subject: Reg:- Create Solr Core Using Command Line
> > >
> > > Hi ,
> > > This question might be very basic, but need to clarify my basic
> > > understanding.
> > > I am using Solr Version 7.2.1
> > > I have one CSV file named as books_data.csv which contains 2
> records.
> > > Now I want to create solr core to start my basic search using Solr UI.
> > > Steps which I Follow :-
> > > 1) Go to bin directory and start solr
> > > C:\solr\bin>solr start -p 8983
> > > 2) books_data.csv is in C:\solr location
> > > 3) Now I try to create solr core.
> > > C:\solr\bin>solr create_core -c books_data -d C:\solr Got Error :- No
> > Conf
> > > Sub folder or Solrconfig.xml file present.
> > > 4) Then I created folder "books_data" in C:\solr location and Created
> > conf
> > > subfolder under books_data folder and put solrconfig.xml inside conf
> > > subfolder.
> > > 5) Again start to execute query
> > > C:\solr\bin>solr create_core -c books_data -d C:\solr\books_data Got
> > Error
> > > :- Already core existed.
> > > When I checked Solr Admin UI , showing error message as SolrCore
> > > Initialization Failures
> > >
> > >- *books_data:*
> > >org.apache.solr.common.SolrException:org.apache.solr.
> > > common.SolrException:
> > >Could not load conf for core books_data: Error loading solr config
> > from
> > >C:\solr\bin\books_data\conf\solrconfig.xml
> > >
> > >
> > > Please tell me where am I doing wrong?
> > > Thanks.
> > >
> >
>

Re: Reg:- Create Solr Core Using Command Line

[abhi Abhishek]

Hello,
I followed the steps outlined in your mail. i was able to get a running
core up fine. only thing I can think of in your case is the config
directory having all the required files for the SOLR Core to get
initialized. can you check if you have all the SOLR config files in the
config directory ( i.e, schema.xml, solrconfig.xml, also various supporting
files referred in the schema.xml) you specified on the command line?

can you share the conf directory, if you can?

Cheers!
Abhishek

On Tue, Feb 6, 2018 at 9:30 AM, @Nandan@ 
wrote:

> Hi Sadiki,
> I checked Sample Techproduct Conf folder. Inside that folder, there are
> numerous files. So Again my question will be how those files came.
> I want to create core from Scratch and want to check and create each and
> every config files from my sides. Then only I can able to understand what
> and which files needs in different solr search function.
> I hope you can understand my query.
>
> Thanks
>
> On Tue, Feb 6, 2018 at 11:48 AM, Sadiki Latty  wrote:
>
> > If I'm not mistaken the command requires that the books_data folder
> > already exists with a conf folder inside and the various required files
> > (solrconfig.xml, solr.xml,etc). To get an idea of what you should have in
> > your conf folder you can check out the included configsets
> > (sample_techproducts_configs for example). These configsets have the
> > required files and you can copy and modify to accommodate your own
> needs. I
> > am not 100% sure where to find them on a windows installation but I
> believe
> > it would be C:\solr\server\configsets\  or another subfolder of the
> server
> > folder.
> >
> > -Original Message-
> > From: @Nandan@ [mailto:nandanpriyadarshi...@gmail.com]
> > Sent: Monday, February 5, 2018 9:46 PM
> > To: solr-user@lucene.apache.org
> > Subject: Reg:- Create Solr Core Using Command Line
> >
> > Hi ,
> > This question might be very basic, but need to clarify my basic
> > understanding.
> > I am using Solr Version 7.2.1
> > I have one CSV file named as books_data.csv which contains 2 records.
> > Now I want to create solr core to start my basic search using Solr UI.
> > Steps which I Follow :-
> > 1) Go to bin directory and start solr
> > C:\solr\bin>solr start -p 8983
> > 2) books_data.csv is in C:\solr location
> > 3) Now I try to create solr core.
> > C:\solr\bin>solr create_core -c books_data -d C:\solr Got Error :- No
> Conf
> > Sub folder or Solrconfig.xml file present.
> > 4) Then I created folder "books_data" in C:\solr location and Created
> conf
> > subfolder under books_data folder and put solrconfig.xml inside conf
> > subfolder.
> > 5) Again start to execute query
> > C:\solr\bin>solr create_core -c books_data -d C:\solr\books_data Got
> Error
> > :- Already core existed.
> > When I checked Solr Admin UI , showing error message as SolrCore
> > Initialization Failures
> >
> >- *books_data:*
> >org.apache.solr.common.SolrException:org.apache.solr.
> > common.SolrException:
> >Could not load conf for core books_data: Error loading solr config
> from
> >C:\solr\bin\books_data\conf\solrconfig.xml
> >
> >
> > Please tell me where am I doing wrong?
> > Thanks.
> >
>

Re: Reg:- Create Solr Core Using Command Line

[@Nandan@]

Hi Sadiki,
I checked Sample Techproduct Conf folder. Inside that folder, there are
numerous files. So Again my question will be how those files came.
I want to create core from Scratch and want to check and create each and
every config files from my sides. Then only I can able to understand what
and which files needs in different solr search function.
I hope you can understand my query.

Thanks

On Tue, Feb 6, 2018 at 11:48 AM, Sadiki Latty  wrote:

> If I'm not mistaken the command requires that the books_data folder
> already exists with a conf folder inside and the various required files
> (solrconfig.xml, solr.xml,etc). To get an idea of what you should have in
> your conf folder you can check out the included configsets
> (sample_techproducts_configs for example). These configsets have the
> required files and you can copy and modify to accommodate your own needs. I
> am not 100% sure where to find them on a windows installation but I believe
> it would be C:\solr\server\configsets\  or another subfolder of the server
> folder.
>
> -Original Message-
> From: @Nandan@ [mailto:nandanpriyadarshi...@gmail.com]
> Sent: Monday, February 5, 2018 9:46 PM
> To: solr-user@lucene.apache.org
> Subject: Reg:- Create Solr Core Using Command Line
>
> Hi ,
> This question might be very basic, but need to clarify my basic
> understanding.
> I am using Solr Version 7.2.1
> I have one CSV file named as books_data.csv which contains 2 records.
> Now I want to create solr core to start my basic search using Solr UI.
> Steps which I Follow :-
> 1) Go to bin directory and start solr
> C:\solr\bin>solr start -p 8983
> 2) books_data.csv is in C:\solr location
> 3) Now I try to create solr core.
> C:\solr\bin>solr create_core -c books_data -d C:\solr Got Error :- No Conf
> Sub folder or Solrconfig.xml file present.
> 4) Then I created folder "books_data" in C:\solr location and Created conf
> subfolder under books_data folder and put solrconfig.xml inside conf
> subfolder.
> 5) Again start to execute query
> C:\solr\bin>solr create_core -c books_data -d C:\solr\books_data Got Error
> :- Already core existed.
> When I checked Solr Admin UI , showing error message as SolrCore
> Initialization Failures
>
>- *books_data:*
>org.apache.solr.common.SolrException:org.apache.solr.
> common.SolrException:
>Could not load conf for core books_data: Error loading solr config from
>C:\solr\bin\books_data\conf\solrconfig.xml
>
>
> Please tell me where am I doing wrong?
> Thanks.
>

RE: Reg:- Create Solr Core Using Command Line

[Sadiki Latty]

If I'm not mistaken the command requires that the books_data folder already 
exists with a conf folder inside and the various required files 
(solrconfig.xml, solr.xml,etc). To get an idea of what you should have in your 
conf folder you can check out the included configsets 
(sample_techproducts_configs for example). These configsets have the required 
files and you can copy and modify to accommodate your own needs. I am not 100% 
sure where to find them on a windows installation but I believe it would be 
C:\solr\server\configsets\  or another subfolder of the server folder.

-Original Message-
From: @Nandan@ [mailto:nandanpriyadarshi...@gmail.com] 
Sent: Monday, February 5, 2018 9:46 PM
To: solr-user@lucene.apache.org
Subject: Reg:- Create Solr Core Using Command Line

Hi ,
This question might be very basic, but need to clarify my basic understanding.
I am using Solr Version 7.2.1
I have one CSV file named as books_data.csv which contains 2 records.
Now I want to create solr core to start my basic search using Solr UI.
Steps which I Follow :-
1) Go to bin directory and start solr
C:\solr\bin>solr start -p 8983
2) books_data.csv is in C:\solr location
3) Now I try to create solr core.
C:\solr\bin>solr create_core -c books_data -d C:\solr Got Error :- No Conf Sub 
folder or Solrconfig.xml file present.
4) Then I created folder "books_data" in C:\solr location and Created conf 
subfolder under books_data folder and put solrconfig.xml inside conf subfolder.
5) Again start to execute query
C:\solr\bin>solr create_core -c books_data -d C:\solr\books_data Got Error :- 
Already core existed.
When I checked Solr Admin UI , showing error message as SolrCore Initialization 
Failures

   - *books_data:*
   org.apache.solr.common.SolrException:org.apache.solr.common.SolrException:
   Could not load conf for core books_data: Error loading solr config from
   C:\solr\bin\books_data\conf\solrconfig.xml


Please tell me where am I doing wrong?
Thanks.

Reg:- Create Solr Core Using Command Line

[@Nandan@]

Hi ,
This question might be very basic, but need to clarify my basic
understanding.
I am using Solr Version 7.2.1
I have one CSV file named as books_data.csv which contains 2 records.
Now I want to create solr core to start my basic search using Solr UI.
Steps which I Follow :-
1) Go to bin directory and start solr
C:\solr\bin>solr start -p 8983
2) books_data.csv is in C:\solr location
3) Now I try to create solr core.
C:\solr\bin>solr create_core -c books_data -d C:\solr
Got Error :- No Conf Sub folder or Solrconfig.xml file present.
4) Then I created folder "books_data" in C:\solr location and Created conf
subfolder under books_data folder and put solrconfig.xml inside conf
subfolder.
5) Again start to execute query
C:\solr\bin>solr create_core -c books_data -d C:\solr\books_data
Got Error :- Already core existed.
When I checked Solr Admin UI , showing error message as
SolrCore Initialization Failures

   - *books_data:*
   org.apache.solr.common.SolrException:org.apache.solr.common.SolrException:
   Could not load conf for core books_data: Error loading solr config from
   C:\solr\bin\books_data\conf\solrconfig.xml


Please tell me where am I doing wrong?
Thanks.

Move the lang-configurations from managed-schema to its own xml file

[S G]

Hi,

I think it would be good to move the lang-configurations from
managed-schema to its own xml file as discussed in
https://issues.apache.org/jira/browse/SOLR-11948

What do other people think?

Thanks
SG

Re: Matching parent docs when two child docs match different criteria

[SuperMattio86]

Thank you! That is interesting. But I think I failed to fully outline the
requirements I have, since I can still contrive an example where this might
not work. If I changed the document as follows:

{
"id":"3",
"_level_":0,
"_version_":1591607734461005824,
"_childDocuments_":[
{
  "id":"3_1",
  "_path_":["p.1"],
  "size":"M",
  "color":"red",
  "_level_":1,
  "_version_":1591607734461005824,
  "_path__str":["p.1"]}]}]
   }
]}

And I made a query similar to what you did: _query_:"{!parent
which=_level_:0}size:M" AND _query_:"{!parent which=_level_:0}size:M"

This document would match. So I'm asking for a different scenario here, but
with the same operation in mind: How do I construct a query that will match
a parent document that has two _different_ child docs, each with size:M. 

Or, in the more general case, how can I construct a query that contains _n_
clauses, where each of the _n_ clauses must match a _different_ child
document. 



--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Min-should-match and Mutli-word synonyms unexpected result

[Nick D]

I have run into an issue with multi-word synonyms and a min-should-match
(MM) of anything other than `0`, *Solr version 6.6.0*.

Here is my example query, first with mm set to zero and the second with a
non-zero value:

With MM set to 0
select?fl=*=on=json=ALL=EIB=ngs_title%20ngs_field_description=false=0

which parse to:

parsedquery_toString":"+(((+ngs_field_description:enterprise
+ngs_field_description:interface +ngs_field_description:builder)
ngs_field_description:eib) | ((+ngs_title:enterprise
+ngs_title:interface +ngs_title:builder) ngs_title:eib))~0.01"

and using my default MM (2<-35%)
select?fl=*=on=json=ALL=EIB=ngs_title%20ngs_field_description=false

which parse to

+ngs_field_description:enterprise +ngs_field_description:interface
+ngs_field_description:builder) ngs_field_description:eib)~2) |
(((+ngs_title:enterprise +ngs_title:interface +ngs_title:builder)
ngs_title:eib)~2))

My synonym here is:
EIB, Enterprise Interface Builder

For my two documents I have the field ngs_title with values "EIB" (Doc 1)
and "enterprise interface builder" (Doc 2)

For both queries the doc 1 is always returned as EIB is matched, but for
doc 2 although I have EIB and Enterprise interface builder defined as
equivalent synonyms when the MM is not set to zero that document is not
returned. From the parsestring I see the ~2 being applied for the MM but my
expectation was that it has been met via the synonyms and the fact that I
am not actaully searching a phrase.

I couldn't find much on the relationship between the two outside of a some
of the things Doug Turnbull had linked to another solr-user question and
this blog post that mentions weirdness around MM and multi-word:

https://lucidworks.com/2017/04/18/multi-word-synonyms-solr-adds-query-time-support/

http://opensourceconnections.com/blog/2013/10/27/why-is-multi-term-synonyms-so-hard-in-solr/

Also looked through the comments here,
https://issues.apache.org/jira/browse/SOLR-9185, but at first glance didn't
see anything that jumped out at me.

Here is the field definition for the ngs_* fields:


  








  
  



  


I am not sure if we cannot use MM anymore for these type of queries or if
there is something I setup incorrectly, any help would be greatly
appreciated.

Nick

Re: 9000+ CLOSE_WAIT connections in solr v6.2.2 causing it to "die"

[Shawn Heisey]

On 2/5/2018 12:29 PM, mmb1234 wrote:
> Maybe this is the issue: https://github.com/eclipse/jetty.project/issues/2169

Could be.  That was one of the Jetty issues I found, and the one that
said 9.4 doesn't have the problem.  Your Solr version would be using
Jetty 9.3, not 9.2.

> The custom client code is:

I don't know a lot about all the code you shared.  But this caught my eye:

> idleConnectionFuture = executor.scheduleAtFixedRate(() -> {
> cmAsync.closeExpiredConnections();
> cmAsync.closeIdleConnections(1, TimeUnit.SECONDS);
> }, 1, 1, TimeUnit.SECONDS);

You're really going to close connections that have been idle for only
one second?  A lot of Solr requests are going to need more than one
second to complete.  As I said in the earlier message, I very much
recommend a socket/idle timeout of at least one minute.  Make it 30
seconds if one minute seems too long.

> Also /solr/admin/cores takes a very long time to respond (QTime = 300secs+). 
> So curl with a timeout also causes an additional CLOSE_WAIT (as expected)
> curl -m 5 'http://:/solr/admin/cores'

It literally takes five minutes for /solr/admin/cores to respond on your
server? That means something is REALLY REALLY wrong.  When I hit it on
one of my servers, which has a few dozen cores, the QTime is 39
milliseconds, and I see the response in the browser very quickly.

> Note there are 100's of solr cores on each solr jvm but only few needed to
> be open at any given time in each solr jvm to avoid heap memory bloat.
> solr.xml has this setting therefore because we're not running solr cloud.
>   ${transientCacheSize:30}

Loading and unloading large cores on a frequent basis is going to make
operations pretty slow.  If your index data is too large to have loaded
all at once, perhaps you need more memory or more servers.

> Also  was removed from solrconfig.xml because we saw 1000's of
> threads BLOCKED on VersionBucket even with.
>name="numVersionBuckets">${solr.ulog.numVersionBuckets:655360}
>
> The side effect is lots of merges [ we'll tackle that when solr stops
> die-ing :-) ]

The presence, absence, or configuration of Solr's transaction log is not
going to affect segment merging in any way.  Segment merging is a
function of Lucene, not Solr, and Lucene has absolutely no knowledge of
Solr features like the transaction log.  I would strongly recommend that
you do NOT disable the transaction log.  Doing so might speed things up
a very small amount, but that speedup is NOT a good tradeoff,
considering that you'd be more likely to lose data in the event of a
serious problem.

Lots of segment merges is a result of doing a lot of indexing.

Thanks,
Shawn

Re: analysis error - cannot change numBytes from 4 to 8

[Erick Erickson]

Steve:

Did this work in the past but just stopped? If so, did you change your
schema at all?

If you changed your schema at all, you probably have to reindex from
scratch. However, you _must_ be absolutely sure you've removed all
traces of your old index, I usually just create a new colletion to
test.

Finally you say "it's potentially multivalued". So you have to specify
the field multiple times, as:
6157731300
6157731301

NOT
6157731300 6157731301

Best,
Erick

On Mon, Feb 5, 2018 at 9:15 AM, Steve Pruitt  wrote:
> I have a collection with a field defined as plongs.  The field schema is:  
>  required="false" stored="true"/>
>
> For each document to add, I build up a HashMap of field values.   Since its 
> multi-valued field, I create an ArrayList of type Long and add the values to 
> it.  The ArrayList is added to the HashMap.
> Next, I create a SolrInputDocument from the HashMap.  I add each 
> SolrInputDocument to a Collection of type SolrInputDocument.
>
> To update, I call  add on my CloudSolrClient instance with the collection.
>
> I'm running with test documents and each document has the same value:   name="ani">6157731300.  Its potentially multivalued.
>
> I get this exception:
>
> Request to collection [explore] failed due to (400) 
> org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error 
> from server at http://myHost:8983/solr/explore_shard2_replica_n2: Exception 
> writing document id 1427388717...1343947742 to the index; possible analysis 
> error: cannot change point numBytes from 4 to 8 for field="ani"
>
> Not sure what this error means exactly.
>
>
> Thanks.
>
> -S

I'm unable to negate a join query using the Spring Java interface to Solr

[Todd Stevenson]

I have the following query:

{!join from=conceptNcid to=ncid fromIndex=hdd_rel_tst}relationshipNcid:364

which I implemented in Spring:

   Criteria criteria = new Criteria("relationshipNcid").is("364");

SimpleQuery search = new SimpleQuery(criteria, pageable);

search.setJoin(Join.from("conceptNcid").fromIndex("hdd_rel_tst").to("ncid"));

Page results = solrTemplate.queryForPage(search, myclass);

This query works great.

However, when I attempt to negate the query (using the minus sign at the 
beginning):

-{!join from=conceptNcid to=ncid fromIndex=hdd_rel_tst}relationshipNcid:364

with this code:

   Criteria criteria = new Criteria("relationshipNcid").is("364").not();

SimpleQuery search = new SimpleQuery(criteria, pageable);

search.setJoin(Join.from("conceptNcid").fromIndex("hdd_rel_tst").to("ncid"));

Page results = solrTemplate.queryForPage(search, myclass);

it does not work.  Notice where I put the 'not()' on the first line.  This 
actual executes this query:

{!join from=conceptNcid to=ncid fromIndex=hdd_rel_tst}-relationshipNcid:364


I know this is the wrong place, but I can't see how to negate the entire query 
in the java code.   It seems to need a 'not()' method for the SimpleQuery class 
since the join is placed on the query,  but I can't see anything in the 
documentation.

Any ideas?

Todd Stevenson

Re: Mixing simple and nested docs in same update?

[Mikhail Khludnev]

Jan, mixing plan docs and blocks are not supported.

On Thu, Jan 11, 2018 at 2:42 AM, Jan Høydahl  wrote:

> Hi,
>
> We index several large nested documents. We found that querying the data
> behaves differently depending on how the documents are indexed.
>
> To reproduce:
>
> solr start
> solr create -c nested
> # Index one plain document, “friend" and a nested one, “mother” and
> “daughter”, in same request:
> curl localhost:8983/solr/nested/update -d ‘
>  
>
>  friend
>  other
>
>
>  mother
>  parent
>  
>daughter
>child
>  
>
>  '
>
> # Query for mother’s children using either child transformer or child
> query parser
> curl "localhost:8983/solr/a/query?q=id:mother=%2A%2C%5Bchild%
> 20parentFilter%3Dtype%3Aparent%5D”
> {
>   "responseHeader":{
> "zkConnected":true,
> "status":0,
> "QTime":4,
> "params":{
>   "q":"id:mother",
>   "fl":"*,[child parentFilter=type:parent]"}},
>   "response":{"numFound":1,"start":0,"docs":[
>   {
> "id":"mother",
> "type":["parent"],
> "_version_":1589249812802306048,
> "type_str":["parent"],
> "_childDocuments_":[
> {
>   "id":"friend",
>   "type":["other"],
>   "_version_":1589249812729954304,
>   "type_str":["other"]},
> {
>   "id":"daughter",
>   "type":["child"],
>   "_version_":1589249812802306048,
>   "type_str":["child"]}]}]
>   }}
>
> As you can see, the “friend” got included as a child of “mother”.
> If you index the exact same request, putting “friend” after “mother” in
> the xml,
> the query works as expected.
>
> Inspecting the index, everything looks correct, and only “daughter” and
> “mother” have _root_=mother.
> Is there a rule that you should start a new update request for each type
> of parent/child relationship
> that you need to index, and not mix them in the same request?
>
> --
> Jan Høydahl, search solution architect
> Cominvent AS - www.cominvent.com
>
>


-- 
Sincerely yours
Mikhail Khludnev

Re: Matching parent docs when two child docs match different criteria

[Mikhail Khludnev]

q=+{!parent .. v=$m} +{!parent .. v=$l}=size:(+M -L)=size:(+L -M)

On Sun, Feb 4, 2018 at 10:56 PM, SuperMattio86 
wrote:

> Hello.
>
> As a toy example, let's assume I have documents that looks like this:
>
> {
>   "id": "1",
>   "_childDocuments_": [
>  {
> "id": "1_2",
> "_path_": "p.1",
>  "size": "M",
>  "color": "red"
>   },
>  {
> "id": "23lk4d23lkj",
> "_path_": "p.2",
>  "size": "L",
>  "color": "blue"
> }
>   ]
> }
>
> Is it possible to construct a query that will match a parent doc that has a
> child doc that say matches 'size:L' and a DIFFERENT child that matches
> 'size:M'? Is such a query possible? What would it look like? Or would I
> need
> to implement a custom query parser to do this?
>
> I can currently do this without sub-documents, by indexing those values as
> arrays, and creating a custom scorer/collector that collects positions in
> addition to docs, but since I am now switching to a version of Solr that
> has
> sub-documents, I was hoping for a better solution that more intuitively
> models my data structures.
>
>
>
>
>
>
> --
> Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
>



-- 
Sincerely yours
Mikhail Khludnev

Re: 9000+ CLOSE_WAIT connections in solr v6.2.2 causing it to "die"

[mmb1234]

Maybe this is the issue: https://github.com/eclipse/jetty.project/issues/2169

I have noticed when number of http requests / sec are increased, CLOSE_WAITS
increase linearly until solr stops accepting socket connections. Netstat
output is
$ netstat -ptan | awk '{print $6 " " $7 }' | sort | uniq -c
   9784 CLOSE_WAIT 1/java
  1 ESTABLISHED -
  3 ESTABLISHED 1/java
  1 Foreign Address
  2 LISTEN -
  4 LISTEN 1/java
 27 TIME_WAIT -
  1 established)


The custom client code is:
RequestConfig config = RequestConfig.custom()
.setConnectionRequestTimeout(4000)
.setConnectTimeout(4000)
.setSocketTimeout(4000)
.build();

ConnectingIOReactor ioReactor = new
DefaultConnectingIOReactor();
PoolingNHttpClientConnectionManager cmAsync = new
PoolingNHttpClientConnectionManager(ioReactor);
cmAsync.setMaxTotal(1);
cmAsync.setDefaultMaxPerRoute(1000);

asynClient = HttpAsyncClients.custom()
.setDefaultRequestConfig(config)
.setConnectionManager(cmAsync).build();
asynClient.start();

executor = Executors.newScheduledThreadPool(1);
idleConnectionFuture = executor.scheduleAtFixedRate(() -> {
cmAsync.closeExpiredConnections();
cmAsync.closeIdleConnections(1, TimeUnit.SECONDS);
}, 1, 1, TimeUnit.SECONDS);


Also /solr/admin/cores takes a very long time to respond (QTime = 300secs+). 
So curl with a timeout also causes an additional CLOSE_WAIT (as expected)
curl -m 5 'http://:/solr/admin/cores' 

customhandler QTime = ~150 (ms) or lower even under max load for the active
cores.
 
Note there are 100's of solr cores on each solr jvm but only few needed to
be open at any given time in each solr jvm to avoid heap memory bloat.
solr.xml has this setting therefore because we're not running solr cloud.
  ${transientCacheSize:30}

Also  was removed from solrconfig.xml because we saw 1000's of
threads BLOCKED on VersionBucket even with.
  ${solr.ulog.numVersionBuckets:655360}

The side effect is lots of merges [ we'll tackle that when solr stops
die-ing :-) ]



--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Re: Solr join query takes too long

[Mikhail Khludnev]

Hello.
There is no way to make it work fast. It executes expensive join operation
for all docs/terms and then post filters with resulting docset.

On Mon, Feb 5, 2018 at 9:53 AM, Aashish Agarwal  wrote:

> Hi
>
> I am using join query that joins 2 cores to get result. Since, number of
> docs in both the cores are too high so I want to use post filter with join
> query. But that is not working by providing cost=100.
>
> Is it implemented for join filter query? I use solr 4.6.0
>
> Thanks,
> Aashish
>



-- 
Sincerely yours
Mikhail Khludnev

Re: SynonymGraphFilterFactory with WordDelimiterGraphFilterFactory usage

[Shawn Heisey]

On 2/5/2018 9:27 AM, Steve Rowe wrote:
> I believe Shawn is wrong.

Not happy to be wrong, but glad for your assist and making sure that the
right information is provided.

Sounds like it's just not possible to use multiple graph-aware filters
together.

Thanks,
Shawn

analysis error - cannot change numBytes from 4 to 8

[Steve Pruitt]

I have a collection with a field defined as plongs.  The field schema is:  


For each document to add, I build up a HashMap of field values.   Since its 
multi-valued field, I create an ArrayList of type Long and add the values to 
it.  The ArrayList is added to the HashMap.
Next, I create a SolrInputDocument from the HashMap.  I add each 
SolrInputDocument to a Collection of type SolrInputDocument.

To update, I call  add on my CloudSolrClient instance with the collection.

I'm running with test documents and each document has the same value:  6157731300.  Its potentially multivalued.

I get this exception:

Request to collection [explore] failed due to (400) 
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error 
from server at http://myHost:8983/solr/explore_shard2_replica_n2: Exception 
writing document id 1427388717...1343947742 to the index; possible analysis 
error: cannot change point numBytes from 4 to 8 for field="ani"

Not sure what this error means exactly.


Thanks.

-S

Re: SynonymGraphFilterFactory with WordDelimiterGraphFilterFactory usage

[Steve Rowe]

Hi Александр,

> On Feb 5, 2018, at 11:19 AM, Shawn Heisey  wrote:
> 
> There should be no problem with using them together.

I believe Shawn is wrong.

From 
:

> NOTE: this cannot consume an incoming graph; results will be undefined.

Unfortunately, the ref guide entry for Synonym Graph Filter 

 doesn’t include a warning about this, but it should, like the warning on Word 
Delimiter Graph Filter 
:

> Note: although this filter produces correct token graphs, it cannot consume 
> an input token graph correctly.

(I’ve just committed a change to the ref guide source to add this also on the 
Synonym Graph Filter and Managed Synonym Graph Filter entries, to be included 
in the ref guide for Solr 7.3.)

In short, the combination of the two filters is not supported, because WDGF 
produces a token graph, which SGF cannot correctly interpret.

Other filters also have this issue, see e.g. 
 for ShingleFilter; this 
issue has gotten some attention recently, and hopefully it will inspire fixes 
elsewhere.

Patches welcome!

--
Steve
www.lucidworks.com


> On Feb 5, 2018, at 11:19 AM, Shawn Heisey  wrote:
> 
> On 2/5/2018 3:55 AM, Александр Шестак wrote:
>> 
>> Hi, I have misunderstanding about usage of SynonymGraphFilterFactory
>> and  WordDelimiterGraphFilterFactory. Can they be used together?
>> 
> 
> There should be no problem with using them together.  But it is always
> possible that the behavior will surprise you, while working 100% as
> designed.
> 
>> I have solr type configured in next way
>> 
>> > autoGeneratePhraseQueries="true">
>>   
>> 
>> > generateWordParts="1" generateNumberParts="1"
>> splitOnNumerics="1"
>> catenateWords="1" catenateNumbers="1" catenateAll="0"
>> preserveOriginal="1" protected="protwords_en.txt"/>
>> 
>>   
>>   
>> 
>> > generateWordParts="1" generateNumberParts="1"
>> splitOnNumerics="1"
>> catenateWords="0" catenateNumbers="0" catenateAll="0"
>> preserveOriginal="1" protected="protwords_en.txt"/>
>> 
>> > synonyms="synonyms_en.txt" ignoreCase="true" expand="true"/>
>>   
>> 
>> 
>> So on query time it uses SynonymGraphFilterFactory after
>> WordDelimiterGraphFilterFactory.
>> Synonyms are configured in next way:
>> b=>b,boron
>> 2=>ii,2
>> 
>> Query in solr analysis tool looks so. It is shown that terms after SGF
>> have positions 3 and 4. Is it correct? I thought that they should had
>> 1 and 2 positions.
>> 
> 
> What matters is the *relative* positions.  The exact position number
> doesn't matter much.  Something new that the Graph implementations use
> is the position length.  That feature is necessary for multi-term
> synonyms to function correctly in phrase queries.
> 
> In your analysis screenshot, WDGF creates three tokens.  The two tokens
> created by splitting the input are at positions 1 and 2, which I think
> is 100% as expected.  It also sets the positionLength of the first term
> to 2, probably because it has split that term into 2 additional terms.
> 
> Then the SGF takes those last two terms and expands them.  Each of the
> synonyms is at the same position as the original term, and the relative
> positions of the two synonym pairs have not changed -- the second one is
> still one higher than the first.  I think the reason that SGF moves the
> positions two higher is because the positionLength on the "b2" term is
> 2, previously set by WDGF.  Someone with more knowledge about the Graph
> implementations may have to speak up as to whether this behavior is correct.
> 
> Because the relative positions of the split terms don't change when SGF
> runs, I think this is probably working as designed.
> 
> Thanks,
> Shawn

Re: [SolrCloud] No config data found after SolrCloud server restart

[Shawn Heisey]

On 2/5/2018 5:13 AM, A.Guptar wrote:
> We are having issues with SolrCloud 5.4.1 instances after the hosting
> servers have been restarted.
>
> Scenario:
> We have 3 servers, on which SolrCloud 5.4.1 was configured and setup to be
> used with an external ZooKeeper ensemble.
> Once configured and setup, it works fine until there is a full server
> restart.
> The error "No config data found at path xxx" is obtained.
> As a workaround, we stopped the ZooKeeper and SolrCloud services and deleted
> all the relevant data before re-creating everything from scratch.
>
> I use the following commands for creating the config and collections in ZK
> and SolrCloud.
>
> function UploadZooKeeperConfig {
> ...
> Start-Process -FilePath ${zkCliPath}\${zkcli} -ArgumentList "-zkhost
> ${zkHost}:${zkPort} -cmd upconfig -confdir ${configPath} -confname
> $confName"
> ...
> }
>
> UploadZooKeeperConfig -configPath "D:\basic_configs\conf" -confName
> "TestCollectionConfig" -zkHost "x.x.x.x"
>
> $expression =
> "http://localhost:8983/solr/admin/collections?action=CREATE=TestCollection=1=3=TestCollectionConfig;
>
> Invoke-WebRequest -Uri $expression
>
> The following command is used to start SolrCloud.
>
> bin\solr start -cloud -z "xxx:2181,yyy:2181,zzz:2181" -m 16g -f

That all looks reasonable.  I don't know what language your collection
build script is written in, but I'm going to assume that the syntax is
correct for that language.

> Below is the zoo.cfg used within SolrCloud.

If you are using an external ZK ensemble, then the zoo.cfg for the
embedded ZK is not going to be used.  So if that zoo.cfg file is part of
your Solr install, it's probably not being used.  The zookeeper config
on your ZK servers is what's going to be important.  If that's what this
is, then I think it's probably OK, but I'm not a ZK expert.  You should
remove the "-cloud" option from the Solr startup, just to be absolutely
sure that you are not starting the embedded ZK.  The -z option is enough
to start in SolrCloud mode.

Can you provide the entire solr.log file from a startup that shows the
error?  If you absolutely need to redact sensitive information from the
logfile, do so.  I don't think there is normally much in the log that is
really sensitive.  If you have changed the default logging level away
from INFO, please set it back to INFO before performing the restart.

Thanks,
Shawn

Re: SynonymGraphFilterFactory with WordDelimiterGraphFilterFactory usage

[Shawn Heisey]

On 2/5/2018 3:55 AM, Александр Шестак wrote:
>
> Hi, I have misunderstanding about usage of SynonymGraphFilterFactory
> and  WordDelimiterGraphFilterFactory. Can they be used together?
>

There should be no problem with using them together.  But it is always
possible that the behavior will surprise you, while working 100% as
designed.

> I have solr type configured in next way
>
>  autoGeneratePhraseQueries="true">
>   
> 
>  generateWordParts="1" generateNumberParts="1"
> splitOnNumerics="1"
> catenateWords="1" catenateNumbers="1" catenateAll="0"
> preserveOriginal="1" protected="protwords_en.txt"/>
> 
>   
>   
> 
>  generateWordParts="1" generateNumberParts="1"
> splitOnNumerics="1"
> catenateWords="0" catenateNumbers="0" catenateAll="0"
> preserveOriginal="1" protected="protwords_en.txt"/>
> 
>  synonyms="synonyms_en.txt" ignoreCase="true" expand="true"/>
>   
> 
>
> So on query time it uses SynonymGraphFilterFactory after
> WordDelimiterGraphFilterFactory.
> Synonyms are configured in next way:
> b=>b,boron
> 2=>ii,2
>
> Query in solr analysis tool looks so. It is shown that terms after SGF
> have positions 3 and 4. Is it correct? I thought that they should had
> 1 and 2 positions.
>

What matters is the *relative* positions.  The exact position number
doesn't matter much.  Something new that the Graph implementations use
is the position length.  That feature is necessary for multi-term
synonyms to function correctly in phrase queries.

In your analysis screenshot, WDGF creates three tokens.  The two tokens
created by splitting the input are at positions 1 and 2, which I think
is 100% as expected.  It also sets the positionLength of the first term
to 2, probably because it has split that term into 2 additional terms.

Then the SGF takes those last two terms and expands them.  Each of the
synonyms is at the same position as the original term, and the relative
positions of the two synonym pairs have not changed -- the second one is
still one higher than the first.  I think the reason that SGF moves the
positions two higher is because the positionLength on the "b2" term is
2, previously set by WDGF.  Someone with more knowledge about the Graph
implementations may have to speak up as to whether this behavior is correct.

Because the relative positions of the split terms don't change when SGF
runs, I think this is probably working as designed.

Thanks,
Shawn

Re: using gatherNodes to produce 'to' -> 'from' pairs

[Joel Bernstein]

There is traceability built in also, but I'm not sure if it provides enough
features for your use case.

In the docs you'll see there is a trackTraversal flag which outputs the
ancestors that point to each node as an array. You can use the
cartesianProduct() expression to expand the ancestors array into tuples and
fetch using the ancestors node IDs.




Joel Bernstein
http://joelsolr.blogspot.com/

On Mon, Feb 5, 2018 at 5:20 AM, Gintautas Sulskus <
gintautas.suls...@gmail.com> wrote:

> Hi Joel,
>
> Thanks. The alternatives will not be as neat, unfortunately.
>
> fetch() does not suit the needs because it cannot trace back the root node
> given only the 'from' value.
> The node pairs below would produce a result set of {'b1', 'b2', 'b1'} and
> there is no way to tell which root element they correspond to.
>
> 'from' - 'to'
> a1 -> b1
> a1 -> b2
> a2 -> b1
>
>
> Best,
> Gintas
>
> On Sun, Feb 4, 2018 at 11:36 PM, Joel Bernstein 
> wrote:
>
> > Currently you can only directly gather nodes and aggregations. The
> fetch()
> > expression can be used to fetch more fields if needed in a secondary
> call.
> >
> > Joel Bernstein
> > http://joelsolr.blogspot.com/
> >
> > On Sun, Feb 4, 2018 at 3:12 PM, Gintautas Sulskus <
> > gintautas.suls...@gmail.com> wrote:
> >
> > > Hi,
> > >
> > > By default, gatherNodes returns only a set of 'to' nodes.
> > > Is there a neat way to return a set of all maplets 'from' -> 'to'?
> > >
> > >
> > > Thanks,
> > > Gin
> > >
> >
>

Re: No Suggestions from SpellCheck when _text_ field tokenizer set to solr.NGramTokenizerFactory

[Alessandro Benedetti]

How is this field type defined : textSpell ?
Can you detail what it is not working as expected ?

Thanks



-
---
Alessandro Benedetti
Search Consultant, R Software Engineer, Director
Sease Ltd. - www.sease.io
--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Re: No Suggestions from SpellCheck when _text_ field tokenizer set to solr.NGramTokenizerFactory

[Anupam Bhattacharya]

Following is the configuration related to Spell check in the
Solr-config.xml file.


  textSpell
  
  title_txt_spellcheck_ja
  title_txt_spellcheck_ja
  true
  true
  ./spellchecker_en
  
  
  title_txt_spellcheck_en
  title_txt_spellcheck_en
  true
  true
  ./spellchecker_de
  
  
  


  

  
  default
  on
  true
  10
  5
  5
  true
  true
  10
  5


  spellcheck

  

Regards,
Anupam

On Mon, Feb 5, 2018 at 5:45 PM, Alessandro Benedetti 
wrote:

> Hi, how is your spellcheck dictionary :
> "spellcheck.dictionary=title_txt_spellcheck_ja" defined in the
> solrconfig.xml?
>
> Regards
>
>
>
> -
> ---
> Alessandro Benedetti
> Search Consultant, R Software Engineer, Director
> Sease Ltd. - www.sease.io
> --
> Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
>



-- 
Thanks & Regards
Anupam Bhattacharya

Re: Using SolrJ for digest authentication

[ddramireddy]

It's running only once. When request went from client from tomcat first-time,
tomcat sent response with "WWW-Authenticate" header with nonce value, asking
to complete the challenge. This challenge is processed. But apache http
client library is checking whether request is "repeatable" or not. This
validation is getting failed and following error is thrown.

"org.apache.http.client.NonRepeatableRequestException: Cannot retry request
with a non-repeatable request entity."

I am able to make POST call by directly calling apache http client library
with the following code. But, when I used solrj, I am getting this error.
"org.apache.http.client.NonRepeatableRequestException: Cannot retry request
with a non-repeatable request entity.". I can see solrj is setting
"isrepeatable" to false in httpsolrclient.java

Working code - When called directly using apache http library:

HttpHost target = HttpHost.create("localhost:9000");
CredentialsProvider credsProvider = new BasicCredentialsProvider();
credsProvider.setCredentials(
new AuthScope(target.getHostName(), target.getPort()),
new UsernamePasswordCredentials("solr", "testpassword"));
CloseableHttpClient httpclient = HttpClients.custom()
.setDefaultCredentialsProvider(credsProvider)
.build();
try {

AuthCache authCache = new BasicAuthCache();
DigestScheme digestAuth = new DigestScheme();
digestAuth.overrideParamter("realm", "Solr");
authCache.put(target, digestAuth);

// Add AuthCache to the execution context
HttpClientContext localContext = HttpClientContext.create();
localContext.setAuthCache(authCache);


HttpPost httpPost = new
HttpPost("http://localhost:9000/solr/account/update?commit=true;);
httpPost.setHeader("Content-type", "text/xml");
StringEntity stringEntity = new
StringEntity("*:*");
httpPost.getRequestLine();
httpPost.setEntity(stringEntity);
System.out.println("Executing request " +
httpPost.getRequestLine() + " to target " + target);
for (int i = 0; i < 3; i++) {
CloseableHttpResponse response =
httpclient.execute(httpPost, localContext);
try {
   
System.out.println("");
System.out.println(response.getStatusLine());
   
System.out.println(EntityUtils.toString(response.getEntity()));
} finally {
response.close();
}
}

} finally {
httpclient.close();
} 


Failing code:


public class TestSolrWithDigest {
public static void main(String[] args) {
try {
HttpSolrClient solrClient = initialiseSOLRClient();
try {

solrClient.deleteByQuery("account", "*:*");
UpdateResponse updateResponse =
solrClient.commit("account");
System.out.println(updateResponse.getStatus());
System.out.println("completed");
final ModifiableSolrParams pStatus = new
ModifiableSolrParams();
pStatus.add("qt", "/dataimport");

final QueryResponse resp = solrClient.query("account",
pStatus, SolrRequest.METHOD.GET);
System.out.println("status:" + resp.getStatus());

} catch (final SolrServerException | IOException exn) {

throw new IllegalStateException(exn);
}

} catch (Exception e) {
e.printStackTrace(System.err);
}


}

private static HttpSolrClient initialiseSOLRClient() {
URL solrURL = null;
try {
solrURL = new URL("http://localhost:9000/solr;);
} catch (MalformedURLException e) {
System.out.println("Cannot parse the SOLR URL!!" +
"http://localhost:9000/solr;);

}
String host = solrURL.getHost();
int port = solrURL.getPort();

AuthScope authScope = new AuthScope(host, port, "Solr", "Digest");

UsernamePasswordCredentials creds = new
UsernamePasswordCredentials("solr", "testpassword");

CredentialsProvider credsProvider = new BasicCredentialsProvider();
credsProvider.setCredentials(
authScope,
creds);

HttpClientBuilder builder = create();
builder.addInterceptorFirst(new PreemptiveAuthInterceptor());
builder.setDefaultCredentialsProvider(credsProvider);
CloseableHttpClient httpClient = builder.build();

return new HttpSolrClient.Builder()
.withBaseSolrUrl("http://localhost:9000/solr;)
.withHttpClient(httpClient)
.build();
}

static class PreemptiveAuthInterceptor implements HttpRequestInterceptor
{

public void process(final 

Re: No Suggestions from SpellCheck when _text_ field tokenizer set to solr.NGramTokenizerFactory

[Alessandro Benedetti]

Hi, how is your spellcheck dictionary :
"spellcheck.dictionary=title_txt_spellcheck_ja" defined in the
solrconfig.xml?

Regards



-
---
Alessandro Benedetti
Search Consultant, R Software Engineer, Director
Sease Ltd. - www.sease.io
--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

[SolrCloud] No config data found after SolrCloud server restart

[A.Guptar]

Hello,

We are having issues with SolrCloud 5.4.1 instances after the hosting
servers have been restarted.

Scenario:
We have 3 servers, on which SolrCloud 5.4.1 was configured and setup to be
used with an external ZooKeeper ensemble.
Once configured and setup, it works fine until there is a full server
restart.
The error "No config data found at path xxx" is obtained.
As a workaround, we stopped the ZooKeeper and SolrCloud services and deleted
all the relevant data before re-creating everything from scratch.

I use the following commands for creating the config and collections in ZK
and SolrCloud.

function UploadZooKeeperConfig {
...
Start-Process -FilePath ${zkCliPath}\${zkcli} -ArgumentList "-zkhost
${zkHost}:${zkPort} -cmd upconfig -confdir ${configPath} -confname
$confName"
...
}

UploadZooKeeperConfig -configPath "D:\basic_configs\conf" -confName
"TestCollectionConfig" -zkHost "x.x.x.x"

$expression =
"http://localhost:8983/solr/admin/collections?action=CREATE=TestCollection=1=3=TestCollectionConfig;

Invoke-WebRequest -Uri $expression

The following command is used to start SolrCloud.

bin\solr start -cloud -z "xxx:2181,yyy:2181,zzz:2181" -m 16g -f

Below is the zoo.cfg used within SolrCloud.

# The number of milliseconds of each tick
tickTime=2000
# The number of ticks that the initial
# synchronization phase can take
initLimit=10
# The number of ticks that can pass between
# sending a request and getting an acknowledgement
syncLimit=5
# the directory where the snapshot is stored.
dataDir=D:\zookeeper\data\1
# NOTE: Solr defaults the dataDir to /zoo_data
# the port at which the clients will connect
clientPort=2181
# NOTE: Solr sets this based on zkRun / zkHost params
server.1=xxx:2888:3888
server.2=yyy:2888:3888
server.3=zzz:2888:3888

Please let me know if there is anything that is required to prevent files to
be deleted upon server restart.

Thanks.



--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

SynonymGraphFilterFactory with WordDelimiterGraphFilterFactory usage

[AlexandrShestak]

Hi, I have misunderstanding about usage of SynonymGraphFilterFactory and 
WordDelimiterGraphFilterFactory. Can they be used together?

I have solr type configured in next way


  



  
  




  


So on query time it uses SynonymGraphFilterFactory after
WordDelimiterGraphFilterFactory.
Synonyms are configured in next way:
b=>b,boron
2=>ii,2

Query in solr analysis tool looks so. It is shown that terms after SGF have
positions 3 and 4. Is it correct? I thought that they should had 1 and 2
positions.

Now when a perform such query "my_field:b2"  then parsedQuery looks so
"my_field:b2 Synonym(my_field:2 my_field:ii)"
But for such query "my_field:2b"  parsedQuery will be such  "my_field:2b
Synonym(my_field:b my_field:boron)"

Synonym works only with last part of my query. 

Am I doing something wrong? Or maybe this filters are incompatible?

 



--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Re: facet.method=uif not working in solr cloud?

[Alessandro Benedetti]

What happens if you set f.color.facet.mincount=1 from the beginning ?
Are you faceting on multiple fields ?

Good news we found out the first mistery though ! :)

Cheers



-
---
Alessandro Benedetti
Search Consultant, R Software Engineer, Director
Sease Ltd. - www.sease.io
--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Re: The Expected Suggestions Is Phrase Instead Of Whole Contents From Field

[Alessandro Benedetti]

I am sorry but what you get is the expected behavior [1]

Maybe you are interested to the free text lookup approach[2].

Cheers

[1] https://sease.io/2015/07/solr-you-complete-me.html
[2]
https://lucene.apache.org/solr/guide/6_6/suggester.html#Suggester-FreeTextLookupFactory





-
---
Alessandro Benedetti
Search Consultant, R Software Engineer, Director
Sease Ltd. - www.sease.io
--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

SynonymGraphFilterFactory with WordDelimiterGraphFilterFactory usage

[Александр Шестак]

Hi, I have misunderstanding about usage of  SynonymGraphFilterFactory and  
WordDelimiterGraphFilterFactory. Can they be used together?

I have solr type configured in next way


  



  
  




  


So on query time it uses SynonymGraphFilterFactory after 
WordDelimiterGraphFilterFactory.
Synonyms are configured in next way:
b=>b,boron
2=>ii,2

Query in solr analysis tool looks so. It is shown that terms after SGF have 
positions 3 and 4. Is it correct? I thought that they should had 1 and 2 
positions.
Now when a perform such query "my_field:b2"  then parsedQuery looks so  " 
my_field:b2 Synonym(my_field:2 my_field:ii)"
But for such query "my_field:2b"  parsedQuery will be such   " my_field:2b 
Synonym( my_field :b  my_field :boron)"
Synonym works only with last part of my query. 
Am I doing something wrong? Or maybe this filters are incompatible?




Now when a perform such query "my_field:b2"  then parsedQuery looks so 
"my_field:b2 Synonym(my_field:2 my_field:ii)"
But for such query "my_field:2b"  parsedQuery will be such  "my_field:2b 
Synonym(my_field:b my_field:boron)"

Re: using gatherNodes to produce 'to' -> 'from' pairs

[Gintautas Sulskus]

Hi Joel,

Thanks. The alternatives will not be as neat, unfortunately.

fetch() does not suit the needs because it cannot trace back the root node
given only the 'from' value.
The node pairs below would produce a result set of {'b1', 'b2', 'b1'} and
there is no way to tell which root element they correspond to.

'from' - 'to'
a1 -> b1
a1 -> b2
a2 -> b1


Best,
Gintas

On Sun, Feb 4, 2018 at 11:36 PM, Joel Bernstein  wrote:

> Currently you can only directly gather nodes and aggregations. The fetch()
> expression can be used to fetch more fields if needed in a secondary call.
>
> Joel Bernstein
> http://joelsolr.blogspot.com/
>
> On Sun, Feb 4, 2018 at 3:12 PM, Gintautas Sulskus <
> gintautas.suls...@gmail.com> wrote:
>
> > Hi,
> >
> > By default, gatherNodes returns only a set of 'to' nodes.
> > Is there a neat way to return a set of all maplets 'from' -> 'to'?
> >
> >
> > Thanks,
> > Gin
> >
>