Re: Filter Query that matches all values of a field

[Chris Hostetter]

:  I have a single type field that can contain zero or more values (comma
: separated values). This field stores some sort of access value.
: 
: In the filter, I am given a list of allowed values for the field and a
: document must be considered if all values contained in its field must be
: present in the allowed values specified in the filter.
: How can i write filter query for this?

My prefered solution is:

1) index the *unique* values as a multivalued StrField 
   (ex: foo)
2) create a second field containing the *count* of unique values, 
   CountFieldValuesUpdateProcessorFactory makes this trivial 
   (ex: foo_count)

3) query/filter using the frange parser with l & h both =0, and the 
function being foo_count minus the sum of the termfreq results for each 
value the user posesses.

Examples query...

:1. Case #1) If the allowed values specified in the filter are (a1, a3,
:a4, a6) --> the document should not be considered since user doesn’t have

fq={!frange l=0 h=0}sub(foo_count,
sum(termfreq(foo,'a1),
termfreq(foo,'a3'),
termfreq(foo,'a4'),
termfreq(foo,'a6')))


-Hoss
http://www.lucidworks.com/

Re: Disabling solr scoring

[Chris Hostetter]

: Can you please elaborate? I am passing user defined sort field and order 
whenever i search.

I think Mikhail just missunderstood your question -- he was giving an 
example of how to override the default sort (which uses score) with one 
that would ensure scores are not computed.

: > Is there any way to completely disable scoring in solr cloud as i am
: > always passing sort parameter whenever i search.

In general, you don't have to do anythign special.  

Solr's internal code looks at the sort specified, and the fields requested 
(via the fl param) to determine if/when scores need to be computed while 
colleting documents.  If scores aren't needed for any reason, then that 
info is passed down to the low level lucene document matching/collection 
code for optimizing the collection so scores aren't computed.


-Hoss
http://www.lucidworks.com/

Re: Integrating Stanford NLP or any other NLP for Natural Language Query

[Alessandro Benedetti]

Hi Puneet,
your requirement :
"I would like users to be able to write queries in natural language rather
than keyword based search."

Is really really vague :(
Can you try to help us with some specific example, starting of course from
the simplest use cases you have initially in mind ?

Moving from keyword based search to natural language is a really complex
task.
Proceeding step by step can help you.

Do you want for example to set up a Q basic system ?
In that case you should take care of query rewriting.
You need basically to identify your base requirement and then build a
specific parser for that.
You can use triple stores and knowledge bases to enrich both your query and
your index, but let's start from the basis, what is your simplest
requirement ?

On Fri, Jul 8, 2016 at 1:56 PM, Jay Urbain  wrote:

> I've added multivalued fields within my SOLR schema for indexing entities
> extracted using NLP methods applied to the text I'm indexing, along with
> fields for other discrete data extracted from relational databases.
>
> A Java application reads data out of multiple relational databases, uses
> NLP on the text and indexes each document (de-normalized) using SOLRJ.
>
> I initially tried doing this with content handlers, but found it much
> easier to just write a Java application.
>
> SOLRJ Java API reference:
> https://cwiki.apache.org/confluence/display/solr/Using+SolrJ
>
> Stanford NLP:
> http://stanfordnlp.github.io/CoreNLP/
>
> Best,
> Jay
>
>
> On Thu, Jul 7, 2016 at 9:52 PM, Puneet Pawaia 
> wrote:
>
> > Hi Jay
> > Any place I can learn more on this method of integration?
> > Thanks
> > Puneet
> >
> > On 8 Jul 2016 02:58, "Jay Urbain"  wrote:
> >
> > > I use Stanford NLP and cTakes (based on OpenNLP) while indexing with a
> > > SOLRJ application.
> > >
> > > Best,
> > > Jay
> > >
> > > On Thu, Jul 7, 2016 at 12:09 PM, Puneet Pawaia <
> puneet.paw...@gmail.com>
> > > wrote:
> > >
> > > > Hi
> > > >
> > > > I am currently using Solr 5.5.x to test but can upgrade to Solr 6.x
> if
> > > > required.
> > > > I am working on a POC for natural language query using Solr. Should I
> > use
> > > > the Stanford libraries or are there any other libraries having
> > > integration
> > > > with Solr already available.
> > > > Any direction in how to do this would be most appreciated. How
> should I
> > > > process the query to give relevant results.
> > > >
> > > > Regards
> > > > Puneet
> > > >
> > >
> >
>



-- 
--

Benedetti Alessandro
Visiting card : http://about.me/alessandro_benedetti

"Tyger, tyger burning bright
In the forests of the night,
What immortal hand or eye
Could frame thy fearful symmetry?"

William Blake - Songs of Experience -1794 England

Re: Searching Home's, Homes and Home

[Alessandro Benedetti]

I second Jamal,
using a soft stemmer for your language should solve the problem.
Specifically to the english language and the cases you mentioned :

1) Minimal English stemmer should be a good solution [1]

2) The english porter stemmer can be valid for your use case as well [2]

3) Not sure if the english possessive is managed by the previous filters,
in case not [3]

Cheers



[1]
https://cwiki.apache.org/confluence/display/solr/Filter+Descriptions#FilterDescriptions-EnglishMinimalStemFilter

[2]
https://cwiki.apache.org/confluence/display/solr/Filter+Descriptions#FilterDescriptions-PorterStemFilter

[3]
https://cwiki.apache.org/confluence/display/solr/Filter+Descriptions#FilterDescriptions-ClassicFilter

On Fri, Jul 8, 2016 at 6:17 PM, Jamal, Sarfaraz <
sarfaraz.ja...@verizonwireless.com.invalid> wrote:

> I would start by looking at the stemming documentation -
>
> It might be of help.
>
> Sas
>
>
> -Original Message-
> From: Surender [mailto:surender.si...@rsystems.com]
> Sent: Friday, July 8, 2016 8:30 AM
> To: solr-user@lucene.apache.org
> Subject: Searching Home's, Homes and Home
>
> User can type keyword for search in many ways an and following are the few
> examples:
> if user types any of  the keywords homes, home, home's then it should be
> able to search the following:
> 1. Home
> 2. Home's
> 3. Homes
>
> If user types Americas, the results should include 1. Americas 2.
> America's 3. America
>
> Please suggest how to send the search query to Solr to include all the
> results.
>
>
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/Searching-Home-s-Homes-and-Home-tp4286341.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>



-- 
--

Benedetti Alessandro
Visiting card : http://about.me/alessandro_benedetti

"Tyger, tyger burning bright
In the forests of the night,
What immortal hand or eye
Could frame thy fearful symmetry?"

William Blake - Songs of Experience -1794 England

Re: How to create highlight search component using Config API

[Cassandra Targett]

If you already have highlighting defined from one of the default
configsets, you can see an example of how the JSON is structured with
a Config API request. I assume you already tried that, but pointing it
out just in case.

Defining a highlighter with the Config API is a bit confusing to be
honest, but I worked out something that works:

{"add-searchcomponent": {"highlight": {"name":"myHighlight",
"class":"solr.HighlightComponent","": {"gap": {"default":"true",
"name": "gap", "class":"solr.highlight.GapFragmenter",
"defaults":{"hl.fragsize":100}}},"html":[{"default": "true","name":
"html","class": "solr.highlight.HtmlFormatter","defaults":
{"hl.simple.pre":"",
"hl.simple.post":""}},{"name": "html","class":
"solr.highlight.HtmlEncoder"}]}}}

Note there is an empty string after the initial class definition
(shown as ""). That lets you then add the fragmenters.

(I tried to prettify that, but my mail client isn't cooperating. I'm
going to add this example to the Solr Ref Guide, though so it might be
easier to see there in a few minutes.)

Hope it helps -
Cassandra

On Wed, Jun 29, 2016 at 8:00 AM, Alexandre Drouin
 wrote:
> Hi,
>
> I'm trying to create a highlight search component using the Config API of 
> Solr 6.0.1 however I cannot figure out how to include the elements 
> fragmenter, formatter, encoder, etc...
>
> Let's say I have the following component:
>
>name="myHighlightingComponent">
> 
>class="solr.highlight.GapFragmenter">
> 
>   100
> 
>   
>class="solr.highlight.HtmlFormatter">
> 
>   
>   
> 
>   
>   
> 
>   
>
> From what I can see from the documentation my JSON should look a bit like 
> this:
>
> {
>   "add-searchcomponent":{
> "name":"myHighlightingComponent",
> "class":"solr.HighlightComponent",
> ??
>   }
> }
>
> However I have no idea how to defines the 2 fragmenters or the encoder.  Any 
> help is appreciated.
>
> Thanks
> Alex
>

RE: Searching Home's, Homes and Home

[Jamal, Sarfaraz]

I would start by looking at the stemming documentation -

It might be of help.

Sas


-Original Message-
From: Surender [mailto:surender.si...@rsystems.com] 
Sent: Friday, July 8, 2016 8:30 AM
To: solr-user@lucene.apache.org
Subject: Searching Home's, Homes and Home

User can type keyword for search in many ways an and following are the few
examples:
if user types any of  the keywords homes, home, home's then it should be able 
to search the following:
1. Home
2. Home's
3. Homes

If user types Americas, the results should include 1. Americas 2. America's 3. 
America

Please suggest how to send the search query to Solr to include all the results.



--
View this message in context: 
http://lucene.472066.n3.nabble.com/Searching-Home-s-Homes-and-Home-tp4286341.html
Sent from the Solr - User mailing list archive at Nabble.com.

Searching Home's, Homes and Home

[Surender]

User can type keyword for search in many ways an and following are the few
examples:
if user types any of  the keywords homes, home, home's then it should be
able to search the following:
1. Home
2. Home's
3. Homes

If user types Americas, the results should include
1. Americas
2. America's
3. America

Please suggest how to send the search query to Solr to include all the
results.



--
View this message in context: 
http://lucene.472066.n3.nabble.com/Searching-Home-s-Homes-and-Home-tp4286341.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: SOLR 6: edismax search query with OR operator does not work as expected

[Erick Erickson]

Also take a look at: https://issues.apache.org/jira/browse/SOLR-8812

On Fri, Jul 8, 2016 at 7:02 AM, Jamal, Sarfaraz
 wrote:
> This sounds like it might be of help -
>
> < solrQueryParser defaultOperator="AND"/>
>
> You can change it from and to or.
>
> (If I understood you) -
>
> Sas
>
>
> -Original Message-
> From: Aleš Gregor [mailto:alg...@gmail.com]
> Sent: Friday, July 8, 2016 9:37 AM
> To: solr-user@lucene.apache.org
> Subject: SOLR 6: edismax search query with OR operator does not work as 
> expected
>
> Hello,
>
> after migrating my index from Solr 4.3 to Solr 6 I noticed that the OR 
> logical operator in search query no longer works as expected.
>
> On Solr 4.3 query - Blue OR Red - brings all documents with Blue or Red or 
> both tokens found.
> On Solr 6 the same query only brings documents with both the tokens, Blue and 
> Red.
>
> I see some difference in the debug of the query but I cannot  make much sense 
> out of it.
>
> Was there any change between Solr 4 and 6 that would cause this?
>
> Thanks
> Ales Gregor

query / joint

[Pithon Philippe]

"docs": [
  {
"id": "...",

"type_s": "ticket",
"customerid_s": "100",
...
...
...
...

  },
  {
"id": "...",

"type_s": "customer",
"customerid_s": "100",
"name_s": "FISHER",
...
...
...
...

  }
]

Hello,
I have two entitys :
- tickets (type_s : ticket)
- customers (type_s : customer)

I want a query to find all tickets for name customer FISHER
In doc ticket, I have id customer but not name customer...

Any ideas ??? joint ???

Thanks
Philippe

RE: SOLR 6: edismax search query with OR operator does not work as expected

[Jamal, Sarfaraz]

This sounds like it might be of help -

< solrQueryParser defaultOperator="AND"/>

You can change it from and to or.

(If I understood you) -

Sas


-Original Message-
From: Aleš Gregor [mailto:alg...@gmail.com] 
Sent: Friday, July 8, 2016 9:37 AM
To: solr-user@lucene.apache.org
Subject: SOLR 6: edismax search query with OR operator does not work as expected

Hello,

after migrating my index from Solr 4.3 to Solr 6 I noticed that the OR logical 
operator in search query no longer works as expected.

On Solr 4.3 query - Blue OR Red - brings all documents with Blue or Red or both 
tokens found.
On Solr 6 the same query only brings documents with both the tokens, Blue and 
Red.

I see some difference in the debug of the query but I cannot  make much sense 
out of it.

Was there any change between Solr 4 and 6 that would cause this?

Thanks
Ales Gregor

Re: Internode communication failed when enable basic authentication Solr 6.1.0

[Aleš Gregor]

Hello,

could this be related to https://issues.apache.org/jira/browse/SOLR-9188 ?

Ales Gregor



2016-06-24 15:15 GMT+02:00 Shankar Ramalingam :

> Hi Team,
>
> Basic Authentication is enabled on Solr cloud and node1 is running on one
> machine and node2 is runnin on second machine, zookeeper installed on
> second machine.  Getting unathorized error when enable basic auth, error
> mostly occure when machine trying access machine 2 solr and also while
> starting solr also i can see the error message.
>
>
> It would be grateful if you help me to resole the issue. I saw some jira
> ticket stating that some internode communication issue and got fixed in
> solr 6, but I am using solr 6 only and also getting isssue, Even-though am
>  login admin user on solr UI, sometime getting Error 401 Unauthorized
> request mostly request go to node1 to node2.
>
>
>
>  [c:adm s:shard2 r:core_node2 x:adm_shard2_replica2  ]
> o.a.s.h.RequestHandlerBase
> org.apache.solr.client.solrj.impl.HttpSolrClient$Re
> moteSolrException:
> Error from server at http://172.16.7.58:8983/solr/adm_shard2_ replica1:
> Expected mime type application/octet-stream but got text/html. 
> 
> 
> Error 401 Unauthorized request, Response code: 401
> 
> HTTP ERROR 401
> Problem accessing /solr/adm_shard2_replica1/select. Reason:
>* Unauthorized request, Response code: 401*
> 
> 
>
>
>
> *Below code define in security.json*
>
> -bash-3.2# curl --user solr:SolrRocks
> http://localhost:8983/solr/admin/authorization
> {
>   "responseHeader":{
> "status":0,
> "QTime":0},
>   "authorization.enabled":true,
>   "authorization":{
> "class":"solr.RuleBasedAuthorizationPlugin",
> "user-role":{"solr":"admin"},
> "permissions":[{
> "name":"security-edit",
> "role":"admin",
> "index":1},
>   {
> "name":"read",
> "role":"admin",
> "index":2},
>   {
> "name":"collection-admin-read ",
> "role":"admin",
> "index":3}],
> "":{"v":50}}}
>
>
>
> Thanks,
> Shankar.
> Contact: 91+9894546732
>

SOLR 6: edismax search query with OR operator does not work as expected

[Aleš Gregor]

Hello,

after migrating my index from Solr 4.3 to Solr 6 I noticed that the OR
logical operator in search query no longer works as expected.

On Solr 4.3 query - Blue OR Red - brings all documents with Blue or Red or
both tokens found.
On Solr 6 the same query only brings documents with both the tokens, Blue
and Red.

I see some difference in the debug of the query but I cannot  make much
sense out of it.

Was there any change between Solr 4 and 6 that would cause this?

Thanks
Ales Gregor

Re: Disabling solr scoring

[Bhaumik Joshi]

Can you please elaborate? I am passing user defined sort field and order 
whenever i search.


Thanks & Regards,

Bhaumik Joshi



From: Mikhail Khludnev 
Sent: Friday, July 8, 2016 4:13 AM
To: solr-user
Subject: Re: Disabling solr scoring

What about
sort=_docid_ asc
?
08  2016 ?. 13:50  "Bhaumik Joshi" <
bhaumik.jo...@outlook.com> ???:

> Hi,
>
>
> Is there any way to completely disable scoring in solr cloud as i am
> always passing sort parameter whenever i search.
>
> And disabling scoring will improve performance?
>
>
> Thanks & Regards,
>
> Bhaumik Joshi
>

Re: Integrating Stanford NLP or any other NLP for Natural Language Query

[Jay Urbain]

I've added multivalued fields within my SOLR schema for indexing entities
extracted using NLP methods applied to the text I'm indexing, along with
fields for other discrete data extracted from relational databases.

A Java application reads data out of multiple relational databases, uses
NLP on the text and indexes each document (de-normalized) using SOLRJ.

I initially tried doing this with content handlers, but found it much
easier to just write a Java application.

SOLRJ Java API reference:
https://cwiki.apache.org/confluence/display/solr/Using+SolrJ

Stanford NLP:
http://stanfordnlp.github.io/CoreNLP/

Best,
Jay


On Thu, Jul 7, 2016 at 9:52 PM, Puneet Pawaia 
wrote:

> Hi Jay
> Any place I can learn more on this method of integration?
> Thanks
> Puneet
>
> On 8 Jul 2016 02:58, "Jay Urbain"  wrote:
>
> > I use Stanford NLP and cTakes (based on OpenNLP) while indexing with a
> > SOLRJ application.
> >
> > Best,
> > Jay
> >
> > On Thu, Jul 7, 2016 at 12:09 PM, Puneet Pawaia 
> > wrote:
> >
> > > Hi
> > >
> > > I am currently using Solr 5.5.x to test but can upgrade to Solr 6.x if
> > > required.
> > > I am working on a POC for natural language query using Solr. Should I
> use
> > > the Stanford libraries or are there any other libraries having
> > integration
> > > with Solr already available.
> > > Any direction in how to do this would be most appreciated. How should I
> > > process the query to give relevant results.
> > >
> > > Regards
> > > Puneet
> > >
> >
>

group.facet=true and facet on field of type int -> org.apache.solr.common.SolrException: Exception during facet.field

[Sebastian Riemer]

Hi all,

are there any limitations in regard to retrieval of facet information, when 
grouping?

When I send the following query where the field to facet ("m_pt_14_s_ns") on is 
of type "string", everything works fine.

"q": "*:*",
"facet.field": "m_pt_14_s_ns",
"indent": "true",
"group.facet": "true",
"fq": "m_id_l:[* TO *]",
"wt": "json",
"facet": "true",
"group.field": "m_id_l",
"group": "true",

 [..]
"facet_fields": {
  "m_pt_14_s_ns": [
"zahlr. Ill.",
7,
"Ill.",
4,
"Ill., graph. Darst.",
2,
"zahlr. Ill. (z.T. farb.))",
   2,
"überw. Ill.",
1,
"zahlr. Cartoons jetzt durchgehend zweifarbig illustriert",
1,
"zahlr. Ill., Kt.",
1,
"zahlr. Ill., graph. Darst.",
1,
"überw. Ill.",
1
  ]
},

When I try the same with a field of type “int” (m_pt_27_i_ns), I get the 
following exception:

"q": "*:*",
"facet.field": "m_pt_27_i_ns",
"indent": "true",
"group.facet": "true",
"fq": "m_id_l:[* TO *]",
"wt": "json",
"facet": "true",
"group.field": "m_id_l",
"group": "true",

 [..]
 "error": {
"metadata": [
  "error-class",
  "org.apache.solr.common.SolrException",
  "root-error-class",
  "java.lang.IllegalStateException"
],
"msg": "Exception during facet.field: m_pt_27_i_ns",
"trace": "org.apache.solr.common.SolrException: Exception during 
facet.field: m_pt_27_i_ns\r\n\tat 
org.apache.solr.request.SimpleFacets$3.call(SimpleFacets.java:700)\r\n\tat 
org.apache.solr.request.SimpleFacets$3.call(SimpleFacets.java:685)\r\n\tat 
java.util.concurrent.FutureTask.run(FutureTask.java:266)\r\n\tat 
org.apache.solr.request.SimpleFacets$2.execute(SimpleFacets.java:639)\r\n\tat 
org.apache.solr.request.SimpleFacets.getFacetFieldCounts(SimpleFacets.java:710)\r\n\tat
 
org.apache.solr.handler.component.FacetComponent.getFacetCounts(FacetComponent.java:294)\r\n\tat
 
org.apache.solr.handler.component.FacetComponent.process(FacetComponent.java:256)\r\n\tat
 
org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:272)\r\n\tat
 
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:155)\r\n\tat
 org.apache.solr.core.SolrCore.execute(SolrCore.java:2082)\r\n\tat 
org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:670)\r\n\tat 
org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:458)\r\n\tat 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:225)\r\n\tat
 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:183)\r\n\tat
 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)\r\n\tat
 
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)\r\n\tat
 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)\r\n\tat
 
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577)\r\n\tat
 
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)\r\n\tat
 
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)\r\n\tat
 
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)\r\n\tat
 
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)\r\n\tat
 
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)\r\n\tat
 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)\r\n\tat
 
org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215)\r\n\tat
 
org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110)\r\n\tat
 
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)\r\n\tat
 org.eclipse.jetty.server.Server.handle(Server.java:499)\r\n\tat 
org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)\r\n\tat 
org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)\r\n\tat
 
org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)\r\n\tat
 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)\r\n\tat
 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)\r\n\tat
 java.lang.Thread.run(Thread.java:745)\r\nCaused by: 
java.lang.IllegalStateException: unexpected docvalues type NUMERIC for field 
'm_pt_27_i_ns' (expected=SORTED). Use UninvertingReader or index with 
docvalues.\r\n\tat 
org.apache.lucene.index.DocValues.checkField(DocValues.java:208)\r\n\tat 
org.apache.lucene.index.DocValues.getSorted(DocValues.java:264)\r\n\tat 
org.apache.lucene.search.grouping.term.TermGroupFacetCollector$SV.doSetNextReader(TermGroupFacetCollector.java:129)\r\n\tat
 
org.apache.lucene.search.SimpleCollector.getLeafCollector(SimpleCollector.java:33)\r\n\tat
 
org.apache.solr.request.SimpleFacets$1.getLeafCollector(SimpleFacets.java:601)\r\n\tat

Re: Suggester Issue

[Alessandro Benedetti]

Taking a look to the code, basically the context field act as the default
field in a Standard Query Parser.
Even if having multiple context fields seems to be supported when building
the auxiliary suggester index, it doesn't seem to work at query time.

If you specify the context only at query time ( through the .cfq) is not
going to work , as the context field is required to build properly the
auxiliary suggesting index.

Any additional insight is welcome,

Cheers

On Thu, Jul 7, 2016 at 1:13 PM, Rajesh Kapur 
wrote:

> Hi,
>
> Any update on this?
>
> Could you please let me know is it possible to pass CFQ parameter on
> multiple fields?
>
> Thanks
>
> On Tue, Jul 5, 2016 at 9:40 AM, Rajesh Kapur 
> wrote:
>
> > Hi,
> >
> >
> >
> > I tried to implement suggester using SOLR 6.0.1 with context field. PFB
> > the configuration we are using to implement suggester
> >
> >
> >
> >
> >
> >class="solr.SuggestComponent">
> >
> > 
> >
> >   mySuggester
> >
> >   
> >
> >   AnalyzingInfixLookupFactory
> >
> >   suggester_infixdata_dir
> >
> >   DocumentDictionaryFactory
> >
> >   SearchSuggestions
> >
> > BrandName
> >
> >   suggest
> >
> >   true
> >
> >   
> >
> > 
> >
> >   
> >
> >
> >
> >   
> >
> > 
> >
> >   true
> >
> >   10
> >
> >   mySuggester
> >
> >   
> >
> > 
> >
> > 
> >
> >   suggest_sitesearch
> >
> > 
> >
> >   
> >
> >
> >
> > But I am not able to get the desired output using suggest.cfq parameter.
> > Could you please help me in getting the correct output.
> >
> >
> >
> > -Thanks,
> >
> > Rajesh Kapur
> >
>



-- 
--

Benedetti Alessandro
Visiting card : http://about.me/alessandro_benedetti

"Tyger, tyger burning bright
In the forests of the night,
What immortal hand or eye
Could frame thy fearful symmetry?"

William Blake - Songs of Experience -1794 England

Re: Disabling solr scoring

[Mikhail Khludnev]

What about
sort=_docid_ asc
?
08 июля 2016 г. 13:50 пользователь "Bhaumik Joshi" <
bhaumik.jo...@outlook.com> написал:

> Hi,
>
>
> Is there any way to completely disable scoring in solr cloud as i am
> always passing sort parameter whenever i search.
>
> And disabling scoring will improve performance?
>
>
> Thanks & Regards,
>
> Bhaumik Joshi
>

Query Elevation

[Swathika]

A new requirement to get particular document as second result in result page. 

For example, If the query is “coal”, this document(id: 222) should come as
second result.

Please let me know if you have any solution.




--
View this message in context: 
http://lucene.472066.n3.nabble.com/Query-Elevation-tp4286332.html
Sent from the Solr - User mailing list archive at Nabble.com.

Disabling solr scoring

[Bhaumik Joshi]

Hi,


Is there any way to completely disable scoring in solr cloud as i am always 
passing sort parameter whenever i search.

And disabling scoring will improve performance?


Thanks & Regards,

Bhaumik Joshi

Re: CDCR (Solr6.x) does not start

[Uwe Reh]

Hi Renaud,

thank you for your response.

You asked for some further information:

1. Log messages at the source cluster:
As mentioned in my addendum "CDCR (Solr6.x) does not start (logfile)". I 
changed the log level for all Handlers to TRACE and I got three Messages 
for each shard caused by "Action LASTPROCESSEDVERSION sent to non-leader 
replica .."

For me this looks like the blocker.

2.

Replication should start even if no commit has been sent to the source cluster.

Thanks for the clarification. It helps me to understand.

3.

The empty queue seems to indicate there is an issue, and that cdcr was unable 
to instantiate the replicator for the target cluster.
Just to be sure, your source cluster has 4 shards, but not replica ? If it has 
replicas, can you ensure that you execute these command on the shard leader.
At the beginning I tried to replicate 4 shards with an replication 
factor of 3. Later on i simplified the environment by omitting the 
replicas. (replication factor = 1)

Do you think having no replicas could the reason for the log messages above?


Regards
Uwe





Am 05.07.2016 um 14:55 schrieb Renaud Delbru:

Hi Uwe,

At first look, your configuration seems correct,
see my comments below.

On 28/06/16 15:36, Uwe Reh wrote:

9. Start CDCR
http://SOURCE:s_port/solr/scoll/cdcr?action=start=json

{"responseHeader":{"status":0,"QTime":13},"status":["process","started","buffer","enabled"]}



! (not even a single query to the target's zookeeper ??)


Indeed, you should have observed a communication between the source
cluster and the target zookeeper. Do you see any errors in the log of
the source cluster ? Or a log message such as:
"Unable to instantiate the log reader for target collection ..."



10. Enter some test data into the SOURCE

11. Explicit commit in SOURCE
http://SOURCE:s_port/solr/scoll/update?commit=true=true
!! (at least now there should be some traffic, or?)


Replication should start even if no commit has been sent to the source
cluster.



12. Check errors and queues
http://SOURCE:s_port/solr/scoll_shard1_replica1/cdcr?action=queues=json


{"responseHeader":{"status":0,"QTime":0},"queues":[],"tlogTotalSize":135,"tlogTotalCount":1,"updateLogSynchronizer":"stopped"}



http://SOURCE:s_port/solr/scoll_shard1_replica1/cdcr?action=errors=json


{"responseHeader":{"status":0,"QTime":0},"errors":[]}

! Why is the element queues is empty


The empty queue seems to indicate there is an issue, and that cdcr was
unable to instantiate the replicator for the target cluster.
Just to be sure, your source cluster has 4 shards, but not replica ? If
it has replicas, can you ensure that you execute these command on the
shard leader.

Kind Regards

RE: File Descriptor/Memory Leak

[Alexandre Rafalovitch]

Is there a firewall between a client and a server by any chance?

CLOSE_WAIT is not a leak, but standard TCP step at the end. So the question
is why sockets are reopened that often or why the other side does not
acknowledge TCP termination packet fast.

I would run Ethereal to troubleshoot that. And truss/strace.

Regards,
Alex
On 8 Jul 2016 4:56 PM, "Mads Tomasgård Bjørgan"  wrote:

FYI - we're using Solr-6.1.0, and the leak seems to be consequent (occurs
every single time when running with SSL).

-Original Message-
From: Anshum Gupta [mailto:ans...@anshumgupta.net]
Sent: torsdag 7. juli 2016 18.14
To: solr-user@lucene.apache.org
Subject: Re: File Descriptor/Memory Leak

I've created a JIRA to track this:
https://issues.apache.org/jira/browse/SOLR-9290

On Thu, Jul 7, 2016 at 8:00 AM, Shai Erera  wrote:

> Shalin, we're seeing that issue too (and actually actively debugging
> it these days). So far I can confirm the following (on a 2-node cluster):
>
> 1) It consistently reproduces on 5.5.1, but *does not* reproduce on
> 5.4.1
> 2) It does not reproduce when SSL is disabled
> 3) Restarting the Solr process (sometimes both need to be restarted),
> the count drops to 0, but if indexing continues, they climb up again
>
> When it does happen, Solr seems stuck. The leader cannot talk to the
> replica, or vice versa, the replica is usually put in DOWN state and
> there's no way to fix it besides restarting the JVM.
>
> Reviewing the changes from 5.4.1 to 5.5.1 I tried reverting some that
> looked suspicious (SOLR-8451 and SOLR-8578), even though the changes
> look legit. That did not help, and honestly I've done that before we
> suspected it might be the SSL. Therefore I think those are "safe", but
just FYI.
>
> When it does happen, the number of CLOSE_WAITS climb very high, to the
> order of 30K+ entries in 'netstat'.
>
> When I say it does not reproduce on 5.4.1 I really mean the numbers
> don't go as high as they do in 5.5.1. Meaning, when running without
> SSL, the number of CLOSE_WAITs is smallish, usually less than a 10 (I
> would separately like to understand why we have any in that state at
> all). When running with SSL and 5.4.1, they stay low at the order of
> hundreds the most.
>
> Unfortunately running without SSL is not an option for us. We will
> likely roll back to 5.4.1, even if the problem exists there, but to a
> lesser degree.
>
> I will post back here when/if we have more info about this.
>
> Shai
>
> On Thu, Jul 7, 2016 at 5:32 PM Shalin Shekhar Mangar <
> shalinman...@gmail.com>
> wrote:
>
> > I have myself seen this CLOSE_WAIT issue at a customer. I am running
> > some tests with different versions trying to pinpoint the cause of this
leak.
> > Once I have some more information and a reproducible test, I'll open
> > a
> jira
> > issue. I'll keep you posted.
> >
> > On Thu, Jul 7, 2016 at 5:13 PM, Mads Tomasgård Bjørgan 
> > wrote:
> >
> > > Hello there,
> > > Our SolrCloud is experiencing a FD leak while running with SSL.
> > > This is occurring on the one machine that our program is sending
> > > data too. We
> > have
> > > a total of three servers running as an ensemble.
> > >
> > > While running without SSL does the FD Count remain quite constant
> > > at around 180 while indexing. Performing a garbage collection also
> > > clears almost the entire JVM-memory.
> > >
> > > However - when indexing with SSL does the FDC grow polynomial. The
> count
> > > increases with a few hundred every five seconds or so, but reaches
> easily
> > > 50 000 within three to four minutes. Performing a GC swipes most
> > > of the memory on the two machines our program isn't transmitting
> > > the data
> > directly
> > > to. The last machine is unaffected by the GC, and both memory nor
> > > FDC doesn't reset before Solr is restarted on that machine.
> > >
> > > Performing a netstat reveals that the FDC mostly consists of
> > > TCP-connections in the state of "CLOSE_WAIT".
> > >
> > >
> > >
> >
> >
> > --
> > Regards,
> > Shalin Shekhar Mangar.
> >
>



--
Anshum Gupta

RE: File Descriptor/Memory Leak

[Mads Tomasgård Bjørgan]

FYI - we're using Solr-6.1.0, and the leak seems to be consequent (occurs every 
single time when running with SSL).

-Original Message-
From: Anshum Gupta [mailto:ans...@anshumgupta.net] 
Sent: torsdag 7. juli 2016 18.14
To: solr-user@lucene.apache.org
Subject: Re: File Descriptor/Memory Leak

I've created a JIRA to track this:
https://issues.apache.org/jira/browse/SOLR-9290

On Thu, Jul 7, 2016 at 8:00 AM, Shai Erera  wrote:

> Shalin, we're seeing that issue too (and actually actively debugging 
> it these days). So far I can confirm the following (on a 2-node cluster):
>
> 1) It consistently reproduces on 5.5.1, but *does not* reproduce on 
> 5.4.1
> 2) It does not reproduce when SSL is disabled
> 3) Restarting the Solr process (sometimes both need to be restarted), 
> the count drops to 0, but if indexing continues, they climb up again
>
> When it does happen, Solr seems stuck. The leader cannot talk to the 
> replica, or vice versa, the replica is usually put in DOWN state and 
> there's no way to fix it besides restarting the JVM.
>
> Reviewing the changes from 5.4.1 to 5.5.1 I tried reverting some that 
> looked suspicious (SOLR-8451 and SOLR-8578), even though the changes 
> look legit. That did not help, and honestly I've done that before we 
> suspected it might be the SSL. Therefore I think those are "safe", but just 
> FYI.
>
> When it does happen, the number of CLOSE_WAITS climb very high, to the 
> order of 30K+ entries in 'netstat'.
>
> When I say it does not reproduce on 5.4.1 I really mean the numbers 
> don't go as high as they do in 5.5.1. Meaning, when running without 
> SSL, the number of CLOSE_WAITs is smallish, usually less than a 10 (I 
> would separately like to understand why we have any in that state at 
> all). When running with SSL and 5.4.1, they stay low at the order of 
> hundreds the most.
>
> Unfortunately running without SSL is not an option for us. We will 
> likely roll back to 5.4.1, even if the problem exists there, but to a 
> lesser degree.
>
> I will post back here when/if we have more info about this.
>
> Shai
>
> On Thu, Jul 7, 2016 at 5:32 PM Shalin Shekhar Mangar < 
> shalinman...@gmail.com>
> wrote:
>
> > I have myself seen this CLOSE_WAIT issue at a customer. I am running 
> > some tests with different versions trying to pinpoint the cause of this 
> > leak.
> > Once I have some more information and a reproducible test, I'll open 
> > a
> jira
> > issue. I'll keep you posted.
> >
> > On Thu, Jul 7, 2016 at 5:13 PM, Mads Tomasgård Bjørgan 
> > wrote:
> >
> > > Hello there,
> > > Our SolrCloud is experiencing a FD leak while running with SSL. 
> > > This is occurring on the one machine that our program is sending 
> > > data too. We
> > have
> > > a total of three servers running as an ensemble.
> > >
> > > While running without SSL does the FD Count remain quite constant 
> > > at around 180 while indexing. Performing a garbage collection also 
> > > clears almost the entire JVM-memory.
> > >
> > > However - when indexing with SSL does the FDC grow polynomial. The
> count
> > > increases with a few hundred every five seconds or so, but reaches
> easily
> > > 50 000 within three to four minutes. Performing a GC swipes most 
> > > of the memory on the two machines our program isn't transmitting 
> > > the data
> > directly
> > > to. The last machine is unaffected by the GC, and both memory nor 
> > > FDC doesn't reset before Solr is restarted on that machine.
> > >
> > > Performing a netstat reveals that the FDC mostly consists of 
> > > TCP-connections in the state of "CLOSE_WAIT".
> > >
> > >
> > >
> >
> >
> > --
> > Regards,
> > Shalin Shekhar Mangar.
> >
>



--
Anshum Gupta