Re: SOl6.3 Alchemy Annotator Not Working

[soumitra80]

This issue has been resolved. Please close this



--
View this message in context: 
http://lucene.472066.n3.nabble.com/SOl6-3-Alchemy-Annotator-Not-Working-tp4307228p4307625.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Solr Analyzer Language

[Chien Nguyen]

many thank to you! @@



--
View this message in context: 
http://lucene.472066.n3.nabble.com/Solr-Analyzer-Language-tp4307585p4307613.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Solrcloud connect/reconnect to Zookeeper (rebooting ZK nodes)

[The Dobbo]

Good news, or bad... I’ve just tried to reproduce in a new environment and 
everything worked as expected. Perhaps something else was at work in the 
environment at the time. I’ll wait until I need to reboot prod zookeeper again 
and I’ll grab logs and zookeeper health whilst I do the reboots, to see if prod 
reproduces the issue. I’ll add in checks on each solr node to check its 
connection with each zookeeper node.
Thanks again for your help,
Brendan  

> On 28 Nov. 2016, at 1:43 pm, Brendan Dobinson  wrote:
> 
> Thanks for the response. I have never noticed any inconsistencies with 
> zookeeper and the leader knows it has two followers (of course this is stats 
> from now, I wish I had looked at this before the restarts)
> 
> #echo mntr | nc 127.0.0.1 2181
> zk_version3.4.8--1, built on 02/06/2016 03:18 GMT
> …
> zk_server_state   leader
> ..
> zk_followers  2
> zk_synced_followers   2
> ..
> 
> Also the solr nodes are connecting effectively with the different ZK nodes:
> (one box for example)
> #echo stat | nc 127.0.0.1 2181
> Zookeeper version: 3.4.8--1, built on 02/06/2016 03:18 GMT
> Clients:
> /192.91.6.40:45450[1](queued=0,recved=598765,sent=601713)
> /192.91.6.27:50060[1](queued=0,recved=595002,sent=598030)
> /192.91.6.204:39008[1](queued=0,recved=625580,sent=628276)
> /127.0.0.1:38748[0](queued=0,recved=1,sent=0)
> 
> Each ZK node lists one or more of the solr boxes as clients.
> 
> Cheers,
> Brendan
> 
> On 28/11/16, 12:11 pm, "Erick Erickson"  wrote:
> 
>This seems very weird. Do the Zookeepers know about each other
>correctly? Some evidence for mis-configured Zookeepers might be if you
>rebooted ZK3 and had this happen again.
> 
>But that's a wild shot in the dark.
> 
>Best,
>Erick
> 
>On Sun, Nov 27, 2016 at 4:42 PM, The Dobbo  wrote:
>> Hi,
>> I have a 3 external node ZK (zookeeper-3.4.8) cluster managing my 6 node 
>> solrcloud (solr 6.1) cluster. Recently due to dirty cow I had to reboot my 
>> Solr and zookeeper clusters. I rebooted the solr nodes one by one and all 
>> was fine. I then rebooted zookeeper nodes 1 and 2 (with at least 10 minute 
>> delay between reboots) and again all was fine - no errors reported in 
>> zookeepers RUOK, solcloud cluster health was all green. When I rebooted ZK 3 
>> solr reported it could no longer connect to ZK and all the leaders lost 
>> their replicas. After a short time solr started rebuilding its replicas (it 
>> recovered all automagically) - I didn’t restart solr. The only issue was a 
>> spike in load on the solr leaders.
>> 
>> My best guess is that solrcloud doesn’t reconnect effectively if a zookeeper 
>> node disappears for a period (zkClientTimeout is set to 15 sec (15000)).
>> 
>> Relevant config in start-up script: -DzkClientTimeout=1500 
>> -DzkHost=zookeeper01:2181,zookeeper02:2181,zookeeper03:2181/solr/production
>> 
>> My questions:
>> Has anyone experienced this upon rebooting zookeeper? Any advice if anything 
>> I did above was wrong? - should I increase zkClientTimeout?
>> Any monitoring that would alert me that solr has an issue connecting to an 
>> individual ZK node (well that would have alerted me before I rebooted ZK3).
>> Any other relevant info from the docs I should be reading? (I believe have 
>> read/looked relatively exhaustively)
>> 
>> Thanks, let me know if further info is required, I unfortunately didn’t 
>> collect logs for this period. My next step is to reproduce in non-prod (but 
>> thought I’d reach out first).
>> - Brendan
>> 
> 
> 
> 

Re: Solrcloud connect/reconnect to Zookeeper (rebooting ZK nodes)

[Brendan Dobinson]

Thanks for the response. I have never noticed any inconsistencies with 
zookeeper and the leader knows it has two followers (of course this is stats 
from now, I wish I had looked at this before the restarts)

#echo mntr | nc 127.0.0.1 2181
zk_version  3.4.8--1, built on 02/06/2016 03:18 GMT
…
zk_server_state leader
..
zk_followers2
zk_synced_followers 2
..

Also the solr nodes are connecting effectively with the different ZK nodes:
(one box for example)
#echo stat | nc 127.0.0.1 2181
Zookeeper version: 3.4.8--1, built on 02/06/2016 03:18 GMT
Clients:
 /192.91.6.40:45450[1](queued=0,recved=598765,sent=601713)
 /192.91.6.27:50060[1](queued=0,recved=595002,sent=598030)
 /192.91.6.204:39008[1](queued=0,recved=625580,sent=628276)
 /127.0.0.1:38748[0](queued=0,recved=1,sent=0)

Each ZK node lists one or more of the solr boxes as clients.

Cheers,
Brendan

On 28/11/16, 12:11 pm, "Erick Erickson"  wrote:

This seems very weird. Do the Zookeepers know about each other
correctly? Some evidence for mis-configured Zookeepers might be if you
rebooted ZK3 and had this happen again.

But that's a wild shot in the dark.

Best,
Erick

On Sun, Nov 27, 2016 at 4:42 PM, The Dobbo  wrote:
> Hi,
> I have a 3 external node ZK (zookeeper-3.4.8) cluster managing my 6 node 
solrcloud (solr 6.1) cluster. Recently due to dirty cow I had to reboot my Solr 
and zookeeper clusters. I rebooted the solr nodes one by one and all was fine. 
I then rebooted zookeeper nodes 1 and 2 (with at least 10 minute delay between 
reboots) and again all was fine - no errors reported in zookeepers RUOK, 
solcloud cluster health was all green. When I rebooted ZK 3 solr reported it 
could no longer connect to ZK and all the leaders lost their replicas. After a 
short time solr started rebuilding its replicas (it recovered all 
automagically) - I didn’t restart solr. The only issue was a spike in load on 
the solr leaders.
>
> My best guess is that solrcloud doesn’t reconnect effectively if a 
zookeeper node disappears for a period (zkClientTimeout is set to 15 sec 
(15000)).
>
> Relevant config in start-up script: -DzkClientTimeout=1500 
-DzkHost=zookeeper01:2181,zookeeper02:2181,zookeeper03:2181/solr/production
>
> My questions:
> Has anyone experienced this upon rebooting zookeeper? Any advice if 
anything I did above was wrong? - should I increase zkClientTimeout?
> Any monitoring that would alert me that solr has an issue connecting to 
an individual ZK node (well that would have alerted me before I rebooted ZK3).
> Any other relevant info from the docs I should be reading? (I believe 
have read/looked relatively exhaustively)
>
> Thanks, let me know if further info is required, I unfortunately didn’t 
collect logs for this period. My next step is to reproduce in non-prod (but 
thought I’d reach out first).
> - Brendan
>

Re: Solrcloud connect/reconnect to Zookeeper (rebooting ZK nodes)

[Erick Erickson]

This seems very weird. Do the Zookeepers know about each other
correctly? Some evidence for mis-configured Zookeepers might be if you
rebooted ZK3 and had this happen again.

But that's a wild shot in the dark.

Best,
Erick

On Sun, Nov 27, 2016 at 4:42 PM, The Dobbo  wrote:
> Hi,
> I have a 3 external node ZK (zookeeper-3.4.8) cluster managing my 6 node 
> solrcloud (solr 6.1) cluster. Recently due to dirty cow I had to reboot my 
> Solr and zookeeper clusters. I rebooted the solr nodes one by one and all was 
> fine. I then rebooted zookeeper nodes 1 and 2 (with at least 10 minute delay 
> between reboots) and again all was fine - no errors reported in zookeepers 
> RUOK, solcloud cluster health was all green. When I rebooted ZK 3 solr 
> reported it could no longer connect to ZK and all the leaders lost their 
> replicas. After a short time solr started rebuilding its replicas (it 
> recovered all automagically) - I didn’t restart solr. The only issue was a 
> spike in load on the solr leaders.
>
> My best guess is that solrcloud doesn’t reconnect effectively if a zookeeper 
> node disappears for a period (zkClientTimeout is set to 15 sec (15000)).
>
> Relevant config in start-up script: -DzkClientTimeout=1500 
> -DzkHost=zookeeper01:2181,zookeeper02:2181,zookeeper03:2181/solr/production
>
> My questions:
> Has anyone experienced this upon rebooting zookeeper? Any advice if anything 
> I did above was wrong? - should I increase zkClientTimeout?
> Any monitoring that would alert me that solr has an issue connecting to an 
> individual ZK node (well that would have alerted me before I rebooted ZK3).
> Any other relevant info from the docs I should be reading? (I believe have 
> read/looked relatively exhaustively)
>
> Thanks, let me know if further info is required, I unfortunately didn’t 
> collect logs for this period. My next step is to reproduce in non-prod (but 
> thought I’d reach out first).
> - Brendan
>

Solrcloud connect/reconnect to Zookeeper (rebooting ZK nodes)

[The Dobbo]

Hi,
I have a 3 external node ZK (zookeeper-3.4.8) cluster managing my 6 node 
solrcloud (solr 6.1) cluster. Recently due to dirty cow I had to reboot my Solr 
and zookeeper clusters. I rebooted the solr nodes one by one and all was fine. 
I then rebooted zookeeper nodes 1 and 2 (with at least 10 minute delay between 
reboots) and again all was fine - no errors reported in zookeepers RUOK, 
solcloud cluster health was all green. When I rebooted ZK 3 solr reported it 
could no longer connect to ZK and all the leaders lost their replicas. After a 
short time solr started rebuilding its replicas (it recovered all 
automagically) - I didn’t restart solr. The only issue was a spike in load on 
the solr leaders. 

My best guess is that solrcloud doesn’t reconnect effectively if a zookeeper 
node disappears for a period (zkClientTimeout is set to 15 sec (15000)).  

Relevant config in start-up script: -DzkClientTimeout=1500 
-DzkHost=zookeeper01:2181,zookeeper02:2181,zookeeper03:2181/solr/production

My questions: 
Has anyone experienced this upon rebooting zookeeper? Any advice if anything I 
did above was wrong? - should I increase zkClientTimeout?
Any monitoring that would alert me that solr has an issue connecting to an 
individual ZK node (well that would have alerted me before I rebooted ZK3).
Any other relevant info from the docs I should be reading? (I believe have 
read/looked relatively exhaustively)  

Thanks, let me know if further info is required, I unfortunately didn’t collect 
logs for this period. My next step is to reproduce in non-prod (but thought I’d 
reach out first).
- Brendan 
 

Re: Solr 6 Performance Suggestions

[Shawn Heisey]

On 11/27/2016 12:51 PM, Florian Gleixner wrote:
> On 22.11.2016 14:54, Max Bridgewater wrote:
>> test cases were exactly the same, the machines where exactly the same
>> and heap settings exactly the same (Xms24g, Xmx24g). Requests were
>> sent with 
> Setting heap too large is a common error. Recent Solr use the
> filesystem cache, so you don't have to set heap to the size of the
> index. The avalible RAM has to be able to run the OS, run the jvm and
> hold most of the index data in filesystem cache. If you have 32GB RAM
> and a 20GB Index, then set -Xms never higher than 10GB. I personally
> would set -Xms to 4GB and omit -Xmx

In my mind, the Xmx setting is much more important than Xms.  Setting
both to the same number avoids any need for Java to detect memory
pressure before increasing the heap size, which can be helpful.

Without Xmx, Java is in control of the max heap size, and it may not
make the correct choice.  It's important to know what your max heap is,
because chances are excellent that the max heap *will* be reached.  Solr
allocates a lot of memory to do its job.

Thanks,
Shawn

Re: Solr Analyzer Language

[Steve Rowe]

Hi Chien,

By “unsigned” I think you mean without diacritics, for example ‘D’ instead of 
‘Đ'.

I think you can get what you want by including ICUFoldingFilterFactory in your 
analyzer - see 
.

--
Steve
www.lucidworks.com

> On Nov 27, 2016, at 11:24 AM, Chien Nguyen  wrote:
> 
> Hi, everyone. 
> Now, i want to search with input is Unsigned Vietnamese. How can i do to get
> result same with Vietnamese input? I hope get a help from u. 
> Thank u so much. 
> 
> 
> 
> --
> View this message in context: 
> http://lucene.472066.n3.nabble.com/Solr-Analyzer-Language-tp4307585.html
> Sent from the Solr - User mailing list archive at Nabble.com.

Re: Solr 6 Performance Suggestions

[Florian Gleixner]

On 22.11.2016 14:54, Max Bridgewater wrote:

> test cases were exactly the same, the machines where exactly the same and
> heap settings exactly the same (Xms24g, Xmx24g). Requests were sent with

Setting heap too large is a common error. Recent Solr use the filesystem
cache, so you don't have to set heap to the size of the index. The
avalible RAM has to be able to run the OS, run the jvm and hold most of
the index data in filesystem cache.
If you have 32GB RAM and a 20GB Index, then set -Xms never higher than
10GB. I personally would set -Xms to 4GB and omit -Xmx




signature.asc
Description: OpenPGP digital signature

Solr Analyzer Language

[Chien Nguyen]

Hi, everyone. 
Now, i want to search with input is Unsigned Vietnamese. How can i do to get
result same with Vietnamese input? I hope get a help from u. 
Thank u so much. 



--
View this message in context: 
http://lucene.472066.n3.nabble.com/Solr-Analyzer-Language-tp4307585.html
Sent from the Solr - User mailing list archive at Nabble.com.

RE: Again : Query formulation help

[Prasanna S. Dhakephalkar]

Hi,

There was another thought in our internal group, I want your opinion will it
give me what I am looking for

In fq give following

{!frange l=2 u=4}sum(
exists(query({!v='code1:'})),
exists(query({!v='code2:'})),
exists(query({!v='code3:'})),
exists(query({!v='code4:'}))
)

Regards,

Prasanna

-Original Message-
From: Michael Kuhlmann [mailto:k...@solr.info] 
Sent: Thursday, November 24, 2016 4:29 PM
To: solr-user@lucene.apache.org
Subject: Re: Again : Query formulation help

Hi Prasanna,

there's no such filter out-of-the-box. It's similar to the mm parameter in
(e)dismax parser, but this only works for full text searches on the same
fields.

So you have to build the query on your own using all possible permutations:

fq=(code1: AND code2:) OR (code1: AND code3:) OR .

Of course, such a query can become huge when there are more than four
constraints.

Best,
Michael

Am 24.11.2016 um 11:40 schrieb Prasanna S. Dhakephalkar:
> Hi,
>
>  
>
> Need to formulate a distinctive field values query on 4 fields with 
> minimum match on 2 fields
>
>  
>
> I have 4 fields in my core
>
> Code 1 : Values between 1001 to 
>
> Code 2 : Values between 1001 to 
>
> Code 3 : Values between 1001 to 
>
> Code 4 : Values between 1001 to 
>
>  
>
> I want to formulate a query in following manner
>
>  
>
> Code 1 : 
>
> Code 2 : 
>
> Code 3 : 
>
> Code 4 : 
>
>  
>
> I want to formulate a query, given above parameters, the result should 
> contain documents where at least 2 of the above match.
>
>  
>
> Thanks and Regards,
>
>  
>
> Prasanna
>
>  
>
>

reindexing a solr collection of nested documents

[adfel70]

Hi
I have a solr collection of nested documents.
I would like to reindex this collection to a new collection ,without running
the original process that created this collection.

If this was not a a collection of nested documents, I would use the /export 
handler to export all the documents and then reindex it.

Can the /export handler be used to retrieve the original nested structure
that was indexed?

Is there another way to do this?

Note that I want to make some schema modifications, so just copying the
index won't do.


Thanks.




--
View this message in context: 
http://lucene.472066.n3.nabble.com/reindexing-a-solr-collection-of-nested-documents-tp4307586.html
Sent from the Solr - User mailing list archive at Nabble.com.

Highlight is Empty for A Matched Query

[Furkan KAMACI]

My content has that line:

 \n \n\n Intelligent En

When I search for *intelligent *it returns 1 response as well. My content
field is defined as:

 

Highlighter is default too. I just make *highlight=on* and *hl.field=content
*However my response does not have any highlights. When I try with
different keywords: Some query keywords has highlight section and some of
them are not. What my be the problem for that? I didn't edit stopwords,
synonyms, etc.

Kind Regards,
Furkan KAMACI

Re: Query parser behavior with AND and negative clause

[Erik Hatcher]

If you start a query with '{' then the entire expression uses that qparser.  
That's why the second one doesn't work - it ignores the AND... stuff.  

Starting a nested query with a lucene qparser character gets it out of that 
rut.  

   Erik

> On Nov 25, 2016, at 14:23, Sandeep Khanzode 
>  wrote:
> 
> WORKS:
> +{!field f=dateRange1 op=Intersects v='[2016-11-22T12:01:00Z TO 
> 2016-11-22T13:59:00Z]'} +(*:* -{!field f=dateRange2 op=Contains 
> v='[2016-11-22T12:01:00Z TO 2016-11-22T13:59:00Z]'})
> 
> 
> +ConstantScore(IntersectsPrefixTreeFilter(fieldName=dateRange1,queryShape=[2016-11-22T12:01
>  TO 2016-11-22T13:59:00],detailLevel=9,prefixGridScanLevel=7)) 
> +(MatchAllDocsQuery(*:*) 
> -ConstantScore(ContainsPrefixTreeFilter(fieldName=dateRange2,queryShape=[2016-11-22T12:01
>  TO 2016-11-22T13:59:00],detailLevel=9,multiOverlappingIndexedShapes=true)))
> 
> 
> 
> 
> DOES NOT WORK :
> {!field f=dateRange1 op=Intersects v='[2016-11-22T12:01:00Z TO 
> 2016-11-22T13:59:00Z]'} AND (*:* -{!field f=dateRange2 op=Contains 
> v='[2016-11-22T12:01:00Z TO 2016-11-22T13:59:00Z]'})
> 
> 
> ConstantScore(IntersectsPrefixTreeFilter(fieldName=dateRange1,queryShape=[2016-11-22T12:01
>  TO 2016-11-22T13:59:00],detailLevel=9,prefixGridScanLevel=7))
>  SRK 
> 
> 
>On Thursday, November 24, 2016 9:02 PM, Alessandro Benedetti 
>  wrote:
> 
> 
> Hey Sandeep,
> can you debug the query ( debugQuery=on) and show how the query is parsed ?
> 
> Cheers
> 
> 
> 
> On Thu, Nov 24, 2016 at 12:38 PM, Sandeep Khanzode <
> sandeep_khanz...@yahoo.com.invalid> wrote:
> 
>> Hi Erick,
>> The example record contains ...dateRange1 = [2016-11-22T18:00:00Z TO
>> 2016-11-22T20:00:00Z], [2016-11-22T06:00:00Z TO 
>> 2016-11-22T14:00:00Z]dateRange2
>> = [2016-11-22T12:00:00Z TO 2016-11-22T14:00:00Z]"
>> The first query works ... which means that it is able to EXCLUDE this
>> record from the result (since the negative dateRange2 clause should return
>> false). Whereas the second query should also work but it does not and
>> actually pulls the record in the result.
>> WORKS:
>> +{!field f=dateRange1 op=Intersects v='[2016-11-22T12:01:00Z TO
>> 2016-11-22T13:59:00Z]'} +(*:* -{!field f=dateRange2 op=Contains
>> v='[2016-11-22T12:01:00Z TO 2016-11-22T13:59:00Z]'})
>> 
>> 
>> DOES NOT WORK :
>> {!field f=dateRange1 op=Intersects v='[2016-11-22T12:01:00Z TO
>> 2016-11-22T13:59:00Z]'} AND (*:* -{!field f=dateRange2 op=Contains
>> v='[2016-11-22T12:01:00Z TO 2016-11-22T13:59:00Z]'})
>>   SRK
>> 
>> On Tuesday, November 22, 2016 9:41 PM, Erick Erickson <
>> erickerick...@gmail.com> wrote:
>> 
>> 
>>   _How_ does it "not work"? You haven't told us what you expect .vs.
>> what you get back.
>> 
>> Plus a sample doc that that violates your expectations (just the
>> dateRange field) would
>> also help.
>> 
>> Best,
>> Erick
>> 
>> On Tue, Nov 22, 2016 at 4:23 AM, Sandeep Khanzode
>>  wrote:
>>> Hi,
>>> I have a simple query that should intersect with dateRange1 and NOT be
>> contained within dateRange2. I have tried the following options:
>>> 
>>> WORKS:
>>> +{!field f=dateRange1 op=Intersects v='[2016-11-22T12:01:00Z TO
>> 2016-11-22T13:59:00Z]'} +(*:* -{!field f=dateRange2 op=Contains
>> v='[2016-11-22T12:01:00Z TO 2016-11-22T13:59:00Z]'})
>>> 
>>> 
>>> DOES NOT WORK :
>>> {!field f=dateRange1 op=Intersects v='[2016-11-22T12:01:00Z TO
>> 2016-11-22T13:59:00Z]'} AND (*:* -{!field f=dateRange2 op=Contains
>> v='[2016-11-22T12:01:00Z TO 2016-11-22T13:59:00Z]'})
>>> 
>>> Why?
>>> 
>>> WILL NOT WORK (because of the negative clause at the top level?):
>>> {!field f=dateRange1 op=Intersects v='[2016-11-22T12:01:00Z TO
>> 2016-11-22T13:59:00Z]'} AND -{!field f=dateRange2 op=Contains
>> v='[2016-11-22T12:01:00Z TO 2016-11-22T13:59:00Z]'}
>>> 
>>> 
>>> SRK
>> 
>> 
>> 
>> 
> 
> 
> 
> -- 
> --
> 
> Benedetti Alessandro
> Visiting card - http://about.me/alessandro_benedetti
> Blog - http://alexbenedetti.blogspot.co.uk
> 
> "Tyger, tyger burning bright
> In the forests of the night,
> What immortal hand or eye
> Could frame thy fearful symmetry?"
> 
> William Blake - Songs of Experience -1794 England
> 
> 

Re: update a document without changing anything

[Dorian Hoxha]

Thanks Ishan.

On Sun, Nov 27, 2016 at 6:42 AM, Ishan Chattopadhyaya <
ichattopadhy...@gmail.com> wrote:

> Maybe do an "inc" of 0 to a numeric field for every document.
> https://cwiki.apache.org/confluence/display/solr/
> Updating+Parts+of+Documents
>
> On Wed, Nov 23, 2016 at 2:13 PM, Dorian Hoxha 
> wrote:
>
> > Hello searcherers,
> >
> > So, I have document that is fully stored. Then I make small change in
> > schema. And now I have to reinsert every document. But I'm afraid of
> doing
> > a get+insert, because something else may change the document in the
> > meantime. So I want to do an "update" of nothing, so internally on the
> > master-shard, the document is updated without changes. Maybe an update
> with
> > no modifiers ?
> >
> > Thank You!
> >
>