Re: Solr 7.6 JoinQueryParser with Muti Threaded Facet stops solr core with Exceptions

[harjagsbby]

SolrQuery used:
q=*:*=edismax=comboid,skuid=0=15=100=A=true=true=!status_facet:inactive=!mode:AF_ONLY=**{!join+from=skuid+to=skuid+fromIndex=promotion_rules}id:promo-opera-test-1000_promo-opera-test-1000**=true=true=1&**facet.threads=32**=dynamiccategory_facet=30=true=ratingandreviewsort+desc=/promo=5000


We are migrating to 7.6. We have been using joins with multithreaded facets
for years now.



--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Solr 7.6 JoinQueryParser with Muti Threaded Facet stops solr core with Exceptions

[harjagsbby]

2019-07-09 17:37:30.053 INFO 
(facetExecutor-67-thread-22-processing-n:127.0.0.1:8983_solr
x:products_comp-v1_shard1_replica_n21 c:products_comp-v1 s:shard1
r:core_node22) [   ] o.a.s.c.SolrCore [promotion_rules-v1_shard1_replica_n1] 
**CLOSING SolrCore org.apache.solr.core.SolrCore**@44de0f89
2019-07-09 17:37:30.054 INFO 
(facetExecutor-67-thread-22-processing-n:127.0.0.1:8983_solr
x:products_comp-v1_shard1_replica_n21 c:products_comp-v1 s:shard1
r:core_node22) [   ] o.a.s.m.SolrMetricManager Closing metric reporters for
registry=solr.core.promotion_rules-v1.shard1.replica_n1, tag=44de0f89
2019-07-09 17:37:30.056 INFO 
(facetExecutor-67-thread-22-processing-n:127.0.0.1:8983_solr
x:products_comp-v1_shard1_replica_n21 c:products_comp-v1 s:shard1
r:core_node22) [   ] o.a.s.m.r.SolrJmxReporter Closing reporter
[org.apache.solr.metrics.reporters.SolrJmxReporter@36a60356: rootName =
null, domain = solr.core.promotion_rules-v1.shard1.replica_n1, service url =
null, agent id = null] for registry
solr.core.promotion_rules-v1.shard1.replica_n1 /
com.codahale.metrics.MetricRegistry@79be463d
2019-07-09 17:37:30.148 INFO 
(facetExecutor-67-thread-22-processing-n:127.0.0.1:8983_solr
x:products_comp-v1_shard1_replica_n21 c:products_comp-v1 s:shard1
r:core_node22) [   ] o.a.s.m.SolrMetricManager Closing metric reporters for
registry=solr.collection.promotion_rules-v1.shard1.leader, tag=44de0f89
2019-07-09 17:37:30.159 INFO 
(facetExecutor-67-thread-22-processing-n:127.0.0.1:8983_solr
x:products_comp-v1_shard1_replica_n21 c:products_comp-v1 s:shard1
r:core_node22) [   ] o.a.s.u.p.DocExpirationUpdateProcessorFactory
Triggering Graceful close of DocExpiration Executor
2019-07-09 17:38:03.062 **ERROR
(facetExecutor-67-thread-26-processing-n:127.0.0.1:8983_solr
x:products_comp-v1_shard1_replica_n21 c:products_comp-v1 s:shard1
r:core_node22) [   ] o.a.s.c.SolrCore Too many close [count:-4] on
org.apache.solr.core.SolrCore@44de0f89. Please report this exception to
solr-user@lucene.apache.org**
2019-07-09 17:38:03.067 ERROR
(facetExecutor-67-thread-28-processing-n:127.0.0.1:8983_solr
x:products_comp-v1_shard1_replica_n21 c:products_comp-v1 s:shard1
r:core_node22) [   ] **o.a.s.c.SolrCore Too many close [count:-5] on
org.apache.solr.core.SolrCore@44de0f89. Please report this exception to
solr-user@lucene.apache.org**
2019-07-09 17:38:06.176 ERROR
(facetExecutor-67-thread-25-processing-n:127.0.0.1:8983_solr
x:products_comp-v1_shard1_replica_n21 c:products_comp-v1 s:shard1
r:core_node22) [   ] o.a.s.c.SolrCore Too many close [count:-7] on
org.apache.solr.core.SolrCore@44de0f89. Please report this exception to
solr-user@lucene.apache.org
2019-07-09 17:38:07.602 ERROR
(facetExecutor-67-thread-24-processing-n:127.0.0.1:8983_solr
x:products_comp-v1_shard1_replica_n21 c:products_comp-v1 s:shard1
r:core_node22) [   ] o.a.s.c.SolrCore Too many close [count:-6] on
org.apache.solr.core.SolrCore@44de0f89. Please report this exception to
solr-user@lucene.apache.org
2019-07-09 17:38:11.868 ERROR
(facetExecutor-67-thread-23-processing-n:127.0.0.1:8983_solr
x:products_comp-v1_shard1_replica_n21 c:products_comp-v1 s:shard1
r:core_node22) [   ] o.a.s.c.SolrCore Too many close [count:-8] on
org.apache.solr.core.SolrCore@44de0f89. Please report this exception to
solr-user@lucene.apache.org
2019-07-09 17:38:32.522 ERROR
(facetExecutor-67-thread-4-processing-n:127.0.0.1:8983_solr
x:products_comp-v1_shard1_replica_n21 c:products_comp-v1 s:shard1
r:core_node22) [   ] **o.a.s.c.SolrCore Too many close [count:-9] on**
org.apache.solr.core.SolrCore@44de0f89. Please report this exception to
solr-user@lucene.apache.org
2019-07-09 17:39:08.862 ERROR
(facetExecutor-67-thread-3-processing-n:127.0.0.1:8983_solr
x:products_comp-v1_shard1_replica_n21 c:products_comp-v1 s:shard1
r:core_node22) [   ] o.a.s.c.SolrCore Too many close [count:-10] on
org.apache.solr.core.SolrCore@44de0f89. Please report this exception to
solr-user@lucene.apache.org
2019-07-09 17:39:08.988 ERROR
(facetExecutor-67-thread-6-processing-n:127.0.0.1:8983_solr
x:products_comp-v1_shard1_replica_n21 c:products_comp-v1 s:shard1
r:core_node22) [   ] o.a.s.c.SolrCore Too many close [count:-12] on
org.apache.solr.core.SolrCore@44de0f89. Please report this exception to
solr-user@lucene.apache.org
2019-07-09 17:39:09.048 ERROR
(facetExecutor-67-thread-5-processing-n:127.0.0.1:8983_solr
x:products_comp-v1_shard1_replica_n21 c:products_comp-v1 s:shard1
r:core_node22) [   ] o.a.s.c.SolrCore Too many close [count:-13] on
org.apache.solr.core.SolrCore@44de0f89. Please report this exception to
solr-user@lucene.apache.org
2019-07-09 17:39:10.474 ERROR
(facetExecutor-67-thread-2-processing-n:127.0.0.1:8983_solr
x:products_comp-v1_shard1_replica_n21 c:products_comp-v1 s:shard1
r:core_node22) [   ] o.a.s.c.SolrCore Too many close [count:-11] on
org.apache.solr.core.SolrCore@44de0f89. Please report this exception to
solr-user@lucene.apache.org
2019-07-09 17:39:10.572 ERROR

Parse multivalued field as list with custom function

[Gregory.Guichard]

Hello,


I'm trying to parse multivalued field (i.e : [8, 6, 9, 50]) as a List in a 
custom function.

I looked all the existing parser here : 
(https://github.com/apache/lucene-solr/tree/master/solr/core/src/java/org/apache/solr/search),
 and I don't find any example of how to parse a multivalued field in a List.


Can you give me example or some leads ?


I thank you in advance.

Using DIH for dynamic list of mailboxes

[Mr Havercamp]

I am currently integrating Solr with a service which uses Zimbra for mail.
I have been tasked with importing inbox content into Solr and have opted to
use DIH.

Because this service adds/removes mailboxes, what would be the best
approach for managing a list of mailboxes for DIH?

My main concern is around having to store the user's password as plaintext;
I'm wondering if there is a workaround.

Interested to hear others' solutions/workarounds to this problem.

Thanks

Re: API to get all the solr nodes (active/down)

[Walter Underwood]

Put this in a script and pass any hostname from the cluster as the argument. I 
use the host name of the load balancer for the cluster. You will need to 
install the “jq” tool. 

https://stedolan.github.io/jq/ 


#!/bin/bash

cluster=$1

echo "Live nodes:"
for host in `curl -s 
"http://${cluster}:8983/solr/admin/collections?action=CLUSTERSTATUS=json; | 
jq -r '.cluster.live_nodes[]' | sort`
do
echo $host
done

echo "All nodes:"
for host in `curl -s 
"http://${cluster}:8983/solr/admin/collections?action=CLUSTERSTATUS=json; | 
jq -r ''.cluster.collections[].shards[].replicas[].node_name'' | sort -u`
do
echo $host
done


wunder
Walter Underwood
wun...@wunderwood.org
http://observer.wunderwood.org/  (my blog)

> On Jul 15, 2019, at 10:55 PM, Fatima Khan  wrote:
> 
> Hi All,
>I am working on a project in which we use Solr for our audit logs. We
> have implemented bootstrapping for the same and we are currently using
> the getLiveNodes()
> method for getting the live Solr nodes but we wanted to know if there is
> any other api in which we
> can get all the Solr nodes irrespective of their status that is even if the
> Solr node is down can we get the details
> of these down nodes.
> 
> 
> *Thanks & Regards ,*
> 
> *Fatima Khan*

Re: Problems using a suggester component in the /select handler in cloud mode

[Alexandros Paramythis]

Hi everyone,

We have a fix for the problem described in the message below. Could 
anyone provide pointers to documentation on how we would go about 
contributing this back?


Thanks in advance for your input,

Alex


On 26/06/2019 10:48, Alexandros Paramythis wrote:

Hi everyone,

Environment:

Solr 7.5.0, cloud mode (but problem should be identical in multiple 
versions, at least in 7.x)


Summary:

We have a Solr configuration that returns suggestions in the course of 
a normal search call (i.e., we have a 'suggest' component added to the 
'last-components' for '/select' request handler). This does not work 
in cloud mode, where we get an NPE in QueryComponent. This problem 
seems to have been reported in various forms in the past -- see for 
example [1] and [2] (links at the end of this email) -- but we 
couldn't find any resolution (or in-depth discussion for that matter).


In more detail:

We have a suggest component configured as follows:

  

    
    default
    name="classname">org.apache.solr.spelling.suggest.Suggester
    name="lookupImpl">org.apache.solr.spelling.suggest.fst.AnalyzingLookupFactory

    dict_default
    text_suggest
    text_suggest
    true
    true
    true
    

    
    suggest_phrase
    name="lookupImpl">org.apache.solr.spelling.suggest.fst.AnalyzingLookupFactory

    dict_suggest_phrase
    name="suggestAnalyzerFieldType">text_suggest_phrase

    suggest_phrase
    true
    true
    true
    

    
    suggest_infix_shingle
    AnalyzingInfixLookupFactory
    suggestInfixShingleDir
    name="suggestAnalyzerFieldType">text_suggest_phrase

    suggest_phrase
    true
    true
    true
    

    
    suggest_prefix
    Suggester
    AnalyzingLookupFactory
    name="suggestAnalyzerFieldType">text_suggest_prefix

    suggest_prefix
    true
    true
    true
    

  


This component works without issued both in standalone and cloud mode, 
when used as the sole component in a handler, such as in the following 
excerpt:


    startup="lazy">

    
    default
        suggest_phrase
    name="suggest.dictionary">suggest_infix_shingle

    suggest_prefix
    true
    10
    false
    
    
    suggest
    
    


It also works when used along with other component in standalone mode, 
such as in the following excerpt, where we use the suggest component 
to get suggestions during a "normal" search call:


    
    
    explicit
    10
    text_search

    edismax

    title^5.0 subtitle^3.0 
abstract^2.0 text_search
    title^5.0 subtitle^3.0 
abstract^2.0 text_search

    4
    on
    default
    true
    10
    name="spellcheck.alternativeTermCount">5
    name="spellcheck.maxResultsForSuggest">5

    true
    name="spellcheck.collateExtendedResults">true

    10
    5

    default
    name="suggest.dictionary">suggest_phrase
    name="suggest.dictionary">suggest_infix_shingle
    name="suggest.dictionary">suggest_prefix

    true
    10
    false
    

    
    suggest
    spellcheck
    
    

However, the above configuration does not work in cloud mode, where we 
get an NPE if a search call is made:


 o.a.s.s.HttpSolrCall null:java.lang.NullPointerException
at 
org.apache.solr.handler.component.QueryComponent.unmarshalSortValues(QueryComponent.java:1034)
at 
org.apache.solr.handler.component.QueryComponent.mergeIds(QueryComponent.java:885)
at 
org.apache.solr.handler.component.QueryComponent.handleRegularResponses(QueryComponent.java:585)
at 
org.apache.solr.handler.component.QueryComponent.handleResponses(QueryComponent.java:564)
at 
org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:426)
at 
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:199)

at org.apache.solr.core.SolrCore.execute(SolrCore.java:2541)
at 

Re: Returning multiple fields in graph streaming expression response documents

[markus kalkbrenner]

You have to perform a „join“ to get more fields.

> Am 16.07.2019 um 13:52 schrieb Ahmed Adel :
> 
> Hi,
> 
> How can multiple fields be returned in graph traversal streaming expression
> response documents? For example, the following query:
> 
> nodes(emails,
>  walk="john...@apache.org->from",
>  gather="to")
> 
> 
> returns these documents in the response:
> 
> {
>  "result-set": {
>"docs": [
>  {
>"node": "sl...@campbell.com",
>"collection": "emails",
>"field": "to",
>"level": 1
>  },
>  {
>"node": "catherine.per...@enron.com",
>"collection": "emails",
>"field": "to",
>"level": 1
>  },
>  {
>"node": "airam.arte...@enron.com",
>"collection": "emails",
>"field": "to",
>"level": 1
>  },
>  {
>"EOF": true,
>"RESPONSE_TIME": 44
>  }
>]
>  }
> }
> 
> How can the query above be modified to return more document fields,
> "subject" for example?
> 
> Best regards,
> 
> A.

Returning multiple fields in graph streaming expression response documents

[Ahmed Adel]

Hi,

How can multiple fields be returned in graph traversal streaming expression
response documents? For example, the following query:

nodes(emails,
  walk="john...@apache.org->from",
  gather="to")


returns these documents in the response:

{
  "result-set": {
"docs": [
  {
"node": "sl...@campbell.com",
"collection": "emails",
"field": "to",
"level": 1
  },
  {
"node": "catherine.per...@enron.com",
"collection": "emails",
"field": "to",
"level": 1
  },
  {
"node": "airam.arte...@enron.com",
"collection": "emails",
"field": "to",
"level": 1
  },
  {
"EOF": true,
"RESPONSE_TIME": 44
  }
]
  }
}

How can the query above be modified to return more document fields,
"subject" for example?

Best regards,

A.

API to get all the solr nodes (active/down)

[Fatima Khan]

Hi All,
I am working on a project in which we use Solr for our audit logs. We
have implemented bootstrapping for the same and we are currently using
the getLiveNodes()
method for getting the live Solr nodes but we wanted to know if there is
any other api in which we
can get all the Solr nodes irrespective of their status that is even if the
Solr node is down can we get the details
of these down nodes.


*Thanks & Regards ,*

*Fatima Khan*

Upload/use a plugin JAR in ZooKeeper

[Richard Walker]

Hi, I'm trying to use a plugin JAR containing
a custom query parser.

I've been able to get this to work the "simple" way,
by putting the JAR in the file system, and specifying
basic

  
  

values in solrconfig.xml. No problem doing it this way.

But I'm running in SolrCloud mode and I'd like to take
advantage of an option that the user guide seems to offer
at this page:

https://lucene.apache.org/solr/guide/8_1/resource-and-plugin-loading.html

But, so far, I don't see how to make it work.

To be specific, I'm trying to use this idea:

"Resources and plugins may be stored:
• in ZooKeeper under a collection’s configset node (SolrCloud only);"

Note: I'm _not_ trying to do the _third_ option listed, i.e.,
"• in Solr’s Blob Store (SolrCloud only)", that uses
the ".system" collection.

The user guide seems to suggest that I can upload the JAR
to the collection's config using zk cp:
"To upload a plugin or resource to a configset
already stored on ZooKeeper, you can use bin/solr zk cp."

So, I've used zk cp to upload the JAR to
zk:/configs/my_collection/my_plugin.jar
(I also tried various other subdirectories such as
zk:/configs/my_collection/lib/my_plugin.jar)

So far, so good. But now how do I refer to the JAR in solrconfig.xml?
The user guide doesn't really say.

I've tried specifying the location of the JAR
with various values of  element.

No success at all; I only get a ClassNotFoundException
for the plugin class.

Could someone please tell me what I'm missing, i.e., what
I need to do to use a plugin JAR stored
"in ZooKeeper under a collection’s configset node"?

Richard.