solrcloud 8.0.0 - debugQuery=on, exception

[derrick cui]

Hi all,
I run a query in solr admin, it's ok if debugQuery=off, but it's throw an 
exception if I set debugQuery=on, please help, thanks
there is debug stack:
2019-06-20 11:33:29.529 ERROR (qtp769429195-2324) [c:article s:shard2 
r:core_node9 x:article_shard2_replica_n6] o.a.s.h.RequestHandlerBase 
java.lang.NullPointerException        at 
java.util.Objects.requireNonNull(Objects.java:203)        at 
org.apache.lucene.search.LeafSimScorer.(LeafSimScorer.java:38)        at 
org.apache.lucene.search.spans.SpanWeight.explain(SpanWeight.java:160)        
at org.apache.lucene.search.BooleanWeight.explain(BooleanWeight.java:81)        
at org.apache.lucene.search.IndexSearcher.explain(IndexSearcher.java:707)       
 at org.apache.lucene.search.IndexSearcher.explain(IndexSearcher.java:684)      
  at 
org.apache.solr.search.SolrIndexSearcher.explain(SolrIndexSearcher.java:2216)   
     at 
org.apache.solr.util.SolrPluginUtils.getExplanations(SolrPluginUtils.java:453)  
      at 
org.apache.solr.util.SolrPluginUtils.doStandardResultsDebug(SolrPluginUtils.java:382)
        at 
org.apache.solr.util.SolrPluginUtils.doStandardDebug(SolrPluginUtils.java:343)  
      at 
org.apache.solr.handler.component.DebugComponent.process(DebugComponent.java:100)
        at 
org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:306)
        at 
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:199)
        at org.apache.solr.core.SolrCore.execute(SolrCore.java:2559)        at 
org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:711)        at 
org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:516)        at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:394)
        at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:340)
        at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1602)
        at 
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:540)      
  at 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146)   
     at 
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)     
   at 
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) 
       at 
org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257)
        at 
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1588)
        at 
org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
        at 
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1345)
        at 
org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)
        at 
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:480)       
 at 
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1557)
        at 
org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)
        at 
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1247)
        at 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)   
     at 
org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:220)
        at 
org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:126)
        at 
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) 
       at 
org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:335)
        at 
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) 
       at org.eclipse.jetty.server.Server.handle(Server.java:502)        at 
org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:364)        at 
org.eclipse.jetty.server.HttpChannel.run(HttpChannel.java:305)        at 
org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333)
        at 
org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310)
        at 
org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
        at 
org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:132)
        at 
org.eclipse.jetty.http2.HTTP2Connection.produce(HTTP2Connection.java:171)       
 at 
org.eclipse.jetty.http2.HTTP2Connection.onFillable(HTTP2Connection.java:126)    
    at 
org.eclipse.jetty.http2.HTTP2Connection$FillableCallback.succeeded(HTTP2Connection.java:338)
        at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)    
    at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118)     
   at 
org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333)
        at 
org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310)
     

8.0 upgrade issue

[Scott Yeadon]

Hi,

I’m running Solr on Ubuntu 18.04 (32-bit) using OpenJDK 10.0.2. Up until now I 
have had no problem with Solr (started running it since 4.x), however after 
upgrading from 7.x to 8.x I am getting serious memory issues.

I have a small repository of 30,000 documents currently using Solr 7.1 for the 
search function (for the last two years without issue). I attempted an upgrade 
to 8.1.1 and tried to perform a full reindex however, it manages about 1000 
documents and then dies from lack of memory (or so it says). I tried 8.1.0 with 
the same result. I then tried 8.0.0 which did successfully manage a full 
reindex but then after performing a couple of search queries died from lack of 
memory. I then tried 7.7.2 which worked fine. I have now gone back to my 
original 7.1 as I can’t risk 8.x in my production system. Has anyone else had 
these issues with 8.x?

Note that I did increase Xmx to 1024m (previously 512m) but that made no 
difference, it may be some other resource than memory, but if it is, it isn’t 
saying so, and it’s such a small repository it doesn’t seem to make sense to be 
running out of memory.

Scott.

Re: How to have Suggester to return part of the content instead of the entire content in Solr version 7.6.0?

[ppunet]

What would you recommend to use for the search field autocomplete
functionality?
Considering I have the whole pdf content, and the user can start searching
for any term from the pdf.



--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Re: SOLR Suggester returns either the full field value or single terms only

[ppunet]

Hey,

As the SuggeterComponent provides the 'entire content' of the field in the
suggestions. How is it possible to have Suggester to return only part of the
content of the field, instead of the entire content, which in my scenario
quite long?



--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Re: How to have Suggester to return part of the content instead of the entire content in Solr version 7.6.0?

[Erick Erickson]

you're mis-using suggester. It’s purpose is to return the entire contents of 
the “document”, to handle, specifically, multi-word suggestions, typically just 
a few words, often 2-4. Putting a large text field instead is outside the 
design.

Also remember that the suggester is looking for the _indexed_ values to return 
best match. So stemming etc. are in the part that matches, and the stored 
portion (which is a big, unanalyzed bunch of text) is all that’s available to 
be returned.

How would suggester know which parts of a large text field to return?

Perhaps you can extract “important” short phrases from the text and put those 
in the suggester as separate documents, but I admit that’s very hand-wavy. This 
is where Named Entity Recognition etc. are sometimes used, but that’s not 
something you get OOB, usually it’s done during the ETL process.

You really have to re-think your expectations about what OOB suggester is good 
for.

Best,
Erick

> On Jun 19, 2019, at 11:58 AM, ppunet  wrote:
> 
> Here is my problem statement and I would really appreciate for your feedback.
> 
> Solr version 7.6.0
> 
> 1. There are 1000's of pdf's with large amount of content are indexed to
> Solr.
> 2. Using AnalyzingInfixSuggester for the suggestions.
> 
> Q. As the SuggeterComponent provides the 'entire content' of the field in
> the suggestions. How is it possible to have Suggester to return only part of
> the content of the field, instead of the entire content, which in my
> scenario quite long?
> 
> 
> Thanks in advance.
> 
> PD
> 
> 
> 
> --
> Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

synonyms.txt -> minimum match (~ mm)

[Kojo]

I have a synonyms.txt mapping some words.

On my Solr 4.9, when I search a word that is in the synonyms.txt, the
debugger shows bellow:

"rawquerystring": "interleucina-6",
"querystring": "interleucina-6",
"parsedquery": "(+DisjunctionMaxQuerytext:interleucin
text:interleucin text:6 text:6)~4/no_coord",
"parsedquery_toString": "+(((text:interleucin text:interleucin
text:6 text:6)~4))",

On my Solr 6.6, the same search doesn´t include the ~4

"rawquerystring":"interleucina-6",
"querystring":"interleucina-6",
"parsedquery":"(+DisjunctionMaxQuery(((text:interleucin
text:interleucin text:6 text:6/no_coord",
"parsedquery_toString":"+((text:interleucin text:interleucin
text:6 text:6))",


My problem is that I don´t know how to configure Solr 6.6 to work as Solr
4.9 to apply the minimum match to the words from the synonyms.txt

On solrconfig.xml I tried to include the parameter 100% on the 
But when I do this, it applies to all the wods that I query, not only those
on the synonyms.txt.

All of the above was tested on Solr dashboard, there is any application
layer transforming the queries.

Could someone point what I am doing wrong?

Thank you very much.

Koji

How to have Suggester to return part of the content instead of the entire content in Solr version 7.6.0?

[ppunet]

Here is my problem statement and I would really appreciate for your feedback.

Solr version 7.6.0

1. There are 1000's of pdf's with large amount of content are indexed to
Solr.
2. Using AnalyzingInfixSuggester for the suggestions.

Q. As the SuggeterComponent provides the 'entire content' of the field in
the suggestions. How is it possible to have Suggester to return only part of
the content of the field, instead of the entire content, which in my
scenario quite long?


Thanks in advance.

PD



--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Re: Solr's suggester results

[ppunet]

Here is my problem statement and I would really appreciate for your feedback.

1. There are 1000's of pdf's with large amount of content are indexed to
Solr.
2. Using AnalyzingInfixSuggester for the suggestions.

Q. As the SuggeterComponent provides the 'entire content' of the field in
the suggestions. How is it possible to have Suggester to return only part of
the content of the field, instead of the entire content, which in my
scenario quite long?


Thanks in advance.

PD



--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

REINDEXCOLLECTION does not work with (basic) authentication

[Colvin Cowie]

Hello

I'm on the Solr 8.1 branch off commit
f26388d034fe5eadca7416aa63b509b8db2c7688 so I have the authentication fixes
from SOLR-13510 (intermittent 401s for internode requests)

When trying to use the new REINDEXCOLLECTION command with basic auth
enabled, the daemon stream fails with repeated 401s when trying to access
the target collection.

This might be the same problem as SOLR-13472, except it applies even with a
single node, and this doesn't require role based configuration.

Repro: I added a reindex request in BasicAuthIntegrationTest and it is
reproducible in there... I don't know what effect it should have on the
auth metrics, if it were working correctly, so I don't know how to update
the test properly. But you can add the request towards the end of
org.apache.solr.security.BasicAuthIntegrationTest.testBasicAuth()



*  CollectionAdminRequest.ReindexCollection reindexReq =
CollectionAdminRequest.reindexCollection(COLLECTION);
reindexReq.setBasicAuthCredentials("harry", "HarryIsUberCool");
cluster.getSolrClient().request(reindexReq, COLLECTION);*

Manual Repro:
run bin/solr -e cloud
Choose 1 node / 1 shard / 1 replica
In browser GET
http://localhost:8983/solr/admin/collections?action=REINDEXCOLLECTION=gettingstarted
will succeed
Enable security: server\scripts\cloud-scripts\zkcli -zkhost localhost:9983
-cmd putfile /security.json 

{
"authentication": {
"blockUnknown": true,
"class": "solr.BasicAuthPlugin",
"credentials": {
"solradmin": "fskh17INKrOTSRCJ8HkamA0L6Uiq1dSMgn4OVy8htME=
/Q4VgOkwVlP6AMVY+ML+IuodbfV81WEfZ3lFb390bws="
}
}
}

In browser authenticate (as solradmin : solradmin) and GET
http://localhost:8983/solr/admin/collections?action=REINDEXCOLLECTION=gettingstarted
will time out after 180 seconds

The solr log will show repeated 401s

Setting "forwardCredentials" : true in the security.json does not appear to
change the outcome.


 responses.txt


 solr.log


 security.json

Re: Solr 5.3 to 6.0

[Erick Erickson]

You cannot go from 5->8 true. However, you also cannot go from 5->6->7->8. As 
of Lucene 8, Lucene will refuse to open an index that has ever been touched by 
Lucene 6, no matter whether you’ve used IndexUpgraderTool, reindexed all your 
data etc.

So you’ll have to plan on re-indexing from your system of record when you go to 
Solr 8 no matter what, so why not now?

Best,
Erick

> On Jun 19, 2019, at 10:02 AM, ilango dhandapani  
> wrote:
> 
> In solr reference guide, I read that need to go to next major version and
> cannot jump ahead next major verison, as indexes will not work. I did not
> attempt from 5.3 to 7.x or 8.x though.
> 
> Thanks,
> Ilango
> 
> 
> 
> --
> Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Re: creating date facets

[Erick Erickson]

There are two approaches I might use, which is really up to you.

- You can do a regex filter. So define your extra fields as you want, then use 
a regex charFilter (NOT filter, you want to transform the entire input) to peel 
out the separate parts. Then copyfield to each one, each with a different regex 
to peel out the correct part.

- use a ScriptUpdateProcessor to do whatever you want in the scripting language 
you’re most comfortable with. Note that the SolrDocument that’s handled by a 
SUP is just a map of key:value pairs and you can modify it as you see fit.

Best,
Erick

> On Jun 19, 2019, at 10:25 AM, Nightingale, Jonathan A (US) 
>  wrote:
> 
> Hi all,
> I'm trying to have a date field automatically generate some facets of itself, 
> like hour of the day and hour of the week as examples, when its stored. I was 
> looking at this tutorial and it deemed to almost do what I wanted
> 
> https://nathanfriend.io/2017/06/26/smart-date-searching-with-solr.html
> 
> I was wondering if there was a way to use these filters and tokenizers to 
> generate values for different fields. Maybe in conjunction with a copyfield?
> 
> I'd like it to work whenever a *_c (calendar) field is indexed it also 
> indexes the hour of the day (hod_d) and hor of week (how_d) fields as well.
> 
> Any thoughts?
> 
> Thanks!

creating date facets

[Nightingale, Jonathan A (US)]

Hi all,
I'm trying to have a date field automatically generate some facets of itself, 
like hour of the day and hour of the week as examples, when its stored. I was 
looking at this tutorial and it deemed to almost do what I wanted

https://nathanfriend.io/2017/06/26/smart-date-searching-with-solr.html

I was wondering if there was a way to use these filters and tokenizers to 
generate values for different fields. Maybe in conjunction with a copyfield?

I'd like it to work whenever a *_c (calendar) field is indexed it also indexes 
the hour of the day (hod_d) and hor of week (how_d) fields as well.

Any thoughts?

Thanks!

Re: Solr 5.3 to 6.0

[ilango dhandapani]

In solr reference guide, I read that need to go to next major version and
cannot jump ahead next major verison, as indexes will not work. I did not
attempt from 5.3 to 7.x or 8.x though.

Thanks,
Ilango



--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Re: Solr 5.3 to 6.0

[Erick Erickson]

Got to the latest 6x release if you must stick with 6x.

Why not leapfrog all the way to 8?

Erick

> On Jun 19, 2019, at 9:08 AM, ilango dhandapani  
> wrote:
> 
> Shawn,
> 
> Is it advisable to goto 6.6 or 6.0 from 5.3.0 ?'
> 
> Thanks,
> Ilango
> 
> 
> 
> --
> Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Re: Solr 5.3 to 6.0

[ilango dhandapani]

Shawn,

Is it advisable to goto 6.6 or 6.0 from 5.3.0 ?'

Thanks,
Ilango



--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Re: Solr 5.3 to 6.0

[ilango dhandapani]

Thanks Shawn. It worked after i explicitly mentioned docValues=False for the
fields am using in managed-schema.
So in case if I need to re index from scratch in new version, is there any
better way of doing that quickly ?

Thanks,
Ilango



--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Re: Delete with Solrj deleteByQuery - Boolean clauses

[Shawn Heisey]

On 6/19/2019 8:33 AM, rgummadi wrote:

Still using Solr 4.6. Terms query parser does not exist in this version
right?


Correct.  It was added in version 4.10.0.

https://issues.apache.org/jira/browse/SOLR-6318

Thanks,
Shawn

Re: Solr 7.7.2 - SolrCloud - SPLITSHARD - Using LINK method fails on disk usage checks

[Andrew Kettmann]

Entered issue: https://issues.apache.org/jira/browse/SOLR-13563


Please let me know if I need to include any other information.


I have to say, props to anyone involved in making the "ant idea" target a 
thing. Makes it ridiculously easy for someone who can code, but not java 
specifically, to look at and suggest possible fixes to the code. 10/10 would 
submit a ticket again!


From: Andrzej Białecki 
Sent: Wednesday, June 19, 2019 7:07:02 AM
To: solr-user@lucene.apache.org
Subject: Re: Solr 7.7.2 - SolrCloud - SPLITSHARD - Using LINK method fails on 
disk usage checks

Hi Andrew,

Please create a JIRA issue and attach this patch, I’ll look into fixing this. 
Thanks!


> On 18 Jun 2019, at 23:19, Andrew Kettmann  
> wrote:
>
> Attached the patch, but that isn't sent out on the mailing list, my mistake. 
> Patch below:
>
>
>
> ### START
>
> diff --git 
> a/solr/core/src/java/org/apache/solr/cloud/api/collections/SplitShardCmd.java 
> b/solr/core/src/java/org/apache/solr/cloud/api/collections/SplitShardCmd.java
> index 24a52eaf97..e018f8a42f 100644
> --- 
> a/solr/core/src/java/org/apache/solr/cloud/api/collections/SplitShardCmd.java
> +++ 
> b/solr/core/src/java/org/apache/solr/cloud/api/collections/SplitShardCmd.java
> @@ -135,7 +135,9 @@ public class SplitShardCmd implements 
> OverseerCollectionMessageHandler.Cmd {
> }
>
> RTimerTree t = timings.sub("checkDiskSpace");
> -checkDiskSpace(collectionName, slice.get(), parentShardLeader);
> +if (splitMethod != SolrIndexSplitter.SplitMethod.LINK) {
> +  checkDiskSpace(collectionName, slice.get(), parentShardLeader);
> +}
> t.stop();
>
> // let's record the ephemeralOwner of the parent leader node
>
> ### END
>
> 
> From: Andrew Kettmann
> Sent: Tuesday, June 18, 2019 3:05:15 PM
> To: solr-user@lucene.apache.org
> Subject: Re: Solr 7.7.2 - SolrCloud - SPLITSHARD - Using LINK method fails on 
> disk usage checks
>
>
> Looks like the disk check here is the problem, I am no Java developer, but 
> this patch ignores the check if you are using the link method for splitting. 
> Attached the patch. This is off of the commit for 7.7.2, d4c30fc285 . The 
> modified version only has to be run on the overseer machine, so there is that 
> at least.
>
> 
> From: Andrew Kettmann
> Sent: Tuesday, June 18, 2019 11:32:43 AM
> To: solr-user@lucene.apache.org
> Subject: Solr 7.7.2 - SolrCloud - SPLITSHARD - Using LINK method fails on 
> disk usage checks
>
>
> Using Solr 7.7.2 Docker image, testing some of the new autoscale features, 
> huge fan so far. Tested with the link method on a 2GB core and found that it 
> took less than 1MB of additional space. Filled the core quite a bit larger, 
> 12GB of a 20GB PVC, and now splitting the shard fails with the following 
> error message on my overseer:
>
>
> 2019-06-18 16:27:41.754 ERROR 
> (OverseerThreadFactory-49-thread-5-processing-n:10.0.192.74:8983_solr) 
> [c:test_autoscale s:shard1  ] o.a.s.c.a.c.OverseerCollectionMessageHandler 
> Collection: test_autoscale operation: splitshard 
> failed:org.apache.solr.common.SolrException: not enough free disk space to 
> perform index split on node 10.0.193.23:8983_solr, required: 
> 23.35038321465254, available: 7.811378479003906
>at 
> org.apache.solr.cloud.api.collections.SplitShardCmd.checkDiskSpace(SplitShardCmd.java:567)
>at 
> org.apache.solr.cloud.api.collections.SplitShardCmd.split(SplitShardCmd.java:138)
>at 
> org.apache.solr.cloud.api.collections.SplitShardCmd.call(SplitShardCmd.java:94)
>at 
> org.apache.solr.cloud.api.collections.OverseerCollectionMessageHandler.processMessage(OverseerCollectionMessageHandler.java:294)
>at 
> org.apache.solr.cloud.OverseerTaskProcessor$Runner.run(OverseerTaskProcessor.java:505)
>at 
> org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209)
>at 
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
>at 
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
>at java.base/java.lang.Thread.run(Thread.java:834)
>
>
>
> I attempted sending the request to the node itself to see if it did anything 
> different, but no luck. My parameters are (Note Python formatting as that is 
> my language of choice):
>
>
>
> splitparams = {'action':'SPLITSHARD',
>   'collection':'test_autoscale',
>   'shard':'shard1',
>   'splitMethod':'link',
>   'timing':'true',
>   'async':'shardsplitasync'}
>
>
> And this is confirmed by the log message from the node itself:
>
>
> 2019-06-18 16:27:41.730 INFO  (qtp1107530534-16) [c:test_autoscale   ] 
> o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections 
> params={async=shardsplitasync=true=SPLITSHARD=test_autoscale=shard1=link}
>  status=0 QTime=20
>
>
> While it 

Re: Issue with Solr 7.7.2 - ClassCastException: org.apache.solr.common.util.ByteArrayUtf8CharSequence

[Jason Gerlowski]

Hi David,

Thanks for the heads up.  We'd hoped to put an end to these issues as
a part of SOLR-13331, but missed some field types as you pointed out.
We're aware of the issue and working on a fix for upcoming Solr
versions.  Anyone interested can watch our progress here:
https://issues.apache.org/jira/browse/SOLR-13539 . (SOLR-13538 also
has some information)

Best,

Jason

On Tue, Jun 11, 2019 at 4:00 PM David Winter  wrote:
>
> Hi,
>
> I would like let you know about server side exceptions for specific field
> types after upgrading to 7.7.2, like ClassCastException:
> org.apache.solr.common.util.ByteArrayUtf8CharSequence.
> For references: https://issues.apache.org/jira/browse/SOLR-13285 and
> https://issues.apache.org/jira/browse/SOLR-13331
>
> You may check out these issues for before upgrading.
>
> java.lang.ClassCastException:
> org.apache.solr.common.util.ByteArrayUtf8CharSequence cannot be cast to
> java.lang.String
> at
> org.apache.solr.schema.TrieDateField.toNativeType(TrieDateField.java:100)
> at
> org.apache.solr.update.processor.AtomicUpdateDocumentMerger.doSet(AtomicUpdateDocumentMerger.java:319)
>
> Mit freundlichen Grüßen / Kind regards
>
> David Winter
>
>
>

Re: Delete with Solrj deleteByQuery - Boolean clauses

[rgummadi]

Hi Erick,
Still using Solr 4.6. Terms query parser does not exist in this version
right?



--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

how to check Solr correction path?

[Wendy2]

Hi,

How to check Solr correction path?
I created a several collections in Solr 7.3.1. and use DIH to index mongoDB.
How can I get the path for the current Solr correction where DIH is running?

Thanks!  



--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

ShardSplit with HDFS

[Joe Obernberger]

Hi All - I'm running an index in HDFS and trying to do a SHARDSPLIT.  It 
is returning that there is "not enough free disk space to perform index 
split".  It looks like it is using the local disk to determine free disk 
space instead of HDFS.


Is there a way around this?  I'm running SolrCloud version 7.6.0 on 4 
nodes.  Thank you!


-Joe Obernberger

Re: CloudSolrClient :java.lang.ClassCastException: java.lang.String cannot be cast to java.util.Map . Related to "router" : "compositeId"

[Shawn Heisey]

On 6/19/2019 7:44 AM, Shawn Heisey wrote:
The version of SolrJ that's included in Spring Boot 1.5.8 is 5.5.4 ... 
CloudSolrClient does not do well when the client version is 
significantly different than the server version.  Pairing a 5.5.4 client 
with a 4.10.3 server could be problematic.


Apologies, I made a typo when I searched for info on Spring Boot.  I got 
info for 1.5.8, you're on 1.5.18.


Searching for the correct version, I get 5.5.5 as the SolrJ version. 
Which is substantially identical to 5.5.4.


Thanks,
Shawn

Re: CloudSolrClient :java.lang.ClassCastException: java.lang.String cannot be cast to java.util.Map . Related to "router" : "compositeId"

[Shawn Heisey]

On 6/19/2019 7:32 AM, Rushikesh Garadade wrote:

I am using CloudSolrClient with Spring boot Solr 1.5.18.RELEASE and Solr
Version is Solr 4.10.3.


When using Spring's packaging of SolrJ, you need to talk to Spring about 
most problems you're having.  They do more than simply include SolrJ and 
make it available, they have their own code running on top of it.


The version of SolrJ that's included in Spring Boot 1.5.8 is 5.5.4 ... 
CloudSolrClient does not do well when the client version is 
significantly different than the server version.  Pairing a 5.5.4 client 
with a 4.10.3 server could be problematic.


Thanks,
Shawn

CloudSolrClient :java.lang.ClassCastException: java.lang.String cannot be cast to java.util.Map . Related to "router" : "compositeId"

[Rushikesh Garadade]

Hello All,

I am using CloudSolrClient with Spring boot Solr 1.5.18.RELEASE and Solr
Version is Solr 4.10.3.

 new
CloudSolrClient.Builder().withZkHost(zkHosts).withZkChroot(solrProperties.getSolrZookeeperLocation()).build();


When I use Solr which comes wth CDH 5.16.1 default
when I save a document I get follwing error :

org.springframework.data.solr.UncategorizedSolrException:
java.lang.ClassCastException: *java.lang.String cannot be cast to
java.util.Map;* nested exception is
org.apache.solr.client.solrj.SolrServerException:
java.lang.ClassCastException: java.lang.String cannot be cast to
java.util.Map

at
org.springframework.data.solr.core.SolrTemplate.execute(SolrTemplate.java:168)

at
org.springframework.data.solr.core.SolrTemplate.saveBeans(SolrTemplate.java:223)

at
com.solix.emailarchiving.email.solr.search.SolrTemplateWrapper.saveBeans(SolrTemplateWrapper.java:33)

at
com.solix.emailarchiving.email.solr.search.EmailSearchRepositoryImpl.saveEmails(EmailSearchRepositoryImpl.java:28)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.lang.reflect.Method.invoke(Method.java:498)

at
org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:338)

at
org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:197)

at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)

at
org.springframework.dao.support.PersistenceExceptionTranslationInterceptor.invoke(PersistenceExceptionTranslationInterceptor.java:139)

at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:185)

at
org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212)

at com.sun.proxy.$Proxy153.saveEmails(Unknown Source)

at
com.solix.emailarchiving.email.EmailSolrServiceImpl.saveEmails(EmailSolrServiceImpl.java:24)

at
com.solix.emailarchiving.email.EmailSolrServiceImpl$$FastClassBySpringCGLIB$$40f18538.invoke()

at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)

at
org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:747)

at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)

at
org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:89)

at
com.solix.emailarchiving.annotation.ServiceTransactionAspect.beforeMethod(ServiceTransactionAspect.java:46)

at sun.reflect.GeneratedMethodAccessor138.invoke(Unknown Source)

at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.lang.reflect.Method.invoke(Method.java:498)

at
org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:643)

at
org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:632)

at
org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:70)

at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:185)

at
org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92)

at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:185)

at
org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:689)

at
com.solix.emailarchiving.email.EmailSolrServiceImpl$$EnhancerBySpringCGLIB$$a84565b1.saveEmails()

at
com.solix.emailarchiving.hbasetosolrsync.EmailsSync.sendEmailsToSolr(EmailsSync.java:157)

at
com.solix.emailarchiving.hbasetosolrsync.EmailsSync.syncMailsFromHbaseToSolr(EmailsSync.java:83)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.lang.reflect.Method.invoke(Method.java:498)

at
org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:338)

at
org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:197)

at
org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:750)

at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)

at
org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92)

at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:185)

at

RE: Is it possible configure a single data-config.xml file for all the environments?

[Hugo Angel Rodriguez]

Thank you very much for your response. Sorry I did not answer before. I am 
validating the suggestions indicated

Cordialmente,

Hugo Rodríguez Rodríguez 
Sr Associate Technical Consultant | Professional Services
 Tel: +  (571) 658-0888 extensión 1858 ▪ hugoangel.rodrig...@sas.com  
SAS Institute Colombia S.A.S. ▪ Calle 7 Sur  No. 42- 70 Bloque 2 Oficina 916 
Edificio Forum ▪ Medellin.
www.sas.com/colombia  - Síganos en    



-Original Message-
From: Shawn Heisey  
Sent: Thursday, June 13, 2019 6:03 AM
To: solr-user@lucene.apache.org
Subject: Re: Is it possible configure a single data-config.xml file for all the 
environments?

EXTERNAL

On 6/12/2019 7:46 PM, Hugo Angel Rodriguez wrote:
> Thanks Shawn for your answers
>
> Regarding your question: " Are these environments on separate Solr instances, 
> separate servers, or are they on the same Solr instance?"
> My answers is: These environments are on separate solr instances, 
> separate servers
>
> Are we dealing with SolrCloud (which is Solr + ZooKeeper), or standalone Solr 
> instances?
> We are dealing with standalone solr instances

I think JNDI is probably your best bet to have the same DIH config everywhere.  
The only documentation I can find about the JNDI method is on the Solr wiki.  
If it's still valid, we should get it in the ref guide.

https://wiki.apache.org/solr/DataImportHandlerFaq#How_do_I_use_a_JNDI_DataSource.3F

You define the JNDI datasource in Jetty, probably in jetty.xml, according to 
Jetty documentation ... and then reference the datasource name in the DIH 
config.

Here's an eclipse wiki page about setting up a JNDI datasource.  I hope it's 
enough -- Jetty is not something I'm super familiar with:

https://wiki.eclipse.org/Jetty/Howto/Configure_JNDI_Datasource

I checked Solr's master codebase, and jndiName is still in the source code, so 
I think it should still work.

Thanks,
Shawn

Re: Solr 7.7.2 - SolrCloud - SPLITSHARD - Using LINK method fails on disk usage checks

[Andrzej Białecki]

Hi Andrew,

Please create a JIRA issue and attach this patch, I’ll look into fixing this. 
Thanks!


> On 18 Jun 2019, at 23:19, Andrew Kettmann  
> wrote:
> 
> Attached the patch, but that isn't sent out on the mailing list, my mistake. 
> Patch below:
> 
> 
> 
> ### START
> 
> diff --git 
> a/solr/core/src/java/org/apache/solr/cloud/api/collections/SplitShardCmd.java 
> b/solr/core/src/java/org/apache/solr/cloud/api/collections/SplitShardCmd.java
> index 24a52eaf97..e018f8a42f 100644
> --- 
> a/solr/core/src/java/org/apache/solr/cloud/api/collections/SplitShardCmd.java
> +++ 
> b/solr/core/src/java/org/apache/solr/cloud/api/collections/SplitShardCmd.java
> @@ -135,7 +135,9 @@ public class SplitShardCmd implements 
> OverseerCollectionMessageHandler.Cmd {
> }
> 
> RTimerTree t = timings.sub("checkDiskSpace");
> -checkDiskSpace(collectionName, slice.get(), parentShardLeader);
> +if (splitMethod != SolrIndexSplitter.SplitMethod.LINK) {
> +  checkDiskSpace(collectionName, slice.get(), parentShardLeader);
> +}
> t.stop();
> 
> // let's record the ephemeralOwner of the parent leader node
> 
> ### END
> 
> 
> From: Andrew Kettmann
> Sent: Tuesday, June 18, 2019 3:05:15 PM
> To: solr-user@lucene.apache.org
> Subject: Re: Solr 7.7.2 - SolrCloud - SPLITSHARD - Using LINK method fails on 
> disk usage checks
> 
> 
> Looks like the disk check here is the problem, I am no Java developer, but 
> this patch ignores the check if you are using the link method for splitting. 
> Attached the patch. This is off of the commit for 7.7.2, d4c30fc285 . The 
> modified version only has to be run on the overseer machine, so there is that 
> at least.
> 
> 
> From: Andrew Kettmann
> Sent: Tuesday, June 18, 2019 11:32:43 AM
> To: solr-user@lucene.apache.org
> Subject: Solr 7.7.2 - SolrCloud - SPLITSHARD - Using LINK method fails on 
> disk usage checks
> 
> 
> Using Solr 7.7.2 Docker image, testing some of the new autoscale features, 
> huge fan so far. Tested with the link method on a 2GB core and found that it 
> took less than 1MB of additional space. Filled the core quite a bit larger, 
> 12GB of a 20GB PVC, and now splitting the shard fails with the following 
> error message on my overseer:
> 
> 
> 2019-06-18 16:27:41.754 ERROR 
> (OverseerThreadFactory-49-thread-5-processing-n:10.0.192.74:8983_solr) 
> [c:test_autoscale s:shard1  ] o.a.s.c.a.c.OverseerCollectionMessageHandler 
> Collection: test_autoscale operation: splitshard 
> failed:org.apache.solr.common.SolrException: not enough free disk space to 
> perform index split on node 10.0.193.23:8983_solr, required: 
> 23.35038321465254, available: 7.811378479003906
>at 
> org.apache.solr.cloud.api.collections.SplitShardCmd.checkDiskSpace(SplitShardCmd.java:567)
>at 
> org.apache.solr.cloud.api.collections.SplitShardCmd.split(SplitShardCmd.java:138)
>at 
> org.apache.solr.cloud.api.collections.SplitShardCmd.call(SplitShardCmd.java:94)
>at 
> org.apache.solr.cloud.api.collections.OverseerCollectionMessageHandler.processMessage(OverseerCollectionMessageHandler.java:294)
>at 
> org.apache.solr.cloud.OverseerTaskProcessor$Runner.run(OverseerTaskProcessor.java:505)
>at 
> org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209)
>at 
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
>at 
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
>at java.base/java.lang.Thread.run(Thread.java:834)
> 
> 
> 
> I attempted sending the request to the node itself to see if it did anything 
> different, but no luck. My parameters are (Note Python formatting as that is 
> my language of choice):
> 
> 
> 
> splitparams = {'action':'SPLITSHARD',
>   'collection':'test_autoscale',
>   'shard':'shard1',
>   'splitMethod':'link',
>   'timing':'true',
>   'async':'shardsplitasync'}
> 
> 
> And this is confirmed by the log message from the node itself:
> 
> 
> 2019-06-18 16:27:41.730 INFO  (qtp1107530534-16) [c:test_autoscale   ] 
> o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections 
> params={async=shardsplitasync=true=SPLITSHARD=test_autoscale=shard1=link}
>  status=0 QTime=20
> 
> 
> While it is true I do not have enough space if I were using the rewrite 
> method, the link method on a 2GB core used an additional less than 1MB of 
> space. Is there something I am missing here? is there an option to disable 
> the disk space check that I need to pass? I can't find anything in the 
> documentation at this point.
> 
> 
> [https://storage.googleapis.com/e24-email-images/e24logonotag.png]
> Andrew Kettmann
> DevOps Engineer
> P: 1.314.596.2836
> [LinkedIn] [Twitter] 
>   

Re: Filtering children of parent doc from the input domain

[Mikhail Khludnev]

Hello, Srijan.
It's hard to grasp the problem without particular query and expected/actual
results.
On top of my head, it's possible to refine children by nesting query facet
instruction

under blockChildren one.

On Tue, Jun 18, 2019 at 11:51 PM Srijan  wrote:

> Hello,
>
> We're on Solr 6.2.1 and have a requirement where we need to facet on nested
> docs. So far we'd been using two pass query approach, where the first query
> searches within the parent domain and gets all the matching nested doc ids
> as facets (parent docs keep track of nested docs they contain) and then a
> second query at the nested doc level that includes the ids of all the
> matching nested docs as boolean clauses found by query 1. We then go back
> and correct the facet count for query 1. But with this approach we've been
> hitting the maxBooleanClauses limit regularly and with the amount of data
> we're seeing in this area, this approach is just not sustainable.
>
> The answer seems to be JSON faceting with blockChildren. The problem with
> this approach, at least in 6.2.1, is that filter clause is not supported
> within the facet domain. blockChildren matches all children of every parent
> doc matched in the query but in reality I only want a subset of the
> children - only the ones that match some criteria. Is there a way to filter
> out children without the need for the filter clause and without having to
> move to Solr 6.4?
>
> Reference:
> http://yonik.com/solr-nested-objects/#faceting
>
> Thanks,
> Srijan
>


-- 
Sincerely yours
Mikhail Khludnev