Re: [EXTERNAL] - Re: HTTP 401 when searching on alias in secured Solr

[Isabelle Giguere]

Hello again;

I have managed to reproduce the issue in a unit test.  I should probably add a 
Jira ticket with a patch for the unit test On Solr 8.5.0, not master.

Meanwhile, for your suggested queries:

  1.  Query on the collection:

curl -i -u admin:admin http://10.5.106.115:8985/solr/test1/select?q=*:*=xml
 HTTP/1.1 200 OK
Content-Security-Policy: default-src 'none'; base-uri 'none'; connect-src 
'self'; form-action 'self'; font-src 'self'; frame-ancestors 'none'; img-src 
'self'; media-src 'self'; style-src 'self' 'unsafe-inline'; script-src 'self'; 
worker-src 'self';
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-XSS-Protection: 1; mode=block
Content-Type: application/xml; charset=UTF-8
Content-Length: 8214





  true
  0
  2
  
*:*
  


Response contains the Solr document, of course


2. Query on the alias

curl -i -u admin:admin 
http://10.5.106.115:8985/solr/test/select?q=*:*=xml
 HTTP/1.1 401 Unauthorized
Content-Security-Policy: default-src 'none'; base-uri 'none'; connect-src 
'self'; form-action 'self'; font-src 'self'; frame-ancestors 'none'; img-src 
'self'; media-src 'self'; style-src 'self' 'unsafe-inline'; script-src 'self'; 
worker-src 'self';
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-XSS-Protection: 1; mode=block
Cache-Control: no-cache, no-store
Pragma: no-cache
Expires: Sat, 01 Jan 2000 01:00:00 GMT
Last-Modified: Fri, 12 Jun 2020 22:30:20 GMT
ETag: "172aaa7c1eb"
Content-Type: application/xml; charset=UTF-8
Content-Length: 1332





  true
  401
  16
  
*:*
  


Error contains the full html HTTP 401 message (with escaped characters, of 
course)
Gist of it : HTTP ERROR 401 require authentication

Thanks;


Isabelle Giguère
Computational Linguist & Java Developer
Linguiste informaticienne & développeur java



De : Jan Høydahl 
Envoyé : 12 juin 2020 17:30
À : solr-user@lucene.apache.org 
Objet : Re: [EXTERNAL] - Re: HTTP 401 when searching on alias in secured Solr

I’d say, try the query with curl and enable http headers

curl -i —user admin:admin http://localhost:8983/solr/mycollection/select?q=*:*
curl -i —user admin:admin http://localhost:8983/solr/myalias/select?q=*:*

Are you saying that you see a difference between the two? What are the headers?

Jan

> 12. jun. 2020 kl. 20:06 skrev Isabelle Giguere 
> :
>
> Hi Jan
>
> Thank you for your time on this.
>
> If I send a /select request directly on the alias (/solr/test/select), the 
> browser asks for credentials, but the Solr response returns status=401 and an 
> html error message with "HTTP ERROR 401 require authentication"
>
> Obviously, my expectation was that some query results would be returned.
>
> Since you can't reproduce the issue, I have to assume it's a configuration 
> issue.
>
> So, if I may, let me provide as much details as I can about my setup.
>
> Can anyone see something wrong here, some incompatibility ?
>
> Solr 8.5.0
>
> solrconfig.xml
> 7.1.0
> 
> 
> 
> 
>
>5
>5
>5
>
>
> schema.xml
> version=1.6
> Some warnings on start-up about Trie* fields and deprecated filters (we 
> should fix that)
>
> security.json in Zookeeper, at the Solr ZK root (provided on this thread)
> blockUnknown : (true|false) = no change in behavior for me, for this issue
> forwardCredentials : (true|false) = no change in behavior for me, for this 
> issue
>
> No SSL
>
> solr.in.sh
> SOLR_AUTH_TYPE="basic"
> SOLR_AUTHENTICATION_OPTS="-Dbasicauth=admin:admin"
>
> start command params:
> solr start -force -c -m 4g -h  -p  -z 
> :/
>
>
> Am I missing anything ?
>
> Thank you.
>
> 
>
> My investigation so far:
>
> I have set logging levels to TRACE for anything related to HTTP, HTTP2, 
> Authorization, Authentication...
>
> Judging by a comment in 
> org.apache.solr.core.CoreContainer.setupHttpClientForAuthPlugin(Object), I 
> should see some logging from PKIAuthenticationPlugin, no matter what plugin 
> is actually used, and regardless if forwardCredentials is true or false:
> Comment:
> // Always register PKI auth interceptor, which will then delegate the 
> decision of who should secure
> // each request to the configured authentication plugin.
>
> Expected log message from 
> org.apache.solr.security.PKIAuthenticationPlugin.setup(Http2SolrClient) 
> and/or from 
> org.apache.solr.security.PKIAuthenticationPlugin.HttpHeaderClientInterceptor.process(HttpRequest,
>  HttpContext)
>
> When running a request on an alias, I only see the expected log message from 
> /admin requests, never for /select requests.
>
> Of course, if my configuration is wrong, then my code and log analysis is 
> useless.
>
> **
>
>
> Isabelle Giguère
> Computational Linguist & Java Developer
> Linguiste informaticienne & développeur java
>
>
> 
> De : Jan Høydahl 
> Envoyé : 12 juin 2020 06:55
> À : solr-user@lucene.apache.org 
> 

Re: [EXTERNAL] - Re: HTTP 401 when searching on alias in secured Solr

[Jan Høydahl]

I’d say, try the query with curl and enable http headers

curl -i —user admin:admin http://localhost:8983/solr/mycollection/select?q=*:*
curl -i —user admin:admin http://localhost:8983/solr/myalias/select?q=*:*

Are you saying that you see a difference between the two? What are the headers?

Jan

> 12. jun. 2020 kl. 20:06 skrev Isabelle Giguere 
> :
> 
> Hi Jan
> 
> Thank you for your time on this.
> 
> If I send a /select request directly on the alias (/solr/test/select), the 
> browser asks for credentials, but the Solr response returns status=401 and an 
> html error message with "HTTP ERROR 401 require authentication"
> 
> Obviously, my expectation was that some query results would be returned.
> 
> Since you can't reproduce the issue, I have to assume it's a configuration 
> issue.
> 
> So, if I may, let me provide as much details as I can about my setup.
> 
> Can anyone see something wrong here, some incompatibility ?
> 
> Solr 8.5.0
> 
> solrconfig.xml
> 7.1.0
> 
> 
> 
> 
>
>5
>5
>5
>
> 
> schema.xml
> version=1.6
> Some warnings on start-up about Trie* fields and deprecated filters (we 
> should fix that)
> 
> security.json in Zookeeper, at the Solr ZK root (provided on this thread)
> blockUnknown : (true|false) = no change in behavior for me, for this issue
> forwardCredentials : (true|false) = no change in behavior for me, for this 
> issue
> 
> No SSL
> 
> solr.in.sh
> SOLR_AUTH_TYPE="basic"
> SOLR_AUTHENTICATION_OPTS="-Dbasicauth=admin:admin"
> 
> start command params:
> solr start -force -c -m 4g -h  -p  -z 
> :/
> 
> 
> Am I missing anything ?
> 
> Thank you.
> 
> 
> 
> My investigation so far:
> 
> I have set logging levels to TRACE for anything related to HTTP, HTTP2, 
> Authorization, Authentication...
> 
> Judging by a comment in 
> org.apache.solr.core.CoreContainer.setupHttpClientForAuthPlugin(Object), I 
> should see some logging from PKIAuthenticationPlugin, no matter what plugin 
> is actually used, and regardless if forwardCredentials is true or false:
> Comment:
> // Always register PKI auth interceptor, which will then delegate the 
> decision of who should secure
> // each request to the configured authentication plugin.
> 
> Expected log message from 
> org.apache.solr.security.PKIAuthenticationPlugin.setup(Http2SolrClient) 
> and/or from 
> org.apache.solr.security.PKIAuthenticationPlugin.HttpHeaderClientInterceptor.process(HttpRequest,
>  HttpContext)
> 
> When running a request on an alias, I only see the expected log message from 
> /admin requests, never for /select requests.
> 
> Of course, if my configuration is wrong, then my code and log analysis is 
> useless.
> 
> **
> 
> 
> Isabelle Giguère
> Computational Linguist & Java Developer
> Linguiste informaticienne & développeur java
> 
> 
> 
> De : Jan Høydahl 
> Envoyé : 12 juin 2020 06:55
> À : solr-user@lucene.apache.org 
> Objet : Re: [EXTERNAL] - Re: HTTP 401 when searching on alias in secured Solr
> 
> Hi
> 
> I tried to reproduce, but I can successfully search both the collection and 
> the alias. Both collection and alias promt for password, and when giving the 
> password the search succeeds.
> 
> What was your expectation?
> 
> Jan
> 
>> 11. jun. 2020 kl. 16:53 skrev Isabelle Giguere 
>> :
>> 
>> Some extra info:
>> Collections have 1 shard, 1 replica.  Only 1 Solr node running.
>> 
>> The HTTP 401 is not intermittent, as reported in SOLR-13421 and SOLR-13510.
>> 
>> Any request to the alias fails.
>> 
>> Thanks;
>> 
>> Isabelle Giguère
>> Computational Linguist & Java Developer
>> Linguiste informaticienne & développeur java
>> 
>> 
>> 
>> De : Isabelle Giguere 
>> Envoyé : 10 juin 2020 16:11
>> À : solr-user@lucene.apache.org 
>> Objet : Re: [EXTERNAL] - Re: HTTP 401 when searching on alias in secured Solr
>> 
>> Hi Jan;
>> 
>> Thank you for your reply.
>> 
>> This is security.json as seen in Zookeeper.  Credentials are admin / admin
>> 
>> {
>> "authentication":{
>>   "blockUnknown":false,
>>   "realm":"MTM Solr",
>>   "forwardCredentials":true,
>>   "class":"solr.BasicAuthPlugin",
>>   "credentials":{"admin":"0rTOgObKYwzSyPoYuj2su2/90eQCfysF1aasxTx+wrc= 
>> +tCMmpawYYtTsp3JfkG9avb8bKZlm/IGTZirsufYvns="},
>>   "":{"v":2}},
>> "authorization":{
>>   "class":"solr.RuleBasedAuthorizationPlugin",
>>   "permissions":[{
>>   "name":"all",
>>   "role":"admin"}],
>>   "user-role":{"admin":"admin"},
>>   "":{"v":8}}}
>> 
>> Thanks for feedback
>> 
>> Isabelle Giguère
>> Computational Linguist & Java Developer
>> Linguiste informaticienne & développeur java
>> 
>> 
>> 
>> De : Jan Høydahl 
>> Envoyé : 10 juin 2020 16:01
>> À : solr-user@lucene.apache.org 
>> Objet : [EXTERNAL] - Re: HTTP 401 when searching on alias in secured Solr
>> 
>> Please share your security.json file
>> 
>> Jan Høydahl
>> 
>>> 10. jun. 2020 kl. 21:53 skrev Isabelle 

eDismax query syntax question

[Webster Homer]

Recently we found strange behavior in a query. We use eDismax as the query 
parser.

This is the query term:
1,3-DIMETHYL-5-(3-PHENYL-ALLYLIDENE)-PYRIMIDINE-2,4,6-TRIONE

It should hit one document in our index. It does not. However, if you use the 
Dismax query parser it does match the record.

The problem seems to involve the parenthesis and the dashes. If you escape the 
dash after the parenthesis it matches
1,3-DIMETHYL-5-(3-PHENYL-ALLYLIDENE)\-PYRIMIDINE-2,4,6-TRIONE

I thought that eDismax and Dismax escaped all lucene special characters before 
passing the query to lucene. Although I also remember reading that + and - can 
have special significance in a query if preceded with white space. I can find 
very little documentation on either query parser in how they work.

Is this expected behavior or is this a bug? If expected, where can I find 
documentation?



This message and any attachment are confidential and may be privileged or 
otherwise protected from disclosure. If you are not the intended recipient, you 
must not copy this message or attachment or disclose the contents to any other 
person. If you have received this transmission in error, please notify the 
sender immediately and delete the message and any attachment from your system. 
Merck KGaA, Darmstadt, Germany and any of its subsidiaries do not accept 
liability for any omissions or errors in this message which may arise as a 
result of E-Mail-transmission or for damages resulting from any unauthorized 
changes of the content of this message and any attachment thereto. Merck KGaA, 
Darmstadt, Germany and any of its subsidiaries do not guarantee that this 
message is free of viruses and does not accept liability for any damages caused 
by any virus transmitted therewith.



Click http://www.merckgroup.com/disclaimer to access the German, French, 
Spanish and Portuguese versions of this disclaimer.

Re: [EXTERNAL] - Re: HTTP 401 when searching on alias in secured Solr

[Isabelle Giguere]

Hi Jan

Thank you for your time on this.

If I send a /select request directly on the alias (/solr/test/select), the 
browser asks for credentials, but the Solr response returns status=401 and an 
html error message with "HTTP ERROR 401 require authentication"

Obviously, my expectation was that some query results would be returned.

Since you can't reproduce the issue, I have to assume it's a configuration 
issue.

So, if I may, let me provide as much details as I can about my setup.

Can anyone see something wrong here, some incompatibility ?

Solr 8.5.0

solrconfig.xml
7.1.0





5
5
5


schema.xml
version=1.6
Some warnings on start-up about Trie* fields and deprecated filters (we should 
fix that)

security.json in Zookeeper, at the Solr ZK root (provided on this thread)
blockUnknown : (true|false) = no change in behavior for me, for this issue
forwardCredentials : (true|false) = no change in behavior for me, for this issue

No SSL

solr.in.sh
SOLR_AUTH_TYPE="basic"
SOLR_AUTHENTICATION_OPTS="-Dbasicauth=admin:admin"

start command params:
solr start -force -c -m 4g -h  -p  -z 
:/


Am I missing anything ?

Thank you.



My investigation so far:

I have set logging levels to TRACE for anything related to HTTP, HTTP2, 
Authorization, Authentication...

Judging by a comment in 
org.apache.solr.core.CoreContainer.setupHttpClientForAuthPlugin(Object), I 
should see some logging from PKIAuthenticationPlugin, no matter what plugin is 
actually used, and regardless if forwardCredentials is true or false:
Comment:
// Always register PKI auth interceptor, which will then delegate the decision 
of who should secure
// each request to the configured authentication plugin.

Expected log message from 
org.apache.solr.security.PKIAuthenticationPlugin.setup(Http2SolrClient) and/or 
from 
org.apache.solr.security.PKIAuthenticationPlugin.HttpHeaderClientInterceptor.process(HttpRequest,
 HttpContext)

When running a request on an alias, I only see the expected log message from 
/admin requests, never for /select requests.

Of course, if my configuration is wrong, then my code and log analysis is 
useless.

**


Isabelle Giguère
Computational Linguist & Java Developer
Linguiste informaticienne & développeur java



De : Jan Høydahl 
Envoyé : 12 juin 2020 06:55
À : solr-user@lucene.apache.org 
Objet : Re: [EXTERNAL] - Re: HTTP 401 when searching on alias in secured Solr

Hi

I tried to reproduce, but I can successfully search both the collection and the 
alias. Both collection and alias promt for password, and when giving the 
password the search succeeds.

What was your expectation?

Jan

> 11. jun. 2020 kl. 16:53 skrev Isabelle Giguere 
> :
>
> Some extra info:
> Collections have 1 shard, 1 replica.  Only 1 Solr node running.
>
> The HTTP 401 is not intermittent, as reported in SOLR-13421 and SOLR-13510.
>
> Any request to the alias fails.
>
> Thanks;
>
> Isabelle Giguère
> Computational Linguist & Java Developer
> Linguiste informaticienne & développeur java
>
>
> 
> De : Isabelle Giguere 
> Envoyé : 10 juin 2020 16:11
> À : solr-user@lucene.apache.org 
> Objet : Re: [EXTERNAL] - Re: HTTP 401 when searching on alias in secured Solr
>
> Hi Jan;
>
> Thank you for your reply.
>
> This is security.json as seen in Zookeeper.  Credentials are admin / admin
>
> {
>  "authentication":{
>"blockUnknown":false,
>"realm":"MTM Solr",
>"forwardCredentials":true,
>"class":"solr.BasicAuthPlugin",
>"credentials":{"admin":"0rTOgObKYwzSyPoYuj2su2/90eQCfysF1aasxTx+wrc= 
> +tCMmpawYYtTsp3JfkG9avb8bKZlm/IGTZirsufYvns="},
>"":{"v":2}},
>  "authorization":{
>"class":"solr.RuleBasedAuthorizationPlugin",
>"permissions":[{
>"name":"all",
>"role":"admin"}],
>"user-role":{"admin":"admin"},
>"":{"v":8}}}
>
> Thanks for feedback
>
> Isabelle Giguère
> Computational Linguist & Java Developer
> Linguiste informaticienne & développeur java
>
>
> 
> De : Jan Høydahl 
> Envoyé : 10 juin 2020 16:01
> À : solr-user@lucene.apache.org 
> Objet : [EXTERNAL] - Re: HTTP 401 when searching on alias in secured Solr
>
> Please share your security.json file
>
> Jan Høydahl
>
>> 10. jun. 2020 kl. 21:53 skrev Isabelle Giguere 
>> :
>>
>> Hi;
>>
>> I'm using Solr 8.5.0.  I have uploaded security.json to Zookeeper.  I can 
>> log in the Solr Admin UI.  I can create collections and aliases, and I can 
>> index documents in Solr.
>>
>> Collections : test1, test2
>> Alias: test (combines test1, test2)
>>
>> Indexed document "solr-word.pdf" in collection test1
>>
>> Searching on a collection works:
>> http://localhost:8983/solr/test1/select?q=*:*=xml
>> 
>>
>> But searching on an alias results in HTTP 401
>> http://localhost:8983/solr/test/select?q=*:*=xml
>>
>> Error from server at null: Expected mime type application/octet-stream but 
>> got 

Re: Facet Performance

[Erick Erickson]

I question whether fiterCache has anything to do with it, I suspect what’s 
really happening is that first time you’re reading the relevant bits from disk 
into memory. And to double check you should have docVaues enabled for all these 
fields. The “uninverting” process  can be very expensive, and docValues 
bypasses that.

As of Solr 7.6, you can define “uninvertible=true” to your field(Type) to “fail 
fast” if Solr needs to uninvert the field.

But that’s an aside. In either case, my claim is that first-time execution does 
“something”, either reads the serialized docValues from disk or uninverts the 
file on Solr’s heap.

You can have this autowarmed by any combination of
1> specifying an autowarm count on your queryResultCache. That’s hit or miss, 
as it replays the most recent N queries which may or may not contain the sorts. 
That said, specifying 10-20 for autowarm count is usually a good idea, assuming 
you’re not committing more than, say, every 30 seconds. I’d add the same to 
filterCache too.

2> specifying a newSearcher or firstSearcher query in solrconfig.xml. The 
difference is that newSearcher is fired every time a commit happens, while 
firstSearcher is only fired when Solr starts, the theory being that there’s no 
cache autowarming available when Solr fist powers up. Usually, people don’t 
bother with firstSearcher or just make it the same as newSearcher. Note that a 
query doesn’t have to be “real” at all. You can just add all the facet fields 
to a *:* query in a single go.

BTW, Trie fields will stay around for a long time even though deprecated. Or at 
least until we find something to replace them with that doesn’t have this 
penalty, so I’d feel pretty safe using those and they’ll be more efficient than 
strings.

Best,
Erick

> On Jun 12, 2020, at 12:39 PM, James Bodkin  
> wrote:
> 
> We've run the performance test after changing the fields to be of the type 
> string. We're seeing improved performance, especially after the first time 
> the query has run. The first run is taking around 1-2 seconds rather than 6-8 
> seconds and when the filter cache is present, the response time is around 
> 400ms.
> Do you have any more suggestions that we could try in order to optimise the 
> performance?
> 
> On 11/06/2020, 14:49, "Erick Erickson"  wrote:
> 
>There’s a lot of confusion about using points-based fields for faceting, 
> see: https://issues.apache.org/jira/browse/SOLR-13227 for instance.
> 
>Two options you might try:
>1> copyField to a string field and facet on that (won’t work, of course, 
> for any kind of interval/range facet)
>2> use the deprecated Trie field instead. You could use the copyField to a 
> Trie field for this too.
> 
>Best,
>Erick
> 

RE: Timeout issue while doing update operations from clients (using SolrJ)

[Kommu, Vinodh K.]

I feel the mentioned issue is more or less relavant to following JIRA. Any idea 
on this?

https://issues.apache.org/jira/browse/SOLR-13458

Thanks & Regards,
Vinodh

From: Kommu, Vinodh K.
Sent: Wednesday, June 10, 2020 10:43 PM
To: solr-user@lucene.apache.org
Subject: RE: Timeout issue while doing update operations from clients (using 
SolrJ)

We are getting following socket timeout exception during this error. Any idea 
on this?

ERROR (updateExecutor-3-thread-1392-processing-n:hostname:1100_solr 
x:TestCollection_shard6_replica_n10 c:TestCollection s:shard6 r:core_node13) 
[c:TestCollection s:shard6 r:core_node13 x:TestCollection_shard6_replica_n10] 
o.a.s.u.SolrCmdDistributor org.apache.solr.client.solrj.SolrServerException: 
Timeout occured while waiting response from server at: 
https://hostname:1100/solr/TestCollection_shard6_replica_n34
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:654)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
at 
org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClient.request(ConcurrentUpdateSolrClient.java:491)
at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1260)
at 
org.apache.solr.update.SolrCmdDistributor.doRequest(SolrCmdDistributor.java:326)
at 
org.apache.solr.update.SolrCmdDistributor.lambda$submit$0(SolrCmdDistributor.java:315)
at 
org.apache.solr.update.SolrCmdDistributor.dt_access$675(SolrCmdDistributor.java)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at 
com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:176)
at 
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209)
at 
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.dt_access$303(ExecutorUtil.java)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
at java.net.SocketInputStream.read(SocketInputStream.java:171)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at sun.security.ssl.InputRecord.readFully(InputRecord.java:465)
at sun.security.ssl.InputRecord.read(InputRecord.java:503)
at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:975)
at sun.security.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:933)
at sun.security.ssl.AppInputStream.read(AppInputStream.java:105)
at 
org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:137)
at 
org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:153)
at 
org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:282)
at 
org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138)
at 
org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56)
at 
org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:259)
at 
org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:163)
at 
org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165)
at 
org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:273)
at 
org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125)
at 
org.apache.solr.util.stats.InstrumentedHttpRequestExecutor.execute(InstrumentedHttpRequestExecutor.java:120)
at 
org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272)
at 
org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185)
at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
at 
org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
at 
org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
at 
org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
at 
org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
at 

Re: Facet Performance

[James Bodkin]

We've run the performance test after changing the fields to be of the type 
string. We're seeing improved performance, especially after the first time the 
query has run. The first run is taking around 1-2 seconds rather than 6-8 
seconds and when the filter cache is present, the response time is around 400ms.
Do you have any more suggestions that we could try in order to optimise the 
performance?

On 11/06/2020, 14:49, "Erick Erickson"  wrote:

There’s a lot of confusion about using points-based fields for faceting, 
see: https://issues.apache.org/jira/browse/SOLR-13227 for instance.

Two options you might try:
1> copyField to a string field and facet on that (won’t work, of course, 
for any kind of interval/range facet)
2> use the deprecated Trie field instead. You could use the copyField to a 
Trie field for this too.

Best,
Erick

Solr cloud backup/restore not working

[yaswanth kumar]

Using solr 8.2.0 and setup a cloud with 2 nodes. (2 replica's for each
collection)
Enabled basic authentication and gave all access to the admin user

Now trying to use solr cloud backup/restore API, backup is working great,
but when trying to invoke restore API its throwing the below error

{
  "responseHeader":{
"status":500,
"QTime":349},
  "Operation restore caused
exception:":"org.apache.solr.common.SolrException:org.apache.solr.common.SolrException:
ADDREPLICA failed to create replica",
  "exception":{
"msg":"ADDREPLICA failed to create replica",
"rspCode":500},
  "error":{
"metadata":[
  "error-class","org.apache.solr.common.SolrException",
  "root-error-class","org.apache.solr.common.SolrException"],
"msg":"ADDREPLICA failed to create replica",
"trace":"org.apache.solr.common.SolrException: ADDREPLICA failed to
create replica\n\tat
org.apache.solr.client.solrj.SolrResponse.getException(SolrResponse.java:53)\n\tat
org.apache.solr.handler.admin.CollectionsHandler.invokeAction(CollectionsHandler.java:280)\n\tat
org.apache.solr.handler.admin.CollectionsHandler.handleRequestBody(CollectionsHandler.java:252)\n\tat
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:199)\n\tat
org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:820)\n\tat
org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:786)\n\tat
org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:546)\n\tat
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:423)\n\tat
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:350)\n\tat
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1602)\n\tat
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:540)\n\tat
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146)\n\tat
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)\n\tat
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)\n\tat
org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257)\n\tat
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1711)\n\tat
org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)\n\tat
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1347)\n\tat
org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)\n\tat
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:480)\n\tat
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1678)\n\tat
org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)\n\tat
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1249)\n\tat
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)\n\tat
org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:220)\n\tat
org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:152)\n\tat
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)\n\tat
org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:335)\n\tat
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)\n\tat
org.eclipse.jetty.server.Server.handle(Server.java:505)\n\tat
org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:370)\n\tat
org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:267)\n\tat
org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:305)\n\tat
org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)\n\tat
org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:427)\n\tat
org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:321)\n\tat
org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:159)\n\tat
org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)\n\tat
org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:117)\n\tat
org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333)\n\tat
org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310)\n\tat
org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)\n\tat
org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)\n\tat
org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)\n\tat
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:781)\n\tat
org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:917)\n\tat
java.base/java.lang.Thread.run(Thread.java:834)\n",
"code":500}}



Can someone please help me if I am missing 

Solr will become an Apache TLP

[Jan Høydahl]

Hi,

Solr has been a sub-project of the Lucene TLP, but wil separate releases, JIRA, 
user-list etc.
The committers have decided to establish Solr as a separate Apache Top Level 
Project (TLP).
Please see the VOTE thread on the DEV mailing list: 
https://s.apache.org/solr-tlp-vote 
Users will not notice much change, other than new website URL, mailing lists 
and download locations.

The PMC will formally announce the change once it starts materializing, but it 
could take several weeks.
Then we’ll also share more detailed information on what will change.

Jan

Re: [EXTERNAL] - Re: HTTP 401 when searching on alias in secured Solr

[Jan Høydahl]

Hi

I tried to reproduce, but I can successfully search both the collection and the 
alias. Both collection and alias promt for password, and when giving the 
password the search succeeds.

What was your expectation?

Jan

> 11. jun. 2020 kl. 16:53 skrev Isabelle Giguere 
> :
> 
> Some extra info:
> Collections have 1 shard, 1 replica.  Only 1 Solr node running.
> 
> The HTTP 401 is not intermittent, as reported in SOLR-13421 and SOLR-13510.
> 
> Any request to the alias fails.
> 
> Thanks;
> 
> Isabelle Giguère
> Computational Linguist & Java Developer
> Linguiste informaticienne & développeur java
> 
> 
> 
> De : Isabelle Giguere 
> Envoyé : 10 juin 2020 16:11
> À : solr-user@lucene.apache.org 
> Objet : Re: [EXTERNAL] - Re: HTTP 401 when searching on alias in secured Solr
> 
> Hi Jan;
> 
> Thank you for your reply.
> 
> This is security.json as seen in Zookeeper.  Credentials are admin / admin
> 
> {
>  "authentication":{
>"blockUnknown":false,
>"realm":"MTM Solr",
>"forwardCredentials":true,
>"class":"solr.BasicAuthPlugin",
>"credentials":{"admin":"0rTOgObKYwzSyPoYuj2su2/90eQCfysF1aasxTx+wrc= 
> +tCMmpawYYtTsp3JfkG9avb8bKZlm/IGTZirsufYvns="},
>"":{"v":2}},
>  "authorization":{
>"class":"solr.RuleBasedAuthorizationPlugin",
>"permissions":[{
>"name":"all",
>"role":"admin"}],
>"user-role":{"admin":"admin"},
>"":{"v":8}}}
> 
> Thanks for feedback
> 
> Isabelle Giguère
> Computational Linguist & Java Developer
> Linguiste informaticienne & développeur java
> 
> 
> 
> De : Jan Høydahl 
> Envoyé : 10 juin 2020 16:01
> À : solr-user@lucene.apache.org 
> Objet : [EXTERNAL] - Re: HTTP 401 when searching on alias in secured Solr
> 
> Please share your security.json file
> 
> Jan Høydahl
> 
>> 10. jun. 2020 kl. 21:53 skrev Isabelle Giguere 
>> :
>> 
>> Hi;
>> 
>> I'm using Solr 8.5.0.  I have uploaded security.json to Zookeeper.  I can 
>> log in the Solr Admin UI.  I can create collections and aliases, and I can 
>> index documents in Solr.
>> 
>> Collections : test1, test2
>> Alias: test (combines test1, test2)
>> 
>> Indexed document "solr-word.pdf" in collection test1
>> 
>> Searching on a collection works:
>> http://localhost:8983/solr/test1/select?q=*:*=xml
>> 
>> 
>> But searching on an alias results in HTTP 401
>> http://localhost:8983/solr/test/select?q=*:*=xml
>> 
>> Error from server at null: Expected mime type application/octet-stream but 
>> got text/html.   > content="text/html;charset=utf-8"/> Error 401 Authentication failed, 
>> Response code: 401  HTTP ERROR 401 Authentication 
>> failed, Response code: 401  
>> URI:/solr/test1_shard1_replica_n1/select 
>> STATUS:401 
>> MESSAGE:Authentication failed, Response code: 401 
>> SERVLET:default   
>> 
>> Even if 
>> https://urldefense.com/v3/__https://issues.apache.org/jira/browse/SOLR-13510__;!!Obbck6kTJA!P6ugA-rw1I80PaH0U_GVasNqn8EXwmVQ33lwcPOU-cvNgTJK6-3zAf8ukzvv3ynJ$
>>   is fixed in Solr 8.5.0, I did try to start Solr with -Dsolr.http1=true, 
>> and I set "forwardCredentials":true in security.json.
>> 
>> Nothing works.  I just cannot use aliases when Solr is secured.
>> 
>> Can anyone confirm if this may be a configuration issue, or if this could 
>> possibly be a bug ?
>> 
>> Thank you;
>> 
>> Isabelle Giguère
>> Computational Linguist & Java Developer
>> Linguiste informaticienne & développeur java
>> 
>>