Implementing security.json is breaking ADDREPLICA
I have been able to reproduce this issue with minimal changes from an
out-of-the-box Zookeeper (3.4.6) and Solr (5.3.1): loading
configsets/basic_configs/conf into Zookeeper, creating the security.json listed
below, creating two nodes (one with a core named xmpl and one without any
core)- I can provide details if helpful.
The security.json is as follows:
{
"authentication":{
"class":"solr.BasicAuthPlugin",
"credentials":{
"solr":"IV0EHq1OnNrj6gvRCwvFwTrZ1+z1oBbnQdiVC3otuq0=
Ndd7LKvVBAaZIF0QAVi1ekCfAJXr1GGfLtRUXhgrF8c=",
"solruser":"VgZX1TAMNHT2IJikoGdKtxQdXc+MbNwfqzf89YqcLEE=
37pPWQ9v4gciIKHuTmFmN0Rv66rnlMOFEWfEy9qjJfY="},
"":{"v":9}},
"authorization":{
"class":"solr.RuleBasedAuthorizationPlugin",
"user-role":{
"solr":[
"admin",
"read",
"xmpladmin",
"xmplgen",
"xmplsel"],
"solruser":[
"read",
"xmplgen",
"xmplsel"]},
"permissions":[
{
"name":"security-edit",
"role":"admin"},
{
"name":"xmpl_admin",
"collection":"xmpl",
"path":"/admin/*",
"role":"xmpladmin"},
{
"name":"xmpl_sel",
"collection":"xmpl",
"path":"/select/*",
"role":null},
{
"name":"xmpl_gen",
"collection":"xmpl",
"path":"/*",
"role":"xmplgen"}],
"":{"v":42}}}
When I then execute admin/collections?action=ADDREPLICA, I get errors such as
the following in the solr.log of the node which was created without a core.
INFO - 2015-11-17 21:03:54.157; [c:xmpl s:shard1 r:core_node2
x:xmpl_shard1_replica1] org.apache.solr.cloud.RecoveryStrategy; Starting
Replication Recovery.
INFO - 2015-11-17 21:03:54.158; [c:xmpl s:shard1 r:core_node2
x:xmpl_shard1_replica1] org.apache.solr.cloud.RecoveryStrategy; Begin buffering
updates.
INFO - 2015-11-17 21:03:54.158; [c:xmpl s:shard1 r:core_node2
x:xmpl_shard1_replica1] org.apache.solr.update.UpdateLog; Starting to buffer
updates. FSUpdateLog{state=ACTIVE, tlog=null}
INFO - 2015-11-17 21:03:54.159; [c:xmpl s:shard1 r:core_node2
x:xmpl_shard1_replica1] org.apache.solr.cloud.RecoveryStrategy; Attempting to
replicate from http://{IP-address-redacted}:4565/solr/xmpl/.
ERROR - 2015-11-17 21:03:54.166; [c:xmpl s:shard1 r:core_node2
x:xmpl_shard1_replica1] org.apache.solr.common.SolrException; Error while
trying to
recover:org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException:
Error from server at http://{IP-address-redacted}:4565/solr/xmpl: Expected mime
type application/octet-stream but got text/html. <html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8"/>
<title>Error 401 Unauthorized request, Response code: 401</title>
</head>
<body><h2>HTTP ERROR 401</h2>
<p>Problem accessing /solr/xmpl/update. Reason:
<pre> Unauthorized request, Response code:
401</pre></p><hr><i><small>Powered by Jetty://</small></i><hr/>
</body>
</html>
at
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:528)
at
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:234)
at
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:226)
at
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:135)
at
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:152)
at
org.apache.solr.cloud.RecoveryStrategy.commitOnLeader(RecoveryStrategy.java:207)
at
org.apache.solr.cloud.RecoveryStrategy.replicate(RecoveryStrategy.java:147)
at
org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:437)
at org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:227)
INFO - 2015-11-17 21:03:54.166; [c:xmpl s:shard1 r:core_node2
x:xmpl_shard1_replica1] org.apache.solr.update.UpdateLog; Dropping buffered
updates FSUpdateLog{state=BUFFERING, tlog=null}
ERROR - 2015-11-17 21:03:54.166; [c:xmpl s:shard1 r:core_node2
x:xmpl_shard1_replica1] org.apache.solr.cloud.RecoveryStrategy; Recovery failed
- trying again... (2)
INFO - 2015-11-17 21:03:54.166; [c:xmpl s:shard1 r:core_node2
x:xmpl_shard1_replica1] org.apache.solr.cloud.RecoveryStrategy; Wait 8.0
seconds before trying to recover again (3)
And (after modifying Logging Levels), the solr.log of the node which already
had a core gets errors such as the following:
2015-11-17 21:03:50.743 DEBUG (qtp59559151-87) [ ] o.e.j.s.Server REQUEST GET
/solr/tpl/cloud.html on
HttpChannelOverHttp@37cf94f4{r=1,c=false,a=DISPATCHED,uri=/solr/tpl/cloud.html}
2015-11-17 21:03:50.744 DEBUG (qtp59559151-87) [ ] o.e.j.s.Server RESPONSE
/solr/tpl/cloud.html 200 handled=true
2015-11-17 21:03:50.802 DEBUG (qtp59559151-91) [ ] o.e.j.s.Server REQUEST GET
/solr/zookeeper on
HttpChannelOverHttp@37cf94f4{r=2,c=false,a=DISPATCHED,uri=/solr/zookeeper}
2015-11-17 21:03:50.803 INFO (qtp59559151-91) [ ] o.a.s.s.HttpSolrCall
userPrincipal: [null] type: [UNKNOWN], collections: [], Path: [/zookeeper]
2015-11-17 21:03:50.831 DEBUG (qtp59559151-91) [ ] o.e.j.s.Server RESPONSE
/solr/zookeeper 200 handled=true
2015-11-17 21:03:50.837 DEBUG (qtp59559151-87) [ ] o.e.j.s.Server REQUEST GET
/solr/zookeeper on
HttpChannelOverHttp@37cf94f4{r=3,c=false,a=DISPATCHED,uri=/solr/zookeeper}
2015-11-17 21:03:50.838 INFO (qtp59559151-87) [ ] o.a.s.s.HttpSolrCall
userPrincipal: [null] type: [UNKNOWN], collections: [], Path: [/zookeeper]
2015-11-17 21:03:50.841 DEBUG (qtp59559151-87) [ ] o.e.j.s.Server RESPONSE
/solr/zookeeper 200 handled=true
2015-11-17 21:03:50.857 DEBUG (qtp59559151-91) [ ] o.e.j.s.Server REQUEST GET
/solr/zookeeper on
HttpChannelOverHttp@37cf94f4{r=4,c=false,a=DISPATCHED,uri=/solr/zookeeper}
2015-11-17 21:03:50.858 INFO (qtp59559151-91) [ ] o.a.s.s.HttpSolrCall
userPrincipal: [null] type: [UNKNOWN], collections: [], Path: [/zookeeper]
2015-11-17 21:03:50.860 DEBUG (qtp59559151-91) [ ] o.e.j.s.Server RESPONSE
/solr/zookeeper 200 handled=true
2015-11-17 21:03:54.162 DEBUG (qtp59559151-87) [ ] o.e.j.s.Server REQUEST
POST /solr/xmpl/update on
HttpChannelOverHttp@1cf967f0{r=1,c=false,a=DISPATCHED,uri=/solr/xmpl/update}
2015-11-17 21:03:54.164 INFO (qtp59559151-87) [c:xmpl s:shard1 r:core_node1
x:xmpl] o.a.s.s.HttpSolrCall userPrincipal: [null] type: [WRITE], collections:
[xmpl,], Path: [/update]
2015-11-17 21:03:54.164 DEBUG (qtp59559151-87) [c:xmpl s:shard1 r:core_node1
x:xmpl] o.e.j.s.Server RESPONSE /solr/xmpl/update 401 handled=true
My impression from Anshum Gupta's 10/16/15 talk in Austin at the Solr
conference was that this was supposed to work. It does seem that one might be
able to add security to replication, but there does not seem to be a way to add
SolrCloud replication to this type of security.
Also, on a side note, I notice that http://hostname:port/solr/ does bring up
the GUI without prompting for a password: the Security team here would like us
to implement security.json in such a way that even the front page of the GUI
will require a password (although they will allow us to allow select access
without a password): I have not yet found a way via security.json to implement
that a password would be required in order to access the GUI front page.
Please advise.
