RE: Re:Re: Implementing security.json is breaking ADDREPLICA
Anshum and Nobel, I've downloaded 5.4, and this seems to be working so far Thanks again -Original Message- From: Anshum Gupta [mailto:ans...@anshumgupta.net] Sent: Tuesday, December 01, 2015 12:52 AM To: solr-user@lucene.apache.org Subject: Re: Re:Re: Implementing security.json is breaking ADDREPLICA Hi Craig, As part of my manual testing for the 5.3 RC, I tried out collection admin request restriction and update restriction on a single node setup. I don't have the manual test steps documented but it wasn't too intensive I'd admit. I think the complications involved in stopping specific nodes and bringing them back up stop us from testing the node restarts as part of the automated tests but we should find a way and fix that. I've just found another issue and opened SOLR-8355 for the same and it involves the "update" permission. As far as patching 5.3.1 go, it's involves more than just this one patch and this patch alone wouldn't help you resolve this issue. You'd certainly need the patch from SOLR-8167. Also, make sure you actually use the 'commit' and not the posted patch as the patch on SOLR-8167 is different from the commit. I don't think you'd need anything other than those patches and whatever comes from 8355 to have a patched 5.3.1. Any help in testing this out would be awesome and thanks for reporting and following up on the issues! On Tue, Dec 1, 2015 at 6:09 AM, Oakley, Craig (NIH/NLM/NCBI) [C] < craig.oak...@nih.gov> wrote: > Thank you, Anshum and Nobel, for your progress on SOLR-8326 > > I have a couple questions to tide me over until 5.4 (hoping to test > security.json a bit further while I wait). > > Given that the seven steps (tar xvzf solr-5.3.1.tgz; tar xvzf > zookeeper-3.4.6.tar.gz; zkServer.sh start zoo_sample.cfg; zkcli.sh -zkhost > localhost:2181 -cmd putfile /security.json ~/security.json; solr start -e > cloud -z localhost:2181; solr stop -p 7574 & solr start -c -p 7574 -s > "example/cloud/node2/solr" -z localhost:2181) demonstrate the problem, are > there a similar set of steps by which one can load _some_ minimal > security.json and still be able to stop & successfully restart one node of > the cluster? (I am wondering what steps were used in the original testing > of 5.3.1) > > Also, has it been verified that the SOLR-8326 patch resolves the > ADDREPLICA bug in addition to the > shutdown-&-restart-one-node-while-keeping-another-node-running bug? > > Also, would it make sense for me to download solr-5.3.1-src.tgz and (in a > test environment) make the changes described in the latest attachment to > SOLR-8326? Or would it be more advisable just to wait for 5.4? I don't know > what may be involved in compiling a new solr.war from the source code. > > Thanks again > > -Original Message- > From: Erick Erickson [mailto:erickerick...@gmail.com] > Sent: Tuesday, November 24, 2015 1:25 PM > To: solr-user <solr-user@lucene.apache.org> > Subject: Re: Re:Re: Implementing security.json is breaking ADDREPLICA > > bq: I don't suppose there is an ETA for 5.4? > > Actually, 5.4 is probably in the works within the next month. I'm not > the one cutting the > release, but there's some rumors that a label will be cut this week, > then the "usual" > process is a week or two (sometimes more if bugs are flushed out) before > the > official release. > > Call it the first of the year for safety's sake, but that's a guess. > > Best, > Erick > > On Tue, Nov 24, 2015 at 10:22 AM, Oakley, Craig (NIH/NLM/NCBI) [C] > <craig.oak...@nih.gov> wrote: > > Thanks for the reply, > > > > I don't suppose there is an ETA for 5.4? > > > > > > Thanks again > > > > -Original Message- > ... > -- Anshum Gupta
RE: Re:Re: Implementing security.json is breaking ADDREPLICA
Thank you, Anshum and Nobel, for your progress on SOLR-8326 I have a couple questions to tide me over until 5.4 (hoping to test security.json a bit further while I wait). Given that the seven steps (tar xvzf solr-5.3.1.tgz; tar xvzf zookeeper-3.4.6.tar.gz; zkServer.sh start zoo_sample.cfg; zkcli.sh -zkhost localhost:2181 -cmd putfile /security.json ~/security.json; solr start -e cloud -z localhost:2181; solr stop -p 7574 & solr start -c -p 7574 -s "example/cloud/node2/solr" -z localhost:2181) demonstrate the problem, are there a similar set of steps by which one can load _some_ minimal security.json and still be able to stop & successfully restart one node of the cluster? (I am wondering what steps were used in the original testing of 5.3.1) Also, has it been verified that the SOLR-8326 patch resolves the ADDREPLICA bug in addition to the shutdown-&-restart-one-node-while-keeping-another-node-running bug? Also, would it make sense for me to download solr-5.3.1-src.tgz and (in a test environment) make the changes described in the latest attachment to SOLR-8326? Or would it be more advisable just to wait for 5.4? I don't know what may be involved in compiling a new solr.war from the source code. Thanks again -Original Message- From: Erick Erickson [mailto:erickerick...@gmail.com] Sent: Tuesday, November 24, 2015 1:25 PM To: solr-user <solr-user@lucene.apache.org> Subject: Re: Re:Re: Implementing security.json is breaking ADDREPLICA bq: I don't suppose there is an ETA for 5.4? Actually, 5.4 is probably in the works within the next month. I'm not the one cutting the release, but there's some rumors that a label will be cut this week, then the "usual" process is a week or two (sometimes more if bugs are flushed out) before the official release. Call it the first of the year for safety's sake, but that's a guess. Best, Erick On Tue, Nov 24, 2015 at 10:22 AM, Oakley, Craig (NIH/NLM/NCBI) [C] <craig.oak...@nih.gov> wrote: > Thanks for the reply, > > I don't suppose there is an ETA for 5.4? > > > Thanks again > > -Original Message- ...
Re: Re:Re: Implementing security.json is breaking ADDREPLICA
Hi Craig, As part of my manual testing for the 5.3 RC, I tried out collection admin request restriction and update restriction on a single node setup. I don't have the manual test steps documented but it wasn't too intensive I'd admit. I think the complications involved in stopping specific nodes and bringing them back up stop us from testing the node restarts as part of the automated tests but we should find a way and fix that. I've just found another issue and opened SOLR-8355 for the same and it involves the "update" permission. As far as patching 5.3.1 go, it's involves more than just this one patch and this patch alone wouldn't help you resolve this issue. You'd certainly need the patch from SOLR-8167. Also, make sure you actually use the 'commit' and not the posted patch as the patch on SOLR-8167 is different from the commit. I don't think you'd need anything other than those patches and whatever comes from 8355 to have a patched 5.3.1. Any help in testing this out would be awesome and thanks for reporting and following up on the issues! On Tue, Dec 1, 2015 at 6:09 AM, Oakley, Craig (NIH/NLM/NCBI) [C] < craig.oak...@nih.gov> wrote: > Thank you, Anshum and Nobel, for your progress on SOLR-8326 > > I have a couple questions to tide me over until 5.4 (hoping to test > security.json a bit further while I wait). > > Given that the seven steps (tar xvzf solr-5.3.1.tgz; tar xvzf > zookeeper-3.4.6.tar.gz; zkServer.sh start zoo_sample.cfg; zkcli.sh -zkhost > localhost:2181 -cmd putfile /security.json ~/security.json; solr start -e > cloud -z localhost:2181; solr stop -p 7574 & solr start -c -p 7574 -s > "example/cloud/node2/solr" -z localhost:2181) demonstrate the problem, are > there a similar set of steps by which one can load _some_ minimal > security.json and still be able to stop & successfully restart one node of > the cluster? (I am wondering what steps were used in the original testing > of 5.3.1) > > Also, has it been verified that the SOLR-8326 patch resolves the > ADDREPLICA bug in addition to the > shutdown-&-restart-one-node-while-keeping-another-node-running bug? > > Also, would it make sense for me to download solr-5.3.1-src.tgz and (in a > test environment) make the changes described in the latest attachment to > SOLR-8326? Or would it be more advisable just to wait for 5.4? I don't know > what may be involved in compiling a new solr.war from the source code. > > Thanks again > > -Original Message- > From: Erick Erickson [mailto:erickerick...@gmail.com] > Sent: Tuesday, November 24, 2015 1:25 PM > To: solr-user <solr-user@lucene.apache.org> > Subject: Re: Re:Re: Implementing security.json is breaking ADDREPLICA > > bq: I don't suppose there is an ETA for 5.4? > > Actually, 5.4 is probably in the works within the next month. I'm not > the one cutting the > release, but there's some rumors that a label will be cut this week, > then the "usual" > process is a week or two (sometimes more if bugs are flushed out) before > the > official release. > > Call it the first of the year for safety's sake, but that's a guess. > > Best, > Erick > > On Tue, Nov 24, 2015 at 10:22 AM, Oakley, Craig (NIH/NLM/NCBI) [C] > <craig.oak...@nih.gov> wrote: > > Thanks for the reply, > > > > I don't suppose there is an ETA for 5.4? > > > > > > Thanks again > > > > -Original Message- > ... > -- Anshum Gupta
Re: Re:Re: Implementing security.json is breaking ADDREPLICA
Yes, it certainly is a PKI issue.
On Tue, Nov 24, 2015 at 7:59 AM, Oakley, Craig (NIH/NLM/NCBI) [C] <
craig.oak...@nih.gov> wrote:
> Thank you for the reply
>
> Trying those exact commands, I'm still getting the same issue
> tar xvzf /net/sybdev11/export/home/sybase/Distr/Solr/solr-5.3.1.tgz
> tar xvzf /net/sybdev11/export/home/sybase/Distr/Solr/zookeeper-3.4.6.tar.gz
> cd zookeeper-3.4.6/
> bin/zkServer.sh start zoo_sample.cfg
> cd ..
> solr-5.3.1/server/scripts/cloud-scripts/zkcli.sh -zkhost localhost:2181
> -cmd putfile /security.json
> PREVsolr-5.3.1/server/scripts/cloud-scripts/security.json
> solr-5.3.1/server/scripts/cloud-scripts/zkcli.sh -zkhost localhost:2181
> -cmd list
> solr-5.3.1/bin/solr start -e cloud -z localhost:2181
> cd solr-5.3.1/
> bin/solr stop -p 7574
> bin/solr start -c -p 7574 -s "example/cloud/node2/solr" -z localhost:2181
> tail -f example/cloud/node2/logs/solr.log
>
> The -cmd list shows
> / (2)
> DATA:
>
> /zookeeper (1)
> DATA:
>
> /security.json (0)
> DATA:
>
>
> {"authentication":{"class":"solr.BasicAuthPlugin","credentials":{"solr":"IV0EHq1OnNrj6gvRCwvFwTrZ1+z1oBbnQdiVC3otuq0=
> Ndd7LKvVBAaZIF0QAVi1ekCfAJXr1GGfLtRUXhgrF8c="}},"authorization":{"class":"solr.RuleBasedAuthorizationPlugin","user-role":
> {"solr":["admin"]}
>
> ,"permissions":[
> {"name":"security-edit","role":"admin"}
>
> ]}}
>
>
> While the output of tail contains
> ERROR - 2015-11-24 10:45:54.796; [c:gettingstarted s:shard1 r:core_node4
> x:gettingstarted_shard1_replica1] org.apache.solr.common.SolrException;
> Error while trying to recover.:java.util.concurrent.ExecutionException:
> org.apache.http.ParseException: Invalid content type:
> at java.util.concurrent.FutureTask.report(FutureTask.java:122)
> at java.util.concurrent.FutureTask.get(FutureTask.java:192)
> at
> org.apache.solr.cloud.RecoveryStrategy.sendPrepRecoveryCmd(RecoveryStrategy.java:598)
> at
> org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:361)
> at
> org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:227)
> Caused by: org.apache.http.ParseException: Invalid content type:
> at org.apache.http.entity.ContentType.parse(ContentType.java:273)
> at
> org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:512)
> at
> org.apache.solr.client.solrj.impl.HttpSolrClient$1.call(HttpSolrClient.java:270)
> at
> org.apache.solr.client.solrj.impl.HttpSolrClient$1.call(HttpSolrClient.java:266)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:210)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
>
>
> -Original Message-
> From: Anshum Gupta [mailto:ans...@anshumgupta.net]
> Sent: Monday, November 23, 2015 7:24 PM
> To: solr-user@lucene.apache.org
> Subject: Re: Re:Re: Implementing security.json is breaking ADDREPLICA
>
> Yes, I see the same issue. I'll update the JIRA and drill down. Thanks.
>
> On Mon, Nov 23, 2015 at 4:18 PM, Anshum Gupta <ans...@anshumgupta.net>
> wrote:
>
> > To restart solr, you should instead use something like:
> > bin/solr start -c -p 8983 -s "example/cloud/node1/solr" -z localhost:2181
> > or
> > bin/solr start -c -p 7574 -s "example/cloud/node2/solr" -z localhost:2181
> >
> > I've seen others report the same exception but never ran into this one
> > myself. Let me try this out.
> >
> >
> >
> > On Mon, Nov 23, 2015 at 2:55 PM, Oakley, Craig (NIH/NLM/NCBI) [C] <
> > craig.oak...@nih.gov> wrote:
> >
> >> FWIW
> >>
> >> I am getting fairly consistent results that if I follow the SOLR-8326
> >> procedure just up through the step of "solr-5.3.1/bin/solr start -e
> cloud
> >> -z localhost:2181": if I then stop just one node (either "./solr stop -p
> >> 7574" or "./solr stop -p 8983") and then restart that same node (using
> the
> >> command suggested by "solr-5.3.1/bin/solr start -e cloud -z
> >> localhost:2181"), then the solr.log for the stopped-and-restarted node
> gets
> >&
RE: Re:Re: Implementing security.json is breaking ADDREPLICA
Thank you for the reply
Trying those exact commands, I'm still getting the same issue
tar xvzf /net/sybdev11/export/home/sybase/Distr/Solr/solr-5.3.1.tgz
tar xvzf /net/sybdev11/export/home/sybase/Distr/Solr/zookeeper-3.4.6.tar.gz
cd zookeeper-3.4.6/
bin/zkServer.sh start zoo_sample.cfg
cd ..
solr-5.3.1/server/scripts/cloud-scripts/zkcli.sh -zkhost localhost:2181 -cmd
putfile /security.json PREVsolr-5.3.1/server/scripts/cloud-scripts/security.json
solr-5.3.1/server/scripts/cloud-scripts/zkcli.sh -zkhost localhost:2181 -cmd
list
solr-5.3.1/bin/solr start -e cloud -z localhost:2181
cd solr-5.3.1/
bin/solr stop -p 7574
bin/solr start -c -p 7574 -s "example/cloud/node2/solr" -z localhost:2181
tail -f example/cloud/node2/logs/solr.log
The -cmd list shows
/ (2)
DATA:
/zookeeper (1)
DATA:
/security.json (0)
DATA:
{"authentication":{"class":"solr.BasicAuthPlugin","credentials":{"solr":"IV0EHq1OnNrj6gvRCwvFwTrZ1+z1oBbnQdiVC3otuq0=
Ndd7LKvVBAaZIF0QAVi1ekCfAJXr1GGfLtRUXhgrF8c="}},"authorization":{"class":"solr.RuleBasedAuthorizationPlugin","user-role":
{"solr":["admin"]}
,"permissions":[
{"name":"security-edit","role":"admin"}
]}}
While the output of tail contains
ERROR - 2015-11-24 10:45:54.796; [c:gettingstarted s:shard1 r:core_node4
x:gettingstarted_shard1_replica1] org.apache.solr.common.SolrException; Error
while trying to recover.:java.util.concurrent.ExecutionException:
org.apache.http.ParseException: Invalid content type:
at java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.util.concurrent.FutureTask.get(FutureTask.java:192)
at
org.apache.solr.cloud.RecoveryStrategy.sendPrepRecoveryCmd(RecoveryStrategy.java:598)
at
org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:361)
at org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:227)
Caused by: org.apache.http.ParseException: Invalid content type:
at org.apache.http.entity.ContentType.parse(ContentType.java:273)
at
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:512)
at
org.apache.solr.client.solrj.impl.HttpSolrClient$1.call(HttpSolrClient.java:270)
at
org.apache.solr.client.solrj.impl.HttpSolrClient$1.call(HttpSolrClient.java:266)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:210)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
-----Original Message-
From: Anshum Gupta [mailto:ans...@anshumgupta.net]
Sent: Monday, November 23, 2015 7:24 PM
To: solr-user@lucene.apache.org
Subject: Re: Re:Re: Implementing security.json is breaking ADDREPLICA
Yes, I see the same issue. I'll update the JIRA and drill down. Thanks.
On Mon, Nov 23, 2015 at 4:18 PM, Anshum Gupta <ans...@anshumgupta.net>
wrote:
> To restart solr, you should instead use something like:
> bin/solr start -c -p 8983 -s "example/cloud/node1/solr" -z localhost:2181
> or
> bin/solr start -c -p 7574 -s "example/cloud/node2/solr" -z localhost:2181
>
> I've seen others report the same exception but never ran into this one
> myself. Let me try this out.
>
>
>
> On Mon, Nov 23, 2015 at 2:55 PM, Oakley, Craig (NIH/NLM/NCBI) [C] <
> craig.oak...@nih.gov> wrote:
>
>> FWIW
>>
>> I am getting fairly consistent results that if I follow the SOLR-8326
>> procedure just up through the step of "solr-5.3.1/bin/solr start -e cloud
>> -z localhost:2181": if I then stop just one node (either "./solr stop -p
>> 7574" or "./solr stop -p 8983") and then restart that same node (using the
>> command suggested by "solr-5.3.1/bin/solr start -e cloud -z
>> localhost:2181"), then the solr.log for the stopped-and-restarted node gets
>> such stack traces as
>> ERROR - 2015-11-23 21:49:28.663; [c:gettingstarted s:shard2 r:core_node3
>> x:gettingstarted_shard2_replica2] org.apache.solr.common.SolrException;
>> Error while trying to recover.:java.util.concurrent.ExecutionException:
>> org.apache.http.ParseException: Invalid content type:
>> at java.util.concurrent.FutureTask.report(FutureTask.java:122)
>> at java.util.concurrent.FutureTask.get(FutureTask.java:192)
>> at
>> org.apache.solr.cloud.RecoveryStrategy.sendPrepRecoveryCmd(RecoveryStrategy.java:598)
>>
RE: Re:Re: Implementing security.json is breaking ADDREPLICA
Thanks for the reply,
I don't suppose there is an ETA for 5.4?
Thanks again
-Original Message-
From: Anshum Gupta [mailto:ans...@anshumgupta.net]
Sent: Tuesday, November 24, 2015 12:31 PM
To: solr-user@lucene.apache.org
Subject: Re: Re:Re: Implementing security.json is breaking ADDREPLICA
Yes, it certainly is a PKI issue.
On Tue, Nov 24, 2015 at 7:59 AM, Oakley, Craig (NIH/NLM/NCBI) [C] <
craig.oak...@nih.gov> wrote:
> Thank you for the reply
>
> Trying those exact commands, I'm still getting the same issue
> tar xvzf /net/sybdev11/export/home/sybase/Distr/Solr/solr-5.3.1.tgz
> tar xvzf /net/sybdev11/export/home/sybase/Distr/Solr/zookeeper-3.4.6.tar.gz
> cd zookeeper-3.4.6/
> bin/zkServer.sh start zoo_sample.cfg
> cd ..
> solr-5.3.1/server/scripts/cloud-scripts/zkcli.sh -zkhost localhost:2181
> -cmd putfile /security.json
> PREVsolr-5.3.1/server/scripts/cloud-scripts/security.json
> solr-5.3.1/server/scripts/cloud-scripts/zkcli.sh -zkhost localhost:2181
> -cmd list
> solr-5.3.1/bin/solr start -e cloud -z localhost:2181
> cd solr-5.3.1/
> bin/solr stop -p 7574
> bin/solr start -c -p 7574 -s "example/cloud/node2/solr" -z localhost:2181
> tail -f example/cloud/node2/logs/solr.log
>
> The -cmd list shows
> / (2)
> DATA:
>
> /zookeeper (1)
> DATA:
>
> /security.json (0)
> DATA:
>
>
> {"authentication":{"class":"solr.BasicAuthPlugin","credentials":{"solr":"IV0EHq1OnNrj6gvRCwvFwTrZ1+z1oBbnQdiVC3otuq0=
> Ndd7LKvVBAaZIF0QAVi1ekCfAJXr1GGfLtRUXhgrF8c="}},"authorization":{"class":"solr.RuleBasedAuthorizationPlugin","user-role":
> {"solr":["admin"]}
>
> ,"permissions":[
> {"name":"security-edit","role":"admin"}
>
> ]}}
>
>
> While the output of tail contains
> ERROR - 2015-11-24 10:45:54.796; [c:gettingstarted s:shard1 r:core_node4
> x:gettingstarted_shard1_replica1] org.apache.solr.common.SolrException;
> Error while trying to recover.:java.util.concurrent.ExecutionException:
> org.apache.http.ParseException: Invalid content type:
> at java.util.concurrent.FutureTask.report(FutureTask.java:122)
> at java.util.concurrent.FutureTask.get(FutureTask.java:192)
> at
> org.apache.solr.cloud.RecoveryStrategy.sendPrepRecoveryCmd(RecoveryStrategy.java:598)
> at
> org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:361)
> at
> org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:227)
> Caused by: org.apache.http.ParseException: Invalid content type:
> at org.apache.http.entity.ContentType.parse(ContentType.java:273)
> at
> org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:512)
> at
> org.apache.solr.client.solrj.impl.HttpSolrClient$1.call(HttpSolrClient.java:270)
> at
> org.apache.solr.client.solrj.impl.HttpSolrClient$1.call(HttpSolrClient.java:266)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:210)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
>
>
> -Original Message-
> From: Anshum Gupta [mailto:ans...@anshumgupta.net]
> Sent: Monday, November 23, 2015 7:24 PM
> To: solr-user@lucene.apache.org
> Subject: Re: Re:Re: Implementing security.json is breaking ADDREPLICA
>
> Yes, I see the same issue. I'll update the JIRA and drill down. Thanks.
>
> On Mon, Nov 23, 2015 at 4:18 PM, Anshum Gupta <ans...@anshumgupta.net>
> wrote:
>
> > To restart solr, you should instead use something like:
> > bin/solr start -c -p 8983 -s "example/cloud/node1/solr" -z localhost:2181
> > or
> > bin/solr start -c -p 7574 -s "example/cloud/node2/solr" -z localhost:2181
> >
> > I've seen others report the same exception but never ran into this one
> > myself. Let me try this out.
> >
> >
> >
> > On Mon, Nov 23, 2015 at 2:55 PM, Oakley, Craig (NIH/NLM/NCBI) [C] <
> > craig.oak...@nih.gov> wrote:
> >
> >> FWIW
> >>
> >> I am getting fairly consistent results that if I follow the SOLR-8326
> >> procedure just up through the step of "solr-5.3.1/bin/solr start -e
> cloud
> >> -z localhost:2181": if I then stop just one node (either "./solr stop -p
> >>
Re: Re:Re: Implementing security.json is breaking ADDREPLICA
bq: I don't suppose there is an ETA for 5.4?
Actually, 5.4 is probably in the works within the next month. I'm not
the one cutting the
release, but there's some rumors that a label will be cut this week,
then the "usual"
process is a week or two (sometimes more if bugs are flushed out) before the
official release.
Call it the first of the year for safety's sake, but that's a guess.
Best,
Erick
On Tue, Nov 24, 2015 at 10:22 AM, Oakley, Craig (NIH/NLM/NCBI) [C]
<craig.oak...@nih.gov> wrote:
> Thanks for the reply,
>
> I don't suppose there is an ETA for 5.4?
>
>
> Thanks again
>
> -Original Message-
> From: Anshum Gupta [mailto:ans...@anshumgupta.net]
> Sent: Tuesday, November 24, 2015 12:31 PM
> To: solr-user@lucene.apache.org
> Subject: Re: Re:Re: Implementing security.json is breaking ADDREPLICA
>
> Yes, it certainly is a PKI issue.
>
> On Tue, Nov 24, 2015 at 7:59 AM, Oakley, Craig (NIH/NLM/NCBI) [C] <
> craig.oak...@nih.gov> wrote:
>
>> Thank you for the reply
>>
>> Trying those exact commands, I'm still getting the same issue
>> tar xvzf /net/sybdev11/export/home/sybase/Distr/Solr/solr-5.3.1.tgz
>> tar xvzf /net/sybdev11/export/home/sybase/Distr/Solr/zookeeper-3.4.6.tar.gz
>> cd zookeeper-3.4.6/
>> bin/zkServer.sh start zoo_sample.cfg
>> cd ..
>> solr-5.3.1/server/scripts/cloud-scripts/zkcli.sh -zkhost localhost:2181
>> -cmd putfile /security.json
>> PREVsolr-5.3.1/server/scripts/cloud-scripts/security.json
>> solr-5.3.1/server/scripts/cloud-scripts/zkcli.sh -zkhost localhost:2181
>> -cmd list
>> solr-5.3.1/bin/solr start -e cloud -z localhost:2181
>> cd solr-5.3.1/
>> bin/solr stop -p 7574
>> bin/solr start -c -p 7574 -s "example/cloud/node2/solr" -z localhost:2181
>> tail -f example/cloud/node2/logs/solr.log
>>
>> The -cmd list shows
>> / (2)
>> DATA:
>>
>> /zookeeper (1)
>> DATA:
>>
>> /security.json (0)
>> DATA:
>>
>>
>> {"authentication":{"class":"solr.BasicAuthPlugin","credentials":{"solr":"IV0EHq1OnNrj6gvRCwvFwTrZ1+z1oBbnQdiVC3otuq0=
>> Ndd7LKvVBAaZIF0QAVi1ekCfAJXr1GGfLtRUXhgrF8c="}},"authorization":{"class":"solr.RuleBasedAuthorizationPlugin","user-role":
>> {"solr":["admin"]}
>>
>> ,"permissions":[
>> {"name":"security-edit","role":"admin"}
>>
>> ]}}
>>
>>
>> While the output of tail contains
>> ERROR - 2015-11-24 10:45:54.796; [c:gettingstarted s:shard1 r:core_node4
>> x:gettingstarted_shard1_replica1] org.apache.solr.common.SolrException;
>> Error while trying to recover.:java.util.concurrent.ExecutionException:
>> org.apache.http.ParseException: Invalid content type:
>> at java.util.concurrent.FutureTask.report(FutureTask.java:122)
>> at java.util.concurrent.FutureTask.get(FutureTask.java:192)
>> at
>> org.apache.solr.cloud.RecoveryStrategy.sendPrepRecoveryCmd(RecoveryStrategy.java:598)
>> at
>> org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:361)
>> at
>> org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:227)
>> Caused by: org.apache.http.ParseException: Invalid content type:
>> at org.apache.http.entity.ContentType.parse(ContentType.java:273)
>> at
>> org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:512)
>> at
>> org.apache.solr.client.solrj.impl.HttpSolrClient$1.call(HttpSolrClient.java:270)
>> at
>> org.apache.solr.client.solrj.impl.HttpSolrClient$1.call(HttpSolrClient.java:266)
>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> at
>> org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:210)
>> at
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>> at
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>> at java.lang.Thread.run(Thread.java:745)
>>
>>
>> -Original Message-
>> From: Anshum Gupta [mailto:ans...@anshumgupta.net]
>> Sent: Monday, November 23, 2015 7:24 PM
>> To: solr-user@lucene.apache.org
>> Subject: Re: Re:Re: Implementing security.json is breaking ADDREPLICA
>>
>> Yes, I see the same issue. I'll update the JIRA and drill down. Thanks.
>>
>> On Mon, Nov 23, 2015 at 4:18 PM, Anshum Gupta <ans...@anshum
RE: Re:Re: Implementing security.json is breaking ADDREPLICA
the other node -- in this case it successfully starts.
Is there some necessary environment tweaking? The symptoms seem similar whether
I use the security.json from SOLR-8326 or the security.json from the Wiki (with
the comma repositioned).
-Original Message-
From: Oakley, Craig (NIH/NLM/NCBI) [C]
Sent: Friday, November 20, 2015 6:59 PM
To: 'solr-user@lucene.apache.org' <solr-user@lucene.apache.org>
Subject: RE: Re:Re: Implementing security.json is breaking ADDREPLICA
Thanks
It seems to work when there is no security.json, so perhaps there's some typo
in the initial version.
I notice that the version you sent is different from the documentation at
cwiki.apache.org/confluence/display/solr/Authentication+and+Authorization+Plugins
in that the Wiki version has "permissions" before "user-role": I also notice
that (at least as of right this moment) the Wiki version has a comma at the end
of '"user-role":{"solr":"admin"},' even though it is at the end: and I notice
that the Wiki version seems to lack a comma between the "permissions" section
and the "user-role" section. I just now also noticed that the version you sent
has '"user-role":{"solr":["admin"]}' (with square brackets) whereas the Wiki
does not have square brackets.
The placement of the comma definitely looks wrong in the Wiki at the moment
(though perhaps someone might correct the Wiki before too long). Other than
that, I don’t know whether the order and/or the square brackets make a
difference. I can try with different permutations.
Thanks again
P.S. for the record, the Wiki currently has
{
"authentication":{
"class":"solr.BasicAuthPlugin",
"credentials":{"solr":"IV0EHq1OnNrj6gvRCwvFwTrZ1+z1oBbnQdiVC3otuq0=
Ndd7LKvVBAaZIF0QAVi1ekCfAJXr1GGfLtRUXhgrF8c="}
},
"authorization":{
"class":"solr.RuleBasedAuthorizationPlugin",
"permissions":[{"name":"security-edit",
"role":"admin"}]
"user-role":{"solr":"admin"},
}}
-Original Message-
From: Anshum Gupta [mailto:ans...@anshumgupta.net]
Sent: Friday, November 20, 2015 6:18 PM
To: solr-user@lucene.apache.org
Subject: Re: Re:Re: Implementing security.json is breaking ADDREPLICA
This seems unrelated and more like a user error somewhere. Can you just
follow the steps, without any security settings i.e. not even uploading
security.json and see if you still see this? Sorry, but I don't have access
to the code right now, I try and look at this later tonight.
On Fri, Nov 20, 2015 at 3:07 PM, Oakley, Craig (NIH/NLM/NCBI) [C] <
craig.oak...@nih.gov> wrote:
> Thank you for opening SOLR-8326
>
> As a side note, in the procedure you listed, even before adding the
> collection-admin-edit authorization, I'm already hitting trouble: stopping
> and restarting a node results in the following
>
> INFO - 2015-11-20 22:48:41.275; [c:solr8326 s:shard2 r:core_node4
> x:solr8326_shard2_replica1] org.apache.solr.cloud.RecoveryStrategy;
> Publishing state of core solr8326_shard2_replica1 as recovering, leader is
> http://{IP-address-redacted}:8983/solr/solr8326_shard2_replica2/ and I am
> http://{IP-address-redacted}:7574/solr/solr8326_shard2_replica1/
> INFO - 2015-11-20 22:48:41.275; [c:solr8326 s:shard2 r:core_node4
> x:solr8326_shard2_replica1] org.apache.solr.cloud.ZkController; publishing
> state=recovering
> INFO - 2015-11-20 22:48:41.278; [c:solr8326 s:shard1 r:core_node3
> x:solr8326_shard1_replica1] org.apache.solr.cloud.RecoveryStrategy;
> Publishing state of core solr8326_shard1_replica1 as recovering, leader is
> http://{IP-address-redacted}:8983/solr/solr8326_shard1_replica2/ and I am
> http://{IP-address-redacted}:7574/solr/solr8326_shard1_replica1/
> INFO - 2015-11-20 22:48:41.280; [c:solr8326 s:shard1 r:core_node3
> x:solr8326_shard1_replica1] org.apache.solr.cloud.ZkController; publishing
> state=recovering
> INFO - 2015-11-20 22:48:41.282; [c:solr8326 s:shard2 r:core_node4
> x:solr8326_shard2_replica1] org.apache.solr.cloud.RecoveryStrategy; Sending
> prep recovery command to http://{IP-address-redacted}:8983/solr;
> WaitForState:
> action=PREPRECOVERY=solr8326_shard2_replica2={IP-address-redacted}%3A7574_solr=core_node4=recovering=true=true=true
> INFO - 2015-11-20 22:48:41.289; [ ]
> org.apache.solr.common.cloud.ZkStateReader$8; A cluster state change:
> WatchedEvent state:SyncConnected type:NodeDataChanged
> path:/collections/solr8326/state.json for collection solr8326 has occurred
> - updating... (live nodes size: 2)
> INFO - 2015-11-20 22:48:41.290; [c:solr8326 s:shard1 r:core_node3
> x:solr8326_shard1_replica1] org.apache.solr.cloud.RecoveryStrategy; Sending
> pre
Re: Re:Re: Implementing security.json is breaking ADDREPLICA
at org.eclipse.jetty.server.Server.handle(Server.java:499)
> at
> org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)
> at
> org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
> at
> org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
> at java.lang.Thread.run(Thread.java:745)
>
> In this case the string is just "r?", but usually it is a longer string of
> control characters.
>
> If I shutdown _both_ nodes and restart _one_, and then allow it to be
> "Waiting until we see more replicas up" until it recognizes itself as
> leader, and _then_ restart the other node -- in this case it successfully
> starts.
>
> Is there some necessary environment tweaking? The symptoms seem similar
> whether I use the security.json from SOLR-8326 or the security.json from
> the Wiki (with the comma repositioned).
>
>
>
> -Original Message-
> From: Oakley, Craig (NIH/NLM/NCBI) [C]
> Sent: Friday, November 20, 2015 6:59 PM
> To: 'solr-user@lucene.apache.org' <solr-user@lucene.apache.org>
> Subject: RE: Re:Re: Implementing security.json is breaking ADDREPLICA
>
> Thanks
>
> It seems to work when there is no security.json, so perhaps there's some
> typo in the initial version.
>
> I notice that the version you sent is different from the documentation at
> cwiki.apache.org/confluence/display/solr/Authentication+and+Authorization+Plugins
> in that the Wiki version has "permissions" before "user-role": I also
> notice that (at least as of right this moment) the Wiki version has a comma
> at the end of '"user-role":{"solr":"admin"},' even though it is at the end:
> and I notice that the Wiki version seems to lack a comma between the
> "permissions" section and the "user-role" section. I just now also noticed
> that the version you sent has '"user-role":{"solr":["admin"]}' (with square
> brackets) whereas the Wiki does not have square brackets.
>
> The placement of the comma definitely looks wrong in the Wiki at the
> moment (though perhaps someone might correct the Wiki before too long).
> Other than that, I don’t know whether the order and/or the square brackets
> make a difference. I can try with different permutations.
>
> Thanks again
>
> P.S. for the record, the Wiki currently has
> {
> "authentication":{
>"class":"solr.BasicAuthPlugin",
>"credentials":{"solr":"IV0EHq1OnNrj6gvRCwvFwTrZ1+z1oBbnQdiVC3otuq0=
> Ndd7LKvVBAaZIF0QAVi1ekCfAJXr1GGfLtRUXhgrF8c="}
> },
> "authorization":{
>"class":"solr.RuleBasedAuthorizationPlugin",
>"permissions":[{"name":"security-edit",
> "role":"admin"}]
>"user-role":{"solr":"admin"},
> }}
>
> -Original Message-
> From: Anshum Gupta [mailto:ans...@anshumgupta.net]
> Sent: Friday, November 20, 2015 6:18 PM
> To: solr-user@lucene.apache.org
> Subject: Re: Re:Re: Implementing security.json is breaking ADDREPLICA
>
> This seems unrelated and more like a user error somewhere. Can you just
> follow the steps, without any security settings i.e. not even uploading
> security.json and see if you still see this? Sorry, but I don't have access
> to the code right now, I try and look at this later tonight.
>
> On Fri, Nov 20, 2015 at 3:07 PM, Oakley, Craig (NIH/NLM/NCBI) [C] <
> craig.oak...@nih.gov> wrote:
>
> > Thank you for opening SOLR-8326
> >
> > As a side note, in the procedure you listed, even before adding the
> > collection-admin-edit authorization, I'm already hitting trouble:
> stopping
> > and restarting a node results in the following
> >
> > INFO - 2015-11-20 22:48:41.275; [c:solr8326 s:shard2 r:core_node4
> > x:solr8326_shard2_replica1] org.apache.solr.cloud.RecoveryStrategy;
> > Publishing state of core solr8326_shard2_replica1 as recovering, leader
> is
> > http://{IP-address-redacted}:8983/solr/solr8326_shard2_replica2/ and I
> am
> > http://{IP-address-redacted}:7574/solr/solr8326_shard2_replica1/
> > INFO - 2015-11-20 22:48:41.275; [c:solr8326 s:shard2 r:core_node4
> > x:solr8326_shard2_replica1] org.apache.solr.cloud.ZkController;
> publishing
> > state=recovering
> > INFO - 2015-11-20 22:48:41.278; [c:solr8326 s:shard1 r:core_node3
> > x:sol
Re: Re:Re: Implementing security.json is breaking ADDREPLICA
gt; org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
>> at
>> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
>> at
>> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215)
>> at
>> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110)
>> at
>> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
>> at org.eclipse.jetty.server.Server.handle(Server.java:499)
>> at
>> org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)
>> at
>> org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
>> at
>> org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)
>> at
>> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
>> at
>> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
>> at java.lang.Thread.run(Thread.java:745)
>>
>> In this case the string is just "r?", but usually it is a longer string
>> of control characters.
>>
>> If I shutdown _both_ nodes and restart _one_, and then allow it to be
>> "Waiting until we see more replicas up" until it recognizes itself as
>> leader, and _then_ restart the other node -- in this case it successfully
>> starts.
>>
>> Is there some necessary environment tweaking? The symptoms seem similar
>> whether I use the security.json from SOLR-8326 or the security.json from
>> the Wiki (with the comma repositioned).
>>
>>
>>
>> -Original Message-
>> From: Oakley, Craig (NIH/NLM/NCBI) [C]
>> Sent: Friday, November 20, 2015 6:59 PM
>> To: 'solr-user@lucene.apache.org' <solr-user@lucene.apache.org>
>> Subject: RE: Re:Re: Implementing security.json is breaking ADDREPLICA
>>
>> Thanks
>>
>> It seems to work when there is no security.json, so perhaps there's some
>> typo in the initial version.
>>
>> I notice that the version you sent is different from the documentation at
>> cwiki.apache.org/confluence/display/solr/Authentication+and+Authorization+Plugins
>> in that the Wiki version has "permissions" before "user-role": I also
>> notice that (at least as of right this moment) the Wiki version has a comma
>> at the end of '"user-role":{"solr":"admin"},' even though it is at the end:
>> and I notice that the Wiki version seems to lack a comma between the
>> "permissions" section and the "user-role" section. I just now also noticed
>> that the version you sent has '"user-role":{"solr":["admin"]}' (with square
>> brackets) whereas the Wiki does not have square brackets.
>>
>> The placement of the comma definitely looks wrong in the Wiki at the
>> moment (though perhaps someone might correct the Wiki before too long).
>> Other than that, I don’t know whether the order and/or the square brackets
>> make a difference. I can try with different permutations.
>>
>> Thanks again
>>
>> P.S. for the record, the Wiki currently has
>> {
>> "authentication":{
>>"class":"solr.BasicAuthPlugin",
>>"credentials":{"solr":"IV0EHq1OnNrj6gvRCwvFwTrZ1+z1oBbnQdiVC3otuq0=
>> Ndd7LKvVBAaZIF0QAVi1ekCfAJXr1GGfLtRUXhgrF8c="}
>> },
>> "authorization":{
>>"class":"solr.RuleBasedAuthorizationPlugin",
>>"permissions":[{"name":"security-edit",
>> "role":"admin"}]
>>"user-role":{"solr":"admin"},
>> }}
>>
>> -Original Message-
>> From: Anshum Gupta [mailto:ans...@anshumgupta.net]
>> Sent: Friday, November 20, 2015 6:18 PM
>> To: solr-user@lucene.apache.org
>> Subject: Re: Re:Re: Implementing security.json is breaking ADDREPLICA
>>
>> This seems unrelated and more like a user error somewhere. Can you just
>> follow the steps, without any security settings i.e. not even uploading
>> security.json and see if you still see this? Sorry, but I don't have
>> access
>> to the code right now, I try and look at this later tonight.
>>
>> On Fri, Nov 20, 2015 at 3:07 PM, Oakley, Craig (NIH/NLM/NCBI) [C] <
>> craig.oak...@nih.gov> wrote:
>>
>> > Thank you for opening SOLR-8326
>> >
>> > As a side note, in the procedur
RE: Re:Re: Implementing security.json is breaking ADDREPLICA
Thank you again for the reply.
Below is the Email I was about to send prior to your reply a moment ago: shall
I try again without "read" in the security.json?
The Collections API method was not discussed in the "Unleashed" class at the
conference in DC in 2014 (probably because it was not yet available), so I was
using the method I knew.
I have now tried again using admin/collections?action=CREATE (using different
port numbers to avoid confusion from the failed previous attempts: the
previously created nodes had been shutdown and their core.properties files
renamed so as not to be discovered), but the results are the same:
INFO - 2015-11-20 16:56:25.283; [c:xmpl3 s:shard1 r:core_node2
x:xmpl3_shard1_replica2] org.apache.solr.cloud.RecoveryStrategy; Starting
Replication Recovery.
INFO - 2015-11-20 16:56:25.284; [c:xmpl3 s:shard1 r:core_node2
x:xmpl3_shard1_replica2] org.apache.solr.cloud.RecoveryStrategy; Begin
buffering updates.
INFO - 2015-11-20 16:56:25.284; [c:xmpl3 s:shard1 r:core_node2
x:xmpl3_shard1_replica2] org.apache.solr.update.UpdateLog; Starting to buffer
updates. FSUpdateLog{state=ACTIVE, tlog=null}
INFO - 2015-11-20 16:56:25.284; [c:xmpl3 s:shard1 r:core_node2
x:xmpl3_shard1_replica2] org.apache.solr.cloud.RecoveryStrategy; Attempting to
replicate from http://{IP-address-redacted}:4685/solr/xmpl3_shard1_replica1/.
ERROR - 2015-11-20 16:56:25.292; [c:xmpl3 s:shard1 r:core_node2
x:xmpl3_shard1_replica2] org.apache.solr.common.SolrException; Error while
trying to
recover:org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException:
Error from server at
http://{IP-address-redacted}:4685/solr/xmpl3_shard1_replica1: Expected mime
type application/octet-stream but got text/html.
Error 401 Unauthorized request, Response code: 401
HTTP ERROR 401
Problem accessing /solr/xmpl3_shard1_replica1/update. Reason:
Unauthorized request, Response code:
401Powered by Jetty://
at
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:528)
at
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:234)
at
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:226)
at
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:135)
at
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:152)
at
org.apache.solr.cloud.RecoveryStrategy.commitOnLeader(RecoveryStrategy.java:207)
at
org.apache.solr.cloud.RecoveryStrategy.replicate(RecoveryStrategy.java:147)
at
org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:437)
at org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:227)
INFO - 2015-11-20 16:56:25.292; [c:xmpl3 s:shard1 r:core_node2
x:xmpl3_shard1_replica2] org.apache.solr.update.UpdateLog; Dropping buffered
updates FSUpdateLog{state=BUFFERING, tlog=null}
ERROR - 2015-11-20 16:56:25.293; [c:xmpl3 s:shard1 r:core_node2
x:xmpl3_shard1_replica2] org.apache.solr.cloud.RecoveryStrategy; Recovery
failed - trying again... (2)
INFO - 2015-11-20 16:56:25.293; [c:xmpl3 s:shard1 r:core_node2
x:xmpl3_shard1_replica2] org.apache.solr.cloud.RecoveryStrategy; Wait 8.0
seconds before trying to recover again (3)
Below is a list of the steps I took.
./zkcli.sh --zkhost localhost:4545 -cmd makepath /solr/xmpl3
./zkcli.sh --zkhost localhost:4545/solr/xmpl3 -cmd putfile /security.json
~/solr/security151119a.json
./zkcli.sh --zkhost localhost:4545/solr/xmpl3 -cmd upconfig -confdir
../../solr/configsets/basic_configs/conf -confname xmpl3
cd ../../../bin/
./solr -c -p 4695 -d ~dbman/solr/straight531outofbox/solr-5.3.1/server/ -z
localhost:4545/solr/xmpl3 -s
~dbman/solr/straight531outofbox/solr-5.3.1/example/solr
./solr -c -p 4685 -d ~dbman/solr/straight531outofbox/solr-5.3.1/server/ -z
localhost:4545/solr/xmpl3 -s
~dbman/solr/straight531outofbox/solr-5.3.1/server/solr
curl -u solr:SolrRocks
'http://nosqltest11:4685/solr/admin/collections?action=CREATE=xmpl3=1=1={IP-address-redacted}:4685_solr'
curl -u solr:SolrRocks
'http://nosqltest11:4685/solr/admin/collections?action=ADDREPLICA=xmpl3=shard1={IP-address-redacted}:4695_solr=json=true'
Can you provide a list of steps to take in an out-of-the-box directory tree
whereby ADDREPLICA _will_ work with security.json already in place?
-Original Message-
From: Anshum Gupta [mailto:ans...@anshumgupta.net]
Sent: Thursday, November 19, 2015 3:44 PM
To: solr-user@lucene.apache.org
Subject: Re: Re:Re: Implementing security.json is breaking ADDREPLICA
I'll try out what you did later in the day, as soon as I get time but why
exactly are you creating cores manually? Seems like you manually create a
core and the try to add a replica. Can you try using the Collections API to
create a collection?
Starting Solr 5.0, the only supported way to create a new collection is via
the Collections API. Creati
Re: Re:Re: Implementing security.json is breaking ADDREPLICA
>From my tests, it seems like the 'read' permission interferes with the
Replication and so the ADDREPLICA also fails. You're also bound to run into
issues if you have 'read' permission setup and restart your cluster,
provided you have a collection that has a replication factor > 1 for at
least one shard.
I'll create a JIRA for this and mark it to be a blocker for 5.4. Thanks for
bringing this up.
On Thu, Nov 19, 2015 at 12:43 PM, Anshum Gupta <ans...@anshumgupta.net>
wrote:
> I'll try out what you did later in the day, as soon as I get time but why
> exactly are you creating cores manually? Seems like you manually create a
> core and the try to add a replica. Can you try using the Collections API to
> create a collection?
>
> Starting Solr 5.0, the only supported way to create a new collection is
> via the Collections API. Creating a core would lead to a collection
> creation but that's not really supported. It was just something that was
> done when there were no Collections API.
>
>
> On Thu, Nov 19, 2015 at 12:36 PM, Oakley, Craig (NIH/NLM/NCBI) [C] <
> craig.oak...@nih.gov> wrote:
>
>> I tried again with the following security.json, but the results were the
>> same:
>>
>> {
>> "authentication":{
>> "class":"solr.BasicAuthPlugin",
>> "credentials":{
>> "solr":"IV0EHq1OnNrj6gvRCwvFwTrZ1+z1oBbnQdiVC3otuq0=
>> Ndd7LKvVBAaZIF0QAVi1ekCfAJXr1GGfLtRUXhgrF8c=",
>> "solruser":"VgZX1TAMNHT2IJikoGdKtxQdXc+MbNwfqzf89YqcLEE=
>> 37pPWQ9v4gciIKHuTmFmN0Rv66rnlMOFEWfEy9qjJfY="},
>> "":{"v":9}},
>> "authorization":{
>> "class":"solr.RuleBasedAuthorizationPlugin",
>> "user-role":{
>> "solr":[
>> "admin",
>> "read",
>> "xmpladmin",
>> "xmplgen",
>> "xmplsel"],
>> "solruser":[
>> "read",
>> "xmplgen",
>> "xmplsel"]},
>> "permissions":[
>> {
>> "name":"security-edit",
>> "role":"admin"},
>> {
>> "name":"xmpl_admin",
>> "collection":"xmpl",
>> "path":"/admin/*",
>> "role":"xmpladmin"},
>> {
>> "name":"xmpl_sel",
>> "collection":"xmpl",
>> "path":"/select/*",
>> "role":null},
>> {
>> "name":"all-admin",
>> "collection":null,
>> "path":"/*",
>> "role":"xmplgen"},
>> {
>> "name":"all-core-handlers",
>> "path":"/*",
>> "role":"xmplgen"}],
>> "":{"v":42}}}
>>
>> -Original Message-
>> From: Oakley, Craig (NIH/NLM/NCBI) [C]
>> Sent: Thursday, November 19, 2015 1:46 PM
>> To: 'solr-user@lucene.apache.org' <solr-user@lucene.apache.org>
>> Subject: RE: Re:Re: Implementing security.json is breaking ADDREPLICA
>>
>> I note that the thread called "Security Problems" (most recent post by
>> Nobel Paul) seems like it may help with much of what I'm trying to do. I
>> will see to what extent that may help.
>>
>
>
>
> --
> Anshum Gupta
>
--
Anshum Gupta
Re: Re:Re: Implementing security.json is breaking ADDREPLICA
core_node2
> x:xmpl3_shard1_replica2] org.apache.solr.cloud.RecoveryStrategy; Starting
> Replication Recovery.
> INFO - 2015-11-20 16:56:25.284; [c:xmpl3 s:shard1 r:core_node2
> x:xmpl3_shard1_replica2] org.apache.solr.cloud.RecoveryStrategy; Begin
> buffering updates.
> INFO - 2015-11-20 16:56:25.284; [c:xmpl3 s:shard1 r:core_node2
> x:xmpl3_shard1_replica2] org.apache.solr.update.UpdateLog; Starting to
> buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
> INFO - 2015-11-20 16:56:25.284; [c:xmpl3 s:shard1 r:core_node2
> x:xmpl3_shard1_replica2] org.apache.solr.cloud.RecoveryStrategy; Attempting
> to replicate from http://
> {IP-address-redacted}:4685/solr/xmpl3_shard1_replica1/.
> ERROR - 2015-11-20 16:56:25.292; [c:xmpl3 s:shard1 r:core_node2
> x:xmpl3_shard1_replica2] org.apache.solr.common.SolrException; Error while
> trying to
> recover:org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException:
> Error from server at
> http://{IP-address-redacted}:4685/solr/xmpl3_shard1_replica1:
> Expected mime type application/octet-stream but got text/html.
>
>
> Error 401 Unauthorized request, Response code: 401
>
> HTTP ERROR 401
> Problem accessing /solr/xmpl3_shard1_replica1/update. Reason:
> Unauthorized request, Response code:
> 401Powered by Jetty://
>
>
>
>
> at
> org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:528)
> at
> org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:234)
> at
> org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:226)
> at
> org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:135)
> at
> org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:152)
> at
> org.apache.solr.cloud.RecoveryStrategy.commitOnLeader(RecoveryStrategy.java:207)
> at
> org.apache.solr.cloud.RecoveryStrategy.replicate(RecoveryStrategy.java:147)
> at
> org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:437)
> at
> org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:227)
>
> INFO - 2015-11-20 16:56:25.292; [c:xmpl3 s:shard1 r:core_node2
> x:xmpl3_shard1_replica2] org.apache.solr.update.UpdateLog; Dropping
> buffered updates FSUpdateLog{state=BUFFERING, tlog=null}
> ERROR - 2015-11-20 16:56:25.293; [c:xmpl3 s:shard1 r:core_node2
> x:xmpl3_shard1_replica2] org.apache.solr.cloud.RecoveryStrategy; Recovery
> failed - trying again... (2)
> INFO - 2015-11-20 16:56:25.293; [c:xmpl3 s:shard1 r:core_node2
> x:xmpl3_shard1_replica2] org.apache.solr.cloud.RecoveryStrategy; Wait 8.0
> seconds before trying to recover again (3)
>
>
> Below is a list of the steps I took.
>
> ./zkcli.sh --zkhost localhost:4545 -cmd makepath /solr/xmpl3
> ./zkcli.sh --zkhost localhost:4545/solr/xmpl3 -cmd putfile /security.json
> ~/solr/security151119a.json
> ./zkcli.sh --zkhost localhost:4545/solr/xmpl3 -cmd upconfig -confdir
> ../../solr/configsets/basic_configs/conf -confname xmpl3
> cd ../../../bin/
> ./solr -c -p 4695 -d ~dbman/solr/straight531outofbox/solr-5.3.1/server/ -z
> localhost:4545/solr/xmpl3 -s
> ~dbman/solr/straight531outofbox/solr-5.3.1/example/solr
> ./solr -c -p 4685 -d ~dbman/solr/straight531outofbox/solr-5.3.1/server/ -z
> localhost:4545/solr/xmpl3 -s
> ~dbman/solr/straight531outofbox/solr-5.3.1/server/solr
> curl -u solr:SolrRocks '
> http://nosqltest11:4685/solr/admin/collections?action=CREATE=xmpl3=1=1={IP-address-redacted}:4685_solr
> '
> curl -u solr:SolrRocks '
> http://nosqltest11:4685/solr/admin/collections?action=ADDREPLICA=xmpl3=shard1={IP-address-redacted}:4695_solr=json=true
> '
>
>
>
>
> Can you provide a list of steps to take in an out-of-the-box directory
> tree whereby ADDREPLICA _will_ work with security.json already in place?
>
>
>
>
> -Original Message-
> From: Anshum Gupta [mailto:ans...@anshumgupta.net]
> Sent: Thursday, November 19, 2015 3:44 PM
> To: solr-user@lucene.apache.org
> Subject: Re: Re:Re: Implementing security.json is breaking ADDREPLICA
>
> I'll try out what you did later in the day, as soon as I get time but why
> exactly are you creating cores manually? Seems like you manually create a
> core and the try to add a replica. Can you try using the Collections API to
> create a collection?
>
> Starting Solr 5.0, the only supported way to create a new collection is via
> the Collections API. Creating a core would lead to a collection creation
> but that's not really supported. It was just something that was done when
> there were no Collections API.
>
>
> On Thu, Nov 19, 2015 at 12:36 PM, Oakley, Craig (NIH/NLM/NCBI) [C] &l
Re: Re:Re: Implementing security.json is breaking ADDREPLICA
trying again... (4)
> INFO - 2015-11-20 22:48:41.300; [c:solr8326 s:shard2 r:core_node4
> x:solr8326_shard2_replica1] org.apache.solr.cloud.RecoveryStrategy; Wait
> 32.0 seconds before trying to recover again (5)
> ERROR - 2015-11-20 22:48:41.300; [c:solr8326 s:shard1 r:core_node3
> x:solr8326_shard1_replica1] org.apache.solr.common.SolrException; Error
> while trying to recover.:java.util.concurrent.ExecutionException:
> org.apache.http.ParseException: Invalid content type:
> at java.util.concurrent.FutureTask.report(FutureTask.java:122)
> at java.util.concurrent.FutureTask.get(FutureTask.java:192)
> at
> org.apache.solr.cloud.RecoveryStrategy.sendPrepRecoveryCmd(RecoveryStrategy.java:598)
> at
> org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:361)
> at
> org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:227)
> Caused by: org.apache.http.ParseException: Invalid content type:
> at org.apache.http.entity.ContentType.parse(ContentType.java:273)
> at
> org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:512)
> at
> org.apache.solr.client.solrj.impl.HttpSolrClient$1.call(HttpSolrClient.java:270)
> at
> org.apache.solr.client.solrj.impl.HttpSolrClient$1.call(HttpSolrClient.java:266)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:210)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
>
> ERROR - 2015-11-20 22:48:41.318; [c:solr8326 s:shard1 r:core_node3
> x:solr8326_shard1_replica1] org.apache.solr.cloud.RecoveryStrategy;
> Recovery failed - trying again... (4)
> INFO - 2015-11-20 22:48:41.318; [ ]
> org.apache.solr.common.cloud.ZkStateReader; Updating data for solr8326 to
> ver 26
> INFO - 2015-11-20 22:48:41.319; [c:solr8326 s:shard1 r:core_node3
> x:solr8326_shard1_replica1] org.apache.solr.cloud.RecoveryStrategy; Wait
> 32.0 seconds before trying to recover again (5)
>
>
> I would not be surprised if this were to be some unrelated issue (the
> symptoms are quite different)
>
>
>
> Thanks again
>
>
> -Original Message-
> From: Anshum Gupta [mailto:ans...@anshumgupta.net]
> Sent: Friday, November 20, 2015 1:31 PM
> To: solr-user@lucene.apache.org
> Subject: Re: Re:Re: Implementing security.json is breaking ADDREPLICA
>
> Collections API were available before November of 2014, if that is when you
> took the class. However, it was only with Solr 5.0 (released in Feb 2015)
> that the only supported mechanism to create a collection was restricted to
> Collections API.
>
> Here are the list of steps that you'd need to run to see that things are
> fine for you without the read permission:
> * Untar and setup Solr, don't start it yet
> * Start clean zookeeper
> * Put the security.json in zk, without anything other than a security-edit
> permission. Find the content of the file below. Upload it using your own zk
> client or through the solr script:
> > solr-5.3.1/server/scripts/cloud-scripts/zkcli.sh -zkhost localhost:2181
> -cmd putfile /security.json ~/security.json
>
> security.json:
>
> {"authentication":{"class":"solr.BasicAuthPlugin","credentials":{"solr":"IV0EHq1OnNrj6gvRCwvFwTrZ1+z1oBbnQdiVC3otuq0=
>
> Ndd7LKvVBAaZIF0QAVi1ekCfAJXr1GGfLtRUXhgrF8c="}},"authorization":{"class":"solr.RuleBasedAuthorizationPlugin","user-role":{"solr":["admin"]},"permissions":[{"name":"security-edit","role":"admin"}]}}
>
> * Start solr:
> > solr-5.3.1/bin/solr start -e cloud -z localhost:2181
>
> You would need to key in a few things e.g. #nodes and ports, leave them at
> the default values of 2 nodes and 8983/7574, unless you want to run Solr on
> a different port. Then let it create a default collection to just make sure
> that everything works fine.
>
> * Add the collection-admin-edit command:
> > curl --user solr:SolrRocks
> http://localhost:8983/solr/admin/authorization
> -H 'Content-type:application/json' -d '{"set-permission" :
> {"name":"collection-admin-edit", "role":"admin"}}'
>
> At this point, everything should be working fine. Restarting the nodes
> should also work fine. You can try 2 things at this point:
> 1. Create a new collection with 1 sha
RE: Re:Re: Implementing security.json is breaking ADDREPLICA
)
at
org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:361)
at org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:227)
Caused by: org.apache.http.ParseException: Invalid content type:
at org.apache.http.entity.ContentType.parse(ContentType.java:273)
at
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:512)
at
org.apache.solr.client.solrj.impl.HttpSolrClient$1.call(HttpSolrClient.java:270)
at
org.apache.solr.client.solrj.impl.HttpSolrClient$1.call(HttpSolrClient.java:266)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:210)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
ERROR - 2015-11-20 22:48:41.318; [c:solr8326 s:shard1 r:core_node3
x:solr8326_shard1_replica1] org.apache.solr.cloud.RecoveryStrategy; Recovery
failed - trying again... (4)
INFO - 2015-11-20 22:48:41.318; [ ]
org.apache.solr.common.cloud.ZkStateReader; Updating data for solr8326 to ver
26
INFO - 2015-11-20 22:48:41.319; [c:solr8326 s:shard1 r:core_node3
x:solr8326_shard1_replica1] org.apache.solr.cloud.RecoveryStrategy; Wait 32.0
seconds before trying to recover again (5)
I would not be surprised if this were to be some unrelated issue (the symptoms
are quite different)
Thanks again
-Original Message-
From: Anshum Gupta [mailto:ans...@anshumgupta.net]
Sent: Friday, November 20, 2015 1:31 PM
To: solr-user@lucene.apache.org
Subject: Re: Re:Re: Implementing security.json is breaking ADDREPLICA
Collections API were available before November of 2014, if that is when you
took the class. However, it was only with Solr 5.0 (released in Feb 2015)
that the only supported mechanism to create a collection was restricted to
Collections API.
Here are the list of steps that you'd need to run to see that things are
fine for you without the read permission:
* Untar and setup Solr, don't start it yet
* Start clean zookeeper
* Put the security.json in zk, without anything other than a security-edit
permission. Find the content of the file below. Upload it using your own zk
client or through the solr script:
> solr-5.3.1/server/scripts/cloud-scripts/zkcli.sh -zkhost localhost:2181
-cmd putfile /security.json ~/security.json
security.json:
{"authentication":{"class":"solr.BasicAuthPlugin","credentials":{"solr":"IV0EHq1OnNrj6gvRCwvFwTrZ1+z1oBbnQdiVC3otuq0=
Ndd7LKvVBAaZIF0QAVi1ekCfAJXr1GGfLtRUXhgrF8c="}},"authorization":{"class":"solr.RuleBasedAuthorizationPlugin","user-role":{"solr":["admin"]},"permissions":[{"name":"security-edit","role":"admin"}]}}
* Start solr:
> solr-5.3.1/bin/solr start -e cloud -z localhost:2181
You would need to key in a few things e.g. #nodes and ports, leave them at
the default values of 2 nodes and 8983/7574, unless you want to run Solr on
a different port. Then let it create a default collection to just make sure
that everything works fine.
* Add the collection-admin-edit command:
> curl --user solr:SolrRocks http://localhost:8983/solr/admin/authorization
-H 'Content-type:application/json' -d '{"set-permission" :
{"name":"collection-admin-edit", "role":"admin"}}'
At this point, everything should be working fine. Restarting the nodes
should also work fine. You can try 2 things at this point:
1. Create a new collection with 1 shard and 1 replica and then try adding a
replica, here's how:
> curl --user solr:SolrRocks
http://localhost:8983/solr/admin/collections?action=CREATE=testcollection=gettingstarted=1
> curl --user solr:SolrRocks
http://localhost:8983/solr/admin/collections?action=ADDREPLICA=testcollection=shard1
This should work fine.
2. After this, try restarting the solr cluster. Here's how you can do so,
assuming you didn't change any of the defaults and you are running zk on
localhost:2181. If not, just change those values below:
> bin/solr stop -all
After this, check that Solr was actually stopped. I'd also suggest you tail
the logs on both nodes when they are coming up to see any errors, if any.
The logs would be here: example/cloud/node1/logs/solr.log
and example/cloud/node2/logs/solr.log
> bin/solr start -c -p 8983 -s "example/cloud/node1/solr" -z localhost:2181
> bin/solr start -c -p 7574 -s "example/cloud/node2/solr" -z localhost:2181
If you get to this checkpoint fine, try adding a read permission.
Add a permission:
> curl --user solr:SolrRocks http://localhost:8983/solr/admin/authorization
-H 'Conte
RE: Re:Re: Implementing security.json is breaking ADDREPLICA
Thanks
It seems to work when there is no security.json, so perhaps there's some typo
in the initial version.
I notice that the version you sent is different from the documentation at
cwiki.apache.org/confluence/display/solr/Authentication+and+Authorization+Plugins
in that the Wiki version has "permissions" before "user-role": I also notice
that (at least as of right this moment) the Wiki version has a comma at the end
of '"user-role":{"solr":"admin"},' even though it is at the end: and I notice
that the Wiki version seems to lack a comma between the "permissions" section
and the "user-role" section. I just now also noticed that the version you sent
has '"user-role":{"solr":["admin"]}' (with square brackets) whereas the Wiki
does not have square brackets.
The placement of the comma definitely looks wrong in the Wiki at the moment
(though perhaps someone might correct the Wiki before too long). Other than
that, I don’t know whether the order and/or the square brackets make a
difference. I can try with different permutations.
Thanks again
P.S. for the record, the Wiki currently has
{
"authentication":{
"class":"solr.BasicAuthPlugin",
"credentials":{"solr":"IV0EHq1OnNrj6gvRCwvFwTrZ1+z1oBbnQdiVC3otuq0=
Ndd7LKvVBAaZIF0QAVi1ekCfAJXr1GGfLtRUXhgrF8c="}
},
"authorization":{
"class":"solr.RuleBasedAuthorizationPlugin",
"permissions":[{"name":"security-edit",
"role":"admin"}]
"user-role":{"solr":"admin"},
}}
-Original Message-
From: Anshum Gupta [mailto:ans...@anshumgupta.net]
Sent: Friday, November 20, 2015 6:18 PM
To: solr-user@lucene.apache.org
Subject: Re: Re:Re: Implementing security.json is breaking ADDREPLICA
This seems unrelated and more like a user error somewhere. Can you just
follow the steps, without any security settings i.e. not even uploading
security.json and see if you still see this? Sorry, but I don't have access
to the code right now, I try and look at this later tonight.
On Fri, Nov 20, 2015 at 3:07 PM, Oakley, Craig (NIH/NLM/NCBI) [C] <
craig.oak...@nih.gov> wrote:
> Thank you for opening SOLR-8326
>
> As a side note, in the procedure you listed, even before adding the
> collection-admin-edit authorization, I'm already hitting trouble: stopping
> and restarting a node results in the following
>
> INFO - 2015-11-20 22:48:41.275; [c:solr8326 s:shard2 r:core_node4
> x:solr8326_shard2_replica1] org.apache.solr.cloud.RecoveryStrategy;
> Publishing state of core solr8326_shard2_replica1 as recovering, leader is
> http://{IP-address-redacted}:8983/solr/solr8326_shard2_replica2/ and I am
> http://{IP-address-redacted}:7574/solr/solr8326_shard2_replica1/
> INFO - 2015-11-20 22:48:41.275; [c:solr8326 s:shard2 r:core_node4
> x:solr8326_shard2_replica1] org.apache.solr.cloud.ZkController; publishing
> state=recovering
> INFO - 2015-11-20 22:48:41.278; [c:solr8326 s:shard1 r:core_node3
> x:solr8326_shard1_replica1] org.apache.solr.cloud.RecoveryStrategy;
> Publishing state of core solr8326_shard1_replica1 as recovering, leader is
> http://{IP-address-redacted}:8983/solr/solr8326_shard1_replica2/ and I am
> http://{IP-address-redacted}:7574/solr/solr8326_shard1_replica1/
> INFO - 2015-11-20 22:48:41.280; [c:solr8326 s:shard1 r:core_node3
> x:solr8326_shard1_replica1] org.apache.solr.cloud.ZkController; publishing
> state=recovering
> INFO - 2015-11-20 22:48:41.282; [c:solr8326 s:shard2 r:core_node4
> x:solr8326_shard2_replica1] org.apache.solr.cloud.RecoveryStrategy; Sending
> prep recovery command to http://{IP-address-redacted}:8983/solr;
> WaitForState:
> action=PREPRECOVERY=solr8326_shard2_replica2={IP-address-redacted}%3A7574_solr=core_node4=recovering=true=true=true
> INFO - 2015-11-20 22:48:41.289; [ ]
> org.apache.solr.common.cloud.ZkStateReader$8; A cluster state change:
> WatchedEvent state:SyncConnected type:NodeDataChanged
> path:/collections/solr8326/state.json for collection solr8326 has occurred
> - updating... (live nodes size: 2)
> INFO - 2015-11-20 22:48:41.290; [c:solr8326 s:shard1 r:core_node3
> x:solr8326_shard1_replica1] org.apache.solr.cloud.RecoveryStrategy; Sending
> prep recovery command to http://{IP-address-redacted}:8983/solr;
> WaitForState:
> action=PREPRECOVERY=solr8326_shard1_replica2={IP-address-redacted}%3A7574_solr=core_node3=recovering=true=true=true
> INFO - 2015-11-20 22:48:41.291; [ ]
> org.apache.solr.common.cloud.ZkStateReader; Updating data for solr8326 to
> ver 25
> ERROR - 2015-11-20 22:48:41.298; [c:solr8326 s:shard2 r:core_node4
> x:solr8326_shard2_replica1] org.apac
RE: Re:Re: Implementing security.json is breaking ADDREPLICA
I note that the thread called "Security Problems" (most recent post by Nobel Paul) seems like it may help with much of what I'm trying to do. I will see to what extent that may help.
Re: Re:Re: Implementing security.json is breaking ADDREPLICA
I'll try out what you did later in the day, as soon as I get time but why
exactly are you creating cores manually? Seems like you manually create a
core and the try to add a replica. Can you try using the Collections API to
create a collection?
Starting Solr 5.0, the only supported way to create a new collection is via
the Collections API. Creating a core would lead to a collection creation
but that's not really supported. It was just something that was done when
there were no Collections API.
On Thu, Nov 19, 2015 at 12:36 PM, Oakley, Craig (NIH/NLM/NCBI) [C] <
craig.oak...@nih.gov> wrote:
> I tried again with the following security.json, but the results were the
> same:
>
> {
> "authentication":{
> "class":"solr.BasicAuthPlugin",
> "credentials":{
> "solr":"IV0EHq1OnNrj6gvRCwvFwTrZ1+z1oBbnQdiVC3otuq0=
> Ndd7LKvVBAaZIF0QAVi1ekCfAJXr1GGfLtRUXhgrF8c=",
> "solruser":"VgZX1TAMNHT2IJikoGdKtxQdXc+MbNwfqzf89YqcLEE=
> 37pPWQ9v4gciIKHuTmFmN0Rv66rnlMOFEWfEy9qjJfY="},
> "":{"v":9}},
> "authorization":{
> "class":"solr.RuleBasedAuthorizationPlugin",
> "user-role":{
> "solr":[
> "admin",
> "read",
> "xmpladmin",
> "xmplgen",
> "xmplsel"],
> "solruser":[
> "read",
> "xmplgen",
> "xmplsel"]},
> "permissions":[
> {
> "name":"security-edit",
> "role":"admin"},
> {
> "name":"xmpl_admin",
> "collection":"xmpl",
> "path":"/admin/*",
> "role":"xmpladmin"},
> {
> "name":"xmpl_sel",
> "collection":"xmpl",
> "path":"/select/*",
> "role":null},
> {
> "name":"all-admin",
> "collection":null,
> "path":"/*",
> "role":"xmplgen"},
> {
> "name":"all-core-handlers",
> "path":"/*",
> "role":"xmplgen"}],
> "":{"v":42}}}
>
> -Original Message-
> From: Oakley, Craig (NIH/NLM/NCBI) [C]
> Sent: Thursday, November 19, 2015 1:46 PM
> To: 'solr-user@lucene.apache.org' <solr-user@lucene.apache.org>
> Subject: RE: Re:Re: Implementing security.json is breaking ADDREPLICA
>
> I note that the thread called "Security Problems" (most recent post by
> Nobel Paul) seems like it may help with much of what I'm trying to do. I
> will see to what extent that may help.
>
--
Anshum Gupta
RE: Re:Re: Implementing security.json is breaking ADDREPLICA
I tried again with the following security.json, but the results were the same:
{
"authentication":{
"class":"solr.BasicAuthPlugin",
"credentials":{
"solr":"IV0EHq1OnNrj6gvRCwvFwTrZ1+z1oBbnQdiVC3otuq0=
Ndd7LKvVBAaZIF0QAVi1ekCfAJXr1GGfLtRUXhgrF8c=",
"solruser":"VgZX1TAMNHT2IJikoGdKtxQdXc+MbNwfqzf89YqcLEE=
37pPWQ9v4gciIKHuTmFmN0Rv66rnlMOFEWfEy9qjJfY="},
"":{"v":9}},
"authorization":{
"class":"solr.RuleBasedAuthorizationPlugin",
"user-role":{
"solr":[
"admin",
"read",
"xmpladmin",
"xmplgen",
"xmplsel"],
"solruser":[
"read",
"xmplgen",
"xmplsel"]},
"permissions":[
{
"name":"security-edit",
"role":"admin"},
{
"name":"xmpl_admin",
"collection":"xmpl",
"path":"/admin/*",
"role":"xmpladmin"},
{
"name":"xmpl_sel",
"collection":"xmpl",
"path":"/select/*",
"role":null},
{
"name":"all-admin",
"collection":null,
"path":"/*",
"role":"xmplgen"},
{
"name":"all-core-handlers",
"path":"/*",
"role":"xmplgen"}],
"":{"v":42}}}
-Original Message-
From: Oakley, Craig (NIH/NLM/NCBI) [C]
Sent: Thursday, November 19, 2015 1:46 PM
To: 'solr-user@lucene.apache.org' <solr-user@lucene.apache.org>
Subject: RE: Re:Re: Implementing security.json is breaking ADDREPLICA
I note that the thread called "Security Problems" (most recent post by Nobel
Paul) seems like it may help with much of what I'm trying to do. I will see to
what extent that may help.
RE: Re:Re: Implementing security.json is breaking ADDREPLICA
Thank you for the reply.
What we are attempting is to require a password for practically everything, so
that even were a hacker to get within the firewall, they would have limited
access to the various services (the Security team even complained that, for
Solr 4.5 servers, attempts to access host:port (without "/solr") resulted in an
error message that gave the full pathname to solr.war)
I am sending the solr.log files directly to Anshum, so as not to clutter up the
Email list.
The steps I used to recreate the problem are as follows:
cd zookeeper-3.4.6/conf/
sed 's/2181/4545/' zoo_sample.cfg | tee zoo_sample4545.cfg
cd ../bin
./zkServer.sh start zoo_sample4545.cfg
cd ../../solr-5.3.1/server/solr
mkdir xmpl
echo 'name=xmpl' | tee xmpl/core.properties
mkdir xmpl/data
mkdir xmpl/data/index
mkdir xmpl/data/tlog
cd ../scripts/cloud-scripts/
./zkcli.sh --zkhost localhost:4545 -cmd makepath /solr
./zkcli.sh --zkhost localhost:4545 -cmd makepath /solr/xmpl
./zkcli.sh --zkhost localhost:4545/solr/xmpl -cmd upconfig -confdir
../../solr/configsets/basic_configs/conf -confname xmpl
mkdir ../../example/solr
cp solr.xml ../../example/solr
./zkcli.sh --zkhost localhost:4545/solr/xmpl -cmd putfile /security.json
~/solr/security151117a.json
cd ../../../bin
mkdir ../example/solr/pid
./solr -c -p 4575 -d ~dbman/solr/straight531outofbox/solr-5.3.1/server/ -z
localhost:4545/solr/xmpl -s
~dbman/solr/straight531outofbox/solr-5.3.1/example/solr
./solr -c -p 4565 -d ~dbman/solr/straight531outofbox/solr-5.3.1/server/ -z
localhost:4545/solr/xmpl -s
~dbman/solr/straight531outofbox/solr-5.3.1/server/solr
curl -u solr:SolrRocks 'http://
{IP-address-redacted}:4575/solr/admin/collections?action=ADDREPLICA=xmpl=shard1={IP-address-redacted}:4575_solr=json=true'
The contents of security151117a.json is included in the original post
If there is a better way using the Well Known Permissions as described at
lucidworks.com/blog/2015/08/17/securing-solr-basic-auth-permission-rules, I'm
open to trying that.
I would like to acknowledge that there definitely seem to be some IMPROVEMENTS
in the security.json implementation: particularly in terms of Core Admin (using
jetty-implemented Authentication in webdefault.xml, anyone who could get into
the GUI front page could rename cores, unless prevented by OS-level permissions
on core.properties).
Thanks again
Re:Re: Implementing security.json is breaking ADDREPLICA
Hi Anshum,
I encounter the same problem after I config my security.json like this:
{ "authentication":{
"class":"solr.BasicAuthPlugin",
"credentials":{"solr":"IV0EHq1OnNrj6gvRCwvFwTrZ1+z1oBbnQdiVC3otuq0=
Ndd7LKvVBAaZIF0QAVi1ekCfAJXr1GGfLtRUXhgrF8c="}},
"authorization":{
"class":"solr.RuleBasedAuthorizationPlugin",
"user-role":{"solr":"admin"},
"permissions":[
{ "name":"security-edit",
"role":"admin"},
{ "name":"read",
"role":"admin"},
{ "name":"update",
"role":"admin"}],
"":{"v":3}}}
I just want to restrict strangers to do select/create/update operations on my
collections and configs like schema.xml or other things related to solr itself
from both Admin UI and sorj. But it is useless and error occurs like this:
ERROR (RecoveryThread-gettingstarted_shard2_replica2) [c:gettingstarted
s:shard2 r:core_node2 x:gettingstarted_shard2_replica2]
o.a.s.c.RecoveryStrategy Error while trying to
recover:org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException:
Error from server at http://(myhost):7574/solr/gettingstarted_shard2_replica1:
Expected mime type application/octet-stream but got text/html.
Error 401 Unauthorized request, Response code: 401
HTTP ERROR 401
Problem accessing /solr/gettingstarted_shard2_replica1/update. Reason:
Unauthorized request, Response code:
401Powered by Jetty://
at
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:528)
at
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:234)
at
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:226)
at
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:135)
at
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:152)
at
org.apache.solr.cloud.RecoveryStrategy.commitOnLeader(RecoveryStrategy.java:207)
at
org.apache.solr.cloud.RecoveryStrategy.replicate(RecoveryStrategy.java:147)
at
org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:437)
at
org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:227)
Error while trying to
recover:org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException:
Error from server at http://(myhost):7574/solr/gettingstarted_shard2_replica1:
Expected mime type application/octet-stream but got text/html.
Kind regards
Byzen Ma
At 2015-11-19 13:46:08, "Anshum Gupta" <ans...@anshumgupta.net> wrote:
>Hi Craig,
>
>Just to be sure that you're using the feature as it should be used, can you
>outline what is it that you're trying to do here? There are a few things
>that aren't clear to me here, e.g. I see permissions for the /admin handler
>for a particular collection.
>
>What are the kind of permissions you're trying to set up.
>
>Solr uses it's internal PKI based mechanism for inter-shard communication
>and so you shouldn't really be hitting this. Can you check your logs and
>tell me if there are any other exceptions you see while bringing the node
>up etc. ? Something from PKI itself.
>
>About restricting the UI, there's another thread in parallel that's been
>discussing exactly that. The thing with the current UI implementation is
>that it bypasses all of this, primarily because most of that content is
>static. I am not saying we should be able to put it behind the
>authentication layer, but just that it's not currently supported through
>this plugin.
>
>On Wed, Nov 18, 2015 at 11:20 AM, Oakley, Craig (NIH/NLM/NCBI) [C] <
>craig.oak...@nih.gov> wrote:
>
>> Implementing security.json is breaking ADDREPLICA
>>
>> I have been able to reproduce this issue with minimal changes from an
>> out-of-the-box Zookeeper (3.4.6) and Solr (5.3.1): loading
>> configsets/basic_configs/conf into Zookeeper, creating the security.json
>> listed below, creating two nodes (one with a core named xmpl and one
>> without any core)- I can provide details if helpful.
>>
>> The security.json is as follows:
>>
>> {
>> "authentication":{
>> "class":"solr.BasicAuthPlugin",
>> "credentials":{
>> "solr":"IV0EHq1OnNrj6gvRCwvFwTrZ1+z1oBbnQdiVC3otuq0=
>> Ndd7LKvVBAaZIF0QAVi1ekCfAJXr1GGfLtRUXhgrF8c=",
>> "solruser":"VgZX1TAMNHT2IJikoGdKtxQdXc+MbNwfqzf89YqcLEE=
>> 37pPWQ9v4gciIKHuTmFmN0Rv66rnlMOFEWfEy9qjJfY="},
>> "":{"v":9}},
>
Implementing security.json is breaking ADDREPLICA
Implementing security.json is breaking ADDREPLICA
I have been able to reproduce this issue with minimal changes from an
out-of-the-box Zookeeper (3.4.6) and Solr (5.3.1): loading
configsets/basic_configs/conf into Zookeeper, creating the security.json listed
below, creating two nodes (one with a core named xmpl and one without any
core)- I can provide details if helpful.
The security.json is as follows:
{
"authentication":{
"class":"solr.BasicAuthPlugin",
"credentials":{
"solr":"IV0EHq1OnNrj6gvRCwvFwTrZ1+z1oBbnQdiVC3otuq0=
Ndd7LKvVBAaZIF0QAVi1ekCfAJXr1GGfLtRUXhgrF8c=",
"solruser":"VgZX1TAMNHT2IJikoGdKtxQdXc+MbNwfqzf89YqcLEE=
37pPWQ9v4gciIKHuTmFmN0Rv66rnlMOFEWfEy9qjJfY="},
"":{"v":9}},
"authorization":{
"class":"solr.RuleBasedAuthorizationPlugin",
"user-role":{
"solr":[
"admin",
"read",
"xmpladmin",
"xmplgen",
"xmplsel"],
"solruser":[
"read",
"xmplgen",
"xmplsel"]},
"permissions":[
{
"name":"security-edit",
"role":"admin"},
{
"name":"xmpl_admin",
"collection":"xmpl",
"path":"/admin/*",
"role":"xmpladmin"},
{
"name":"xmpl_sel",
"collection":"xmpl",
"path":"/select/*",
"role":null},
{
"name":"xmpl_gen",
"collection":"xmpl",
"path":"/*",
"role":"xmplgen"}],
"":{"v":42}}}
When I then execute admin/collections?action=ADDREPLICA, I get errors such as
the following in the solr.log of the node which was created without a core.
INFO - 2015-11-17 21:03:54.157; [c:xmpl s:shard1 r:core_node2
x:xmpl_shard1_replica1] org.apache.solr.cloud.RecoveryStrategy; Starting
Replication Recovery.
INFO - 2015-11-17 21:03:54.158; [c:xmpl s:shard1 r:core_node2
x:xmpl_shard1_replica1] org.apache.solr.cloud.RecoveryStrategy; Begin buffering
updates.
INFO - 2015-11-17 21:03:54.158; [c:xmpl s:shard1 r:core_node2
x:xmpl_shard1_replica1] org.apache.solr.update.UpdateLog; Starting to buffer
updates. FSUpdateLog{state=ACTIVE, tlog=null}
INFO - 2015-11-17 21:03:54.159; [c:xmpl s:shard1 r:core_node2
x:xmpl_shard1_replica1] org.apache.solr.cloud.RecoveryStrategy; Attempting to
replicate from http://{IP-address-redacted}:4565/solr/xmpl/.
ERROR - 2015-11-17 21:03:54.166; [c:xmpl s:shard1 r:core_node2
x:xmpl_shard1_replica1] org.apache.solr.common.SolrException; Error while
trying to
recover:org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException:
Error from server at http://{IP-address-redacted}:4565/solr/xmpl: Expected mime
type application/octet-stream but got text/html.
Error 401 Unauthorized request, Response code: 401
HTTP ERROR 401
Problem accessing /solr/xmpl/update. Reason:
Unauthorized request, Response code:
401Powered by Jetty://
at
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:528)
at
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:234)
at
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:226)
at
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:135)
at
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:152)
at
org.apache.solr.cloud.RecoveryStrategy.commitOnLeader(RecoveryStrategy.java:207)
at
org.apache.solr.cloud.RecoveryStrategy.replicate(RecoveryStrategy.java:147)
at
org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:437)
at org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:227)
INFO - 2015-11-17 21:03:54.166; [c:xmpl s:shard1 r:core_node2
x:xmpl_shard1_replica1] org.apache.solr.update.UpdateLog; Dropping buffered
updates FSUpdateLog{state=BUFFERING, tlog=null}
ERROR - 2015-11-17 21:03:54.166; [c:xmpl s:shard1 r:core_node2
x:xmpl_shard1_replica1] org.apache.solr.cloud.RecoveryStrategy; Recovery failed
- trying again... (2)
INFO - 2015-11-17 21:03:54.166; [c:xmpl s:shard1 r:core_node2
x:xmpl_shard1_replica1] org.apache.solr.cloud.RecoveryStrategy; Wait 8.0
seconds before trying to recover again (3)
And (after modifying Logging Levels), the solr.log of the node which already
had a core gets errors such as the following:
2015-11-17 21:03:50.743 DEBUG (qtp59559151-87) [ ] o.e.j.s.Server REQUEST GET
/solr/tpl/cloud.html on
HttpChannelO
Re: Implementing security.json is breaking ADDREPLICA
Hi Craig,
Just to be sure that you're using the feature as it should be used, can you
outline what is it that you're trying to do here? There are a few things
that aren't clear to me here, e.g. I see permissions for the /admin handler
for a particular collection.
What are the kind of permissions you're trying to set up.
Solr uses it's internal PKI based mechanism for inter-shard communication
and so you shouldn't really be hitting this. Can you check your logs and
tell me if there are any other exceptions you see while bringing the node
up etc. ? Something from PKI itself.
About restricting the UI, there's another thread in parallel that's been
discussing exactly that. The thing with the current UI implementation is
that it bypasses all of this, primarily because most of that content is
static. I am not saying we should be able to put it behind the
authentication layer, but just that it's not currently supported through
this plugin.
On Wed, Nov 18, 2015 at 11:20 AM, Oakley, Craig (NIH/NLM/NCBI) [C] <
craig.oak...@nih.gov> wrote:
> Implementing security.json is breaking ADDREPLICA
>
> I have been able to reproduce this issue with minimal changes from an
> out-of-the-box Zookeeper (3.4.6) and Solr (5.3.1): loading
> configsets/basic_configs/conf into Zookeeper, creating the security.json
> listed below, creating two nodes (one with a core named xmpl and one
> without any core)- I can provide details if helpful.
>
> The security.json is as follows:
>
> {
> "authentication":{
> "class":"solr.BasicAuthPlugin",
> "credentials":{
> "solr":"IV0EHq1OnNrj6gvRCwvFwTrZ1+z1oBbnQdiVC3otuq0=
> Ndd7LKvVBAaZIF0QAVi1ekCfAJXr1GGfLtRUXhgrF8c=",
> "solruser":"VgZX1TAMNHT2IJikoGdKtxQdXc+MbNwfqzf89YqcLEE=
> 37pPWQ9v4gciIKHuTmFmN0Rv66rnlMOFEWfEy9qjJfY="},
> "":{"v":9}},
> "authorization":{
> "class":"solr.RuleBasedAuthorizationPlugin",
> "user-role":{
> "solr":[
> "admin",
> "read",
> "xmpladmin",
> "xmplgen",
> "xmplsel"],
> "solruser":[
> "read",
> "xmplgen",
> "xmplsel"]},
> "permissions":[
> {
> "name":"security-edit",
> "role":"admin"},
> {
> "name":"xmpl_admin",
> "collection":"xmpl",
> "path":"/admin/*",
> "role":"xmpladmin"},
> {
> "name":"xmpl_sel",
> "collection":"xmpl",
> "path":"/select/*",
> "role":null},
> {
> "name":"xmpl_gen",
> "collection":"xmpl",
> "path":"/*",
> "role":"xmplgen"}],
> "":{"v":42}}}
>
>
>
>
>
> When I then execute admin/collections?action=ADDREPLICA, I get errors such
> as the following in the solr.log of the node which was created without a
> core.
>
> INFO - 2015-11-17 21:03:54.157; [c:xmpl s:shard1 r:core_node2
> x:xmpl_shard1_replica1] org.apache.solr.cloud.RecoveryStrategy; Starting
> Replication Recovery.
> INFO - 2015-11-17 21:03:54.158; [c:xmpl s:shard1 r:core_node2
> x:xmpl_shard1_replica1] org.apache.solr.cloud.RecoveryStrategy; Begin
> buffering updates.
> INFO - 2015-11-17 21:03:54.158; [c:xmpl s:shard1 r:core_node2
> x:xmpl_shard1_replica1] org.apache.solr.update.UpdateLog; Starting to
> buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
> INFO - 2015-11-17 21:03:54.159; [c:xmpl s:shard1 r:core_node2
> x:xmpl_shard1_replica1] org.apache.solr.cloud.RecoveryStrategy; Attempting
> to replicate from http://{IP-address-redacted}:4565/solr/xmpl/.
> ERROR - 2015-11-17 21:03:54.166; [c:xmpl s:shard1 r:core_node2
> x:xmpl_shard1_replica1] org.apache.solr.common.SolrException; Error while
> trying to
> recover:org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException:
> Error from server at http://{IP-address-redacted}:4565/solr/xmpl:
> Expected mime type application/octet-stream but got text/html.
>
>
> Error 401 Unauthorized request, Response code: 401
>
> HTTP ERROR 401
> Problem accessing /solr/xmpl/update. Reason:
> Unauthorized request, Response code:
> 401Powered by Jetty://
>
>
>
>
> at
> org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:5
