Re: CloudSolrClient Connect To Zookeeper with ACL Protected files

[Kevin Lee]

Thanks Alan!

That works!  I was looking for a programatic way to do it, but this will work 
for now as it doesn’t seem to be supported.

- Kevin

> On Nov 18, 2015, at 1:24 AM, Alan Woodward <a...@flax.co.uk> wrote:
> 
> At the moment it seems that it's only settable via System properties - see 
> https://cwiki.apache.org/confluence/display/solr/ZooKeeper+Access+Control.  
> But it would be nice to do this programmatically as well, maybe worth opening 
> a JIRA ticket?
> 
> Alan Woodward
> www.flax.co.uk
> 
> 
> On 17 Nov 2015, at 16:44, Kevin Lee wrote:
> 
>> Does anyone know if it is possible to set the ACL credentials in 
>> CloudSolrClient needed to access a protected resource in Zookeeper?
>> 
>> Thanks!
>> 
>>> On Nov 13, 2015, at 1:20 PM, Kevin Lee <kgle...@yahoo.com.INVALID> wrote:
>>> 
>>> Hi,
>>> 
>>> Is there a way to use CloudSolrClient and connect to a Zookeeper instance 
>>> where ACL is enabled and resources/files like /live_nodes, etc are ACL 
>>> protected?  Couldn’t find a way to set the ACL credentials.
>>> 
>>> Thanks,
>>> Kevin
>> 
> 

Re: CloudSolrClient Connect To Zookeeper with ACL Protected files

[Kevin Lee]

Does anyone know if it is possible to set the ACL credentials in 
CloudSolrClient needed to access a protected resource in Zookeeper?

Thanks!

> On Nov 13, 2015, at 1:20 PM, Kevin Lee <kgle...@yahoo.com.INVALID> wrote:
> 
> Hi,
> 
> Is there a way to use CloudSolrClient and connect to a Zookeeper instance 
> where ACL is enabled and resources/files like /live_nodes, etc are ACL 
> protected?  Couldn’t find a way to set the ACL credentials.
> 
> Thanks,
> Kevin

CloudSolrClient Connect To Zookeeper with ACL Protected files

[Kevin Lee]

Hi,

Is there a way to use CloudSolrClient and connect to a Zookeeper instance where 
ACL is enabled and resources/files like /live_nodes, etc are ACL protected?  
Couldn’t find a way to set the ACL credentials.

Thanks,
Kevin

Re: Issue Using Solr 5.3 Authentication and Authorization Plugins

[Kevin Lee]

Thanks Dan!  Please let us know what you find.  I’m interested to know if this 
is an issue with anyone else’s setup or if I have an issue in my local 
configuration that is still preventing it to work on start/restart.

- Kevin

> On Sep 5, 2015, at 8:45 AM, Dan Davis <dansm...@gmail.com> wrote:
> 
> Kevin & Noble,
> 
> I'll take it on to test this.   I've built from source before, and I've
> wanted this authorization capability for awhile.
> 
> On Fri, Sep 4, 2015 at 9:59 AM, Kevin Lee <kgle...@yahoo.com.invalid> wrote:
> 
>> Noble,
>> 
>> Does SOLR-8000 need to be re-opened?  Has anyone else been able to test
>> the restart fix?
>> 
>> At startup, these are the log messages that say there is no security
>> configuration and the plugins aren’t being used even though security.json
>> is in Zookeeper:
>> 2015-09-04 08:06:21.205 INFO  (main) [   ] o.a.s.c.CoreContainer Security
>> conf doesn't exist. Skipping setup for authorization module.
>> 2015-09-04 08:06:21.205 INFO  (main) [   ] o.a.s.c.CoreContainer No
>> authentication plugin used.
>> 
>> Thanks,
>> Kevin
>> 
>>> On Sep 4, 2015, at 5:47 AM, Noble Paul <noble.p...@gmail.com> wrote:
>>> 
>>> There are no download links for 5.3.x branch  till we do a bug fix
>> release
>>> 
>>> If you wish to download the trunk nightly (which is not same as 5.3.0)
>>> check here
>> https://builds.apache.org/job/Solr-Artifacts-trunk/lastSuccessfulBuild/artifact/solr/package/
>>> 
>>> If you wish to get the binaries for 5.3 branch you will have to make it
>>> (you will need to install svn and ant)
>>> 
>>> Here are the steps
>>> 
>>> svn checkout
>> http://svn.apache.org/repos/asf/lucene/dev/branches/lucene_solr_5_3/
>>> cd lucene_solr_5_3/solr
>>> ant server
>>> 
>>> 
>>> 
>>> On Fri, Sep 4, 2015 at 4:11 PM, davidphilip cherian
>>> <davidphilipcher...@gmail.com> wrote:
>>>> Hi Kevin/Noble,
>>>> 
>>>> What is the download link to take the latest? What are the steps to
>> compile
>>>> it, test and use?
>>>> We also have a use case to have this feature in solr too. Therefore,
>> wanted
>>>> to test and above info would help a lot to get started.
>>>> 
>>>> Thanks.
>>>> 
>>>> 
>>>> On Fri, Sep 4, 2015 at 1:45 PM, Kevin Lee <kgle...@yahoo.com.invalid>
>> wrote:
>>>> 
>>>>> Thanks, I downloaded the source and compiled it and replaced the jar
>> file
>>>>> in the dist and solr-webapp’s WEB-INF/lib directory.  It does seem to
>> be
>>>>> protecting the Collections API reload command now as long as I upload
>> the
>>>>> security.json after startup of the Solr instances.  If I shutdown and
>> bring
>>>>> the instances back up, the security is no longer in place and I have to
>>>>> upload the security.json again for it to take effect.
>>>>> 
>>>>> - Kevin
>>>>> 
>>>>>> On Sep 3, 2015, at 10:29 PM, Noble Paul <noble.p...@gmail.com> wrote:
>>>>>> 
>>>>>> Both these are committed. If you could test with the latest 5.3 branch
>>>>>> it would be helpful
>>>>>> 
>>>>>> On Wed, Sep 2, 2015 at 5:11 PM, Noble Paul <noble.p...@gmail.com>
>> wrote:
>>>>>>> I opened a ticket for the same
>>>>>>> https://issues.apache.org/jira/browse/SOLR-8004
>>>>>>> 
>>>>>>> On Wed, Sep 2, 2015 at 1:36 PM, Kevin Lee <kgle...@yahoo.com.invalid
>>> 
>>>>> wrote:
>>>>>>>> I’ve found that completely exiting Chrome or Firefox and opening it
>>>>> back up re-prompts for credentials when they are required.  It was
>>>>> re-prompting with the /browse path where authentication was working
>> each
>>>>> time I completely exited and started the browser again, however it
>> won’t
>>>>> re-prompt unless you exit completely and close all running instances
>> so I
>>>>> closed all instances each time to test.
>>>>>>>> 
>>>>>>>> However, to make sure I ran it via the command line via curl as
>>>>> suggested and it still does not give any authentication error when
>> trying
>>>>> to issue the command via curl.  I get a success re

Re: Issue Using Solr 5.3 Authentication and Authorization Plugins

[Kevin Lee]

Noble,

Does SOLR-8000 need to be re-opened?  Has anyone else been able to test the 
restart fix?  

At startup, these are the log messages that say there is no security 
configuration and the plugins aren’t being used even though security.json is in 
Zookeeper:
2015-09-04 08:06:21.205 INFO  (main) [   ] o.a.s.c.CoreContainer Security conf 
doesn't exist. Skipping setup for authorization module.
2015-09-04 08:06:21.205 INFO  (main) [   ] o.a.s.c.CoreContainer No 
authentication plugin used.

Thanks,
Kevin

> On Sep 4, 2015, at 5:47 AM, Noble Paul <noble.p...@gmail.com> wrote:
> 
> There are no download links for 5.3.x branch  till we do a bug fix release
> 
> If you wish to download the trunk nightly (which is not same as 5.3.0)
> check here 
> https://builds.apache.org/job/Solr-Artifacts-trunk/lastSuccessfulBuild/artifact/solr/package/
> 
> If you wish to get the binaries for 5.3 branch you will have to make it
> (you will need to install svn and ant)
> 
> Here are the steps
> 
> svn checkout 
> http://svn.apache.org/repos/asf/lucene/dev/branches/lucene_solr_5_3/
> cd lucene_solr_5_3/solr
> ant server
> 
> 
> 
> On Fri, Sep 4, 2015 at 4:11 PM, davidphilip cherian
> <davidphilipcher...@gmail.com> wrote:
>> Hi Kevin/Noble,
>> 
>> What is the download link to take the latest? What are the steps to compile
>> it, test and use?
>> We also have a use case to have this feature in solr too. Therefore, wanted
>> to test and above info would help a lot to get started.
>> 
>> Thanks.
>> 
>> 
>> On Fri, Sep 4, 2015 at 1:45 PM, Kevin Lee <kgle...@yahoo.com.invalid> wrote:
>> 
>>> Thanks, I downloaded the source and compiled it and replaced the jar file
>>> in the dist and solr-webapp’s WEB-INF/lib directory.  It does seem to be
>>> protecting the Collections API reload command now as long as I upload the
>>> security.json after startup of the Solr instances.  If I shutdown and bring
>>> the instances back up, the security is no longer in place and I have to
>>> upload the security.json again for it to take effect.
>>> 
>>> - Kevin
>>> 
>>>> On Sep 3, 2015, at 10:29 PM, Noble Paul <noble.p...@gmail.com> wrote:
>>>> 
>>>> Both these are committed. If you could test with the latest 5.3 branch
>>>> it would be helpful
>>>> 
>>>> On Wed, Sep 2, 2015 at 5:11 PM, Noble Paul <noble.p...@gmail.com> wrote:
>>>>> I opened a ticket for the same
>>>>> https://issues.apache.org/jira/browse/SOLR-8004
>>>>> 
>>>>> On Wed, Sep 2, 2015 at 1:36 PM, Kevin Lee <kgle...@yahoo.com.invalid>
>>> wrote:
>>>>>> I’ve found that completely exiting Chrome or Firefox and opening it
>>> back up re-prompts for credentials when they are required.  It was
>>> re-prompting with the /browse path where authentication was working each
>>> time I completely exited and started the browser again, however it won’t
>>> re-prompt unless you exit completely and close all running instances so I
>>> closed all instances each time to test.
>>>>>> 
>>>>>> However, to make sure I ran it via the command line via curl as
>>> suggested and it still does not give any authentication error when trying
>>> to issue the command via curl.  I get a success response from all the Solr
>>> instances that the reload was successful.
>>>>>> 
>>>>>> Not sure why the pre-canned permissions aren’t working, but the one to
>>> the request handler at the /browse path is.
>>>>>> 
>>>>>> 
>>>>>>> On Sep 1, 2015, at 11:03 PM, Noble Paul <noble.p...@gmail.com> wrote:
>>>>>>> 
>>>>>>> " However, after uploading the new security.json and restarting the
>>>>>>> web browser,"
>>>>>>> 
>>>>>>> The browser remembers your login , So it is unlikely to prompt for the
>>>>>>> credentials again.
>>>>>>> 
>>>>>>> Why don't you try the RELOAD operation using command line (curl) ?
>>>>>>> 
>>>>>>> On Tue, Sep 1, 2015 at 10:31 PM, Kevin Lee <kgle...@yahoo.com.invalid>
>>> wrote:
>>>>>>>> The restart issues aside, I’m trying to lockdown usage of the
>>> Collections API, but that also does not seem to be working either.
>>>>>>>> 
>>

Re: Config error mystery

[Kevin Lee]

Are you using a single instance or cloud?  What version of Solr are you using?  
In your solrconfig.xml is the path to where you copied your library specified 
in a  tag?  Do you have a jar file for the Postgres JDBC driver in your 
lib directory as well?

For simple setup in 5.x I copy the jars to a lib directory under the 
core/collection directory.  For example, under server/solr//lib I 
would have the solr-dataimporthandler-.jar and the jdbc driver jar 
file.  These should be automatically picked up without having to add anything 
to the solrconfig.xml in terms of  tags.

For a production cloud deployment, I create a lib directory outside of the 
core/collection directory somewhere else on the file system so that it is easy 
to install without having to wait for a directory to be created by the 
Collections CREATE command and add the appropriate entry to the solrconfig.xml. 
 Then stick both jars in that directory.

Your error may be different, but as long as I have both jars in one of the two 
places mentioned above with the appropriate entry in solrconfig.xml if needed, 
then it has been working in my setups.

- Kevin


> On Sep 4, 2015, at 9:40 AM, Mark Fenbers  wrote:
> 
> Greetings,
> 
> I'm moving on from the tutorials and trying to setup an index for my own data 
> (from a database).  All I did was add the following to the solrconfig.xml 
> (taken verbatim from the example in Solr documentation, except for the 
> name="config" pathname) and I get an error in the web-based UI.
> 
>   class="org.apache.solr.handler.dataimport.DataImportHandler" >
>
>/localapps/dev/EventLog/data-config.xml
>
>  
> 
> Because of this error, no /dataimport page is available in the Admin user 
> interface; therefore, I cannot visit the page 
> http://localhost:8983/solr/dataimport.  The actual error is:
> 
> org.apache.solr.common.SolrException: Error Instantiating requestHandler, 
> org.apache.solr.handler.dataimport.DataImportHandler failed to instantiate 
> org.apache.solr.request.SolrRequestHandler
>at org.apache.solr.core.SolrCore.(SolrCore.java:820)
>at org.apache.solr.core.SolrCore.(SolrCore.java:659)
>at org.apache.solr.core.CoreContainer.create(CoreContainer.java:727)
>at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:447)
>at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:438)
>at java.util.concurrent.FutureTask.run(FutureTask.java:262)
>at 
> org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:210)
>at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>at java.lang.Thread.run(Thread.java:745)
> Caused by: org.apache.solr.common.SolrException: Error Instantiating 
> requestHandler, org.apache.solr.handler.dataimport.DataImportHandler failed 
> to instantiate org.apache.solr.request.SolrRequestHandler
>at org.apache.solr.core.SolrCore.createInstance(SolrCore.java:588)
>at org.apache.solr.core.PluginBag.createPlugin(PluginBag.java:122)
>at org.apache.solr.core.PluginBag.init(PluginBag.java:217)
>at 
> org.apache.solr.core.RequestHandlers.initHandlersFromConfig(RequestHandlers.java:130)
>at org.apache.solr.core.SolrCore.(SolrCore.java:773)
>... 9 more
> Caused by: java.lang.ClassCastException: class 
> org.apache.solr.handler.dataimport.DataImportHandler
>at java.lang.Class.asSubclass(Class.java:3208)
>at 
> org.apache.solr.core.SolrResourceLoader.findClass(SolrResourceLoader.java:475)
>at 
> org.apache.solr.core.SolrResourceLoader.findClass(SolrResourceLoader.java:422)
>at org.apache.solr.core.SolrCore.createInstance(SolrCore.java:567)
>... 13 more
> 
> 
> If I remove the  section and restart Solr, the error goes 
> away.  As best I can tell, the contents of
> /localapps/dev/EventLog/data-config.xml look fine, too.  See it here:
> 
> 
> url="jdbc:postgresql://dx1f/OHRFC" user="awips" />
>
>deltaQuery="SELECT posttime FROM eventlogtext WHERE 
> lastmodtime > '${dataimporter.last_index_time}'">
>
>
>
>
> 
> 
> It seems to me that this problem could be a classpath issue, but I copied the 
> appropriate jar file into the solr/lib directory to be sure.  This made the 
> (slightly different) initial error go away, but now I cannot make this one go 
> away.
> 
> Any ideas?
> 
> Mark
> 
> 
> 

Re: Issue Using Solr 5.3 Authentication and Authorization Plugins

[Kevin Lee]

Thanks, I downloaded the source and compiled it and replaced the jar file in 
the dist and solr-webapp’s WEB-INF/lib directory.  It does seem to be 
protecting the Collections API reload command now as long as I upload the 
security.json after startup of the Solr instances.  If I shutdown and bring the 
instances back up, the security is no longer in place and I have to upload the 
security.json again for it to take effect.

- Kevin

> On Sep 3, 2015, at 10:29 PM, Noble Paul <noble.p...@gmail.com> wrote:
> 
> Both these are committed. If you could test with the latest 5.3 branch
> it would be helpful
> 
> On Wed, Sep 2, 2015 at 5:11 PM, Noble Paul <noble.p...@gmail.com> wrote:
>> I opened a ticket for the same
>> https://issues.apache.org/jira/browse/SOLR-8004
>> 
>> On Wed, Sep 2, 2015 at 1:36 PM, Kevin Lee <kgle...@yahoo.com.invalid> wrote:
>>> I’ve found that completely exiting Chrome or Firefox and opening it back up 
>>> re-prompts for credentials when they are required.  It was re-prompting 
>>> with the /browse path where authentication was working each time I 
>>> completely exited and started the browser again, however it won’t re-prompt 
>>> unless you exit completely and close all running instances so I closed all 
>>> instances each time to test.
>>> 
>>> However, to make sure I ran it via the command line via curl as suggested 
>>> and it still does not give any authentication error when trying to issue 
>>> the command via curl.  I get a success response from all the Solr instances 
>>> that the reload was successful.
>>> 
>>> Not sure why the pre-canned permissions aren’t working, but the one to the 
>>> request handler at the /browse path is.
>>> 
>>> 
>>>> On Sep 1, 2015, at 11:03 PM, Noble Paul <noble.p...@gmail.com> wrote:
>>>> 
>>>> " However, after uploading the new security.json and restarting the
>>>> web browser,"
>>>> 
>>>> The browser remembers your login , So it is unlikely to prompt for the
>>>> credentials again.
>>>> 
>>>> Why don't you try the RELOAD operation using command line (curl) ?
>>>> 
>>>> On Tue, Sep 1, 2015 at 10:31 PM, Kevin Lee <kgle...@yahoo.com.invalid> 
>>>> wrote:
>>>>> The restart issues aside, I’m trying to lockdown usage of the Collections 
>>>>> API, but that also does not seem to be working either.
>>>>> 
>>>>> Here is my security.json.  I’m using the “collection-admin-edit” 
>>>>> permission and assigning it to the “adminRole”.  However, after uploading 
>>>>> the new security.json and restarting the web browser, it doesn’t seem to 
>>>>> be requiring credentials when calling the RELOAD action on the 
>>>>> Collections API.  The only thing that seems to work is the custom 
>>>>> permission “browse” which is requiring authentication before allowing me 
>>>>> to pull up the page.  Am I using the permissions correctly for the 
>>>>> RuleBasedAuthorizationPlugin?
>>>>> 
>>>>> {
>>>>>   "authentication":{
>>>>>  "class":"solr.BasicAuthPlugin",
>>>>>  "credentials": {
>>>>>   "admin”:” ",
>>>>>   "user": ” "
>>>>>   }
>>>>>   },
>>>>>   "authorization":{
>>>>>  "class":"solr.RuleBasedAuthorizationPlugin",
>>>>>  "permissions": [
>>>>>   {
>>>>>   "name":"security-edit",
>>>>>   "role":"adminRole"
>>>>>   },
>>>>>   {
>>>>>   "name":"collection-admin-edit”,
>>>>>   "role":"adminRole"
>>>>>   },
>>>>>   {
>>>>>   "name":"browse",
>>>>>   "collection": "inventory",
>>>>>   "path": "/browse",
>>>>>   "rol

Re: Error in creating a new collection

[Kevin Lee]

Configuration upload to zookeeper and collection creation are two separate 
things, although they can be accompished at the same time using /bin/solr.  You 
can upload configurations before you create collections and you can have 
mutiple configurations uploaded to zookeeper at the same time.  I typically 
upload my configurations using zkcli.sh in the server/scripts/cloud-scripts 
directory in Solr and then use curl to send a request to create the collection 
based on a configuration I uploaded to Zookeeper instead of using the /bin/solr 
script to do the upload and creation all at once.  So you may have several 
configs uploaded, but you may not necessarily have created collections for 
them.  

If you need to update the configuration, just re-upload using zkcli.sh (from 
Solr server/scripts/cloud-scripts not ZkCli.sh in zookeeper/bin) and then try 
the collection creation again using curl or the browser to issue the 
Collections API create command.  Re-uploading will overwrite the config in 
Zookeeper.

I don't believe what you are seeing is a bug, as you can upload anything you 
want to Zookeeper, but it doesn't mean it's valid.  Only Solr knows that once 
it tries to load what you've uploaded to Zookeeper.

- Kevin

> On Sep 3, 2015, at 12:36 AM, shacky  wrote:
> 
> Hi Shalin,
> thank you very much for your answer.
> 
> I found out and managed in recreating the problem.
> 
> I created a new collection, with the wrong configset. I got the error
> and the collection was not created, good.
> But after that I continue to see the "SolrCore Initialization
> Failures" in the Solr Admin web interface on all three nodes.
> I had to restart Solr to remove that error.
> 
> From this point if I try to change the configset and recreate the same
> collection I still continue to get the same previous error.
> 
> So I discovered that the configuration is still in Zookeeper:
> 
> root@index1:~# /usr/share/zookeeper/bin/zkCli.sh
> Connecting to localhost:2181
> Welcome to ZooKeeper!
> JLine support is enabled
> 
> WATCHER::
> 
> WatchedEvent state:SyncConnected type:None path:null
> [zk: localhost:2181(CONNECTED) 0] ls /solr/configs
> [test, test2]
> 
> Even if the "test2" collection was not created due to the previous error.
> 
> I had to remove (rmr) the configuration in Zookeeper to be able to
> recreate the collection.
> 
> I think this is a bug, isn't it?
> The configuration should be removed from Zookeeper if the collection
> was not created due to an error...

Re: Issue Using Solr 5.3 Authentication and Authorization Plugins

[Kevin Lee]

I’ve found that completely exiting Chrome or Firefox and opening it back up 
re-prompts for credentials when they are required.  It was re-prompting with 
the /browse path where authentication was working each time I completely exited 
and started the browser again, however it won’t re-prompt unless you exit 
completely and close all running instances so I closed all instances each time 
to test.

However, to make sure I ran it via the command line via curl as suggested and 
it still does not give any authentication error when trying to issue the 
command via curl.  I get a success response from all the Solr instances that 
the reload was successful.

Not sure why the pre-canned permissions aren’t working, but the one to the 
request handler at the /browse path is.


> On Sep 1, 2015, at 11:03 PM, Noble Paul <noble.p...@gmail.com> wrote:
> 
> " However, after uploading the new security.json and restarting the
> web browser,"
> 
> The browser remembers your login , So it is unlikely to prompt for the
> credentials again.
> 
> Why don't you try the RELOAD operation using command line (curl) ?
> 
> On Tue, Sep 1, 2015 at 10:31 PM, Kevin Lee <kgle...@yahoo.com.invalid> wrote:
>> The restart issues aside, I’m trying to lockdown usage of the Collections 
>> API, but that also does not seem to be working either.
>> 
>> Here is my security.json.  I’m using the “collection-admin-edit” permission 
>> and assigning it to the “adminRole”.  However, after uploading the new 
>> security.json and restarting the web browser, it doesn’t seem to be 
>> requiring credentials when calling the RELOAD action on the Collections API. 
>>  The only thing that seems to work is the custom permission “browse” which 
>> is requiring authentication before allowing me to pull up the page.  Am I 
>> using the permissions correctly for the RuleBasedAuthorizationPlugin?
>> 
>> {
>>"authentication":{
>>   "class":"solr.BasicAuthPlugin",
>>   "credentials": {
>>"admin”:” ",
>>"user": ” "
>>}
>>},
>>"authorization":{
>>   "class":"solr.RuleBasedAuthorizationPlugin",
>>   "permissions": [
>>{
>>"name":"security-edit",
>>"role":"adminRole"
>>},
>>{
>>"name":"collection-admin-edit”,
>>"role":"adminRole"
>>},
>>{
>>"name":"browse",
>>"collection": "inventory",
>>"path": "/browse",
>>"role":"browseRole"
>>}
>>],
>>   "user-role": {
>>"admin": [
>>"adminRole",
>>"browseRole"
>>],
>>"user": [
>>"browseRole"
>>]
>>}
>>}
>> }
>> 
>> Also tried adding the permission using the Authorization API, but no effect, 
>> still isn’t protecting the Collections API from being invoked without a 
>> username password.  I do see in the Solr logs that it sees the updates 
>> because it outputs the messages “Updating /security.json …”, “Security node 
>> changed”, “Initializing authorization plugin: 
>> solr.RuleBasedAuthorizationPlugin” and “Authentication plugin class obtained 
>> from ZK: solr.BasicAuthPlugin”.
>> 
>> Thanks,
>> Kevin
>> 
>>> On Sep 1, 2015, at 12:31 AM, Noble Paul <noble.p...@gmail.com> wrote:
>>> 
>>> I'm investigating why restarts or first time start does not read the
>>> security.json
>>> 
>>> On Tue, Sep 1, 2015 at 1:00 PM, Noble Paul <noble.p...@gmail.com> wrote:
>>>> I removed that statement
>>>> 
>>>> "If activating the authorization plugin doesn't protect the admin ui,
>>>> how does one protect access to it?"
>>>> 
>>>> On

Using bq param for negative boost

[Kevin Lee]

Hi,

I’m trying to boost all results using the bq param with edismax where termA and 
termB do not appear in the field, but if phraseC appears it doesn’t matter if 
termA and termB appear.

The following works and boosts everything that doesn’t have termA and termB in 
myField so the effect is that all documents with termA and termB are pushed to 
the bottom of the result list.

myField:(*:* -termA -termB)^1

How would you add the second part where if phraseC is present, then termA and 
termB can be present?

Tried doing something like the following, but it is not working.

myField:(*:* ((-termA -termB) OR +”phraseC”))^1

Thanks!

Re: Issue Using Solr 5.3 Authentication and Authorization Plugins

[Kevin Lee]

The restart issues aside, I’m trying to lockdown usage of the Collections API, 
but that also does not seem to be working either.

Here is my security.json.  I’m using the “collection-admin-edit” permission and 
assigning it to the “adminRole”.  However, after uploading the new 
security.json and restarting the web browser, it doesn’t seem to be requiring 
credentials when calling the RELOAD action on the Collections API.  The only 
thing that seems to work is the custom permission “browse” which is requiring 
authentication before allowing me to pull up the page.  Am I using the 
permissions correctly for the RuleBasedAuthorizationPlugin?

{
"authentication":{
   "class":"solr.BasicAuthPlugin",
   "credentials": {
"admin”:” ",
"user": ” "
}
},
"authorization":{
   "class":"solr.RuleBasedAuthorizationPlugin",
   "permissions": [
{
"name":"security-edit", 
"role":"adminRole"
},
{
"name":"collection-admin-edit”,
"role":"adminRole"
},
{
"name":"browse", 
"collection": "inventory", 
"path": "/browse", 
"role":"browseRole"
}
],
   "user-role": {
"admin": [
"adminRole",
"browseRole"
],
"user": [
"browseRole"
]
}
}
}

Also tried adding the permission using the Authorization API, but no effect, 
still isn’t protecting the Collections API from being invoked without a 
username password.  I do see in the Solr logs that it sees the updates because 
it outputs the messages “Updating /security.json …”, “Security node changed”, 
“Initializing authorization plugin: solr.RuleBasedAuthorizationPlugin” and 
“Authentication plugin class obtained from ZK: solr.BasicAuthPlugin”.

Thanks,
Kevin

> On Sep 1, 2015, at 12:31 AM, Noble Paul <noble.p...@gmail.com> wrote:
> 
> I'm investigating why restarts or first time start does not read the
> security.json
> 
> On Tue, Sep 1, 2015 at 1:00 PM, Noble Paul <noble.p...@gmail.com> wrote:
>> I removed that statement
>> 
>> "If activating the authorization plugin doesn't protect the admin ui,
>> how does one protect access to it?"
>> 
>> One does not need to protect the admin UI. You only need to protect
>> the relevant API calls . I mean it's OK to not protect the CSS and
>> HTML stuff.  But if you perform an action to create a core or do a
>> query through admin UI , it automatically will prompt you for
>> credentials (if those APIs are protected)
>> 
>> On Tue, Sep 1, 2015 at 12:41 PM, Kevin Lee <kgle...@yahoo.com.invalid> wrote:
>>> Thanks for the clarification!
>>> 
>>> So is the wiki page incorrect at
>>> https://cwiki.apache.org/confluence/display/solr/Basic+Authentication+Plugin
>>>  which says that the admin ui will require authentication once the 
>>> authorization plugin is activated?
>>> 
>>> "An authorization plugin is also available to configure Solr with 
>>> permissions to perform various activities in the system. Once activated, 
>>> access to the Solr Admin UI and all requests will need to be authenticated 
>>> and users will be required to have the proper authorization for all 
>>> requests, including using the Admin UI and making any API calls."
>>> 
>>> If activating the authorization plugin doesn't protect the admin ui, how 
>>> does one protect access to it?
>>> 
>>> Also, the issue I'm having is not just at restart.  According to the docs 
>>> security.json should be uploaded to Zookeeper before starting any of the 
>>> Solr instances.  However, I tried to upload security.json before starting 
>>> any of the Solr instances, but it would not pick up the security config 
>>> until after the Solr instances are already running and then uploading the 
>>> security.json again.  I 

Re: Issue Using Solr 5.3 Authentication and Authorization Plugins

[Kevin Lee]

Thanks for the clarification!  

So is the wiki page incorrect at 
https://cwiki.apache.org/confluence/display/solr/Basic+Authentication+Plugin 
which says that the admin ui will require authentication once the authorization 
plugin is activated?

"An authorization plugin is also available to configure Solr with permissions 
to perform various activities in the system. Once activated, access to the Solr 
Admin UI and all requests will need to be authenticated and users will be 
required to have the proper authorization for all requests, including using the 
Admin UI and making any API calls."

If activating the authorization plugin doesn't protect the admin ui, how does 
one protect access to it?

Also, the issue I'm having is not just at restart.  According to the docs 
security.json should be uploaded to Zookeeper before starting any of the Solr 
instances.  However, I tried to upload security.json before starting any of the 
Solr instances, but it would not pick up the security config until after the 
Solr instances are already running and then uploading the security.json again.  
I can see in the logs at startup that the Solr instances don't see any plugin 
enabled even though security.json is already in zookeeper and then after they 
are started and the security.json is uploaded again I see it reconfigure to use 
the plugin.

Thanks,
Kevin

> On Aug 31, 2015, at 11:22 PM, Noble Paul <noble.p...@gmail.com> wrote:
> 
> Admin UI is not protected by any of these permissions. Only if you try
> to perform a protected operation , it asks for a password.
> 
> I'll investigate the restart problem and report my  findings
> 
>> On Tue, Sep 1, 2015 at 3:10 AM, Kevin Lee <kgle...@yahoo.com.invalid> wrote:
>> Anyone else running into any issues trying to get the authentication and 
>> authorization plugins in 5.3 working?
>> 
>>> On Aug 29, 2015, at 2:30 AM, Kevin Lee <kgle...@yahoo.com.INVALID> wrote:
>>> 
>>> Hi,
>>> 
>>> I’m trying to use the new basic auth plugin for Solr 5.3 and it doesn’t 
>>> seem to be working quite right.  Not sure if I’m missing steps or there is 
>>> a bug.  I am able to get it to protect access to a URL under a collection, 
>>> but am unable to get it to secure access to the Admin UI.  In addition, 
>>> after stopping the Solr and Zookeeper instances, the security.json is still 
>>> in Zookeeper, however Solr is allowing access to everything again like the 
>>> security configuration isn’t in place.
>>> 
>>> Contents of security.json taken from wiki page, but edited to produce valid 
>>> JSON.  Had to move comma after 3rd from last “}” up to just after the last 
>>> “]”.
>>> 
>>> {
>>> "authentication":{
>>> "class":"solr.BasicAuthPlugin",
>>> "credentials":{"solr":"IV0EHq1OnNrj6gvRCwvFwTrZ1+z1oBbnQdiVC3otuq0= 
>>> Ndd7LKvVBAaZIF0QAVi1ekCfAJXr1GGfLtRUXhgrF8c="}
>>> },
>>> "authorization":{
>>> "class":"solr.RuleBasedAuthorizationPlugin",
>>> "permissions":[{"name":"security-edit",
>>>"role":"admin"}],
>>> "user-role":{"solr":"admin"}
>>> }}
>>> 
>>> Here are the steps I followed:
>>> 
>>> Upload security.json to zookeeper
>>> ./zkcli.sh -z localhost:2181,localhost:2182,localhost:2183 -cmd putfile 
>>> /security.json ~/solr/security.json
>>> 
>>> Use zkCli.sh from Zookeeper to ensure the security.json is in Zookeeper at 
>>> /security.json.  It is there and looks like what was originally uploaded.
>>> 
>>> Start Solr Instances
>>> 
>>> Attempt to create a permission, however get the following error:
>>> {
>>> "responseHeader":{
>>>  "status":400,
>>>  "QTime":0},
>>> "error":{
>>>  "msg":"No authorization plugin configured",
>>>  "code":400}}
>>> 
>>> Upload security.json again.
>>> ./zkcli.sh -z localhost:2181,localhost:2182,localhost:2183 -cmd putfile 
>>> /security.json ~/solr/security.json
>>> 
>>> Issue the following to try to create the permission again and this time 
>>> it’s successful.
>>> // Create a permission for mysearch endpoint
>>>  curl --user solr:SolrRocks -H 'Content-type:application/json' -d 
>>> '{"set-permission": {"name":"mycollection-search","collection": 
>>> “mycollection"

Re: DataImportHandler scheduling

[Kevin Lee]

While it may be useful to have a scheduler for simple cases, I think there are 
too many variables to make it useful for everyone's case.  For example, I 
recently wrote a script that uses the data import handler api to get the 
status, kick off the import, etc.  However, before allowing it to just kick 
off, I needed to query the database where the data was coming from to make sure 
it had finished it's daily load and then if it hadn't finished, wait for awhile 
to see if it would, then the script could do the load.  After the load is 
finished it does another check to ensure the expected number of docs was 
actually loaded by Solr based on the data from the database.

If a scheduler were built into Solr it probably would only cover the simple 
case and for production you'd probably need to write your own scripts and use 
your own scheduler anyways to ensure the loads are starting/completing as 
expected.

> On Sep 1, 2015, at 1:09 PM, William Bell  wrote:
> 
> We should add a simple scheduler in the UI. It is very useful. To schedule
> various actions:
> 
> - Full index
> - Delta Index
> - Replicate
> 
> 
> 
> 
>> On Tue, Sep 1, 2015 at 12:41 PM, Shawn Heisey  wrote:
>> 
>>> On 9/1/2015 11:45 AM, Troy Edwards wrote:
>>> My initial thought was to use scheduling built with DIH:
>>> http://wiki.apache.org/solr/DataImportHandler#Scheduling
>>> 
>>> But I think just a cron job should do the same for me.
>> 
>> The dataimport scheduler does not exist in any Solr version.  This is a
>> proposed feature, with the enhancement issue open for more than four years:
>> 
>> https://issues.apache.org/jira/browse/SOLR-2305
>> 
>> I have updated the wiki page to state the fact that the scheduler is a
>> proposed improvement, not a usable feature.
>> 
>> Thanks,
>> Shawn
> 
> 
> -- 
> Bill Bell
> billnb...@gmail.com
> cell 720-256-8076

Re: Issue Using Solr 5.3 Authentication and Authorization Plugins

[Kevin Lee]

Anyone else running into any issues trying to get the authentication and 
authorization plugins in 5.3 working?

> On Aug 29, 2015, at 2:30 AM, Kevin Lee <kgle...@yahoo.com.INVALID> wrote:
> 
> Hi,
> 
> I’m trying to use the new basic auth plugin for Solr 5.3 and it doesn’t seem 
> to be working quite right.  Not sure if I’m missing steps or there is a bug.  
> I am able to get it to protect access to a URL under a collection, but am 
> unable to get it to secure access to the Admin UI.  In addition, after 
> stopping the Solr and Zookeeper instances, the security.json is still in 
> Zookeeper, however Solr is allowing access to everything again like the 
> security configuration isn’t in place.
> 
> Contents of security.json taken from wiki page, but edited to produce valid 
> JSON.  Had to move comma after 3rd from last “}” up to just after the last 
> “]”.
> 
> {
> "authentication":{
>   "class":"solr.BasicAuthPlugin",
>   "credentials":{"solr":"IV0EHq1OnNrj6gvRCwvFwTrZ1+z1oBbnQdiVC3otuq0= 
> Ndd7LKvVBAaZIF0QAVi1ekCfAJXr1GGfLtRUXhgrF8c="}
> },
> "authorization":{
>   "class":"solr.RuleBasedAuthorizationPlugin",
>   "permissions":[{"name":"security-edit",
>  "role":"admin"}],
>   "user-role":{"solr":"admin"}
> }}
> 
> Here are the steps I followed:
> 
> Upload security.json to zookeeper
> ./zkcli.sh -z localhost:2181,localhost:2182,localhost:2183 -cmd putfile 
> /security.json ~/solr/security.json
> 
> Use zkCli.sh from Zookeeper to ensure the security.json is in Zookeeper at 
> /security.json.  It is there and looks like what was originally uploaded.
> 
> Start Solr Instances
> 
> Attempt to create a permission, however get the following error:
> {
>  "responseHeader":{
>"status":400,
>"QTime":0},
>  "error":{
>"msg":"No authorization plugin configured",
>"code":400}}
> 
> Upload security.json again.
> ./zkcli.sh -z localhost:2181,localhost:2182,localhost:2183 -cmd putfile 
> /security.json ~/solr/security.json
> 
> Issue the following to try to create the permission again and this time it’s 
> successful.
> // Create a permission for mysearch endpoint
>curl --user solr:SolrRocks -H 'Content-type:application/json' -d 
> '{"set-permission": {"name":"mycollection-search","collection": 
> “mycollection","path":”/mysearch","role": "search-user"}}' 
> http://localhost:8983/solr/admin/authorization
>
>{
>  "responseHeader":{
>"status":0,
>"QTime":7}}
>
> Issue the following commands to add users
> curl --user solr:SolrRocks http://localhost:8983/solr/admin/authentication -H 
> 'Content-type:application/json' -d '{"set-user": {"admin" : “password" }}’
> curl --user solr:SolrRocks http://localhost:8983/solr/admin/authentication -H 
> 'Content-type:application/json' -d '{"set-user": {"user" : “password" }}'
> 
> Issue the following command to add permission to users
> curl -u solr:SolrRocks -H 'Content-type:application/json' -d '{ 
> "set-user-role" : {"admin": ["search-user", "admin"]}}' 
> http://localhost:8983/solr/admin/authorization
> curl -u solr:SolrRocks -H 'Content-type:application/json' -d '{ 
> "set-user-role" : {"user": ["search-user"]}}' 
> http://localhost:8983/solr/admin/authorization
> 
> After executing the above, access to /mysearch is protected until I restart 
> the Solr and Zookeeper instances.  However, the admin UI is never protected 
> like the Wiki page says it should be once activated.
> 
> https://cwiki.apache.org/confluence/display/solr/Rule-Based+Authorization+Plugin
>  
> <https://cwiki.apache.org/confluence/display/solr/Rule-Based+Authorization+Plugin>
> 
> Why does the authentication and authorization plugin not stay activated after 
> restart and why is the Admin UI never protected?  Am I missing any steps?
> 
> Thanks,
> Kevin

Issue Using Solr 5.3 Authentication and Authorization Plugins

[Kevin Lee]

Hi,

I’m trying to use the new basic auth plugin for Solr 5.3 and it doesn’t seem to 
be working quite right.  Not sure if I’m missing steps or there is a bug.  I am 
able to get it to protect access to a URL under a collection, but am unable to 
get it to secure access to the Admin UI.  In addition, after stopping the Solr 
and Zookeeper instances, the security.json is still in Zookeeper, however Solr 
is allowing access to everything again like the security configuration isn’t in 
place.

Contents of security.json taken from wiki page, but edited to produce valid 
JSON.  Had to move comma after 3rd from last “}” up to just after the last “]”.

{
authentication:{
   class:solr.BasicAuthPlugin,
   credentials:{solr:IV0EHq1OnNrj6gvRCwvFwTrZ1+z1oBbnQdiVC3otuq0= 
Ndd7LKvVBAaZIF0QAVi1ekCfAJXr1GGfLtRUXhgrF8c=}
},
authorization:{
   class:solr.RuleBasedAuthorizationPlugin,
   permissions:[{name:security-edit,
  role:admin}],
   user-role:{solr:admin}
}}

Here are the steps I followed:

Upload security.json to zookeeper
./zkcli.sh -z localhost:2181,localhost:2182,localhost:2183 -cmd putfile 
/security.json ~/solr/security.json

Use zkCli.sh from Zookeeper to ensure the security.json is in Zookeeper at 
/security.json.  It is there and looks like what was originally uploaded.

Start Solr Instances

Attempt to create a permission, however get the following error:
{
  responseHeader:{
status:400,
QTime:0},
  error:{
msg:No authorization plugin configured,
code:400}}

Upload security.json again.
./zkcli.sh -z localhost:2181,localhost:2182,localhost:2183 -cmd putfile 
/security.json ~/solr/security.json

Issue the following to try to create the permission again and this time it’s 
successful.
// Create a permission for mysearch endpoint
curl --user solr:SolrRocks -H 
'Content-type:application/json' -d '{set-permission: 
{name:mycollection-search,collection: 
“mycollection,path:”/mysearch,role: search-user}}' 
http://localhost:8983/solr/admin/authorization

{
responseHeader:{
  status:0,
 QTime:7}}

Issue the following commands to add users
curl --user solr:SolrRocks http://localhost:8983/solr/admin/authentication -H 
'Content-type:application/json' -d '{set-user: {admin : “password }}’
curl --user solr:SolrRocks http://localhost:8983/solr/admin/authentication -H 
'Content-type:application/json' -d '{set-user: {user : “password }}'

Issue the following command to add permission to users
curl -u solr:SolrRocks -H 'Content-type:application/json' -d '{ set-user-role 
: {admin: [search-user, admin]}}' 
http://localhost:8983/solr/admin/authorization
curl -u solr:SolrRocks -H 'Content-type:application/json' -d '{ set-user-role 
: {user: [search-user]}}' http://localhost:8983/solr/admin/authorization

After executing the above, access to /mysearch is protected until I restart the 
Solr and Zookeeper instances.  However, the admin UI is never protected like 
the Wiki page says it should be once activated.

https://cwiki.apache.org/confluence/display/solr/Rule-Based+Authorization+Plugin
 
https://cwiki.apache.org/confluence/display/solr/Rule-Based+Authorization+Plugin

Why does the authentication and authorization plugin not stay activated after 
restart and why is the Admin UI never protected?  Am I missing any steps?

Thanks,
Kevin

Re: Difficulties in getting Solrcloud running

[Kevin Lee]

Hi,

Have you created a collection yet?  If not, then there won’t be a graph yet.  
It doesn’t show up until there is at least one collection.

- Kevin

 On Aug 19, 2015, at 5:48 AM, Merlin Morgenstern 
 merlin.morgenst...@gmail.com wrote:
 
 HI everybody,
 
 I am trying to setup solrcloud on ubuntu and somehow the graph on the admin
 interface does not show up. It is simply blanck. The tree is available.
 
 This is a test installation on one machine.
 
 There are 3 zookeepers running.
 
 I start two solr nodes like this:
 
 solr-5.2.1$ bin/solr start -cloud -s server/solr1 -p 8983 -z
 zk1:2181,zk1:2182,zk1:2183 -noprompt
 
 solr-5.2.1$ bin/solr start -cloud -s server/solr2 -p 8984 -z
 zk1:2181,zk1:2182,zk1:2183 -noprompt
 
 zk1 is a local interface with 10.0.0.120
 
 it all looks OK, no error messages.
 
 Thank you in advance for any help on this

Re: Upload core.properties to ZooKeeper

[Kevin Lee]

You should be able to use user defined properties within core.properties.  
However, it sounds like you are uploading core.properties to Zookeeper.  In 
SolrCloud, core.properties is not uploaded to Zookeeper.  You place 
core.properties within your core’s top level directory and the cores are 
automatically discovered.  Your configuration set which includes your 
solrconfig.xml and schema.xml would be uploaded to Zookeeper.

See the following around core discovery and the very last sentence on the page 
at the first link which speaks to “user defined” properties and the second link 
which discusses the different ways to use user defined properties.
https://cwiki.apache.org/confluence/display/solr/Defining+core.properties 
https://cwiki.apache.org/confluence/display/solr/Defining+core.properties
https://cwiki.apache.org/confluence/display/solr/Configuring+solrconfig.xml#Configuringsolrconfig.xml-SubstitutingPropertiesinSolrConfigFiles
 
https://cwiki.apache.org/confluence/display/solr/Configuring+solrconfig.xml#Configuringsolrconfig.xml-SubstitutingPropertiesinSolrConfigFiles

I just tried a quick test and defining a property in core.properties does get 
picked up and substituted in solrconfig.xml.  However, I tried to use 
solrcore.properties as described at the link above, but it did not work.  So 
you may need to just ensure your config set is uploaded (without 
core.properties) and make sure your core.properties exist in the root directory 
of each core with your custom properties.

See the section on “Upload a configuration directory” below for how to upload 
to Zookeeper.
https://cwiki.apache.org/confluence/display/solr/Command+Line+Utilities 
https://cwiki.apache.org/confluence/display/solr/Command+Line+Utilities

- Kevin

 On Aug 6, 2015, at 4:37 AM, Upayavira u...@odoko.co.uk wrote:
 
 Have you looked at the collections API? It has the ability to set
 properties against collections. I wonder if that'll achieve the same
 thing as adding them to core.properties? I've never used it myself, but
 wonder if it'll solve your issue.
 
 Upayavira
 
 On Thu, Aug 6, 2015, at 12:35 PM, marotosg wrote:
 Hi,
 
 I am in the process of migrating my master, slave Solr infraestructure to
 SolrCloud.
 At the moment I have several cores inside a folder with this structure
 /MyCores
 /MyCores/Core1
 /MyCores/Core1/conf
 /MyCores/Core1/core.properties
 /MyCores/Core2
 /MyCores/Core2/conf
 /MyCores/Core1/core.properties
 
 As you can see I have several cores with a core.properties file inside
 which
 contain several variables to populate solrconfig.xml and schema.xml.
 
 When uploading this info into Zookeeper it fails during the creation of a
 collection because it's not able to resolve the core.properties
 variables.
 
 Do you have any idea? Is it possible to use core.properties with
 SolrCloud?
 
 Thanks
 Sergio
 
 
 
 
 --
 View this message in context:
 http://lucene.472066.n3.nabble.com/Upload-core-properties-to-ZooKeeper-tp4221259.html
 Sent from the Solr - User mailing list archive at Nabble.com.