Re: Loading of zkCredentialsProvider has changed in Solr 7 or 8?
I realize that attachments might not work on the mailing list, so here is the log on Drive https://drive.google.com/file/d/0B7mypFpwbHptWkp0X2U0azU2dGREb1k2WGlpeUM3MlRIWmRB/view?usp=sharing On Tue, 11 Jun 2019 at 11:21, Colvin Cowie wrote: > Hello all > > I hit another problem in moving from Solr 6 to 8. > > We secure our ZooKeeper entirely (there's a restrictive ACL for every > znode) > To pass the ZooKeeper credentials to Solr we implemented > ZkCredentialsProvider and ZkACLProvider to load the credentials from a file > on disk, which has the credentials in an encoded format (the details don't > really matter, just that we have our own implementations of the providers) > > In Solr 6 we were able to specify the implementation classes in the > /solr.xml (zkCredentialsProvider & zkACLProvider) in the home directory for > each node and provided the implementation in a jar in the /lib directory of > the node, and this worked without issue. > > In Solr 8.1.1, this still works for *some* of the requests that are made > to ZooKeeper, but other requests made by the SolrZkClient (via > SolrDispatchFilter) do not use the ZkCredentialsProvider specified in > solr.xml, instead falling back to the DefaultZkCredentialsProvider, or a > provider specified by the system property *-DzkCredentialsProvider.* > >- If Solr makes requests using the DefaultZkCredentialsProvider, they >will fail with *NoAuth for /clusterprops.json.* >- If I specify the *-DzkCredentialsProvider* then the jars in /lib are >not visible to the SolrZkClient making the requests, resulting in a >ClassNotFoundException (log attached). >- If I add the jar directly to >solr\server\solr-webapp\webapp\WEB-INF\lib then it is loadable, but I don't >want to mess around with the lib directory Solr provides. >- I know that the solrconfig.xml allows you to configure the >directive for the classpath, but Solr won't be able to retrieve that >solrconfig.xml from ZooKeeper without authenticating... Catch-22. > > > I've not had a chance to properly read the Solr 8 Reference Guide yet, but > it does still refer to the zkCredentialsProvider & zkACLProvider properties > being supported > https://lucene.apache.org/solr/guide/8_0/format-of-solr-xml.html#the-solr-element > I didn't see anything particular in the upgrade notes for Solr 7 or 8 > about changes to zkCredentialsProvider > > So is there something wrong with what we were doing in Solr 6? Or has > there been a regression? > > Thanks for any advice > Colvin >
Loading of zkCredentialsProvider has changed in Solr 7 or 8?
Hello all I hit another problem in moving from Solr 6 to 8. We secure our ZooKeeper entirely (there's a restrictive ACL for every znode) To pass the ZooKeeper credentials to Solr we implemented ZkCredentialsProvider and ZkACLProvider to load the credentials from a file on disk, which has the credentials in an encoded format (the details don't really matter, just that we have our own implementations of the providers) In Solr 6 we were able to specify the implementation classes in the /solr.xml (zkCredentialsProvider & zkACLProvider) in the home directory for each node and provided the implementation in a jar in the /lib directory of the node, and this worked without issue. In Solr 8.1.1, this still works for *some* of the requests that are made to ZooKeeper, but other requests made by the SolrZkClient (via SolrDispatchFilter) do not use the ZkCredentialsProvider specified in solr.xml, instead falling back to the DefaultZkCredentialsProvider, or a provider specified by the system property *-DzkCredentialsProvider.* - If Solr makes requests using the DefaultZkCredentialsProvider, they will fail with *NoAuth for /clusterprops.json.* - If I specify the *-DzkCredentialsProvider* then the jars in /lib are not visible to the SolrZkClient making the requests, resulting in a ClassNotFoundException (log attached). - If I add the jar directly to solr\server\solr-webapp\webapp\WEB-INF\lib then it is loadable, but I don't want to mess around with the lib directory Solr provides. - I know that the solrconfig.xml allows you to configure the directive for the classpath, but Solr won't be able to retrieve that solrconfig.xml from ZooKeeper without authenticating... Catch-22. I've not had a chance to properly read the Solr 8 Reference Guide yet, but it does still refer to the zkCredentialsProvider & zkACLProvider properties being supported https://lucene.apache.org/solr/guide/8_0/format-of-solr-xml.html#the-solr-element I didn't see anything particular in the upgrade notes for Solr 7 or 8 about changes to zkCredentialsProvider So is there something wrong with what we were doing in Solr 6? Or has there been a regression? Thanks for any advice Colvin 2019-06-11 10:04:42.621 INFO (main) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port 8983 2019-06-11 10:04:42.621 INFO (main) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: C:\solr 2019-06-11 10:04:42.622 INFO (main) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|Start time: 2019-06-11T10:04:42.621Z 2019-06-11 10:04:42.644 INFO (main) [ ] o.a.s.c.SolrResourceLoader Using system property solr.solr.home: C:/data/solr/clusters/is_cluster/nodes/node1 2019-06-11 10:04:42.656 INFO (main) [ ] o.a.s.c.c.SolrZkClient Using ZkCredentialsProvider: xxx.yyy.zzz.EncodedZkCredentialsProvider 2019-06-11 10:04:42.657 WARN (main) [ ] o.a.s.c.c.SolrZkClient VM param zkCredentialsProvider does not point to a class implementing ZkCredentialsProvider and with a non-arg constructor => java.lang.ClassNotFoundException: xxx.yyy.zzz.EncodedZkCredentialsProvider at java.net.URLClassLoader.findClass(URLClassLoader.java:382) java.lang.ClassNotFoundException: xxx.yyy.zzz.EncodedZkCredentialsProvider at java.net.URLClassLoader.findClass(URLClassLoader.java:382) ~[?:1.8.0_211] at java.lang.ClassLoader.loadClass(ClassLoader.java:424) ~[?:1.8.0_211] at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ~[?:1.8.0_211] at org.eclipse.jetty.webapp.WebAppClassLoader.loadClass(WebAppClassLoader.java:556) ~[jetty-webapp-9.4.14.v20181114.jar:9.4.14.v20181114] at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ~[?:1.8.0_211] at java.lang.Class.forName0(Native Method) ~[?:1.8.0_211] at java.lang.Class.forName(Class.java:264) ~[?:1.8.0_211] at org.apache.solr.common.cloud.SolrZkClient.createZkCredentialsToAddAutomatically(SolrZkClient.java:225) ~[?:?] at org.apache.solr.common.cloud.SolrZkClient.(SolrZkClient.java:144) ~[?:?] at org.apache.solr.common.cloud.SolrZkClient.(SolrZkClient.java:126) ~[?:?] at org.apache.solr.common.cloud.SolrZkClient.(SolrZkClient.java:121) ~[?:?] at org.apache.solr.common.cloud.SolrZkClient.(SolrZkClient.java:108) ~[?:?] at org.apache.solr.servlet.SolrDispatchFilter.loadNodeConfig(SolrDispatchFilter.java:276) ~[?:?] at org.apache.solr.servlet.SolrDispatchFilter.createCoreContainer(SolrDispatchFilter.java:253) ~[?:?] at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:175) ~[?:?] at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:136) ~[jetty-servlet-9.4.14.v20181114.jar:9.4.14.v20181114] at org.eclipse.jetty.servlet.ServletHandler.lambda$initialize$0(ServletHandler.java:750) ~[jetty-servlet-9.4.14.v20181114.jar:9.4.14.v20181114] at
