laminelam opened a new pull request, #826: URL: https://github.com/apache/solr/pull/826
https://issues.apache.org/jira/browse/SOLR-15857 **Problem** Solr uses a list of credentials to connect to Zookeeper and to handle ACLs. - 1- In the current implementation, the credentials are passed through command line (system props) or read from a clear text file stored in all cluster hosts. Needless to say this is not safe enough. - 2- On the other hand, the same code to load the credentials is called twice, first by _ZkCredentialsProvider_ to connect to Zookeeper and a second time by _ZkACLProvider_ to create ACLs. The code is also duplicated, although it's only reading from system props. Adding a custom pair of _ZkCredentialsProvider_/_ZkACLProvider_ to load the credentials from another source (ex a Secret Manager) would also require duplicate the code and make repetitive calls to extract the same credentials. - 3- There is no way to customize how credentials are passed without recompiling. **Proposed solution** Let’s start with problem 2). **Problem 2** - Refactor the way how the credentials are injected by passing them as a dependency. One code, called once and injected into the client class. Here the client classes are _ZkCredentialsProvider_ and _ZkACLProvider_. - Favor composition over inheritance to inject custom credentials loaders without changing the composing (container) class. - Add a third interface _ZkCredentialsInjector_ whose implementations load ZK credentials from a credentials source to be injected into _ZkCredentialsProvider_ and _ZkACLProvider_ - The workflow is: Credentials source —> _ZkCredentialsInjector_ --> _ZkCredentialsProvider_/_ZkACLProvider_ --> Zookeeper - The _ZkCredentialsInjector_ gets the creds from an external source which get injected into zkCredentialsProvider and zkACLProvider. The "_external source_" here can be system props, a file, a Secret Manager, or any other local or remote source. ``` public interface ZkCredentialsInjector { List<ZkCredential> getZkCredentials(); ... } ``` - Any class implementing _ZkCredentialsInjector_ can be injected via system props in _solr.ini.sh/cmd_. In the below example _VMParamsZkCredentialsInjector_ is injected. Note: _VMParamsAllAndReadonlyDigestZkACLProvider_ and _VMParamsSingleSetCredentialsDigestZkCredentialsProvider_ would be deprecated and replaced with a combination of _DigestZkACLProvider_/_DigestZkCredentialsProvider_ and _VMParamsZkCredentialsInjector_. ``` SOLR_ZK_CREDS_AND_ACLS=“ -DzkACLProvider=org.apache.solr.common.cloud.acl.DigestZkACLProvider \ -DzkCredentialsProvider=org.apache.solr.common.cloud.acl.DigestZkCredentialsProvider \ -DzkCredentialsInjector=org.apache.solr.common.cloud.acl.VMParamsZkCredentialsInjector \ -DzkDigestUsername=admin-user -DzkDigestPassword=CHANGEME-ADMIN-PASSWORD \ -DzkDigestReadonlyUsername=readonly-user -DzkDigestReadonlyPassword=CHANGEME-READONLY-PASSWORD" SOLR_OPTS="$SOLR_OPTS $SOLR_ZK_CREDS_AND_ACLS" ``` - Add _DigestZkACLProvider_/_DigestZkCredentialsProvider_ classes to support _Digest_ based scheme ZK authentication/authorization ``` Class DigestZkACLProvider implements ZkACLProvider{ CredentialsInjector credentialsInjector; ... } Class DigestZkCredentialsProvider implements ZkCredentialsProvider{ CredentialsInjector credentialsInjector; ... } ``` This concept can be generalized to non-digest schemes (a kind of Strategy pattern) but that would require more refactoring, it can be achieved in a future contribution if this one is accepted. Now apply this new feature and add a custom injector to solve problem 1). **Problem 1** - Store the credentials in a Secret Manager to have Solr pull them out at startup. - Add _SecretCredentialInjector_ class that contains a dependency interface (_SecretCredentialsProvider_) whose implementation pulls zk credentials from a Secret Manager and delegate the _getZkCredentials_ call. ``` public class SecretCredentialInjector implements ZkCredentialsInjector { ... private SecretCredentialsProvider secretCredentialProvider; public List<ZkCredential> getZkCredentials() { ... return secretCredentialProvider.getZkCredentials(secretName); } ... } ``` - In this contribution the offered implementating class is _AWSSecretCredentialsProvider_ that gets zk credentials from AWS Secret Manager. Tu support any other Secret Manager provider all you need to do is add a class implementing _SecretCredentialsProvider_ and pass it through system props (-_DzkSecretCredentialsProvider_) ``` SOLR_ZK_CREDS_AND_ACLS="-DzkACLProvider=org.apache.solr.common.cloud.acl.DigestZkACLProvider \ -DzkCredentialsProvider=org.apache.solr.common.cloud.acl.DigestZkCredentialsProvider \ -DzkCredentialsInjector=org.apache.solr.common.cloud.acl.SecretCredentialInjector \ -DzkSecretCredentialsProvider=org.apache.solr.secret.zk.AWSSecretCredentialsProvider \ -DzkSecretCredentialSecretName=zkSecret \ -DzkCredentialsAWSSecretRegion=us-west-2" SOLR_OPTS="$SOLR_OPTS $SOLR_ZK_CREDS_AND_ACLS" ``` **Problem 3** A new _contrib_ module (_secret-provider_) is added where _SecretCredentialsProvider_ implementing classes can be added without the need to add a new dependency to Solr core. All one needs to do after adding a new class is to pass it through system props via _solr.ini.sh/cmd_ file. This module can be used in the future for other secrets injections, not specifically related to zk. Thank you in advance for your time and your comments. # Checklist Please review the following and check all that apply: - [x] I have reviewed the guidelines for [How to Contribute](https://wiki.apache.org/solr/HowToContribute) and my code conforms to the standards described there to the best of my ability. - [x] I have created a Jira issue and added the issue ID to my pull request title. - [x] I have given Solr maintainers [access](https://help.github.com/en/articles/allowing-changes-to-a-pull-request-branch-created-from-a-fork) to contribute to my PR branch. (optional but recommended) - [x] I have developed this patch against the `main` branch. - [x] I have run `./gradlew check`. - [x] I have added tests for my changes. - [x] I have added documentation for the [Reference Guide](https://github.com/apache/solr/tree/main/solr/solr-ref-guide) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
