[
https://issues.apache.org/jira/browse/SOLR-15857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lamine updated SOLR-15857:
--------------------------
Description:
*{*}Problem{*}*
Solr uses a list of credentials to connect to Zookeeper and to handle ACLs.
- 1- In the current implementation, the credentials are passed through command
line (system props) or read from a clear text file stored in all cluster hosts.
Needless to say this is not safe enough.
- 2- On the other hand, the same code to load the credentials is called twice,
first by _ZkCredentialsProvider_ to connect to Zookeeper and a second time by
_ZkACLProvider_ to create ACLs. The code is also duplicated, although it's only
reading from system props.
Adding a custom pair of {_}ZkCredentialsProvider{_}/{_}ZkACLProvider{_} to
load the credentials from another source (ex a Secret Manager) would also
require duplicate the code and make repetitive calls to extract the same
credentials.
- 3- There is no way to customize how credentials are passed without
recompiling.
*{*}Proposed solution{*}*
Let’s start with problem 2).
*{*}Problem 2{*}*
- Refactor the way how the credentials are injected by passing them as a
dependency. One code, called once and injected into the client class. Here the
client classes are _ZkCredentialsProvider_ and {_}ZkACLProvider{_}.
- Favor composition over inheritance to inject custom credentials loaders
without changing the composing (container) class.
- Add a third interface _ZkCredentialsInjector_ whose implementations load ZK
credentials from a credentials source to be injected into
_ZkCredentialsProvider_ and _ZkACLProvider_
- The workflow is: Credentials source —> _ZkCredentialsInjector_ -->
{_}ZkCredentialsProvider{_}/{_}ZkACLProvider{_} --> Zookeeper
- The _ZkCredentialsInjector_ gets the creds from an external source which get
injected into zkCredentialsProvider and zkACLProvider. The "{_}external
source{_}" here can be system props, a file, a Secret Manager, or any other
local or remote source.
{code:java}
public interface ZkCredentialsInjector{
List<ZkCredential> getZkCredentials();
...
}
{code}
- Any class implementing _ZkCredentialsInjector_ can be injected via system
props in {_}solr.ini.sh/cmd{_}.
In the below example _VMParamsZkCredentialsInjector_ is injected.
Note: _VMParamsAllAndReadonlyDigestZkACLProvider_ and
_VMParamsSingleSetCredentialsDigestZkCredentialsProvider_ would be deprecated
and replaced with a combination of
{_}DigestZkACLProvider{_}/{_}DigestZkCredentialsProvider{_} and
{_}VMParamsZkCredentialsInjector{_}.
{code:java}
SOLR_ZK_CREDS_AND_ACLS=“
-DzkACLProvider=org.apache.solr.common.cloud.acl.DigestZkACLProvider \
-DzkCredentialsProvider=org.apache.solr.common.cloud.acl.DigestZkCredentialsProvider
\
-DzkCredentialsInjector=org.apache.solr.common.cloud.acl.VMParamsZkCredentialsInjector
\
-DzkDigestUsername=admin-user -DzkDigestPassword=CHANGEME-ADMIN-PASSWORD \
-DzkDigestReadonlyUsername=readonly-user
-DzkDigestReadonlyPassword=CHANGEME-READONLY-PASSWORD"
SOLR_OPTS="$SOLR_OPTS $SOLR_ZK_CREDS_AND_ACLS"{code}
- Add {_}DigestZkACLProvider{_}/{_}DigestZkCredentialsProvider{_} classes to
support _Digest_ based scheme ZK authentication/authorization
{code:java}
Class DigestZkACLProvider implements ZkACLProvider { CredentialsInjector
credentialsInjector;
...
}
Class DigestZkCredentialsProvider implements ZkCredentialsProvider{
CredentialsInjector credentialsInjector;
...
}{code}
This concept can be generalized to non-digest schemes (a kind of Strategy
pattern) but that would require more refactoring, it can be achieved in a
future contribution if this one is accepted.
Now apply this new feature and add a custom injector to solve problem 1).
*{*}Problem 1{*}*
- Store the credentials in a Secret Manager to have Solr pull them out at
startup.
- Add _SecretCredentialInjector_ class that contains a dependency interface
({_}SecretCredentialsProvider{_}) whose implementation pulls zk credentials
from a Secret Manager and delegate the _getZkCredentials_ call.
{code:java}
public class SecretCredentialInjector implements ZkCredentialsInjector {
...
private SecretCredentialsProvider secretCredentialProvider;
public List<ZkCredential> getZkCredentials() {
...
return secretCredentialProvider.getZkCredentials(secretName);
}
...
}
{code}
- In this contribution the offered implementating class is
_AWSSecretCredentialsProvider_ that gets zk credentials from AWS Secret
Manager. Tu support any other Secret Manager provider all you need to do is add
a class implementing _SecretCredentialsProvider_ and pass it through system
props (-{_}DzkSecretCredentialsProvider{_})
{code:java}
SOLR_ZK_CREDS_AND_ACLS="-DzkACLProvider=org.apache.solr.common.cloud.acl.DigestZkACLProvider
\
-DzkCredentialsProvider=org.apache.solr.common.cloud.acl.DigestZkCredentialsProvider
\
-DzkCredentialsInjector=org.apache.solr.common.cloud.acl.SecretCredentialInjector
\
-DzkSecretCredentialsProvider=org.apache.solr.secret.zk.AWSSecretCredentialsProvider
\
-DzkSecretCredentialSecretName=zkSecret \
-DzkCredentialsAWSSecretRegion=us-west-2"
SOLR_OPTS="$SOLR_OPTS $SOLR_ZK_CREDS_AND_ACLS"
{code}
*{*}Problem 3{*}*
A new _contrib_ module ({_}secret-provider{_}) is added where
_SecretCredentialsProvider_ implementing classes can be added without the need
to add a new dependency to Solr core. All one needs to do after adding a new
class is to pass it through system props via _solr.ini.sh/cmd_ file.
This module can be used in the future for other secrets injections, not
specifically related to zk.
Thank you in advance for your time and your comments.
was:
https://issues.apache.org/jira/browse/SOLR-15857
**Problem**
Solr uses a list of credentials to connect to Zookeeper and to handle ACLs.
- 1- In the current implementation, the credentials are passed through command
line (system props) or read from a clear text file stored in all cluster hosts.
Needless to say this is not safe enough.
- 2- On the other hand, the same code to load the credentials is called twice,
first by _ZkCredentialsProvider_ to connect to Zookeeper and a second time by
_ZkACLProvider_ to create ACLs. The code is also duplicated, although it's only
reading from system props.
Adding a custom pair of _ZkCredentialsProvider_/_ZkACLProvider_ to load the
credentials from another source (ex a Secret Manager) would also require
duplicate the code and make repetitive calls to extract the same credentials.
- 3- There is no way to customize how credentials are passed without
recompiling.
**Proposed solution**
Let’s start with problem 2).
**Problem 2**
- Refactor the way how the credentials are injected by passing them as a
dependency. One code, called once and injected into the client class. Here the
client classes are _ZkCredentialsProvider_ and _ZkACLProvider_.
- Favor composition over inheritance to inject custom credentials loaders
without changing the composing (container) class.
- Add a third interface _ZkCredentialsInjector_ whose implementations load ZK
credentials from a credentials source to be injected into
_ZkCredentialsProvider_ and _ZkACLProvider_
- The workflow is: Credentials source —> _ZkCredentialsInjector_ -->
_ZkCredentialsProvider_/_ZkACLProvider_ --> Zookeeper
- The _ZkCredentialsInjector_ gets the creds from an external source which get
injected into zkCredentialsProvider and zkACLProvider. The "_external source_"
here can be system props, a file, a Secret Manager, or any other local or
remote source.
```
public interface ZkCredentialsInjector {
List<ZkCredential> getZkCredentials();
...
}
```
- Any class implementing _ZkCredentialsInjector_ can be injected via system
props in _solr.ini.sh/cmd_.
In the below example _VMParamsZkCredentialsInjector_ is injected.
Note: _VMParamsAllAndReadonlyDigestZkACLProvider_ and
_VMParamsSingleSetCredentialsDigestZkCredentialsProvider_ would be deprecated
and replaced with a combination of
_DigestZkACLProvider_/_DigestZkCredentialsProvider_ and
_VMParamsZkCredentialsInjector_.
```
SOLR_ZK_CREDS_AND_ACLS=“
-DzkACLProvider=org.apache.solr.common.cloud.acl.DigestZkACLProvider \
-DzkCredentialsProvider=org.apache.solr.common.cloud.acl.DigestZkCredentialsProvider
\
-DzkCredentialsInjector=org.apache.solr.common.cloud.acl.VMParamsZkCredentialsInjector
\
-DzkDigestUsername=admin-user -DzkDigestPassword=CHANGEME-ADMIN-PASSWORD \
-DzkDigestReadonlyUsername=readonly-user
-DzkDigestReadonlyPassword=CHANGEME-READONLY-PASSWORD"
SOLR_OPTS="$SOLR_OPTS $SOLR_ZK_CREDS_AND_ACLS"
```
- Add _DigestZkACLProvider_/_DigestZkCredentialsProvider_ classes to support
_Digest_ based scheme ZK authentication/authorization
```
Class DigestZkACLProvider implements ZkACLProvider{
CredentialsInjector credentialsInjector;
...
}
Class DigestZkCredentialsProvider implements ZkCredentialsProvider{
CredentialsInjector credentialsInjector;
...
}
```
This concept can be generalized to non-digest schemes (a kind of Strategy
pattern) but that would require more refactoring, it can be achieved in a
future contribution if this one is accepted.
Now apply this new feature and add a custom injector to solve problem 1).
**Problem 1**
- Store the credentials in a Secret Manager to have Solr pull them out at
startup.
- Add _SecretCredentialInjector_ class that contains a dependency interface
(_SecretCredentialsProvider_) whose implementation pulls zk credentials from a
Secret Manager and delegate the _getZkCredentials_ call.
```
public class SecretCredentialInjector implements ZkCredentialsInjector {
...
private SecretCredentialsProvider secretCredentialProvider;
public List<ZkCredential> getZkCredentials() {
...
return secretCredentialProvider.getZkCredentials(secretName);
}
...
}
```
- In this contribution the offered implementating class is
_AWSSecretCredentialsProvider_ that gets zk credentials from AWS Secret
Manager. Tu support any other Secret Manager provider all you need to do is add
a class implementing _SecretCredentialsProvider_ and pass it through system
props (-_DzkSecretCredentialsProvider_)
```
SOLR_ZK_CREDS_AND_ACLS="-DzkACLProvider=org.apache.solr.common.cloud.acl.DigestZkACLProvider
\
-DzkCredentialsProvider=org.apache.solr.common.cloud.acl.DigestZkCredentialsProvider
\
-DzkCredentialsInjector=org.apache.solr.common.cloud.acl.SecretCredentialInjector
\
-DzkSecretCredentialsProvider=org.apache.solr.secret.zk.AWSSecretCredentialsProvider
\
-DzkSecretCredentialSecretName=zkSecret \
-DzkCredentialsAWSSecretRegion=us-west-2"
SOLR_OPTS="$SOLR_OPTS $SOLR_ZK_CREDS_AND_ACLS"
```
**Problem 3**
A new _contrib_ module (_secret-provider_) is added where
_SecretCredentialsProvider_ implementing classes can be added without the need
to add a new dependency to Solr core. All one needs to do after adding a new
class is to pass it through system props via _solr.ini.sh/cmd_ file.
This module can be used in the future for other secrets injections, not
specifically related to zk.
Thank you in advance for your time and your comments.
> Add Secret Manager support for ZK ACL credentials
> -------------------------------------------------
>
> Key: SOLR-15857
> URL: https://issues.apache.org/jira/browse/SOLR-15857
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Reporter: Lamine
> Priority: Minor
> Time Spent: 10m
> Remaining Estimate: 0h
>
>
> *{*}Problem{*}*
> Solr uses a list of credentials to connect to Zookeeper and to handle ACLs.
> - 1- In the current implementation, the credentials are passed through
> command line (system props) or read from a clear text file stored in all
> cluster hosts. Needless to say this is not safe enough.
> - 2- On the other hand, the same code to load the credentials is called
> twice, first by _ZkCredentialsProvider_ to connect to Zookeeper and a second
> time by _ZkACLProvider_ to create ACLs. The code is also duplicated, although
> it's only reading from system props.
> Adding a custom pair of {_}ZkCredentialsProvider{_}/{_}ZkACLProvider{_} to
> load the credentials from another source (ex a Secret Manager) would also
> require duplicate the code and make repetitive calls to extract the same
> credentials.
> - 3- There is no way to customize how credentials are passed without
> recompiling.
>
> *{*}Proposed solution{*}*
> Let’s start with problem 2).
> *{*}Problem 2{*}*
> - Refactor the way how the credentials are injected by passing them as a
> dependency. One code, called once and injected into the client class. Here
> the client classes are _ZkCredentialsProvider_ and {_}ZkACLProvider{_}.
> - Favor composition over inheritance to inject custom credentials loaders
> without changing the composing (container) class.
> - Add a third interface _ZkCredentialsInjector_ whose implementations load
> ZK credentials from a credentials source to be injected into
> _ZkCredentialsProvider_ and _ZkACLProvider_
> - The workflow is: Credentials source —> _ZkCredentialsInjector_ -->
> {_}ZkCredentialsProvider{_}/{_}ZkACLProvider{_} --> Zookeeper
> - The _ZkCredentialsInjector_ gets the creds from an external source which
> get injected into zkCredentialsProvider and zkACLProvider. The "{_}external
> source{_}" here can be system props, a file, a Secret Manager, or any other
> local or remote source.
>
> {code:java}
> public interface ZkCredentialsInjector{
> List<ZkCredential> getZkCredentials();
> ...
> }
> {code}
>
>
> - Any class implementing _ZkCredentialsInjector_ can be injected via system
> props in {_}solr.ini.sh/cmd{_}.
> In the below example _VMParamsZkCredentialsInjector_ is injected.
> Note: _VMParamsAllAndReadonlyDigestZkACLProvider_ and
> _VMParamsSingleSetCredentialsDigestZkCredentialsProvider_ would be deprecated
> and replaced with a combination of
> {_}DigestZkACLProvider{_}/{_}DigestZkCredentialsProvider{_} and
> {_}VMParamsZkCredentialsInjector{_}.
>
> {code:java}
> SOLR_ZK_CREDS_AND_ACLS=“
> -DzkACLProvider=org.apache.solr.common.cloud.acl.DigestZkACLProvider \
>
> -DzkCredentialsProvider=org.apache.solr.common.cloud.acl.DigestZkCredentialsProvider
> \
>
> -DzkCredentialsInjector=org.apache.solr.common.cloud.acl.VMParamsZkCredentialsInjector
> \
> -DzkDigestUsername=admin-user -DzkDigestPassword=CHANGEME-ADMIN-PASSWORD
> \
> -DzkDigestReadonlyUsername=readonly-user
> -DzkDigestReadonlyPassword=CHANGEME-READONLY-PASSWORD"
> SOLR_OPTS="$SOLR_OPTS $SOLR_ZK_CREDS_AND_ACLS"{code}
>
> - Add {_}DigestZkACLProvider{_}/{_}DigestZkCredentialsProvider{_} classes
> to support _Digest_ based scheme ZK authentication/authorization
>
> {code:java}
> Class DigestZkACLProvider implements ZkACLProvider { CredentialsInjector
> credentialsInjector;
> ...
> }
> Class DigestZkCredentialsProvider implements ZkCredentialsProvider{
> CredentialsInjector credentialsInjector;
> ...
> }{code}
>
> This concept can be generalized to non-digest schemes (a kind of Strategy
> pattern) but that would require more refactoring, it can be achieved in a
> future contribution if this one is accepted.
> Now apply this new feature and add a custom injector to solve problem 1).
> *{*}Problem 1{*}*
> - Store the credentials in a Secret Manager to have Solr pull them out at
> startup.
> - Add _SecretCredentialInjector_ class that contains a dependency interface
> ({_}SecretCredentialsProvider{_}) whose implementation pulls zk credentials
> from a Secret Manager and delegate the _getZkCredentials_ call.
> {code:java}
> public class SecretCredentialInjector implements ZkCredentialsInjector {
> ...
> private SecretCredentialsProvider secretCredentialProvider;
>
> public List<ZkCredential> getZkCredentials() {
> ...
> return secretCredentialProvider.getZkCredentials(secretName);
> }
> ...
> }
> {code}
>
> - In this contribution the offered implementating class is
> _AWSSecretCredentialsProvider_ that gets zk credentials from AWS Secret
> Manager. Tu support any other Secret Manager provider all you need to do is
> add a class implementing _SecretCredentialsProvider_ and pass it through
> system props (-{_}DzkSecretCredentialsProvider{_})
>
>
> {code:java}
> SOLR_ZK_CREDS_AND_ACLS="-DzkACLProvider=org.apache.solr.common.cloud.acl.DigestZkACLProvider
> \
>
> -DzkCredentialsProvider=org.apache.solr.common.cloud.acl.DigestZkCredentialsProvider
> \
>
> -DzkCredentialsInjector=org.apache.solr.common.cloud.acl.SecretCredentialInjector
> \
>
> -DzkSecretCredentialsProvider=org.apache.solr.secret.zk.AWSSecretCredentialsProvider
> \
> -DzkSecretCredentialSecretName=zkSecret \
> -DzkCredentialsAWSSecretRegion=us-west-2"
> SOLR_OPTS="$SOLR_OPTS $SOLR_ZK_CREDS_AND_ACLS"
>
> {code}
>
> *{*}Problem 3{*}*
> A new _contrib_ module ({_}secret-provider{_}) is added where
> _SecretCredentialsProvider_ implementing classes can be added without the
> need to add a new dependency to Solr core. All one needs to do after adding a
> new class is to pass it through system props via _solr.ini.sh/cmd_ file.
> This module can be used in the future for other secrets injections, not
> specifically related to zk.
>
> Thank you in advance for your time and your comments.
>
>
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
