[
https://issues.apache.org/jira/browse/SOLR-15857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lamine updated SOLR-15857:
--------------------------
Description:
_{*}Note{*}: Attached a copy of a ref guide describing this feature in more
details and with usage examples._
*Problem*
Solr uses a list of credentials to connect to Zookeeper and to handle ACLs.
- 1- In the current implementation, the credentials are passed through command
line (system props) or read from a clear text file stored in all cluster hosts.
Needless to say this is not safe enough.
- 2- On the other hand, the same code to load the credentials is called twice,
first by _ZkCredentialsProvider_ to connect to Zookeeper and a second time by
_ZkACLProvider_ to create ACLs. The code is also duplicated, although it's only
reading from system props.
Adding a custom pair of {_}ZkCredentialsProvider{_}/{_}ZkACLProvider{_} to
load the credentials from another source (ex a Secret Manager) would also
require duplicate the code and make repetitive calls to extract the same
credentials.
- 3- There is no way to customize how credentials are passed without
recompiling.
*Proposed solution*
Let’s start with problem 2).
*Problem 2*
- Refactor the way how the credentials are injected by passing them as a
dependency. One code, called once and injected into the client class. Here the
client classes are _ZkCredentialsProvider_ and {_}ZkACLProvider{_}.
- Favor composition over inheritance to inject custom credentials loaders
without changing the composing (container) class.
- Add a third interface _ZkCredentialsInjector_ whose implementations load ZK
credentials from a credentials source to be injected into
_ZkCredentialsProvider_ and _ZkACLProvider_
- The workflow is: Credentials source —> _ZkCredentialsInjector_ -->
{_}ZkCredentialsProvider{_}/{_}ZkACLProvider{_} --> Zookeeper
- The _ZkCredentialsInjector_ gets the creds from an external source which get
injected into zkCredentialsProvider and zkACLProvider. The "{_}external
source{_}" here can be system props, a file, a Secret Manager, or any other
local or remote source.
{code:java}
public interface ZkCredentialsInjector{
List<ZkCredential> getZkCredentials();
...
}
{code}
- Any class implementing _ZkCredentialsInjector_ can be injected via system
props in {_}solr.ini.sh/cmd{_}.
In the below example _VMParamsZkCredentialsInjector_ is injected.
Note: _VMParamsAllAndReadonlyDigestZkACLProvider_ and
_VMParamsSingleSetCredentialsDigestZkCredentialsProvider_ would be deprecated
and replaced with a combination of
{_}DigestZkACLProvider{_}/{_}DigestZkCredentialsProvider{_} and
{_}VMParamsZkCredentialsInjector{_}.
{code:java}
SOLR_ZK_CREDS_AND_ACLS=“
-DzkACLProvider=org.apache.solr.common.cloud.acl.DigestZkACLProvider \
-DzkCredentialsProvider=org.apache.solr.common.cloud.acl.DigestZkCredentialsProvider
\
-DzkCredentialsInjector=org.apache.solr.common.cloud.acl.VMParamsZkCredentialsInjector
\
-DzkDigestUsername=admin-user -DzkDigestPassword=CHANGEME-ADMIN-PASSWORD \
-DzkDigestReadonlyUsername=readonly-user
-DzkDigestReadonlyPassword=CHANGEME-READONLY-PASSWORD"
SOLR_OPTS="$SOLR_OPTS $SOLR_ZK_CREDS_AND_ACLS"{code}
- Add {_}DigestZkACLProvider{_}/{_}DigestZkCredentialsProvider{_} classes to
support _Digest_ based scheme ZK authentication/authorization
{code:java}
Class DigestZkACLProvider implements ZkACLProvider { CredentialsInjector
credentialsInjector;
...
}
Class DigestZkCredentialsProvider implements ZkCredentialsProvider{
CredentialsInjector credentialsInjector;
...
}{code}
This concept can be generalized to non-digest schemes (a kind of Strategy
pattern) but that would require more refactoring, it can be achieved in a
future contribution if this one is accepted.
Now apply this new feature and add a custom injector to solve problem 1).
*Problem 1*
- Store the credentials in a Secret Manager to have Solr pull them out at
startup.
- Add _SecretCredentialInjector_ class that contains a dependency interface
({_}SecretCredentialsProvider{_}) whose implementation pulls zk credentials
from a Secret Manager and delegate the _getZkCredentials_ call.
{code:java}
public class SecretCredentialInjector implements ZkCredentialsInjector {
...
private SecretCredentialsProvider secretCredentialProvider;
public List<ZkCredential> getZkCredentials() {
...
return secretCredentialProvider.getZkCredentials(secretName);
}
...
}
{code}
- In this contribution the offered implementating class is
_AWSSecretCredentialsProvider_ that gets zk credentials from AWS Secret
Manager. Tu support any other Secret Manager provider all you need to do is add
a class implementing _SecretCredentialsProvider_ and pass it through system
props (-{_}DzkSecretCredentialsProvider{_})
{code:java}
SOLR_ZK_CREDS_AND_ACLS="-DzkACLProvider=org.apache.solr.common.cloud.acl.DigestZkACLProvider
\
-DzkCredentialsProvider=org.apache.solr.common.cloud.acl.DigestZkCredentialsProvider
\
-DzkCredentialsInjector=org.apache.solr.common.cloud.acl.SecretCredentialInjector
\
-DzkSecretCredentialsProvider=org.apache.solr.secret.zk.AWSSecretCredentialsProvider
\
-DzkSecretCredentialSecretName=zkSecret \
-DzkCredentialsAWSSecretRegion=us-west-2"
SOLR_OPTS="$SOLR_OPTS $SOLR_ZK_CREDS_AND_ACLS"
{code}
*Problem 3*
A new _contrib_ module ({_}secret-provider{_}) is added where
_SecretCredentialsProvider_ implementing classes can be added without the need
to add a new dependency to Solr core. All one needs to do after adding a new
class is to pass it through system props via _solr.ini.sh/cmd_ file.
This module can be used in the future for other secrets injections, not
specifically related to zk.
Thank you in advance for your time and your comments.
was:
_{*}Note{*}: Attached a copy of a ref guide describing this feature in more
details and examples._
*Problem*
Solr uses a list of credentials to connect to Zookeeper and to handle ACLs.
- 1- In the current implementation, the credentials are passed through command
line (system props) or read from a clear text file stored in all cluster hosts.
Needless to say this is not safe enough.
- 2- On the other hand, the same code to load the credentials is called twice,
first by _ZkCredentialsProvider_ to connect to Zookeeper and a second time by
_ZkACLProvider_ to create ACLs. The code is also duplicated, although it's only
reading from system props.
Adding a custom pair of {_}ZkCredentialsProvider{_}/{_}ZkACLProvider{_} to
load the credentials from another source (ex a Secret Manager) would also
require duplicate the code and make repetitive calls to extract the same
credentials.
- 3- There is no way to customize how credentials are passed without
recompiling.
*Proposed solution*
Let’s start with problem 2).
*Problem 2*
- Refactor the way how the credentials are injected by passing them as a
dependency. One code, called once and injected into the client class. Here the
client classes are _ZkCredentialsProvider_ and {_}ZkACLProvider{_}.
- Favor composition over inheritance to inject custom credentials loaders
without changing the composing (container) class.
- Add a third interface _ZkCredentialsInjector_ whose implementations load ZK
credentials from a credentials source to be injected into
_ZkCredentialsProvider_ and _ZkACLProvider_
- The workflow is: Credentials source —> _ZkCredentialsInjector_ -->
{_}ZkCredentialsProvider{_}/{_}ZkACLProvider{_} --> Zookeeper
- The _ZkCredentialsInjector_ gets the creds from an external source which get
injected into zkCredentialsProvider and zkACLProvider. The "{_}external
source{_}" here can be system props, a file, a Secret Manager, or any other
local or remote source.
{code:java}
public interface ZkCredentialsInjector{
List<ZkCredential> getZkCredentials();
...
}
{code}
- Any class implementing _ZkCredentialsInjector_ can be injected via system
props in {_}solr.ini.sh/cmd{_}.
In the below example _VMParamsZkCredentialsInjector_ is injected.
Note: _VMParamsAllAndReadonlyDigestZkACLProvider_ and
_VMParamsSingleSetCredentialsDigestZkCredentialsProvider_ would be deprecated
and replaced with a combination of
{_}DigestZkACLProvider{_}/{_}DigestZkCredentialsProvider{_} and
{_}VMParamsZkCredentialsInjector{_}.
{code:java}
SOLR_ZK_CREDS_AND_ACLS=“
-DzkACLProvider=org.apache.solr.common.cloud.acl.DigestZkACLProvider \
-DzkCredentialsProvider=org.apache.solr.common.cloud.acl.DigestZkCredentialsProvider
\
-DzkCredentialsInjector=org.apache.solr.common.cloud.acl.VMParamsZkCredentialsInjector
\
-DzkDigestUsername=admin-user -DzkDigestPassword=CHANGEME-ADMIN-PASSWORD \
-DzkDigestReadonlyUsername=readonly-user
-DzkDigestReadonlyPassword=CHANGEME-READONLY-PASSWORD"
SOLR_OPTS="$SOLR_OPTS $SOLR_ZK_CREDS_AND_ACLS"{code}
- Add {_}DigestZkACLProvider{_}/{_}DigestZkCredentialsProvider{_} classes to
support _Digest_ based scheme ZK authentication/authorization
{code:java}
Class DigestZkACLProvider implements ZkACLProvider { CredentialsInjector
credentialsInjector;
...
}
Class DigestZkCredentialsProvider implements ZkCredentialsProvider{
CredentialsInjector credentialsInjector;
...
}{code}
This concept can be generalized to non-digest schemes (a kind of Strategy
pattern) but that would require more refactoring, it can be achieved in a
future contribution if this one is accepted.
Now apply this new feature and add a custom injector to solve problem 1).
*Problem 1*
- Store the credentials in a Secret Manager to have Solr pull them out at
startup.
- Add _SecretCredentialInjector_ class that contains a dependency interface
({_}SecretCredentialsProvider{_}) whose implementation pulls zk credentials
from a Secret Manager and delegate the _getZkCredentials_ call.
{code:java}
public class SecretCredentialInjector implements ZkCredentialsInjector {
...
private SecretCredentialsProvider secretCredentialProvider;
public List<ZkCredential> getZkCredentials() {
...
return secretCredentialProvider.getZkCredentials(secretName);
}
...
}
{code}
- In this contribution the offered implementating class is
_AWSSecretCredentialsProvider_ that gets zk credentials from AWS Secret
Manager. Tu support any other Secret Manager provider all you need to do is add
a class implementing _SecretCredentialsProvider_ and pass it through system
props (-{_}DzkSecretCredentialsProvider{_})
{code:java}
SOLR_ZK_CREDS_AND_ACLS="-DzkACLProvider=org.apache.solr.common.cloud.acl.DigestZkACLProvider
\
-DzkCredentialsProvider=org.apache.solr.common.cloud.acl.DigestZkCredentialsProvider
\
-DzkCredentialsInjector=org.apache.solr.common.cloud.acl.SecretCredentialInjector
\
-DzkSecretCredentialsProvider=org.apache.solr.secret.zk.AWSSecretCredentialsProvider
\
-DzkSecretCredentialSecretName=zkSecret \
-DzkCredentialsAWSSecretRegion=us-west-2"
SOLR_OPTS="$SOLR_OPTS $SOLR_ZK_CREDS_AND_ACLS"
{code}
*Problem 3*
A new _contrib_ module ({_}secret-provider{_}) is added where
_SecretCredentialsProvider_ implementing classes can be added without the need
to add a new dependency to Solr core. All one needs to do after adding a new
class is to pass it through system props via _solr.ini.sh/cmd_ file.
This module can be used in the future for other secrets injections, not
specifically related to zk.
Thank you in advance for your time and your comments.
> Add Secret Manager support for ZK ACL credentials
> -------------------------------------------------
>
> Key: SOLR-15857
> URL: https://issues.apache.org/jira/browse/SOLR-15857
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Reporter: Lamine
> Priority: Minor
> Attachments: ZooKeeper_Access_Control.pdf
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> _{*}Note{*}: Attached a copy of a ref guide describing this feature in more
> details and with usage examples._
> *Problem*
> Solr uses a list of credentials to connect to Zookeeper and to handle ACLs.
> - 1- In the current implementation, the credentials are passed through
> command line (system props) or read from a clear text file stored in all
> cluster hosts. Needless to say this is not safe enough.
> - 2- On the other hand, the same code to load the credentials is called
> twice, first by _ZkCredentialsProvider_ to connect to Zookeeper and a second
> time by _ZkACLProvider_ to create ACLs. The code is also duplicated, although
> it's only reading from system props.
> Adding a custom pair of {_}ZkCredentialsProvider{_}/{_}ZkACLProvider{_} to
> load the credentials from another source (ex a Secret Manager) would also
> require duplicate the code and make repetitive calls to extract the same
> credentials.
> - 3- There is no way to customize how credentials are passed without
> recompiling.
>
> *Proposed solution*
> Let’s start with problem 2).
> *Problem 2*
> - Refactor the way how the credentials are injected by passing them as a
> dependency. One code, called once and injected into the client class. Here
> the client classes are _ZkCredentialsProvider_ and {_}ZkACLProvider{_}.
> - Favor composition over inheritance to inject custom credentials loaders
> without changing the composing (container) class.
> - Add a third interface _ZkCredentialsInjector_ whose implementations load
> ZK credentials from a credentials source to be injected into
> _ZkCredentialsProvider_ and _ZkACLProvider_
> - The workflow is: Credentials source —> _ZkCredentialsInjector_ -->
> {_}ZkCredentialsProvider{_}/{_}ZkACLProvider{_} --> Zookeeper
> - The _ZkCredentialsInjector_ gets the creds from an external source which
> get injected into zkCredentialsProvider and zkACLProvider. The "{_}external
> source{_}" here can be system props, a file, a Secret Manager, or any other
> local or remote source.
>
> {code:java}
> public interface ZkCredentialsInjector{
> List<ZkCredential> getZkCredentials();
> ...
> }
> {code}
>
>
> - Any class implementing _ZkCredentialsInjector_ can be injected via system
> props in {_}solr.ini.sh/cmd{_}.
> In the below example _VMParamsZkCredentialsInjector_ is injected.
> Note: _VMParamsAllAndReadonlyDigestZkACLProvider_ and
> _VMParamsSingleSetCredentialsDigestZkCredentialsProvider_ would be deprecated
> and replaced with a combination of
> {_}DigestZkACLProvider{_}/{_}DigestZkCredentialsProvider{_} and
> {_}VMParamsZkCredentialsInjector{_}.
>
> {code:java}
> SOLR_ZK_CREDS_AND_ACLS=“
> -DzkACLProvider=org.apache.solr.common.cloud.acl.DigestZkACLProvider \
>
> -DzkCredentialsProvider=org.apache.solr.common.cloud.acl.DigestZkCredentialsProvider
> \
>
> -DzkCredentialsInjector=org.apache.solr.common.cloud.acl.VMParamsZkCredentialsInjector
> \
> -DzkDigestUsername=admin-user -DzkDigestPassword=CHANGEME-ADMIN-PASSWORD
> \
> -DzkDigestReadonlyUsername=readonly-user
> -DzkDigestReadonlyPassword=CHANGEME-READONLY-PASSWORD"
> SOLR_OPTS="$SOLR_OPTS $SOLR_ZK_CREDS_AND_ACLS"{code}
>
> - Add {_}DigestZkACLProvider{_}/{_}DigestZkCredentialsProvider{_} classes
> to support _Digest_ based scheme ZK authentication/authorization
>
> {code:java}
> Class DigestZkACLProvider implements ZkACLProvider { CredentialsInjector
> credentialsInjector;
> ...
> }
> Class DigestZkCredentialsProvider implements ZkCredentialsProvider{
> CredentialsInjector credentialsInjector;
> ...
> }{code}
>
> This concept can be generalized to non-digest schemes (a kind of Strategy
> pattern) but that would require more refactoring, it can be achieved in a
> future contribution if this one is accepted.
> Now apply this new feature and add a custom injector to solve problem 1).
> *Problem 1*
> - Store the credentials in a Secret Manager to have Solr pull them out at
> startup.
> - Add _SecretCredentialInjector_ class that contains a dependency interface
> ({_}SecretCredentialsProvider{_}) whose implementation pulls zk credentials
> from a Secret Manager and delegate the _getZkCredentials_ call.
> {code:java}
> public class SecretCredentialInjector implements ZkCredentialsInjector {
> ...
> private SecretCredentialsProvider secretCredentialProvider;
>
> public List<ZkCredential> getZkCredentials() {
> ...
> return secretCredentialProvider.getZkCredentials(secretName);
> }
> ...
> }
> {code}
>
> - In this contribution the offered implementating class is
> _AWSSecretCredentialsProvider_ that gets zk credentials from AWS Secret
> Manager. Tu support any other Secret Manager provider all you need to do is
> add a class implementing _SecretCredentialsProvider_ and pass it through
> system props (-{_}DzkSecretCredentialsProvider{_})
>
>
> {code:java}
> SOLR_ZK_CREDS_AND_ACLS="-DzkACLProvider=org.apache.solr.common.cloud.acl.DigestZkACLProvider
> \
>
> -DzkCredentialsProvider=org.apache.solr.common.cloud.acl.DigestZkCredentialsProvider
> \
>
> -DzkCredentialsInjector=org.apache.solr.common.cloud.acl.SecretCredentialInjector
> \
>
> -DzkSecretCredentialsProvider=org.apache.solr.secret.zk.AWSSecretCredentialsProvider
> \
> -DzkSecretCredentialSecretName=zkSecret \
> -DzkCredentialsAWSSecretRegion=us-west-2"
> SOLR_OPTS="$SOLR_OPTS $SOLR_ZK_CREDS_AND_ACLS"
>
> {code}
>
> *Problem 3*
> A new _contrib_ module ({_}secret-provider{_}) is added where
> _SecretCredentialsProvider_ implementing classes can be added without the
> need to add a new dependency to Solr core. All one needs to do after adding a
> new class is to pass it through system props via _solr.ini.sh/cmd_ file.
> This module can be used in the future for other secrets injections, not
> specifically related to zk.
>
> Thank you in advance for your time and your comments.
>
>
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
