[jira] [Updated] (RANGER-3630) Support wildcards, group short names, and list of memberof attribute DNs for computing user search filter
[ https://issues.apache.org/jira/browse/RANGER-3630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sailaja Polavarapu updated RANGER-3630: --- Attachment: RANGER-3630_proposal.pdf > Support wildcards, group short names, and list of memberof attribute DNs for > computing user search filter > - > > Key: RANGER-3630 > URL: https://issues.apache.org/jira/browse/RANGER-3630 > Project: Ranger > Issue Type: New Feature > Components: Ranger, usersync >Reporter: Sailaja Polavarapu >Assignee: Sailaja Polavarapu >Priority: Major > Attachments: RANGER-3630_proposal.pdf > > > Ranger Usersync provides multiple configuration properties to sync users & > groups from AD/LDAP. One of the key configuration properties is the User > Search filter (ranger.usersync.ldap.user.searchfilter). Currently, the value > of user search filter must be a valid ldap search filter and is used by > ranger usersync “as is” to limit the no. of users to be sync’d from AD/LDAP. > Example values include: > # samaccountname=* > ** Syncs all users from a given user search base > # (|(memberof=CN=finance,ou=Hadoop > Groups,dc=apache,dc=org)(memberof=CN=eng_dev,ou=Hadoop > Groups,dc=apache,dc=org)(memberof=CN=eng_testing,ou=Hadoop > Groups,dc=apache,dc=org)) > ** Sync users that are members of finance, eng_dev, and eng_testing groups > According to [Microsoft > documentation|https://social.technet.microsoft.com/wiki/contents/articles/5392.active-directory-ldap-syntax-filters.aspx], > the wildcard character * is not allowed when the is a DN > attribute. Examples of DN attributes are distinguishedName, manager, > directReports, member, and memberOf. If users need to be sync'd from multiple > Active Directory groups with memberOf filters, this value can quickly become > a long string of OR concatenated group DNs. A single misplaced character in > this cryptic string results in all users failing to sync. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] (RANGER-3630) Support wildcards, group short names, and list of memberof attribute DNs for computing user search filter
[ https://issues.apache.org/jira/browse/RANGER-3630 ]
Sailaja Polavarapu deleted comment on RANGER-3630:
was (Author: spolavarapu):
*Proposal:*
Majority of the use cases to filter users using the “memberof” attribute fall
into two categories:
# Unique pattern for the group name - example, “eng_dev” and “finance”
# Group names with wildcard character - example, “eng_dev” and “eng_testing”
As noted down earlier, since Active directory doesn’t support either wildcards
or short names of the groups with memberof attributes, Ranger usersync must be
improved to generate user search filter internally by taking list of individual
group names or group names with wildcard character.
Instead of configuring user search filter as one big string, a new
configuration “ranger.usersync.ldap.groupnames” can be introduced for usersync.
Values can be either list of DN of the groups, list of short name of the
groups, or list of group names with wildcard character with ";" separated like
below:
# DN of the groups - "memberof=CN=finance,ou=Hadoop
Groups,dc=apache,dc=org{color:#ff}{*};{*}{color:#172b4d}memberof=CN=eng_dev,ou=Hadoop
Groups,dc=apache,dc=org{color}{color}{color:#de350b}*;*{color}memberof=CN=eng_testing,ou=Hadoop
Groups,dc=apache,dc=org"
# Short names of the groups -
"CN=finance{color:#de350b}*;*{color}CN=eng_dev{color:#de350b}*;*{color}CN=eng_testing"
# Group names with wildcard character -
CN=eng*{color:#de350b}*;*{color}CN=finance"
*Usersync Changes:*
Usersync reads these new configuration values and determines the format of the
specified values as DN of the groups, short names of the groups, or group names
with wildcard character.
# Values specified as DN of the groups
# In this case ranger usersync builds the user search filter by concatenating
each DN with an OR (|) operator
# Example - (|(memberof=CN=finance,ou=Hadoop
Groups,dc=apache,dc=org)(memberof=CN=eng_dev,ou=Hadoop
Groups,dc=apache,dc=org)(memberof=CN=eng_testing,ou=Hadoop
Groups,dc=apache,dc=org))
# Values specified as short names of the groups or with wildcard character
# In this case ranger usersync first contacts AD/LDAP server to retrieve the
DN of the specified groups.
# Build user search filter by prepending each DN with “memberof=” and
concatenating with and OR(|) operator
Notes:
# This new configuration(ranger.usersync.ldap.groupnames) is read by usersync
only when “ranger.usersync.ldap.user.searchfilter” configuration value is
empty.
# When “ranger.usersync.ldap.user.searchfilter” configuration value is not
empty, then usersync will ignore the value for
“ranger.usersync.ldap.groupnames” configuration.
# All the configured group names(ranger.usersync.ldap.groupnames) are
concatenated with only OR (|) operator and are hardcoded for “memberof”
attribute.
> Support wildcards, group short names, and list of memberof attribute DNs for
> computing user search filter
> -
>
> Key: RANGER-3630
> URL: https://issues.apache.org/jira/browse/RANGER-3630
> Project: Ranger
> Issue Type: New Feature
> Components: Ranger, usersync
>Reporter: Sailaja Polavarapu
>Assignee: Sailaja Polavarapu
>Priority: Major
>
> Ranger Usersync provides multiple configuration properties to sync users &
> groups from AD/LDAP. One of the key configuration properties is the User
> Search filter (ranger.usersync.ldap.user.searchfilter). Currently, the value
> of user search filter must be a valid ldap search filter and is used by
> ranger usersync “as is” to limit the no. of users to be sync’d from AD/LDAP.
> Example values include:
> # samaccountname=*
> ** Syncs all users from a given user search base
> # (|(memberof=CN=finance,ou=Hadoop
> Groups,dc=apache,dc=org)(memberof=CN=eng_dev,ou=Hadoop
> Groups,dc=apache,dc=org)(memberof=CN=eng_testing,ou=Hadoop
> Groups,dc=apache,dc=org))
> ** Sync users that are members of finance, eng_dev, and eng_testing groups
> According to [Microsoft
> documentation|https://social.technet.microsoft.com/wiki/contents/articles/5392.active-directory-ldap-syntax-filters.aspx],
> the wildcard character * is not allowed when the is a DN
> attribute. Examples of DN attributes are distinguishedName, manager,
> directReports, member, and memberOf. If users need to be sync'd from multiple
> Active Directory groups with memberOf filters, this value can quickly become
> a long string of OR concatenated group DNs. A single misplaced character in
> this cryptic string results in all users failing to sync.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Comment Edited] (RANGER-3630) Support wildcards, group short names, and list of memberof attribute DNs for computing user search filter
[
https://issues.apache.org/jira/browse/RANGER-3630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17493992#comment-17493992
]
Sailaja Polavarapu edited comment on RANGER-3630 at 2/17/22, 3:05 PM:
--
*Proposal:*
Majority of the use cases to filter users using the “memberof” attribute fall
into two categories:
# Unique pattern for the group name - example, “eng_dev” and “finance”
# Group names with wildcard character - example, “eng_dev” and “eng_testing”
As noted down earlier, since Active directory doesn’t support either wildcards
or short names of the groups with memberof attributes, Ranger usersync must be
improved to generate user search filter internally by taking list of individual
group names or group names with wildcard character.
Instead of configuring user search filter as one big string, a new
configuration “ranger.usersync.ldap.groupnames” can be introduced for usersync.
Values can be either list of DN of the groups, list of short name of the
groups, or list of group names with wildcard character with ";" separated like
below:
# DN of the groups - "memberof=CN=finance,ou=Hadoop
Groups,dc=apache,dc=org{color:#ff}*;*
{color:#172b4d}memberof=CN=eng_dev,ou=Hadoop
Groups,dc=apache,dc=org{color}{color}{color:#de350b}*;*{color}memberof=CN=eng_testing,ou=Hadoop
Groups,dc=apache,dc=org"
# Short names of the groups - "CN=finance{*};{*}CN=eng_dev{*};{*}CN=eng_testing"
# Group names with wildcard character -
CN=eng*{color:#de350b}*;*{color}CN=finance"
*Usersync Changes:*
Usersync reads these new configuration values and determines the format of the
specified values as DN of the groups, short names of the groups, or group names
with wildcard character.
# Values specified as DN of the groups
# In this case ranger usersync builds the user search filter by concatenating
each DN with an OR (|) operator
# Example - (|(memberof=CN=finance,ou=Hadoop
Groups,dc=apache,dc=org)(memberof=CN=eng_dev,ou=Hadoop
Groups,dc=apache,dc=org)(memberof=CN=eng_testing,ou=Hadoop
Groups,dc=apache,dc=org))
# Values specified as short names of the groups or with wildcard character
# In this case ranger usersync first contacts AD/LDAP server to retrieve the
DN of the specified groups.
# Build user search filter by prepending each DN with “memberof=” and
concatenating with and OR(|) operator
Notes:
# This new configuration(ranger.usersync.ldap.groupnames) is read by usersync
only when “ranger.usersync.ldap.user.searchfilter” configuration value is
empty.
# When “ranger.usersync.ldap.user.searchfilter” configuration value is not
empty, then usersync will ignore the value for
“ranger.usersync.ldap.groupnames” configuration.
# All the configured group names(ranger.usersync.ldap.groupnames) are
concatenated with only OR (|) operator and are hardcoded for “memberof”
attribute.
was (Author: spolavarapu):
*Proposal:*
Majority of the use cases to filter users using the “memberof” attribute fall
into two categories:
# Unique pattern for the group name - example, “eng_dev” and “finance”
# Group names with wildcard character - example, “eng_dev” and “eng_testing”
As noted down earlier, since Active directory doesn’t support either wildcards
or short names of the groups with memberof attributes, Ranger usersync must be
improved to generate user search filter internally by taking list of individual
group names or group names with wildcard character.
Instead of configuring user search filter as one big string, a new
configuration “ranger.usersync.ldap.groupnames” can be introduced for usersync.
Values can be either list of DN of the groups, list of short name of the
groups, or list of group names with wildcard character with ";" separated like
below:
# DN of the groups - "memberof=CN=finance,ou=Hadoop
Groups,dc=apache,dc=org{color:#ff}{*};{*}{color:#172b4d}memberof=CN=eng_dev,ou=Hadoop
Groups,dc=apache,dc=org{color}{color}{*};{*}memberof=CN=eng_testing,ou=Hadoop
Groups,dc=apache,dc=org"
# Short names of the groups -
"CN=finance{*};{*}CN=eng_dev{color:#ff}*;*{color}CN=eng_testing"
# Group names with wildcard character -
CN=eng*{color:#de350b}*;*{color}CN=finance"
*Usersync Changes:*
Usersync reads these new configuration values and determines the format of the
specified values as DN of the groups, short names of the groups, or group names
with wildcard character.
# Values specified as DN of the groups
# In this case ranger usersync builds the user search filter by concatenating
each DN with an OR (|) operator
# Example - (|(memberof=CN=finance,ou=Hadoop
Groups,dc=apache,dc=org)(memberof=CN=eng_dev,ou=Hadoop
Groups,dc=apache,dc=org)(memberof=CN=eng_testing,ou=Hadoop
Groups,dc=apache,dc=org))
# Values specified as short names of the groups or with wildcard character
# In this case ranger usersync first contacts AD/LDAP server to retrieve the
DN of
[jira] [Comment Edited] (RANGER-3630) Support wildcards, group short names, and list of memberof attribute DNs for computing user search filter
[
https://issues.apache.org/jira/browse/RANGER-3630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17493992#comment-17493992
]
Sailaja Polavarapu edited comment on RANGER-3630 at 2/17/22, 3:05 PM:
--
*Proposal:*
Majority of the use cases to filter users using the “memberof” attribute fall
into two categories:
# Unique pattern for the group name - example, “eng_dev” and “finance”
# Group names with wildcard character - example, “eng_dev” and “eng_testing”
As noted down earlier, since Active directory doesn’t support either wildcards
or short names of the groups with memberof attributes, Ranger usersync must be
improved to generate user search filter internally by taking list of individual
group names or group names with wildcard character.
Instead of configuring user search filter as one big string, a new
configuration “ranger.usersync.ldap.groupnames” can be introduced for usersync.
Values can be either list of DN of the groups, list of short name of the
groups, or list of group names with wildcard character with ";" separated like
below:
# DN of the groups - "memberof=CN=finance,ou=Hadoop
Groups,dc=apache,dc=org{color:#ff}{*};{*}{color:#172b4d}memberof=CN=eng_dev,ou=Hadoop
Groups,dc=apache,dc=org{color}{color}{color:#de350b}*;*{color}memberof=CN=eng_testing,ou=Hadoop
Groups,dc=apache,dc=org"
# Short names of the groups -
"CN=finance{color:#de350b}*;*{color}CN=eng_dev{color:#de350b}*;*{color}CN=eng_testing"
# Group names with wildcard character -
CN=eng*{color:#de350b}*;*{color}CN=finance"
*Usersync Changes:*
Usersync reads these new configuration values and determines the format of the
specified values as DN of the groups, short names of the groups, or group names
with wildcard character.
# Values specified as DN of the groups
# In this case ranger usersync builds the user search filter by concatenating
each DN with an OR (|) operator
# Example - (|(memberof=CN=finance,ou=Hadoop
Groups,dc=apache,dc=org)(memberof=CN=eng_dev,ou=Hadoop
Groups,dc=apache,dc=org)(memberof=CN=eng_testing,ou=Hadoop
Groups,dc=apache,dc=org))
# Values specified as short names of the groups or with wildcard character
# In this case ranger usersync first contacts AD/LDAP server to retrieve the
DN of the specified groups.
# Build user search filter by prepending each DN with “memberof=” and
concatenating with and OR(|) operator
Notes:
# This new configuration(ranger.usersync.ldap.groupnames) is read by usersync
only when “ranger.usersync.ldap.user.searchfilter” configuration value is
empty.
# When “ranger.usersync.ldap.user.searchfilter” configuration value is not
empty, then usersync will ignore the value for
“ranger.usersync.ldap.groupnames” configuration.
# All the configured group names(ranger.usersync.ldap.groupnames) are
concatenated with only OR (|) operator and are hardcoded for “memberof”
attribute.
was (Author: spolavarapu):
*Proposal:*
Majority of the use cases to filter users using the “memberof” attribute fall
into two categories:
# Unique pattern for the group name - example, “eng_dev” and “finance”
# Group names with wildcard character - example, “eng_dev” and “eng_testing”
As noted down earlier, since Active directory doesn’t support either wildcards
or short names of the groups with memberof attributes, Ranger usersync must be
improved to generate user search filter internally by taking list of individual
group names or group names with wildcard character.
Instead of configuring user search filter as one big string, a new
configuration “ranger.usersync.ldap.groupnames” can be introduced for usersync.
Values can be either list of DN of the groups, list of short name of the
groups, or list of group names with wildcard character with ";" separated like
below:
# DN of the groups - "memberof=CN=finance,ou=Hadoop
Groups,dc=apache,dc=org{color:#ff}*;*
{color:#172b4d}memberof=CN=eng_dev,ou=Hadoop
Groups,dc=apache,dc=org{color}{color}{color:#de350b}*;*{color}memberof=CN=eng_testing,ou=Hadoop
Groups,dc=apache,dc=org"
# Short names of the groups - "CN=finance{*};{*}CN=eng_dev{*};{*}CN=eng_testing"
# Group names with wildcard character -
CN=eng*{color:#de350b}*;*{color}CN=finance"
*Usersync Changes:*
Usersync reads these new configuration values and determines the format of the
specified values as DN of the groups, short names of the groups, or group names
with wildcard character.
# Values specified as DN of the groups
# In this case ranger usersync builds the user search filter by concatenating
each DN with an OR (|) operator
# Example - (|(memberof=CN=finance,ou=Hadoop
Groups,dc=apache,dc=org)(memberof=CN=eng_dev,ou=Hadoop
Groups,dc=apache,dc=org)(memberof=CN=eng_testing,ou=Hadoop
Groups,dc=apache,dc=org))
# Values specified as short names of the groups or with wildcard character
# In this case ranger usersync first contacts
[jira] [Comment Edited] (RANGER-3630) Support wildcards, group short names, and list of memberof attribute DNs for computing user search filter
[
https://issues.apache.org/jira/browse/RANGER-3630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17493992#comment-17493992
]
Sailaja Polavarapu edited comment on RANGER-3630 at 2/17/22, 3:04 PM:
--
*Proposal:*
Majority of the use cases to filter users using the “memberof” attribute fall
into two categories:
# Unique pattern for the group name - example, “eng_dev” and “finance”
# Group names with wildcard character - example, “eng_dev” and “eng_testing”
As noted down earlier, since Active directory doesn’t support either wildcards
or short names of the groups with memberof attributes, Ranger usersync must be
improved to generate user search filter internally by taking list of individual
group names or group names with wildcard character.
Instead of configuring user search filter as one big string, a new
configuration “ranger.usersync.ldap.groupnames” can be introduced for usersync.
Values can be either list of DN of the groups, list of short name of the
groups, or list of group names with wildcard character with ";" separated like
below:
# DN of the groups - "memberof=CN=finance,ou=Hadoop
Groups,dc=apache,dc=org{color:#ff}{*};{*}{color:#172b4d}memberof=CN=eng_dev,ou=Hadoop
Groups,dc=apache,dc=org{color}{color}{*};{*}memberof=CN=eng_testing,ou=Hadoop
Groups,dc=apache,dc=org"
# Short names of the groups -
"CN=finance{*};{*}CN=eng_dev{color:#ff}*;*{color}CN=eng_testing"
# Group names with wildcard character -
CN=eng*{color:#de350b}*;*{color}CN=finance"
*Usersync Changes:*
Usersync reads these new configuration values and determines the format of the
specified values as DN of the groups, short names of the groups, or group names
with wildcard character.
# Values specified as DN of the groups
# In this case ranger usersync builds the user search filter by concatenating
each DN with an OR (|) operator
# Example - (|(memberof=CN=finance,ou=Hadoop
Groups,dc=apache,dc=org)(memberof=CN=eng_dev,ou=Hadoop
Groups,dc=apache,dc=org)(memberof=CN=eng_testing,ou=Hadoop
Groups,dc=apache,dc=org))
# Values specified as short names of the groups or with wildcard character
# In this case ranger usersync first contacts AD/LDAP server to retrieve the
DN of the specified groups.
# Build user search filter by prepending each DN with “memberof=” and
concatenating with and OR(|) operator
Notes:
# This new configuration(ranger.usersync.ldap.groupnames) is read by usersync
only when “ranger.usersync.ldap.user.searchfilter” configuration value is
empty.
# When “ranger.usersync.ldap.user.searchfilter” configuration value is not
empty, then usersync will ignore the value for
“ranger.usersync.ldap.groupnames” configuration.
# All the configured group names(ranger.usersync.ldap.groupnames) are
concatenated with only OR (|) operator and are hardcoded for “memberof”
attribute.
was (Author: spolavarapu):
*Proposal:*
Majority of the use cases to filter users using the “memberof” attribute fall
into two categories:
# Unique pattern for the group name - example, “eng_dev” and “finance”
# Group names with wildcard character - example, “eng_dev” and “eng_testing”
As noted down earlier, since Active directory doesn’t support either wildcards
or short names of the groups with memberof attributes, Ranger usersync must be
improved to generate user search filter internally by taking list of individual
group names or group names with wildcard character.
Instead of configuring user search filter as one big string, a new
configuration “ranger.usersync.ldap.groupnames” can be introduced for usersync.
Values can be either list of DN of the groups, list of short name of the
groups, or list of group names with wildcard character with ";" separated like
below:
# DN of the groups - "memberof=CN=finance,ou=Hadoop
Groups,dc=apache,dc=org{color:#FF}{*};{*}{color:#172b4d}memberof=CN=eng_dev,ou=Hadoop
Groups,dc=apache,dc=org{color}{color}{color:#FF}{*};{*}{color:#172b4d}memberof=CN=eng_testing,ou=Hadoop
Groups,dc=apache,dc=org{color}{color}"
# Short names of the groups -
"CN=finance{color:#FF}*;*{color}CN=eng_dev{color:#FF}*;*{color}CN=eng_testing"
# Group names with wildcard character -
CN=eng*{color:#de350b}*;*{color}CN=finance"
*Usersync Changes:*
Usersync reads these new configuration values and determines the format of the
specified values as DN of the groups, short names of the groups, or group names
with wildcard character.
# Values specified as DN of the groups
# In this case ranger usersync builds the user search filter by concatenating
each DN with an OR (|) operator
# Example - (|(memberof=CN=finance,ou=Hadoop
Groups,dc=apache,dc=org)(memberof=CN=eng_dev,ou=Hadoop
Groups,dc=apache,dc=org)(memberof=CN=eng_testing,ou=Hadoop
Groups,dc=apache,dc=org))
# Values specified as short names of the groups or with wildcard character
# In this case
[jira] [Commented] (RANGER-3630) Support wildcards, group short names, and list of memberof attribute DNs for computing user search filter
[
https://issues.apache.org/jira/browse/RANGER-3630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17493992#comment-17493992
]
Sailaja Polavarapu commented on RANGER-3630:
*Proposal:*
Majority of the use cases to filter users using the “memberof” attribute fall
into two categories:
# Unique pattern for the group name - example, “eng_dev” and “finance”
# Group names with wildcard character - example, “eng_dev” and “eng_testing”
As noted down earlier, since Active directory doesn’t support either wildcards
or short names of the groups with memberof attributes, Ranger usersync must be
improved to generate user search filter internally by taking list of individual
group names or group names with wildcard character.
Instead of configuring user search filter as one big string, a new
configuration “ranger.usersync.ldap.groupnames” can be introduced for usersync.
Values can be either list of DN of the groups, list of short name of the
groups, or list of group names with wildcard character with ";" separated like
below:
# DN of the groups - "memberof=CN=finance,ou=Hadoop
Groups,dc=apache,dc=org{color:#FF}{*};{*}{color:#172b4d}memberof=CN=eng_dev,ou=Hadoop
Groups,dc=apache,dc=org{color}{color}{color:#FF}{*};{*}{color:#172b4d}memberof=CN=eng_testing,ou=Hadoop
Groups,dc=apache,dc=org{color}{color}"
# Short names of the groups -
"CN=finance{color:#FF}*;*{color}CN=eng_dev{color:#FF}*;*{color}CN=eng_testing"
# Group names with wildcard character -
CN=eng*{color:#de350b}*;*{color}CN=finance"
*Usersync Changes:*
Usersync reads these new configuration values and determines the format of the
specified values as DN of the groups, short names of the groups, or group names
with wildcard character.
# Values specified as DN of the groups
# In this case ranger usersync builds the user search filter by concatenating
each DN with an OR (|) operator
# Example - (|(memberof=CN=finance,ou=Hadoop
Groups,dc=apache,dc=org)(memberof=CN=eng_dev,ou=Hadoop
Groups,dc=apache,dc=org)(memberof=CN=eng_testing,ou=Hadoop
Groups,dc=apache,dc=org))
# Values specified as short names of the groups or with wildcard character
# In this case ranger usersync first contacts AD/LDAP server to retrieve the
DN of the specified groups.
# Build user search filter by prepending each DN with “memberof=” and
concatenating with and OR(|) operator
Notes:
# This new configuration(ranger.usersync.ldap.groupnames) is read by usersync
only when “ranger.usersync.ldap.user.searchfilter” configuration value is
empty.
# When “ranger.usersync.ldap.user.searchfilter” configuration value is not
empty, then usersync will ignore the value for
“ranger.usersync.ldap.groupnames” configuration.
# All the configured group names(ranger.usersync.ldap.groupnames) are
concatenated with only OR (|) operator and are hardcoded for “memberof”
attribute.
> Support wildcards, group short names, and list of memberof attribute DNs for
> computing user search filter
> -
>
> Key: RANGER-3630
> URL: https://issues.apache.org/jira/browse/RANGER-3630
> Project: Ranger
> Issue Type: New Feature
> Components: Ranger, usersync
>Reporter: Sailaja Polavarapu
>Assignee: Sailaja Polavarapu
>Priority: Major
>
> Ranger Usersync provides multiple configuration properties to sync users &
> groups from AD/LDAP. One of the key configuration properties is the User
> Search filter (ranger.usersync.ldap.user.searchfilter). Currently, the value
> of user search filter must be a valid ldap search filter and is used by
> ranger usersync “as is” to limit the no. of users to be sync’d from AD/LDAP.
> Example values include:
> # samaccountname=*
> ** Syncs all users from a given user search base
> # (|(memberof=CN=finance,ou=Hadoop
> Groups,dc=apache,dc=org)(memberof=CN=eng_dev,ou=Hadoop
> Groups,dc=apache,dc=org)(memberof=CN=eng_testing,ou=Hadoop
> Groups,dc=apache,dc=org))
> ** Sync users that are members of finance, eng_dev, and eng_testing groups
> According to [Microsoft
> documentation|https://social.technet.microsoft.com/wiki/contents/articles/5392.active-directory-ldap-syntax-filters.aspx],
> the wildcard character * is not allowed when the is a DN
> attribute. Examples of DN attributes are distinguishedName, manager,
> directReports, member, and memberOf. If users need to be sync'd from multiple
> Active Directory groups with memberOf filters, this value can quickly become
> a long string of OR concatenated group DNs. A single misplaced character in
> this cryptic string results in all users failing to sync.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Created] (RANGER-3630) Support wildcards, group short names, and list of memberof attribute DNs for computing user search filter
Sailaja Polavarapu created RANGER-3630: -- Summary: Support wildcards, group short names, and list of memberof attribute DNs for computing user search filter Key: RANGER-3630 URL: https://issues.apache.org/jira/browse/RANGER-3630 Project: Ranger Issue Type: New Feature Components: Ranger, usersync Reporter: Sailaja Polavarapu Ranger Usersync provides multiple configuration properties to sync users & groups from AD/LDAP. One of the key configuration properties is the User Search filter (ranger.usersync.ldap.user.searchfilter). Currently, the value of user search filter must be a valid ldap search filter and is used by ranger usersync “as is” to limit the no. of users to be sync’d from AD/LDAP. Example values include: # samaccountname=* ** Syncs all users from a given user search base # (|(memberof=CN=finance,ou=Hadoop Groups,dc=apache,dc=org)(memberof=CN=eng_dev,ou=Hadoop Groups,dc=apache,dc=org)(memberof=CN=eng_testing,ou=Hadoop Groups,dc=apache,dc=org)) ** Sync users that are members of finance, eng_dev, and eng_testing groups According to [Microsoft documentation|https://social.technet.microsoft.com/wiki/contents/articles/5392.active-directory-ldap-syntax-filters.aspx], the wildcard character * is not allowed when the is a DN attribute. Examples of DN attributes are distinguishedName, manager, directReports, member, and memberOf. If users need to be sync'd from multiple Active Directory groups with memberOf filters, this value can quickly become a long string of OR concatenated group DNs. A single misplaced character in this cryptic string results in all users failing to sync. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Assigned] (RANGER-3630) Support wildcards, group short names, and list of memberof attribute DNs for computing user search filter
[ https://issues.apache.org/jira/browse/RANGER-3630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sailaja Polavarapu reassigned RANGER-3630: -- Assignee: Sailaja Polavarapu > Support wildcards, group short names, and list of memberof attribute DNs for > computing user search filter > - > > Key: RANGER-3630 > URL: https://issues.apache.org/jira/browse/RANGER-3630 > Project: Ranger > Issue Type: New Feature > Components: Ranger, usersync >Reporter: Sailaja Polavarapu >Assignee: Sailaja Polavarapu >Priority: Major > > Ranger Usersync provides multiple configuration properties to sync users & > groups from AD/LDAP. One of the key configuration properties is the User > Search filter (ranger.usersync.ldap.user.searchfilter). Currently, the value > of user search filter must be a valid ldap search filter and is used by > ranger usersync “as is” to limit the no. of users to be sync’d from AD/LDAP. > Example values include: > # samaccountname=* > ** Syncs all users from a given user search base > # (|(memberof=CN=finance,ou=Hadoop > Groups,dc=apache,dc=org)(memberof=CN=eng_dev,ou=Hadoop > Groups,dc=apache,dc=org)(memberof=CN=eng_testing,ou=Hadoop > Groups,dc=apache,dc=org)) > ** Sync users that are members of finance, eng_dev, and eng_testing groups > According to [Microsoft > documentation|https://social.technet.microsoft.com/wiki/contents/articles/5392.active-directory-ldap-syntax-filters.aspx], > the wildcard character * is not allowed when the is a DN > attribute. Examples of DN attributes are distinguishedName, manager, > directReports, member, and memberOf. If users need to be sync'd from multiple > Active Directory groups with memberOf filters, this value can quickly become > a long string of OR concatenated group DNs. A single misplaced character in > this cryptic string results in all users failing to sync. -- This message was sent by Atlassian Jira (v8.20.1#820001)
Re: Review Request 73852: RANGER-3595, refactor the file layout of ranger-xxx-kms.tar.gz
> On 二月 17, 2022, 7:54 a.m., bhavik patel wrote:
> > Thanks for the deatiled info and cleaning up the packing.
> >
> > can you please verify zone operations, import/export keys to jceks file and
> > masterkey import/export operation.
> >
> > I just wan to make we are not breaking the existing functionality’s.
>
> Kirby Zhou wrote:
> Tested:
>
> # Basic APIs passed
>
> 1. TestConnection by rangeradmin
> 2. Browes existing keys by rangeradmin
> 2. CreateKey by rangeradmin
> 3. KeyRoll by rangeradmin
> 4. generateeek by curl
> 5. decrypteek by curl
>
> # export passed
> ]$ ./exportKeysToJCEKS.sh mkeybak jceks
> Enter Password for the keystore FILE :
> Enter Password for the KEY(s) stored in the keystore:
> Keys from Ranger KMS Database has been successfully exported into mkeybak
> ]$ keytool -list -keystore mkeybak -storetype jceks
> Enter keystore password:
> Keystore type: JCEKS
> Keystore provider: SunJCE
>
> Your keystore contains 12 entries
>
> hell, Feb 17, 2022, SecretKeyEntry,
> hell-1-2-31, Feb 17, 2022, SecretKeyEntry,
> hell-1-2-31@0, Feb 17, 2022, SecretKeyEntry,
> hell-1-2-31@1, Feb 17, 2022, SecretKeyEntry,
> hell@0, Feb 17, 2022, SecretKeyEntry,
> hell@1, Feb 17, 2022, SecretKeyEntry,
> hell@2, Feb 17, 2022, SecretKeyEntry,
> hello-world, Feb 17, 2022, SecretKeyEntry,
> hello-world@0, Feb 17, 2022, SecretKeyEntry,
> hello-world@1, Feb 17, 2022, SecretKeyEntry,
> paladin, Feb 17, 2022, SecretKeyEntry,
> paladin@0, Feb 17, 2022, SecretKeyEntry,
>
>
>
> # import passed with a bit of accident
> bash -c 'function java() { /usr/bin/java -Djceks.key.serialFilter= "$@" ;
> } ; source ./importJCEKSKeys.sh mkeybak jceks'
> Enter Password for the keystore FILE :
> Enter Password for the KEY(s) stored in the keystore:
> Keys from mkeybak has been successfully imported into RangerDB.
>
> # Why -Djceks.key.serialFilter appears?
>
> See
> https://www.oracle.com/java/technologies/javase/8u171-relnotes.html#JDK-8189997
>
> Since jdk-8u171, importJCEKSKeys is broken without
> -Djceks.key.serialFilter=.
> I tested it at tag-ranger-2.2, it is broken too.
> But that is another story, I wont fix it in that patch here.
>
> Kirby Zhou wrote:
> Let us check "RangerKeyStoreProvider.java"
>
> ```
> KeyVersion innerSetKeyVersion(...) {
> if (azureKeyVaultEnabled || this.isGCPEnabled) {
> // ...
> } else {
> dbStore.addKeyEntry(versionName, new
> SecretKeySpec(material,
> cipher), masterKey, cipher,
> bitLength, description,
> version, attribute);
> }
> }
>
>
> public void flush(...) {
> if (azureKeyVaultEnabled ||
> this.isGCPEnabled) {
> // ...
> } else {
>
> dbStore.addKeyEntry(entry.getKey(), new KeyMetadata(
> metadata),
> masterKey, metadata.getAlgorithm(),
>
> metadata.getBitLength(), metadata
>
> .getDescription(), metadata
>
> .getVersions(), attributes);
> }
> }
>
> ```
>
> innerSetKeyVersion is called by createKey and rollNewVersion to create
> versioned Key "@0" "@1", and flush is called after
> createKey/rollNewVersion, it will create "".
>
> You can see innerSetKeyVersion call new SecretKeySpec(...) and flush call
> new KeyMetadata(...).
> And the SecretKeySpec can not be unsealed without
> "-Djceks.key.serialFilter="
>
> I donot know the reason of store type choice between KeyMetadata and
> SecretKeySpec.
> Maybe someone can tell me?
>
> The code is writteb by @dhavalshah9131 , commited by @mehulbparikh
>
> 02640d3cf520ee5a0dae13dd46b91ff266359e77
>
> RANGER-2497 : Support Azure Key Vault for storing master keys of Ranger
> KMS
>
> bhavik patel wrote:
> Thanks for the verification.
>
> "-Djceks.key.serialFilte" is not the issue with kms code I think it's the
> issue with some Java version(Security policies)
But who knows why we use both KeyMetadata and SecretKeySpec to store a key?
- Kirby
---
This is an automatically generated e-mail. To reply, visit:
[jira] [Commented] (RANGER-3580) Support Ranger KMS integration with TencentKMS
[ https://issues.apache.org/jira/browse/RANGER-3580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17493933#comment-17493933 ] Dhaval Shah commented on RANGER-3580: - Apache Master Commit: https://github.com/apache/ranger/commit/605de3cba8d311e983f0aee3490e67266f6668a9 Apache ranger-2.3 commit: https://github.com/apache/ranger/commit/55c04f2f47bf749ede969804b34a3837f8520937 > Support Ranger KMS integration with TencentKMS > -- > > Key: RANGER-3580 > URL: https://issues.apache.org/jira/browse/RANGER-3580 > Project: Ranger > Issue Type: New Feature > Components: kms >Affects Versions: 3.0.0 >Reporter: kirby zhou >Priority: Major > Fix For: 3.0.0 > > Attachments: 0001-add-TencentKMS-as-MasterKeyProvider-1.patch, > 0001-add-TencentKMS-as-MasterKeyProvider.patch > > > Want Ranger KMS can work with more Key Vault Provider as its backend. > Such as Tencent KMS, AliCloud KMS, AWS KMS. -- This message was sent by Atlassian Jira (v8.20.1#820001)
Re: Review Request 73842: RANGER-3620 : Ranger - Upgrade tomcat to 8.5.75
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/73842/#review224071 --- Ship it! Ship It! - Dhaval Shah On Feb. 14, 2022, 1:21 p.m., Mateen Mansoori wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/73842/ > --- > > (Updated Feb. 14, 2022, 1:21 p.m.) > > > Review request for ranger, Dhaval Shah, Mehul Parikh, Pradeep Agrawal, and > Velmurugan Periasamy. > > > Bugs: RANGER-3620 > https://issues.apache.org/jira/browse/RANGER-3620 > > > Repository: ranger > > > Description > --- > > Upgraded embedded-tomcat to 8.5.75 > > > Diffs > - > > pom.xml 8a19c2de4 > > > Diff: https://reviews.apache.org/r/73842/diff/1/ > > > Testing > --- > > - Build succeeded - mvn clean compile test verify install > - Verified service, user CRUD on local VM. > > > Thanks, > > Mateen Mansoori > >
Re: Review Request 73852: RANGER-3595, refactor the file layout of ranger-xxx-kms.tar.gz
> On Feb. 17, 2022, 7:54 a.m., bhavik patel wrote:
> > Thanks for the deatiled info and cleaning up the packing.
> >
> > can you please verify zone operations, import/export keys to jceks file and
> > masterkey import/export operation.
> >
> > I just wan to make we are not breaking the existing functionality’s.
>
> Kirby Zhou wrote:
> Tested:
>
> # Basic APIs passed
>
> 1. TestConnection by rangeradmin
> 2. Browes existing keys by rangeradmin
> 2. CreateKey by rangeradmin
> 3. KeyRoll by rangeradmin
> 4. generateeek by curl
> 5. decrypteek by curl
>
> # export passed
> ]$ ./exportKeysToJCEKS.sh mkeybak jceks
> Enter Password for the keystore FILE :
> Enter Password for the KEY(s) stored in the keystore:
> Keys from Ranger KMS Database has been successfully exported into mkeybak
> ]$ keytool -list -keystore mkeybak -storetype jceks
> Enter keystore password:
> Keystore type: JCEKS
> Keystore provider: SunJCE
>
> Your keystore contains 12 entries
>
> hell, Feb 17, 2022, SecretKeyEntry,
> hell-1-2-31, Feb 17, 2022, SecretKeyEntry,
> hell-1-2-31@0, Feb 17, 2022, SecretKeyEntry,
> hell-1-2-31@1, Feb 17, 2022, SecretKeyEntry,
> hell@0, Feb 17, 2022, SecretKeyEntry,
> hell@1, Feb 17, 2022, SecretKeyEntry,
> hell@2, Feb 17, 2022, SecretKeyEntry,
> hello-world, Feb 17, 2022, SecretKeyEntry,
> hello-world@0, Feb 17, 2022, SecretKeyEntry,
> hello-world@1, Feb 17, 2022, SecretKeyEntry,
> paladin, Feb 17, 2022, SecretKeyEntry,
> paladin@0, Feb 17, 2022, SecretKeyEntry,
>
>
>
> # import passed with a bit of accident
> bash -c 'function java() { /usr/bin/java -Djceks.key.serialFilter= "$@" ;
> } ; source ./importJCEKSKeys.sh mkeybak jceks'
> Enter Password for the keystore FILE :
> Enter Password for the KEY(s) stored in the keystore:
> Keys from mkeybak has been successfully imported into RangerDB.
>
> # Why -Djceks.key.serialFilter appears?
>
> See
> https://www.oracle.com/java/technologies/javase/8u171-relnotes.html#JDK-8189997
>
> Since jdk-8u171, importJCEKSKeys is broken without
> -Djceks.key.serialFilter=.
> I tested it at tag-ranger-2.2, it is broken too.
> But that is another story, I wont fix it in that patch here.
>
> Kirby Zhou wrote:
> Let us check "RangerKeyStoreProvider.java"
>
> ```
> KeyVersion innerSetKeyVersion(...) {
> if (azureKeyVaultEnabled || this.isGCPEnabled) {
> // ...
> } else {
> dbStore.addKeyEntry(versionName, new
> SecretKeySpec(material,
> cipher), masterKey, cipher,
> bitLength, description,
> version, attribute);
> }
> }
>
>
> public void flush(...) {
> if (azureKeyVaultEnabled ||
> this.isGCPEnabled) {
> // ...
> } else {
>
> dbStore.addKeyEntry(entry.getKey(), new KeyMetadata(
> metadata),
> masterKey, metadata.getAlgorithm(),
>
> metadata.getBitLength(), metadata
>
> .getDescription(), metadata
>
> .getVersions(), attributes);
> }
> }
>
> ```
>
> innerSetKeyVersion is called by createKey and rollNewVersion to create
> versioned Key "@0" "@1", and flush is called after
> createKey/rollNewVersion, it will create "".
>
> You can see innerSetKeyVersion call new SecretKeySpec(...) and flush call
> new KeyMetadata(...).
> And the SecretKeySpec can not be unsealed without
> "-Djceks.key.serialFilter="
>
> I donot know the reason of store type choice between KeyMetadata and
> SecretKeySpec.
> Maybe someone can tell me?
>
> The code is writteb by @dhavalshah9131 , commited by @mehulbparikh
>
> 02640d3cf520ee5a0dae13dd46b91ff266359e77
>
> RANGER-2497 : Support Azure Key Vault for storing master keys of Ranger
> KMS
Thanks for the verification.
"-Djceks.key.serialFilte" is not the issue with kms code I think it's the issue
with some Java version(Security policies)
- bhavik
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73852/#review224066
---
On Feb. 16, 2022, 10:29
Re: Review Request 73852: RANGER-3595, refactor the file layout of ranger-xxx-kms.tar.gz
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/73852/#review224070 --- Ship it! Ship It! - bhavik patel On Feb. 16, 2022, 10:29 a.m., Kirby Zhou wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/73852/ > --- > > (Updated Feb. 16, 2022, 10:29 a.m.) > > > Review request for ranger, Dhaval Shah, Dineshkumar Yadav, Gautam Borad, > Jayendra Parab, Kishor Gollapalliwar, Abhay Kulkarni, Mateen Mansoori, Mehul > Parikh, pengjianhua, Pradeep Agrawal, VaradreawiZTV VaradreawiZTV, Vishal > Suvagia, Velmurugan Periasamy, and Qiang Zhang. > > > Bugs: RANGER-3595 > https://issues.apache.org/jira/browse/RANGER-3595 > > > Repository: ranger > > > Description > --- > > There are lots of .class files under ews/webapp/, and never used. > > > 1. place web.xml at correct location > 2. setup.sh want to patch hadoop-common.jar at runtime, it requires some file > inside ranger-kms.jar. But the patching of hadoop-common.jar is unnecessary. > > Regular webapp should have its own class files under > ews/webapp/WEB-INF/classes, and dependencies under ews/webapp/WEB-INF/lib, > and the Container should put its libraries under ews/lib. But at current, we > use directories sucn as ews/webapp/lib, ews/webapp/WEB-INF/classes/lib. It > looks dirty and ugly. > > > My patch here makes KMS no longer bring ranger-kms.jar, and place classes and > web.xml at correct location. as a alternative of > https://reviews.apache.org/r/73816/ > > > Now: > ews/lib contains ews bootstrap jars, > ews/webapp/WEB-INF/classes contains KMS app itself, > ews/webapp/WEB-INF/lib contains KMS dependencies, > ews/webapp/WEB-INF/lib/ranger-kms-plugin-impl contains ranger-kms-plugin. > > Additionaly, kms/pom.xml even depends on original hadoop-kms, which can > confuse developers, so I removed it. > > BTW: the bootstrap embedded server looks like too heavy and too much > dependeices. > > > Diffs > - > > distro/src/main/assembly/kms.xml 983a43e59 > kms/pom.xml 7a4f98df7 > kms/scripts/DBMK2HSM.sh 001199d97 > kms/scripts/DBMKTOAZUREKEYVAULT.sh cfe5a6b5e > kms/scripts/DBMKTOKEYSECURE.sh c0aa6e58c > kms/scripts/HSMMK2DB.sh 6c77f7340 > kms/scripts/KEYSECUREMKTOKMSDB.sh 340e05e2c > kms/scripts/VerifyIsDBMasterkeyCorrect.sh 1c9a2e148 > kms/scripts/exportKeysToJCEKS.sh f3205789b > kms/scripts/importJCEKSKeys.sh 5d4fe978f > kms/scripts/ranger-kms 429a31e5a > kms/scripts/setup.sh 2051df59a > kms/src/main/java/org/apache/hadoop/crypto/key/kms/server/KMSWebApp.java > c899bdf98 > kms/src/main/resources/META-INF/context.xml > kms/src/main/resources/WEB-INF/web.xml 5e2d489fe > > > Diff: https://reviews.apache.org/r/73852/diff/1/ > > > Testing > --- > > mvn clean pacakge > fresh install and upgrade from 2.2.0 > > > Thanks, > > Kirby Zhou > >
[jira] [Updated] (RANGER-3628) Support fine grain authorization for different solr objects
[ https://issues.apache.org/jira/browse/RANGER-3628?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mateen N Mansoori updated RANGER-3628: -- Description: Modify ranger solr plugin to allow granting the following privileges: - QUERY - read only privilege on an object - UPDATE - write only privilege on an object - Both - read and write access Privileges can be defined on the following objects: admin * collections * cores * metrics * autoscaling * security - collection - config - schema was: Modify ranger solr plugin to allow granting the following privileges: - QUERY - read only privilege on an object - UPDATE - write only privilege on an object - Both - read and write access Privileges can be defined on the following objects: - admin * collections * cores * metrics * autoscaling * security - collection - config - schema > Support fine grain authorization for different solr objects > --- > > Key: RANGER-3628 > URL: https://issues.apache.org/jira/browse/RANGER-3628 > Project: Ranger > Issue Type: Improvement > Components: Ranger >Affects Versions: 3.0.0, 2.3.0 >Reporter: Mateen N Mansoori >Priority: Major > Fix For: 3.0.0, 2.3.0 > > > Modify ranger solr plugin to allow granting the following privileges: > - QUERY - read only privilege on an object > - UPDATE - write only privilege on an object > - Both - read and write access > Privileges can be defined on the following objects: > admin > * collections > * cores > * metrics > * autoscaling > * security > - collection > - config > - schema -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (RANGER-3628) Support fine grain authorization for different solr objects
[ https://issues.apache.org/jira/browse/RANGER-3628?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mateen N Mansoori updated RANGER-3628: -- Description: Modify ranger solr plugin to allow granting the following privileges: - QUERY - read only privilege on an object - UPDATE - write only privilege on an object - Both - read and write access Privileges can be defined on the following objects: - admin * collections * cores * metrics * autoscaling * security - collection - config - schema was: Modify ranger solr plugin to allow granting the following privileges: * QUERY - read only privilege on an object * UPDATE - write only privilege on an object * * - read and write access Privileges can be defined on the following objects: admin * collections * cores * metrics * autoscaling * security - collection - config - schema > Support fine grain authorization for different solr objects > --- > > Key: RANGER-3628 > URL: https://issues.apache.org/jira/browse/RANGER-3628 > Project: Ranger > Issue Type: Improvement > Components: Ranger >Affects Versions: 3.0.0, 2.3.0 >Reporter: Mateen N Mansoori >Priority: Major > Fix For: 3.0.0, 2.3.0 > > > Modify ranger solr plugin to allow granting the following privileges: > - QUERY - read only privilege on an object > - UPDATE - write only privilege on an object > - Both - read and write access > Privileges can be defined on the following objects: > - admin > * collections > * cores > * metrics > * autoscaling > * security > - collection > - config > - schema -- This message was sent by Atlassian Jira (v8.20.1#820001)
Re: Review Request 73852: RANGER-3595, refactor the file layout of ranger-xxx-kms.tar.gz
> On 二月 17, 2022, 7:54 a.m., bhavik patel wrote:
> > Thanks for the deatiled info and cleaning up the packing.
> >
> > can you please verify zone operations, import/export keys to jceks file and
> > masterkey import/export operation.
> >
> > I just wan to make we are not breaking the existing functionality’s.
>
> Kirby Zhou wrote:
> Tested:
>
> # Basic APIs passed
>
> 1. TestConnection by rangeradmin
> 2. Browes existing keys by rangeradmin
> 2. CreateKey by rangeradmin
> 3. KeyRoll by rangeradmin
> 4. generateeek by curl
> 5. decrypteek by curl
>
> # export passed
> ]$ ./exportKeysToJCEKS.sh mkeybak jceks
> Enter Password for the keystore FILE :
> Enter Password for the KEY(s) stored in the keystore:
> Keys from Ranger KMS Database has been successfully exported into mkeybak
> ]$ keytool -list -keystore mkeybak -storetype jceks
> Enter keystore password:
> Keystore type: JCEKS
> Keystore provider: SunJCE
>
> Your keystore contains 12 entries
>
> hell, Feb 17, 2022, SecretKeyEntry,
> hell-1-2-31, Feb 17, 2022, SecretKeyEntry,
> hell-1-2-31@0, Feb 17, 2022, SecretKeyEntry,
> hell-1-2-31@1, Feb 17, 2022, SecretKeyEntry,
> hell@0, Feb 17, 2022, SecretKeyEntry,
> hell@1, Feb 17, 2022, SecretKeyEntry,
> hell@2, Feb 17, 2022, SecretKeyEntry,
> hello-world, Feb 17, 2022, SecretKeyEntry,
> hello-world@0, Feb 17, 2022, SecretKeyEntry,
> hello-world@1, Feb 17, 2022, SecretKeyEntry,
> paladin, Feb 17, 2022, SecretKeyEntry,
> paladin@0, Feb 17, 2022, SecretKeyEntry,
>
>
>
> # import passed with a bit of accident
> bash -c 'function java() { /usr/bin/java -Djceks.key.serialFilter= "$@" ;
> } ; source ./importJCEKSKeys.sh mkeybak jceks'
> Enter Password for the keystore FILE :
> Enter Password for the KEY(s) stored in the keystore:
> Keys from mkeybak has been successfully imported into RangerDB.
>
> # Why -Djceks.key.serialFilter appears?
>
> See
> https://www.oracle.com/java/technologies/javase/8u171-relnotes.html#JDK-8189997
>
> Since jdk-8u171, importJCEKSKeys is broken without
> -Djceks.key.serialFilter=.
> I tested it at tag-ranger-2.2, it is broken too.
> But that is another story, I wont fix it in that patch here.
Let us check "RangerKeyStoreProvider.java"
```
KeyVersion innerSetKeyVersion(...) {
if (azureKeyVaultEnabled || this.isGCPEnabled) {
// ...
} else {
dbStore.addKeyEntry(versionName, new
SecretKeySpec(material,
cipher), masterKey, cipher,
bitLength, description,
version, attribute);
}
}
public void flush(...) {
if (azureKeyVaultEnabled ||
this.isGCPEnabled) {
// ...
} else {
dbStore.addKeyEntry(entry.getKey(), new KeyMetadata(
metadata),
masterKey, metadata.getAlgorithm(),
metadata.getBitLength(), metadata
.getDescription(), metadata
.getVersions(), attributes);
}
}
```
innerSetKeyVersion is called by createKey and rollNewVersion to create
versioned Key "@0" "@1", and flush is called after
createKey/rollNewVersion, it will create "".
You can see innerSetKeyVersion call new SecretKeySpec(...) and flush call new
KeyMetadata(...).
And the SecretKeySpec can not be unsealed without "-Djceks.key.serialFilter="
I donot know the reason of store type choice between KeyMetadata and
SecretKeySpec.
Maybe someone can tell me?
The code is writteb by @dhavalshah9131 , commited by @mehulbparikh
02640d3cf520ee5a0dae13dd46b91ff266359e77
RANGER-2497 : Support Azure Key Vault for storing master keys of Ranger KMS
- Kirby
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73852/#review224066
---
On 二月 16, 2022, 10:29 a.m., Kirby Zhou wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/73852/
> ---
>
> (Updated 二月 16, 2022, 10:29 a.m.)
>
>
> Review request for ranger, Dhaval Shah,
Re: Review Request 73852: RANGER-3595, refactor the file layout of ranger-xxx-kms.tar.gz
> On 二月 17, 2022, 7:54 a.m., bhavik patel wrote:
> > Thanks for the deatiled info and cleaning up the packing.
> >
> > can you please verify zone operations, import/export keys to jceks file and
> > masterkey import/export operation.
> >
> > I just wan to make we are not breaking the existing functionality’s.
Tested:
# Basic APIs passed
1. TestConnection by rangeradmin
2. Browes existing keys by rangeradmin
2. CreateKey by rangeradmin
3. KeyRoll by rangeradmin
4. generateeek by curl
5. decrypteek by curl
# export passed
]$ ./exportKeysToJCEKS.sh mkeybak jceks
Enter Password for the keystore FILE :
Enter Password for the KEY(s) stored in the keystore:
Keys from Ranger KMS Database has been successfully exported into mkeybak
]$ keytool -list -keystore mkeybak -storetype jceks
Enter keystore password:
Keystore type: JCEKS
Keystore provider: SunJCE
Your keystore contains 12 entries
hell, Feb 17, 2022, SecretKeyEntry,
hell-1-2-31, Feb 17, 2022, SecretKeyEntry,
hell-1-2-31@0, Feb 17, 2022, SecretKeyEntry,
hell-1-2-31@1, Feb 17, 2022, SecretKeyEntry,
hell@0, Feb 17, 2022, SecretKeyEntry,
hell@1, Feb 17, 2022, SecretKeyEntry,
hell@2, Feb 17, 2022, SecretKeyEntry,
hello-world, Feb 17, 2022, SecretKeyEntry,
hello-world@0, Feb 17, 2022, SecretKeyEntry,
hello-world@1, Feb 17, 2022, SecretKeyEntry,
paladin, Feb 17, 2022, SecretKeyEntry,
paladin@0, Feb 17, 2022, SecretKeyEntry,
# import passed with a bit of accident
bash -c 'function java() { /usr/bin/java -Djceks.key.serialFilter= "$@" ; } ;
source ./importJCEKSKeys.sh mkeybak jceks'
Enter Password for the keystore FILE :
Enter Password for the KEY(s) stored in the keystore:
Keys from mkeybak has been successfully imported into RangerDB.
# Why -Djceks.key.serialFilter appears?
See
https://www.oracle.com/java/technologies/javase/8u171-relnotes.html#JDK-8189997
Since jdk-8u171, importJCEKSKeys is broken without -Djceks.key.serialFilter=.
I tested it at tag-ranger-2.2, it is broken too.
But that is another story, I wont fix it in that patch here.
- Kirby
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73852/#review224066
---
On 二月 16, 2022, 10:29 a.m., Kirby Zhou wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/73852/
> ---
>
> (Updated 二月 16, 2022, 10:29 a.m.)
>
>
> Review request for ranger, Dhaval Shah, Dineshkumar Yadav, Gautam Borad,
> Jayendra Parab, Kishor Gollapalliwar, Abhay Kulkarni, Mateen Mansoori, Mehul
> Parikh, pengjianhua, Pradeep Agrawal, VaradreawiZTV VaradreawiZTV, Vishal
> Suvagia, Velmurugan Periasamy, and Qiang Zhang.
>
>
> Bugs: RANGER-3595
> https://issues.apache.org/jira/browse/RANGER-3595
>
>
> Repository: ranger
>
>
> Description
> ---
>
> There are lots of .class files under ews/webapp/, and never used.
>
>
> 1. place web.xml at correct location
> 2. setup.sh want to patch hadoop-common.jar at runtime, it requires some file
> inside ranger-kms.jar. But the patching of hadoop-common.jar is unnecessary.
>
> Regular webapp should have its own class files under
> ews/webapp/WEB-INF/classes, and dependencies under ews/webapp/WEB-INF/lib,
> and the Container should put its libraries under ews/lib. But at current, we
> use directories sucn as ews/webapp/lib, ews/webapp/WEB-INF/classes/lib. It
> looks dirty and ugly.
>
>
> My patch here makes KMS no longer bring ranger-kms.jar, and place classes and
> web.xml at correct location. as a alternative of
> https://reviews.apache.org/r/73816/
>
>
> Now:
> ews/lib contains ews bootstrap jars,
> ews/webapp/WEB-INF/classes contains KMS app itself,
> ews/webapp/WEB-INF/lib contains KMS dependencies,
> ews/webapp/WEB-INF/lib/ranger-kms-plugin-impl contains ranger-kms-plugin.
>
> Additionaly, kms/pom.xml even depends on original hadoop-kms, which can
> confuse developers, so I removed it.
>
> BTW: the bootstrap embedded server looks like too heavy and too much
> dependeices.
>
>
> Diffs
> -
>
> distro/src/main/assembly/kms.xml 983a43e59
> kms/pom.xml 7a4f98df7
> kms/scripts/DBMK2HSM.sh 001199d97
> kms/scripts/DBMKTOAZUREKEYVAULT.sh cfe5a6b5e
> kms/scripts/DBMKTOKEYSECURE.sh c0aa6e58c
> kms/scripts/HSMMK2DB.sh 6c77f7340
> kms/scripts/KEYSECUREMKTOKMSDB.sh 340e05e2c
> kms/scripts/VerifyIsDBMasterkeyCorrect.sh 1c9a2e148
> kms/scripts/exportKeysToJCEKS.sh f3205789b
> kms/scripts/importJCEKSKeys.sh 5d4fe978f
> kms/scripts/ranger-kms 429a31e5a
> kms/scripts/setup.sh 2051df59a
> kms/src/main/java/org/apache/hadoop/crypto/key/kms/server/KMSWebApp.java
> c899bdf98
> kms/src/main/resources/META-INF/context.xml
> kms/src/main/resources/WEB-INF/web.xml 5e2d489fe
>
