[jira] [Commented] (RANGER-4026) Provide option to update group memberships when same users/groups are synced from different sync sources
[ https://issues.apache.org/jira/browse/RANGER-4026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17721871#comment-17721871 ] Abhishek Kumar commented on RANGER-4026: PR: [https://github.com/apache/ranger/pull/254] > Provide option to update group memberships when same users/groups are synced > from different sync sources > > > Key: RANGER-4026 > URL: https://issues.apache.org/jira/browse/RANGER-4026 > Project: Ranger > Issue Type: Improvement > Components: usersync >Reporter: Sailaja Polavarapu >Assignee: Abhishek Kumar >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > RANGER-3254 implemented a change in user/group mapping so that sync source is > taken into account when a group name matches multiple sources. LDAP users > belonging to a group like "CN=mygroup" will not be synced in Ranger if there > is an existing "mygroup" that was imported by UnixUserGroupBuilder. > This breaks a very common use case where posix users and groups are synced to > the OS from an LDAP backend using SSSD, Centrify, or similar utilities. In > those cases, both the linux OS and LDAP/AD are using the same identity > repository. If Ranger imported a set of users and groups from one sync > source, and then later switches to another, group mappings break and users > don't get all of their groups. > Provide an option for customers to treat users/groups from multiple sync > sources as same for updating group memberships. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [ranger] kumaab opened a new pull request, #254: RANGER-4026: Allow sync source updates for existing users & groups synced via different sync sources
kumaab opened a new pull request, #254: URL: https://github.com/apache/ranger/pull/254 ## What changes were proposed in this pull request? [RANGER-3254](https://issues.apache.org/jira/browse/RANGER-3254) implemented a change in user/group mapping so that sync source is taken into account when a group name matches multiple sources. LDAP users belonging to a group like "CN=mygroup" will not be synced in Ranger if there is an existing "mygroup" that was imported by UnixUserGroupBuilder. This breaks a very common use case where posix users and groups are synced to the OS from an LDAP backend. In those cases, both the linux OS and LDAP/AD are using the same identity repository. If Ranger imported a set of users and groups from one sync source, and then later switches to another, group mappings break and users don't get all of their groups. Provide an option to treat users/groups from multiple sync sources as same for updating group memberships. ## How was this patch tested? Tested changes by changing sync source & restarting usersync. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@ranger.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (RANGER-4235) Optimize security-zone create/update to avoid unnecessary ref table entries
[
https://issues.apache.org/jira/browse/RANGER-4235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17721800#comment-17721800
]
Madhan Neethiraj commented on RANGER-4235:
--
{noformat}
commit 54a2cd0a452aa8eeb3e54b3ba4a1ca6150b79791 (HEAD -> master, origin/master,
origin/HEAD)
Author: Madhan Neethiraj
Date: Thu May 11 00:49:07 2023 -0700
RANGER-4235: security-zone persistence optimized to avoid creation of
unnecessary ref table entries
{noformat}
{noformat}
commit 5f7c504392276ab3060dc473908c0aec54ece755 (HEAD -> ranger-2.4,
origin/ranger-2.4)
Author: Madhan Neethiraj
Date: Thu May 11 00:49:07 2023 -0700
RANGER-4235: security-zone persistence optimized to avoid creation of
unnecessary ref table entries
(cherry picked from commit 54a2cd0a452aa8eeb3e54b3ba4a1ca6150b79791)
{noformat}
> Optimize security-zone create/update to avoid unnecessary ref table entries
> ---
>
> Key: RANGER-4235
> URL: https://issues.apache.org/jira/browse/RANGER-4235
> Project: Ranger
> Issue Type: Improvement
> Components: admin
>Reporter: Madhan Neethiraj
>Assignee: Madhan Neethiraj
>Priority: Major
> Fix For: 3.0.0, 2.4.1
>
>
> Security zone create/update operations create ref-table entries for each
> resource included in the zone. Instead of creating ref-table entries for each
> resource, it is enough to create one ref table entry for each resource-type -
> like database, table, column, path. This will result in significantly less
> number of ref-table entries for security-zones having large number of
> resources, which in turn will reduce the time taken to update such security
> zones.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
Re: Review Request 74404: RANGER-4165:API to find whether a user/group is authorized to the given operation on any resource of give type
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/74404/ --- (Updated May 11, 2023, 1:58 p.m.) Review request for ranger, Abhay Kulkarni and Madhan Neethiraj. Bugs: RANGER-4165 https://issues.apache.org/jira/browse/RANGER-4165 Repository: ranger Description --- RANGER-4165:API to find whether a user/group is authorized to the given operation on any resource of give type Diffs - agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/RangerTagEnricher.java e0a86c398 agents-common/src/main/java/org/apache/ranger/plugin/model/validation/RangerSecurityZoneValidator.java ca899979a agents-common/src/main/java/org/apache/ranger/plugin/policyengine/PolicyEngine.java 3864f30d2 agents-common/src/main/java/org/apache/ranger/plugin/policyengine/RangerPolicyEngineImpl.java e75bb722c agents-common/src/main/java/org/apache/ranger/plugin/policyengine/RangerPolicyRepository.java b5b26702c agents-common/src/main/java/org/apache/ranger/plugin/policyengine/RangerResourceTrie.java f89d51e35 agents-common/src/main/java/org/apache/ranger/plugin/resourcematcher/RangerAbstractResourceMatcher.java 032d4487c agents-common/src/main/java/org/apache/ranger/plugin/resourcematcher/RangerDefaultResourceMatcher.java c421388e7 agents-common/src/main/java/org/apache/ranger/plugin/resourcematcher/ResourceMatcher.java 5df4f1e3a agents-common/src/main/java/org/apache/ranger/plugin/util/RangerAccessRequestUtil.java b505f495b agents-common/src/main/java/org/apache/ranger/plugin/util/RangerResourceEvaluatorsRetriever.java e60fe055b agents-common/src/test/java/org/apache/ranger/plugin/policyengine/TestPolicyEngine.java b2a5151e5 agents-common/src/test/resources/policyengine/test_policyengine_kafka.json PRE-CREATION agents-common/src/test/resources/resourcematcher/test_resourcematcher_default.json 779f57a0b security-admin/src/main/java/org/apache/ranger/biz/RangerPolicyAdminImpl.java 97a384f30 Diff: https://reviews.apache.org/r/74404/diff/1/ Testing (updated) --- - Testing done with TestCase. -- Request has to set the resource as "*_any*_" and add a request context "RESOURCE_TYPE" = "". example: resource => "_any_topic", context => "topic" , operation => consume, user => "user1" -- Policy maintained => user1 will have access to consume on several topics, this call should result in "ALLOWED". -- Testing done with new tests in agents-common/src/test/resources/policyengine/test_policyengine_kafka.json -- Ran all the PolicyEngine and plugin tests. Thanks, Ramesh Mani
Re: Review Request 74431: Add flag based support for mounting db volume in dev-support scripts
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/74431/ --- (Updated May 11, 2023, 1:12 p.m.) Review request for ranger and Jai Patel. Repository: ranger Description --- Currently, the DB which gets used via dev-support scripts is not mounted to any local storage. As a result, we lose all data if docker containers are shutdown and brought up again. Raising this ticket to have a flag based support for mounting the db volumes. Diffs (updated) - dev-support/ranger-docker/.env 3f795abde dev-support/ranger-docker/docker-compose.ranger-postgres-mounted.yml PRE-CREATION ranger_in_docker 2057bde3e Diff: https://reviews.apache.org/r/74431/diff/2/ Changes: https://reviews.apache.org/r/74431/diff/1-2/ Testing --- tested and verified in UI. mounted db changes are visible Thanks, Mohit Ambalkar
Review Request 74431: Add flag based support for mounting db volume in dev-support scripts
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/74431/ --- Review request for ranger and Jai Patel. Repository: ranger Description --- Currently, the DB which gets used via dev-support scripts is not mounted to any local storage. As a result, we lose all data if docker containers are shutdown and brought up again. Raising this ticket to have a flag based support for mounting the db volumes. Diffs - dev-support/ranger-docker/.env 3f795abde dev-support/ranger-docker/docker-compose.ranger-postgres-mounted.yml PRE-CREATION ranger_in_docker 2057bde3e Diff: https://reviews.apache.org/r/74431/diff/1/ Testing --- tested and verified in UI. mounted db changes are visible Thanks, Mohit Ambalkar
Re: Review Request 74428: RANGER-4235: security-zone persistence optimized to avoid creation of unnecessary ref table entries
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/74428/#review225449 --- Ship it! Ship It! - Ankita Sinha On May 11, 2023, 8:51 a.m., Madhan Neethiraj wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/74428/ > --- > > (Updated May 11, 2023, 8:51 a.m.) > > > Review request for ranger, Abhishek Kumar, Ankita Sinha, Abhay Kulkarni, > Mehul Parikh, Monika Kachhadiya, Pradeep Agrawal, Ramesh Mani, Sailaja > Polavarapu, Subhrat Chaudhary, and Velmurugan Periasamy. > > > Bugs: RANGER-4235 > https://issues.apache.org/jira/browse/RANGER-4235 > > > Repository: ranger > > > Description > --- > > Optimized to create only one ref-table entry per resource-type. With this > optimization, for a security-zone containing 10,000 Hive tables, 2 ref-table > entries are created instead of 20,000 ref-table entries. > > > Diffs > - > > > security-admin/src/main/java/org/apache/ranger/biz/SecurityZoneRefUpdater.java > 4cfe62701 > > > Diff: https://reviews.apache.org/r/74428/diff/1/ > > > Testing > --- > > - create and updated security zones having 10 resources to 50,000 resources > - creating security zone with 10,000 resources took 10m46s; with this > optimization, the same operation took less than 2 seconds > - updating security zone having 1,000 resources with 5,000 resources took > 6m3s; with this optimization the same operation took less than 0.5 second > > > Thanks, > > Madhan Neethiraj > >
Review Request 74428: RANGER-4235: security-zone persistence optimized to avoid creation of unnecessary ref table entries
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/74428/ --- Review request for ranger, Abhishek Kumar, Ankita Sinha, Abhay Kulkarni, Mehul Parikh, Monika Kachhadiya, Pradeep Agrawal, Ramesh Mani, Sailaja Polavarapu, Subhrat Chaudhary, and Velmurugan Periasamy. Bugs: RANGER-4235 https://issues.apache.org/jira/browse/RANGER-4235 Repository: ranger Description --- Optimized to create only one ref-table entry per resource-type. With this optimization, for a security-zone containing 10,000 Hive tables, 2 ref-table entries are created instead of 20,000 ref-table entries. Diffs - security-admin/src/main/java/org/apache/ranger/biz/SecurityZoneRefUpdater.java 4cfe62701 Diff: https://reviews.apache.org/r/74428/diff/1/ Testing --- - create and updated security zones having 10 resources to 50,000 resources - creating security zone with 10,000 resources took 10m46s; with this optimization, the same operation took less than 2 seconds - updating security zone having 1,000 resources with 5,000 resources took 6m3s; with this optimization the same operation took less than 0.5 second Thanks, Madhan Neethiraj
[jira] [Created] (RANGER-4235) Optimize security-zone create/update to avoid unnecessary ref table entries
Madhan Neethiraj created RANGER-4235: Summary: Optimize security-zone create/update to avoid unnecessary ref table entries Key: RANGER-4235 URL: https://issues.apache.org/jira/browse/RANGER-4235 Project: Ranger Issue Type: Improvement Components: admin Reporter: Madhan Neethiraj Assignee: Madhan Neethiraj Security zone create/update operations create ref-table entries for each resource included in the zone. Instead of creating ref-table entries for each resource, it is enough to create one ref table entry for each resource-type - like database, table, column, path. This will result in significantly less number of ref-table entries for security-zones having large number of resources, which in turn will reduce the time taken to update such security zones. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [ranger] kumaab commented on a diff in pull request #253: RANGER-4230: Add REST APIs to delete all external users & groups
kumaab commented on code in PR #253:
URL: https://github.com/apache/ranger/pull/253#discussion_r1190729008
##
security-admin/src/main/java/org/apache/ranger/rest/XUserREST.java:
##
@@ -165,6 +166,10 @@ public class XUserREST {
static final Logger logger = LoggerFactory.getLogger(XUserMgr.class);
+ static volatile boolean usersDeletionInProgress = false;
Review Comment:
Sure, makes sense. Sometimes these calls might take long time to get a
response, the idea was to get a response indicating the progress(like 30%
deletion complete or deletion in progress check back later) in the interim via
another simultaneous call.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscr...@ranger.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [ranger] kumaab commented on a diff in pull request #253: RANGER-4230: Add REST APIs to delete all external users & groups
kumaab commented on code in PR #253:
URL: https://github.com/apache/ranger/pull/253#discussion_r1190726336
##
security-admin/src/main/java/org/apache/ranger/rest/XUserREST.java:
##
@@ -863,6 +868,49 @@ public void deleteXUserByUserName(@PathParam("userName")
String userName,
xUserMgr.deleteXUser(vxUser.getId(), forceDelete);
}
+
+ /**
+* Proceed with caution: Force deletes users in bulk from the ranger db,
+* essentially serves as a cleanup Op for external users.
+* Delete happens one at a time with immediate commit on the
transaction.
+*/
+ @DELETE
+ @Path("/delete/external/users")
+ @PreAuthorize("hasRole('ROLE_SYS_ADMIN')")
+ @Produces({ "application/json" })
+ public Response deleteXUsers() {
Review Comment:
thanks for the suggestion, changes included in the latest patch.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscr...@ranger.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [ranger] kumaab commented on a diff in pull request #253: RANGER-4230: Add REST APIs to delete all external users & groups
kumaab commented on code in PR #253:
URL: https://github.com/apache/ranger/pull/253#discussion_r1190725954
##
security-admin/src/main/java/org/apache/ranger/rest/XUserREST.java:
##
@@ -863,6 +868,49 @@ public void deleteXUserByUserName(@PathParam("userName")
String userName,
xUserMgr.deleteXUser(vxUser.getId(), forceDelete);
}
+
+ /**
+* Proceed with caution: Force deletes users in bulk from the ranger db,
+* essentially serves as a cleanup Op for external users.
+* Delete happens one at a time with immediate commit on the
transaction.
+*/
+ @DELETE
+ @Path("/delete/external/users")
+ @PreAuthorize("hasRole('ROLE_SYS_ADMIN')")
+ @Produces({ "application/json" })
+ public Response deleteXUsers() {
+ if (!usersDeletionInProgress){
+ synchronized (XUserREST.class) {
+ usersDeletionInProgress = true;
+ xUserMgr.bulkDeleteUsers();
+ usersDeletionInProgress = false;
+ }
+ return Response.ok("External users deleted
successfully").build();
+ }
+ return Response.ok("Users Deletion in progress, check ranger
admin logs!").build();
+ }
+
+ /**
+* Proceed with caution: Force deletes groups in bulk from the ranger
db,
+* essentially serves as a cleanup Op for external groups.
+* Delete happens one at a time with immediate commit on the
transaction.
+*/
+ @DELETE
+ @Path("/delete/external/groups")
+ @PreAuthorize("hasRole('ROLE_SYS_ADMIN')")
+ @Produces({ "application/json" })
+ public Response deleteXGroups() {
Review Comment:
thanks for the suggestion, changes included in the latest patch.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscr...@ranger.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
