----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/75197/ -----------------------------------------------------------
Review request for ranger, Anand Nadar, Asit Vadhavkar, madhan, Madhan Neethiraj, Monika Kachhadiya, and Siddhesh Phatak. Bugs: https://issues.apache.org/jira/browse/RANGER-4907 https://issues.apache.org/jira/browse/https://issues.apache.org/jira/browse/RANGER-4907 Repository: ranger Description ------- If a user has large number of associations with groups, policies, the user delete operation can take large amount of time, especially while updating the policies. Since to update a policy, we have large number of validations in place. This can be even worse, if multiple user delete requests are received at the same time. We can add following optimizations to improve the user delete performance: 1.Remove foreign key relation of x_user with x_auth_sess, with this we can skip updating the user references in x_auth_sess when a user is deleted. 2.Currently all policy reference for a user are cleaned up and updated ones are added again. We can remove policy reference only for the user being deleted. 3.Added new policy update method, take list of policies, update only the policyText and push policy updates in bulk. Diffs ----- security-admin/db/mysql/optimized/current/ranger_core_db_mysql.sql 80af2c8ce security-admin/db/oracle/optimized/current/ranger_core_db_oracle.sql 8d528e4ee security-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql 6bb9c525a security-admin/db/sqlanywhere/optimized/current/ranger_core_db_sqlanywhere.sql a37c9eb8e security-admin/db/sqlserver/optimized/current/ranger_core_db_sqlserver.sql 151f6804b security-admin/src/main/java/org/apache/ranger/biz/PolicyRefUpdater.java ed0992604 security-admin/src/main/java/org/apache/ranger/biz/ServiceDBStore.java c934fdd7c security-admin/src/main/java/org/apache/ranger/biz/XUserMgr.java 5ba6c14b9 security-admin/src/main/java/org/apache/ranger/db/XXAuthSessionDao.java 645c27cbd security-admin/src/main/java/org/apache/ranger/db/XXGroupUserDao.java 76cdd93f5 security-admin/src/main/java/org/apache/ranger/db/XXPolicyRefGroupDao.java 3ce371306 security-admin/src/main/java/org/apache/ranger/db/XXPolicyRefRoleDao.java 29d2bc4c9 security-admin/src/main/java/org/apache/ranger/db/XXPolicyRefUserDao.java 07d94ceb9 security-admin/src/main/java/org/apache/ranger/service/XPortalUserService.java 9ab886e43 security-admin/src/main/resources/META-INF/jpa_named_queries.xml 1672c148e security-admin/src/test/java/org/apache/ranger/biz/TestXUserMgr.java de342e994 Diff: https://reviews.apache.org/r/75197/diff/1/ Testing ------- Validations done with PostgresDB: All the existing JUnits are passing Case: Delete User 1.Time taken to delete a user with large no of linked groups and policies, before and after optmization Without optmization Groups: 10k, Policies: 10k, User deletion time: 4.32 min Version: Before update: Policy: 1011 Userstore: 2590 After update: Policy: 2011 Userstore: 2590 With optmization Groups: 10k, Policies: 10k, User deletion time: 7.81 sec Version: Before update: Policy: 10011 Userstore: 10003 After update: Policy: 10012 Userstore: 10003 2. Confirm policy version is incremented 3. Confirm admin audit is generated for the user deletion Admin audits generated for user and user profile deletion Case: Delete Group 1.Time taken to delete a group with large no of linked users and policies, before and after optmization Without optmization Users: 1.5k, Policies: 1k, Group deletion time: 6.59 min Version: Before update: Policy: 1011 Userstore: 1589 After update: Policy: 2011 Userstore: 1589 With optmization Users: 10k, Policies: 10k, Group deletion time: 6.37 sec Version: Before update: Policy: 10011 Userstore: 20005 After update: Policy: 10012 Userstore: 10003 2. Confirm policy version is incremented 3. Confirm admin audit is generated for the group deletion Admin audits generated for group deletion Thanks, Subhrat Chaudhary
