----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/74245/#review225167 -----------------------------------------------------------
I'm not a committer but I was the author of the original PR a few years back - happy to see this come back and hope it can be merged soon! - Andrew Luo On Jan. 4, 2023, 7:14 a.m., Ramachandran Krishnan wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/74245/ > ----------------------------------------------------------- > > (Updated Jan. 4, 2023, 7:14 a.m.) > > > Review request for ranger, Don Bosco Durai, Kirby Zhou, Abhay Kulkarni, > Madhan Neethiraj, Mehul Parikh, Nikhil P, Pradeep Agrawal, Ramesh Mani, > Selvamohan Neethiraj, Sailaja Polavarapu, Subhrat Chaudhary, and Velmurugan > Periasamy. > > > Bugs: RANGER-3899 > https://issues.apache.org/jira/browse/RANGER-3899 > > > Repository: ranger > > > Description > ------- > > Analysis is when policy has 500 users with 5 policy items. > > we perform total 526 DB read calls and 504 DB write, 3 DB Batch write calls. > This is the reason for policy creation time where 1000 plus DB calls are done > sequentially. > Time is taken to create the policy :4~8 seconds > > 1st#Improvement: > > After Fix :After changing to batch create of XXPolicyRefUser instead of > calling 1 DB write XXPolicyRefUser for every user of the policy: > > When Policy contains 500 users,5 access Types (permissions),12 resources > (1DB,1Table,10 columns) > > we perform total 526 DB read calls and 6 DB write calls, 5 DB Batch write > calls. > > Time is taken to create the policy :2.6~4 seconds > > 2nd#Improvement: > Getting the ID for each role/group/user one-by-one is slow, especially for > large policies with many roles/groups/users. Batching significantly improves > performance. > > Fixing this part will reduce the DB read calls to a very minimum which will > improve performance drastically. > > The policy contains 500 users,5 access Types (permissions),12 resources > (1DB,1Table,10 columns) > > After fix: > DB Read count:27 > > Time taken to create the policy: 1.4 to 2 secs (The policy contains 500 > users,5 access Types (permissions),12 resources (1DB,1Table,10 columns)) > > DB read calls will be reduced to 27 from 526 > > Time is taken to create the policy :1.4~2 seconds > > > Diffs > ----- > > security-admin/src/main/java/org/apache/ranger/biz/PolicyRefUpdater.java > 6cc3509d8 > security-admin/src/main/java/org/apache/ranger/db/XXGroupDao.java 8ca4f62bb > security-admin/src/main/java/org/apache/ranger/db/XXRoleDao.java 4e5b692b5 > security-admin/src/main/java/org/apache/ranger/db/XXUserDao.java 97bc2680a > security-admin/src/main/resources/META-INF/jpa_named_queries.xml 85c8b6213 > > > Diff: https://reviews.apache.org/r/74245/diff/2/ > > > Testing > ------- > > Tested Created policies via load testing > > > Thanks, > > Ramachandran Krishnan > >
