[
https://issues.apache.org/jira/browse/RANGER-3900?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ramachandran updated RANGER-3900:
---------------------------------
Description:
*{color:#0000ff}Role Deletion Steps in Apache Ranger:{color}*{color:#0000ff}
{color}
{code:java}
1. Getting XXRole by roleName:
XXRole xxRole = daoMgr.getXXRole().findByRoleName(roleName); → 1DB Read call
2. Ensuring role is not in policy
Long roleRefPolicyCount =
daoMgr.getXXPolicyRefRole().findRoleRefPolicyCount(roleName); → 1DB Read call
3. Ensuring role is not in another role
Long roleRefRoleCount =
daoMgr.getXXRoleRefRole().findRoleRefRoleCount(roleName);→ 1DB Read call
4. Fetching RangerRole by id
RangerRole role = roleService.read(xxRole.getId()); → 1DB Read call
5. Dereferencing all the users from the role.
for (XXRoleRefUser xxRoleRefUser : xRoleUserDao.findByRoleId(roleId)) { → 1DB
Read call
xRoleUserDao.remove(xxRoleRefUser); → 1DB Write call for each xxRoleRefUser
}
6. Dereferencing all the groups from the role
for (XXRoleRefGroup xxRoleRefGroup : xRoleGroupDao.findByRoleId(roleId)) { →
1DB Read call
xRoleGroupDao.remove(xxRoleRefGroup); → 1DB Write call for each xxRoleRefGroup
}
7. Dereferencing all other roles from the role.
for (XXRoleRefRole xxRoleRefRole : xRoleRoleDao.findByRoleId(roleId)) {→ 1DB
Read call
xRoleRoleDao.remove(xxRoleRefRole);→ 1DB Write call for each xxRoleRefRole
}
8. Delete the rangerRole in DB
roleService.delete(role); → 1DB Write call
9. Create TrxLog
bizUtil.createTrxLog(trxLogList) → 1DB Write call
{code}
*{color:#0000ff}Total number of DB calls involved for the below role deletion
in Apache Ranger:{color}*
*Deleting the role which contains more users:*
1. role contains 100 users
||DB Write count||DB Read count||DB Batch Write count||Time is taken to delete
the role||
|102|6|0|1.01 seconds|
1 role contains 500 users
||DB Write count||DB Read count||DB Batch Write count||Time is taken to delete
the role||
|502|6|0| 2.241 seconds|
1 role contains 1000 users
||DB Write count||DB Read count||DB Batch Write count||Time is taken to delete
the role||
|1002|6|0|3.14 seconds|
1 role contains 1000 users ,1000 roles,1000 groups
||DB Write count||DB Read count||DB Batch Write count||Time taken to delete the
role||
|3002|6|0|11.577 seconds|
{color:#ff0000}Proposal{color}:
{color:#ff0000}We can try a DB batch write call instead of 1 DB write call for
every user deference{color}
After Fix
1 role contains 500 users
||DB Write count||DB Read count||DB Batch Write count||Time taken to delete the
role||
|3|6|0|0.666 seconds |
1 role contains 1000 users
||DB Write count||DB Read count||DB Batch Write count||Time taken to delete the
role||
|3|6|0|0.835 seconds|
1 role contains 1000 users ,1000 roles,1000 groups
||DB Write count||DB Read count||DB Batch Write count||Time taken to delete the
role||
|5|6|0|1.021 seconds|
{color:#ff0000}Recommendation :{color}
When we reduce the number of users added to the role -→ Total number of DB
calls will be reduced
Instead of adding roles to the users, can we add those users into the group.
And then link that group to the role
if we use groups instead of directly adding users into the role, the total
number of DB write calls will be reduced to 2 from 1002 for 1k users
was:
*{color:#0000ff}Role Deletion Steps in Apache Ranger:{color}*{color:#0000ff}
{color}
{code:java}
1. Getting XXRole by roleName:
XXRole xxRole = daoMgr.getXXRole().findByRoleName(roleName); → 1DB Read call
2. Ensuring role is not in policy
Long roleRefPolicyCount =
daoMgr.getXXPolicyRefRole().findRoleRefPolicyCount(roleName); → 1DB Read call
3. Ensuring role is not in another role
Long roleRefRoleCount =
daoMgr.getXXRoleRefRole().findRoleRefRoleCount(roleName);→ 1DB Read call
4. Fetching RangerRole by id
RangerRole role = roleService.read(xxRole.getId()); → 1DB Read call
5. Dereferencing all the users from the role.
for (XXRoleRefUser xxRoleRefUser : xRoleUserDao.findByRoleId(roleId)) { → 1DB
Read call
xRoleUserDao.remove(xxRoleRefUser); → 1DB Write call for each xxRoleRefUser
}
6. Dereferencing all the groups from the role
for (XXRoleRefGroup xxRoleRefGroup : xRoleGroupDao.findByRoleId(roleId)) { →
1DB Read call
xRoleGroupDao.remove(xxRoleRefGroup); → 1DB Write call for each xxRoleRefGroup
}
7. Dereferencing all other roles from the role.
for (XXRoleRefRole xxRoleRefRole : xRoleRoleDao.findByRoleId(roleId)) {→ 1DB
Read call
xRoleRoleDao.remove(xxRoleRefRole);→ 1DB Write call for each xxRoleRefRole
}
8. Delete the rangerRole in DB
roleService.delete(role); → 1DB Write call
9. Create TrxLog
bizUtil.createTrxLog(trxLogList) → 1DB Write call
{code}
*{color:#0000ff}Total number of DB calls involved for the below role deletion
in Apache Ranger:{color}*
*Deleting the role which contains more users:*
1. role contains 100 users
||DB Write count||DB Read count||DB Batch Write count||Time is taken to delete
the role||
|102|6|0|1.01 seconds|
1 role contains 500 users
||DB Write count||DB Read count||DB Batch Write count||Time is taken to delete
the role||
|502|6|0| 2.241 seconds|
1 role contains 1000 users
||DB Write count||DB Read count||DB Batch Write count||Time is taken to delete
the role||
|1002|6|0|3.14 seconds|
{color:#ff0000}Recommendation :{color}
When we reduce the number of users added to the role -→ Total number of DB
calls will be reduced
Instead of adding roles to the users, can we add those users into the group.
And then link that group to the role
if we use groups instead of directly adding users into the role, the total
number of DB write calls will be reduced to 2 from 1002 for 1k users
{color:#ff0000}Proposal{color}:
{color:#ff0000}We can try a DB batch write call instead of 1 DB write call for
every user deference{color}
After Fix
1 role contains 500 users
||DB Write count||DB Read count||DB Batch Write count||Time taken to delete the
role||
|3|6|0|0.666 seconds |
1 role contains 1000 users
||DB Write count||DB Read count||DB Batch Write count||Time taken to delete the
role||
|3|6|0|0.835 seconds|
1 role contains 1000 users ,1000 roles,1000 groups
||DB Write count||DB Read count||DB Batch Write count||Time taken to delete the
role||
|5|6|0|1.021 seconds|
> Roles deletion Takes time in Apache Ranger when there are more
> users,groups,roles
> ---------------------------------------------------------------------------------
>
> Key: RANGER-3900
> URL: https://issues.apache.org/jira/browse/RANGER-3900
> Project: Ranger
> Issue Type: Improvement
> Components: Ranger
> Affects Versions: 3.0.0
> Reporter: Ramachandran
> Assignee: Ramachandran
> Priority: Major
>
> *{color:#0000ff}Role Deletion Steps in Apache Ranger:{color}*{color:#0000ff}
> {color}
>
> {code:java}
>
> 1. Getting XXRole by roleName:
> XXRole xxRole = daoMgr.getXXRole().findByRoleName(roleName); → 1DB Read call
> 2. Ensuring role is not in policy
> Long roleRefPolicyCount =
> daoMgr.getXXPolicyRefRole().findRoleRefPolicyCount(roleName); → 1DB Read call
> 3. Ensuring role is not in another role
> Long roleRefRoleCount =
> daoMgr.getXXRoleRefRole().findRoleRefRoleCount(roleName);→ 1DB Read call
> 4. Fetching RangerRole by id
> RangerRole role = roleService.read(xxRole.getId()); → 1DB Read call
> 5. Dereferencing all the users from the role.
> for (XXRoleRefUser xxRoleRefUser : xRoleUserDao.findByRoleId(roleId)) { →
> 1DB Read call
> xRoleUserDao.remove(xxRoleRefUser); → 1DB Write call for each xxRoleRefUser
> }
> 6. Dereferencing all the groups from the role
> for (XXRoleRefGroup xxRoleRefGroup : xRoleGroupDao.findByRoleId(roleId)) { →
> 1DB Read call
> xRoleGroupDao.remove(xxRoleRefGroup); → 1DB Write call for each
> xxRoleRefGroup
> }
> 7. Dereferencing all other roles from the role.
> for (XXRoleRefRole xxRoleRefRole : xRoleRoleDao.findByRoleId(roleId)) {→ 1DB
> Read call
> xRoleRoleDao.remove(xxRoleRefRole);→ 1DB Write call for each xxRoleRefRole
> }
> 8. Delete the rangerRole in DB
> roleService.delete(role); → 1DB Write call
> 9. Create TrxLog
> bizUtil.createTrxLog(trxLogList) → 1DB Write call
>
> {code}
>
> *{color:#0000ff}Total number of DB calls involved for the below role deletion
> in Apache Ranger:{color}*
>
> *Deleting the role which contains more users:*
> 1. role contains 100 users
> ||DB Write count||DB Read count||DB Batch Write count||Time is taken to
> delete the role||
> |102|6|0|1.01 seconds|
>
> 1 role contains 500 users
> ||DB Write count||DB Read count||DB Batch Write count||Time is taken to
> delete the role||
> |502|6|0| 2.241 seconds|
> 1 role contains 1000 users
> ||DB Write count||DB Read count||DB Batch Write count||Time is taken to
> delete the role||
> |1002|6|0|3.14 seconds|
>
> 1 role contains 1000 users ,1000 roles,1000 groups
> ||DB Write count||DB Read count||DB Batch Write count||Time taken to delete
> the role||
> |3002|6|0|11.577 seconds|
>
> {color:#ff0000}Proposal{color}:
> {color:#ff0000}We can try a DB batch write call instead of 1 DB write call
> for every user deference{color}
>
> After Fix
> 1 role contains 500 users
> ||DB Write count||DB Read count||DB Batch Write count||Time taken to delete
> the role||
> |3|6|0|0.666 seconds |
> 1 role contains 1000 users
> ||DB Write count||DB Read count||DB Batch Write count||Time taken to delete
> the role||
> |3|6|0|0.835 seconds|
>
> 1 role contains 1000 users ,1000 roles,1000 groups
> ||DB Write count||DB Read count||DB Batch Write count||Time taken to delete
> the role||
> |5|6|0|1.021 seconds|
> {color:#ff0000}Recommendation :{color}
> When we reduce the number of users added to the role -→ Total number of DB
> calls will be reduced
> Instead of adding roles to the users, can we add those users into the group.
> And then link that group to the role
> if we use groups instead of directly adding users into the role, the total
> number of DB write calls will be reduced to 2 from 1002 for 1k users
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
