-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74498/
-----------------------------------------------------------
Review request for ranger, Ankita Sinha, Abhay Kulkarni, Mehul Parikh, Pradeep
Agrawal, Ramesh Mani, Sailaja Polavarapu, Subhrat Chaudhary, and Velmurugan
Periasamy.
Bugs: RANGER-4303
https://issues.apache.org/jira/browse/RANGER-4303
Repository: ranger
Description
-------
- added ranger-mem-sizing.sh to ranger-tools
- the tool creates a RangerPlugin object from policies/tags/roles/userStore
loaded from files
Diffs
-----
distro/src/main/assembly/ranger-tools.xml b8713d806
ranger-tools/conf/logback-mem-sizing.xml PRE-CREATION
ranger-tools/scripts/ranger-mem-sizing.sh PRE-CREATION
ranger-tools/src/main/java/org/apache/ranger/sizing/PerfMemTimeTracker.java
PRE-CREATION
ranger-tools/src/main/java/org/apache/ranger/sizing/RangerMemSizing.java
PRE-CREATION
ranger-tools/testdata/test_roles_hive.json PRE-CREATION
ranger-tools/testdata/test_userstore_hive.json PRE-CREATION
Diff: https://reviews.apache.org/r/74498/diff/1/
Testing
-------
- verified that ranger-mem-sizing.sh processes policies/tags/roles/userStore
loaded from files and prints the memory taken up by the plugin
- Usage:
./ranger-mem-sizing.sh -p testdata/test_servicepolicies_hive.json -t
testdata/test_servicetags_hive.json -r testdata/test_roles_hive.json -u
testdata/test_userstore_hive.json
2023-06-26 22:56:35.524: loading
policies(file=testdata/test_servicepolicies_hive.json)
2023-06-26 22:56:35.948: loaded
policies(file=testdata/test_servicepolicies_hive.json, size=5343709):
policyCount=2002
2023-06-26 22:56:35.976: loading tags(file=testdata/test_servicetags_hive.json)
2023-06-26 22:56:36.86: DeDupTags(duplicateTags=0)
2023-06-26 22:56:36.140: loaded tags(file=testdata/test_servicetags_hive.json,
size=472563): tagDefCount=499, tagCount499, resourceCount=499
2023-06-26 22:56:36.160: loading roles(file=testdata/test_roles_hive.json)
2023-06-26 22:56:36.185: loaded roles(file=testdata/test_roles_hive.json,
size=7771): roleCount=10
2023-06-26 22:56:36.204: loading
userStore(file=testdata/test_userstore_hive.json)
2023-06-26 22:56:36.293: loaded
userStore(file=testdata/test_userstore_hive.json, size=3288): users=20,
groups=20, userGroupMappings=20)
2023-06-26 22:56:36.361: Initializing RangerBasePlugin...
2023-06-26 22:56:36.932: Initialized RangerBasePlugin.
Parameters:
Policies: file=testdata/test_servicepolicies_hive.json, size=5343709,
policyCount=2002
Tags: file=testdata/test_servicetags_hive.json, size=472563,
tagDefCount=499, tagCount499, resourceCount=499
Roles: file=testdata/test_roles_hive.json, size=7771, roleCount=10
UserStore: file=testdata/test_userstore_hive.json, size=3288, users=20,
groups=20, userGroupMappings=20
DeDup: true
OptMode: space
Results:
*****************************
Task|Time (ms)|Memory (bytes)
RangerMemSizing|876|18299960
Load policies|314|5387856
Read policies|294|10872064
DeDupStrings|19|-5026928
Load tags|25|732000
Read tags|18|1214160
DeDupTags|3|-26168
DuDupStrings|2|-454296
Load roles|5|60952
Load userStore|3|38856
Read userStore|3|58968
DeDupStrings(userStore)|0|-11192
RangerBasePlugin initialization|502|12920976
*****************************
Thanks,
Madhan Neethiraj
