-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73715/
-----------------------------------------------------------
Review request for ranger, Madhan Neethiraj, Pradeep Agrawal, Ramesh Mani,
Sailaja Polavarapu, and Velmurugan Periasamy.
Bugs: RANGER-3519
https://issues.apache.org/jira/browse/RANGER-3519
Repository: ranger
Description
-------
When the number of policies (and/or tagged resources) is large, the data
structures used by Ranger as indexes for policies (and/or tagged resources) may
need a very large heap memory because they are optimized for fast lookup. It is
desirable to be able to configure Ranger to have these structures optimized for
space in order to keep the heap requirements within acceptable limit at the
cost of somewhat slower lookup.
---
There are two new configuration variables that control the memory
characteristics of Trie objects used by Ranger policy-engine.
<property-prefix>.policyengine.option.optimize.trie.for.space - if true, Trie
objects for searching policies are optimized for space
<property-prefix>.policyengine.option.optimize.tag.trie.for.space - if true,
Trie objects for searching tagged resources are optimized for space.
where <property-prefix> is "ranger.admin" - for policy engines used within
Ranger Admin server, and it is "ranger.plugin.<service-type>" for policy
engines used in the plug-in.
Diffs
-----
agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/RangerTagEnricher.java
c8346d369
agents-common/src/main/java/org/apache/ranger/plugin/policyengine/RangerPolicyEngineOptions.java
07d0a3917
agents-common/src/main/java/org/apache/ranger/plugin/policyengine/RangerPolicyRepository.java
008ee7719
agents-common/src/main/java/org/apache/ranger/plugin/policyengine/RangerResourceTrie.java
7c37e05f9
Diff: https://reviews.apache.org/r/73715/diff/1/
Testing
-------
- Ran all unit tests successfully.
- Tested with a large (100K or more) number of Hive tagged resources, and
verified that the heap size requirement is substantially reduced when
"ranger.plugin.hive.policyengine.option.optimize.tag.trie.for.space" is set to
true for HiveServer2 service.
Thanks,
Abhay Kulkarni
