-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73715/
-----------------------------------------------------------
(Updated Nov. 20, 2021, 3:55 a.m.)
Review request for ranger, Madhan Neethiraj, Pradeep Agrawal, Ramesh Mani,
Sailaja Polavarapu, and Velmurugan Periasamy.
Changes
-------
More updates to test tools
Bugs: RANGER-3519
https://issues.apache.org/jira/browse/RANGER-3519
Repository: ranger
Description
-------
When the number of policies (and/or tagged resources) is large, the data
structures used by Ranger as indexes for policies (and/or tagged resources) may
need a very large heap memory because they are optimized for fast lookup. It is
desirable to be able to configure Ranger to have these structures optimized for
space in order to keep the heap requirements within acceptable limit at the
cost of somewhat slower lookup.
---
There are two new configuration variables that control the memory
characteristics of Trie objects used by Ranger policy-engine.
<property-prefix>.policyengine.option.optimize.trie.for.space - if true, Trie
objects for searching policies are optimized for space
<property-prefix>.policyengine.option.optimize.tag.trie.for.space - if true,
Trie objects for searching tagged resources are optimized for space.
where <property-prefix> is "ranger.admin" - for policy engines used within
Ranger Admin server, and it is "ranger.plugin.<service-type>" for policy
engines used in the plug-in.
Diffs (updated)
-----
agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/RangerTagEnricher.java
c8346d369
agents-common/src/main/java/org/apache/ranger/plugin/policyengine/RangerPolicyEngineOptions.java
07d0a3917
agents-common/src/main/java/org/apache/ranger/plugin/policyengine/RangerPolicyRepository.java
008ee7719
agents-common/src/main/java/org/apache/ranger/plugin/policyengine/RangerResourceTrie.java
7c37e05f9
distro/src/main/assembly/ranger-tools.xml 3ae890c1e
ranger-tools/scripts/create_requests.py PRE-CREATION
ranger-tools/src/main/java/org/apache/ranger/policyengine/PerfTestConfiguration.java
PRE-CREATION
ranger-tools/src/main/java/org/apache/ranger/policyengine/PerfTestEngine.java
37ac9ea74
ranger-tools/src/main/java/org/apache/ranger/policyengine/PerfTestOptions.java
0379fe6a6
ranger-tools/src/main/java/org/apache/ranger/policyengine/RangerPolicyenginePerfTester.java
760ebb9ee
ranger-tools/src/test/resources/testdata/ranger-config.xml 933d6705a
ranger-tools/src/test/resources/testdata/test_requests_hive.json 0db7207ab
ranger-tools/testdata/ranger-config.xml 933d6705a
Diff: https://reviews.apache.org/r/73715/diff/3/
Changes: https://reviews.apache.org/r/73715/diff/2-3/
Testing
-------
- Ran all unit tests successfully.
- Tested with a large (100K or more) number of Hive tagged resources, and
verified that the heap size requirement is substantially reduced when
"ranger.plugin.hive.policyengine.option.optimize.tag.trie.for.space" is set to
true for HiveServer2 service.
Thanks,
Abhay Kulkarni
