----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/73715/#review223754 -----------------------------------------------------------
Ship it! Ship It! - Madhan Neethiraj On Nov. 20, 2021, 3:55 a.m., Abhay Kulkarni wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/73715/ > ----------------------------------------------------------- > > (Updated Nov. 20, 2021, 3:55 a.m.) > > > Review request for ranger, Madhan Neethiraj, Pradeep Agrawal, Ramesh Mani, > Sailaja Polavarapu, and Velmurugan Periasamy. > > > Bugs: RANGER-3519 > https://issues.apache.org/jira/browse/RANGER-3519 > > > Repository: ranger > > > Description > ------- > > When the number of policies (and/or tagged resources) is large, the data > structures used by Ranger as indexes for policies (and/or tagged resources) > may need a very large heap memory because they are optimized for fast lookup. > It is desirable to be able to configure Ranger to have these structures > optimized for space in order to keep the heap requirements within acceptable > limit at the cost of somewhat slower lookup. > > --- > > There are two new configuration variables that control the memory > characteristics of Trie objects used by Ranger policy-engine. > > <property-prefix>.policyengine.option.optimize.trie.for.space - if true, Trie > objects for searching policies are optimized for space > <property-prefix>.policyengine.option.optimize.tag.trie.for.space - if true, > Trie objects for searching tagged resources are optimized for space. > > where <property-prefix> is "ranger.admin" - for policy engines used within > Ranger Admin server, and it is "ranger.plugin.<service-type>" for policy > engines used in the plug-in. > > > Diffs > ----- > > > agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/RangerTagEnricher.java > c8346d369 > > agents-common/src/main/java/org/apache/ranger/plugin/policyengine/RangerPolicyEngineOptions.java > 07d0a3917 > > agents-common/src/main/java/org/apache/ranger/plugin/policyengine/RangerPolicyRepository.java > 008ee7719 > > agents-common/src/main/java/org/apache/ranger/plugin/policyengine/RangerResourceTrie.java > 7c37e05f9 > distro/src/main/assembly/ranger-tools.xml 3ae890c1e > ranger-tools/scripts/create_requests.py PRE-CREATION > > ranger-tools/src/main/java/org/apache/ranger/policyengine/PerfTestConfiguration.java > PRE-CREATION > > ranger-tools/src/main/java/org/apache/ranger/policyengine/PerfTestEngine.java > 37ac9ea74 > > ranger-tools/src/main/java/org/apache/ranger/policyengine/PerfTestOptions.java > 0379fe6a6 > > ranger-tools/src/main/java/org/apache/ranger/policyengine/RangerPolicyenginePerfTester.java > 760ebb9ee > ranger-tools/src/test/resources/testdata/ranger-config.xml 933d6705a > ranger-tools/src/test/resources/testdata/test_requests_hive.json 0db7207ab > ranger-tools/testdata/ranger-config.xml 933d6705a > > > Diff: https://reviews.apache.org/r/73715/diff/3/ > > > Testing > ------- > > - Ran all unit tests successfully. > - Tested with a large (100K or more) number of Hive tagged resources, and > verified that the heap size requirement is substantially reduced when > "ranger.plugin.hive.policyengine.option.optimize.tag.trie.for.space" is set > to true for HiveServer2 service. > > > Thanks, > > Abhay Kulkarni > >
