-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/75194/
-----------------------------------------------------------
(Updated Aug. 29, 2024, 9:54 p.m.)
Review request for ranger, Abhishek Kumar, Fateh Singh, madhan, Madhan
Neethiraj, Mehul Parikh, Pradeep Agrawal, Ramesh Mani, Sailaja Polavarapu,
Velmurugan Periasamy, and Vyom Tiwari.
Changes
-------
Updated with Apache JIRA details
Summary (updated)
-----------------
RANGER-4922: Reduce time to find tags associated with multi-level resource
Bugs: RANGER-4922
https://issues.apache.org/jira/browse/RANGER-4922
Repository: ranger
Description (updated)
-------
With the following use case:
Service supports resource hierarchy with more than one level
Large number of tags are associated with the resources, with majority of tagged
resources with values for all levels in resource hierarchy
Accessed resource does not have values for all levels in the resource hierarchy
the time required to find the tags associated with the accessed resource is
significant.
When tested with a large number of tagged Ozone resources (~ 629,000) with
approximately 20 tagged volumes and 103 tagged buckets and the rest being keys,
the access evaluation times are:
(volume, bucket, key) : requestCount=629118, avgTimeTaken=49911ns
(volume, bucket) : requestCount=103, avgTimeTaken=10738069ns
(volume) :
- requestCount=20, avgTimeTaken=21968890ns
- requestCount=1056, avgTimeTaken=13763978ns (repeated requests in previous run
multiple times)
This patch, using filtering and caching technique attempts to reduce this time.
Diffs
-----
agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/RangerServiceResourceMatcher.java
e696db518
agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/RangerTagEnricher.java
a8fbc0215
agents-common/src/main/java/org/apache/ranger/plugin/model/validation/RangerZoneResourceMatcher.java
1a8a867a0
agents-common/src/main/java/org/apache/ranger/plugin/policyengine/RangerResourceTrie.java
abfc8f0aa
agents-common/src/main/java/org/apache/ranger/plugin/policyengine/gds/GdsSharedResourceEvaluator.java
c2773c9e0
agents-common/src/main/java/org/apache/ranger/plugin/policyevaluator/RangerAbstractPolicyEvaluator.java
549dc8f5a
agents-common/src/main/java/org/apache/ranger/plugin/policyresourcematcher/RangerResourceEvaluator.java
014bdd528
agents-common/src/main/java/org/apache/ranger/plugin/util/RangerRESTClient.java
e5461c2e6
agents-common/src/main/java/org/apache/ranger/plugin/util/RangerResourceEvaluatorsRetriever.java
69cb4417d
agents-common/src/main/java/org/apache/ranger/plugin/util/ServiceDefUtil.java
ed1ab5bda
agents-common/src/test/java/org/apache/ranger/plugin/contextenricher/TestTagEnricher.java
86985177a
agents-common/src/test/java/org/apache/ranger/plugin/policyengine/TestPathResourceTrie.java
d23509f86
distro/src/main/assembly/ranger-tools.xml 6893366d3
ranger-tools/conf/logback-mem-sizing.xml c3b601451
ranger-tools/pom.xml 440f3711e
Diff: https://reviews.apache.org/r/75194/diff/2/
Testing (updated)
-------
-- Passed all unit tests
-- Tested with the same data as before:
Average time required for accessed resource containing
(volume, bucket, key) : requestCount=629118, avgTimeTaken=39443ns
(volume, bucket) : requestCount=103, avgTimeTaken=14044457ns
(volume) :
- requestCount=20, avgTimeTaken=5569833ns
- requestCount=1056, avgTimeTaken=383638ns (repeated requests in previous run
multiple times)
Thanks,
Abhay Kulkarni
