----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/74175/#review224809 -----------------------------------------------------------
agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/RangerAdminTagRetriever.java Lines 37 (patched) <https://reviews.apache.org/r/74175/#comment313639> Is there a rough estimate of how much de-duping of tags save on the plugin (and admin) side? agents-common/src/main/java/org/apache/ranger/plugin/util/RangerCommonConstants.java Line 53 (original), 53 (patched) <https://reviews.apache.org/r/74175/#comment313640> How is this constant used? There is another constant in RangerAdminTagRetriever which seems to control de-duping as well. agents-common/src/main/java/org/apache/ranger/plugin/util/RangerServiceTagsDeltaUtil.java Line 64 (original), 65 (patched) <https://reviews.apache.org/r/74175/#comment313638> Please evaluate if this shallow copy can be done only in the Ranger admin process. In the plugin code, 1. making a copy may not be necessary, and 2. if 1 is correct, then copying will incur excessive overhead. (It is assumed that the time taken to process service-tags in the plugin is much less than the tag download interval). agents-common/src/main/java/org/apache/ranger/plugin/util/RangerServiceTagsDeltaUtil.java Line 253 (original), 269 (patched) <https://reviews.apache.org/r/74175/#comment313641> It may be useful to keep GUIDs of tag-definitions, tags and service-resources around. If the exported (or downloaded) serviceTags are to be used to import them into another (parallel - for DR use-case) system, then having same GUIDs of same objects is desirable. - Abhay Kulkarni On Oct. 17, 2022, 11:42 p.m., Madhan Neethiraj wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/74175/ > ----------------------------------------------------------- > > (Updated Oct. 17, 2022, 11:42 p.m.) > > > Review request for ranger, Ankita Sinha, Kishor Gollapalliwar, Abhay > Kulkarni, Mehul Parikh, Pradeep Agrawal, Ramesh Mani, Sailaja Polavarapu, > Subhrat Chaudhary, and Velmurugan Periasamy. > > > Bugs: RANGER-3951 > https://issues.apache.org/jira/browse/RANGER-3951 > > > Repository: ranger > > > Description > ------- > > - skipped initialization of collection members (Map, List) with new > collection instance; these members will be set to null by default > - skipped serializing empty collections, this helps to reduce size of JSON > downloaded to plugins > - fixed potential concurrent modification to ServiceTags instance at Ranger > admin > - updated tag-retriever to deDup tags, so that the plugins will have reduced > memory usage irrespective of the optimization at the server side > - updated server to dedup tags by default i.e. default value for > configuration ranger.admin.supports.tags.dedup is changed from false to true > > > Diffs > ----- > > > agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/RangerAdminTagRetriever.java > 1df5587cb > > agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/RangerTagForEval.java > 1c634b393 > > agents-common/src/main/java/org/apache/ranger/plugin/model/RangerServiceResource.java > 47d006048 > agents-common/src/main/java/org/apache/ranger/plugin/model/RangerTag.java > 9e620c32c > > agents-common/src/main/java/org/apache/ranger/plugin/model/RangerTagDef.java > f1a83cfaf > > agents-common/src/main/java/org/apache/ranger/plugin/policyengine/RangerPolicyRepository.java > ff83c7434 > > agents-common/src/main/java/org/apache/ranger/plugin/util/RangerCommonConstants.java > 86f881f80 > > agents-common/src/main/java/org/apache/ranger/plugin/util/RangerServiceTagsDeltaUtil.java > 0365fc992 > agents-common/src/main/java/org/apache/ranger/plugin/util/ServiceTags.java > b4f250038 > > security-admin/src/main/java/org/apache/ranger/biz/RangerTagDBRetriever.java > c6b8ab937 > security-admin/src/main/java/org/apache/ranger/biz/TagDBStore.java > 4d3a56911 > > security-admin/src/main/java/org/apache/ranger/common/RangerServiceTagsCache.java > 576546f29 > > security-admin/src/main/java/org/apache/ranger/patch/PatchForUpdatingTagsJson_J10020.java > 301d5c30f > > security-admin/src/main/java/org/apache/ranger/rest/ServiceTagsProcessor.java > 1d6c48a4e > > > Diff: https://reviews.apache.org/r/74175/diff/1/ > > > Testing > ------- > > - verified that all tests pass successfully > - in an environment with 1m resources and ~3m tags, observed following > improvements: > -- 45% reduction in JSON file downloaded to plugins > -- 33% reduction in memory required for tags in plugins and Ranger admin > > > Thanks, > > Madhan Neethiraj > >
