> On May 6, 2024, 11:15 p.m., Abhay Kulkarni wrote: > > agents-common/src/main/java/org/apache/ranger/plugin/service/RangerBasePlugin.java > > Lines 96 (patched) > > <https://reviews.apache.org/r/74966/diff/5/?file=2287597#file2287597line96> > > > > The default value seems to be OptimalNullSafeSupplier. Should it be > > DefaultNullSafeSupplier?
I suggest to use OptimalNullSafeSupplier as the default. Any application that requires the current behavior, which would use more memory, can set the configuration to switch to current behavior. Also, to avoid potential confustion, I will rename DefaultNullSafeSupplier => NullSafeSupplierV1, OptimalNullSafeSupplier => NullSafeSupplierV2 - Madhan ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/74966/#review226417 ----------------------------------------------------------- On May 5, 2024, 8:43 a.m., Madhan Neethiraj wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/74966/ > ----------------------------------------------------------- > > (Updated May 5, 2024, 8:43 a.m.) > > > Review request for ranger, Abhishek Kumar, Asit Vadhavkar, Fateh Singh, > Kishor Gollapalliwar, Abhay Kulkarni, Pradeep Agrawal, Ramesh Mani, Sailaja > Polavarapu, Subhrat Chaudhary, and Velmurugan Periasamy. > > > Bugs: RANGER-4787 > https://issues.apache.org/jira/browse/RANGER-4787 > > > Repository: ranger > > > Description > ------- > > Model package classes like RangerPolicy initialize collection members with a > new collection object (ArrayList/HashMap/HashSet). Many of these members > would likely remain empty, resulting in the initialization to be unnecessary. > Instead of creating a new collection object, it will help to initialize with > Collections.emptyList()/emptySet()/emptyMap() - to save memory while > retaining non-null value for collection members. > > Also, set() on these members currently create a new collection object with a > copy of the value to be set. This can be avoided by using the given value. > > Updated RangerPolicy and RangerOptimizedPolicyEvaluator classes with above > optimizations, and updated all refereces to collection members to safely add > items to the collections. > > > Diffs > ----- > > > agents-common/src/main/java/org/apache/ranger/plugin/model/RangerBaseModelObject.java > e59ea2345 > > agents-common/src/main/java/org/apache/ranger/plugin/model/RangerPolicy.java > ec0618421 > > agents-common/src/main/java/org/apache/ranger/plugin/policyevaluator/RangerAbstractPolicyItemEvaluator.java > 2190ad281 > > agents-common/src/main/java/org/apache/ranger/plugin/policyevaluator/RangerDefaultPolicyEvaluator.java > 9745dc64f > > agents-common/src/main/java/org/apache/ranger/plugin/policyevaluator/RangerOptimizedPolicyEvaluator.java > 6d6169309 > > agents-common/src/main/java/org/apache/ranger/plugin/service/RangerBasePlugin.java > 97da473a8 > > agents-common/src/main/java/org/apache/ranger/plugin/service/RangerBaseService.java > eaee00b80 > > agents-common/src/test/java/org/apache/ranger/plugin/model/TestRangerPolicy.java > 57e710e61 > > agents-common/src/test/java/org/apache/ranger/plugin/util/ServiceDefUtilTest.java > 36f0b6af6 > > hbase-agent/src/main/java/org/apache/ranger/services/hbase/RangerServiceHBase.java > 962be1a55 > > hdfs-agent/src/main/java/org/apache/ranger/services/hdfs/RangerServiceHdfs.java > e6411b3b0 > > hive-agent/src/main/java/org/apache/ranger/services/hive/RangerServiceHive.java > a8d71772a > > knox-agent/src/main/java/org/apache/ranger/services/knox/RangerServiceKnox.java > eafd0e3fe > > plugin-atlas/src/main/java/org/apache/ranger/services/atlas/RangerServiceAtlas.java > fee2179b5 > > plugin-elasticsearch/src/main/java/org/apache/ranger/services/elasticsearch/RangerServiceElasticsearch.java > a8953e1e7 > > plugin-kafka/src/main/java/org/apache/ranger/services/kafka/RangerServiceKafka.java > f683e608e > > plugin-kms/src/main/java/org/apache/ranger/services/kms/RangerServiceKMS.java > d0c2e260a > > plugin-ozone/src/main/java/org/apache/ranger/services/ozone/RangerServiceOzone.java > 704412246 > > plugin-presto/src/main/java/org/apache/ranger/services/presto/RangerServicePresto.java > 9e773f160 > > plugin-solr/src/main/java/org/apache/ranger/services/solr/RangerServiceSolr.java > 3a8f9b462 > > plugin-trino/src/main/java/org/apache/ranger/services/trino/RangerServiceTrino.java > 936fb7ac8 > > plugin-yarn/src/main/java/org/apache/ranger/services/yarn/RangerServiceYarn.java > 48a63ff56 > ranger-tools/src/main/java/org/apache/ranger/sizing/PerfMemTimeTracker.java > 917c0da9c > > security-admin/src/main/java/org/apache/ranger/biz/RangerPolicyAdminImpl.java > 84ee31ba2 > security-admin/src/main/java/org/apache/ranger/biz/ServiceDBStore.java > cccc47fbe > security-admin/src/main/java/org/apache/ranger/biz/XUserMgr.java d202b2184 > security-admin/src/main/java/org/apache/ranger/common/ServiceUtil.java > 180124ca5 > > security-admin/src/main/java/org/apache/ranger/patch/PatchForAtlasToAddTypeRead_J10040.java > 31fb1b1d3 > > security-admin/src/main/java/org/apache/ranger/patch/PatchForHBaseDefaultPolicyUpdate_J10045.java > 302a26602 > > security-admin/src/main/java/org/apache/ranger/patch/PatchForOzoneDefaultPoliciesUpdate_J10044.java > d686be485 > > security-admin/src/main/java/org/apache/ranger/patch/PatchForOzoneServiceDefUpdate_J10041.java > 0a085f659 > > security-admin/src/main/java/org/apache/ranger/patch/PatchForUpdatingPolicyJson_J10019.java > e7d10883d > > security-admin/src/main/java/org/apache/ranger/patch/PatchForUpdatingTagsJson_J10020.java > c8516d2b7 > > security-admin/src/main/java/org/apache/ranger/patch/PatchMigration_J10002.java > c8cd92c7e > security-admin/src/main/java/org/apache/ranger/rest/ServiceREST.java > a6c759234 > security-admin/src/main/java/org/apache/ranger/rest/ServiceRESTUtil.java > 60e34c0c7 > security-admin/src/test/java/org/apache/ranger/biz/TestPolicyAdmin.java > 41c360dec > security-admin/src/test/java/org/apache/ranger/rest/TestServiceREST.java > ff5fe218a > > storm-agent/src/main/java/org/apache/ranger/services/storm/RangerServiceStorm.java > 23e71ba01 > > > Diff: https://reviews.apache.org/r/74966/diff/5/ > > > Testing > ------- > > - verified that all unit tests pass successfully > - built Ranger successfully > - using docker setup, built and started Ranger admin/usersync/tagsync/plugins > successfully > - Following memory savings are seen with creation of 1m policies each having > one resource, one policyItem(one-group, one-access): > -------------------------------------------------------------------------------- > bytes before bytes after > Class optimization optimization Savings > -------------------------------------------------------------------------------- > RangerPolicy 996,053,872 708,059,576 287,994,296 > (28.91%) > RangerOptimizedPolicyEvaluator 1,068,156,376 748,153,544 320,002,832 > (29.96%) > -------------------------------------------------------------------------------- > Total 2,064,210,248 1,456,213,120 607,997,128 > (29.45%) > > -------------------------------------------------------------------------------- > > > Thanks, > > Madhan Neethiraj > >
