----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/74144/#review224719 -----------------------------------------------------------
agents-common/src/main/java/org/apache/ranger/plugin/util/RangerServiceTagsDeltaUtil.java Line 107 (original), 101 (patched) <https://reviews.apache.org/r/74144/#comment313502> This conversion will be done every time applyDeltas() is called. This may be expensive operation. Is it possible to cache this map in ServiceTags but ignore it when serializing ServiceTags object? agents-common/src/main/java/org/apache/ranger/plugin/util/RangerServiceTagsDeltaUtil.java Line 142 (original), 130 (patched) <https://reviews.apache.org/r/74144/#comment313503> If any resources are deleted or updated, this code iterates over all service-resources. Will this be expensive, as before? - Abhay Kulkarni On Sept. 28, 2022, 10:58 p.m., Madhan Neethiraj wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/74144/ > ----------------------------------------------------------- > > (Updated Sept. 28, 2022, 10:58 p.m.) > > > Review request for ranger, Ankita Sinha, Kishor Gollapalliwar, Abhay > Kulkarni, Mehul Parikh, Pradeep Agrawal, Ramesh Mani, Sailaja Polavarapu, > Subhrat Chaudhary, and Velmurugan Periasamy. > > > Bugs: RANGER-3934 > https://issues.apache.org/jira/browse/RANGER-3934 > > > Repository: ranger > > > Description > ------- > > - updated several retrieval of XXService JPA object by replacing with > retrival of serviceId > - replaced several instances of Map<Long, Long> with Set<Long>, as these > instances had same value for key and value > - avoided expensive calls to RangerTagResourceMapService.getByTagId(tagId) > and RangerTagResourceMapService.getTagIdsForResourceId(serviceResourceId) > - updated several methods in XXTagResourceMapDao to avoid loading of > XXTagResourceMap JPA object; instead created XXTagResourceMap object from > individial fields retrieved from query > > > Diffs > ----- > > > agents-common/src/main/java/org/apache/ranger/plugin/util/RangerServiceTagsDeltaUtil.java > 8d9241c1c > security-admin/src/main/java/org/apache/ranger/biz/TagDBStore.java > d8154b7de > > security-admin/src/main/java/org/apache/ranger/db/XXRMSServiceResourceDao.java > afa754ba2 > security-admin/src/main/java/org/apache/ranger/db/XXServiceDao.java > 3cc3d9cef > security-admin/src/main/java/org/apache/ranger/db/XXTagResourceMapDao.java > 3f8b5b718 > > security-admin/src/main/java/org/apache/ranger/patch/PatchForAtlasServiceDefUpdate_J10013.java > b0f71e138 > security-admin/src/main/java/org/apache/ranger/rest/TagREST.java c7cf3bfb8 > security-admin/src/main/resources/META-INF/jpa_named_queries.xml e4a2354b0 > > > Diff: https://reviews.apache.org/r/74144/diff/2/ > > > Testing > ------- > > - with Ranger database having ~1m service-resources, ~2m tags and delta of > 20k resource & 40k tags: > -- before these optmizations, tag-cache update from delta didn't complete > event after a long time (1h35m) > -- with these optmizations, tag-cache update from the same delta completed > within 10 minutes > - there is scope for further optimization > RangerServiceTagsDeltaUtil.applyDelta() - as this took almost 97% time in > updating the cache (588 seconds); compare this to > TagDBStore.getServiceTagsDelta() which completed in 13 seeconds > > > Thanks, > > Madhan Neethiraj > >
