----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/71236/#review217065 -----------------------------------------------------------
security-admin/src/main/java/org/apache/ranger/biz/ServiceDBStore.java Line 2126 (original) <https://reviews.apache.org/r/71236/#comment304314> Reverting changes done in RANGER-2493 (to fix bulk delete policy issue) RR link: https://reviews.apache.org/r/71007/ - Pradeep Agrawal On Aug. 4, 2019, 8:14 a.m., Pradeep Agrawal wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/71236/ > ----------------------------------------------------------- > > (Updated Aug. 4, 2019, 8:14 a.m.) > > > Review request for ranger, Ankita Sinha, bhavik patel, Gautam Borad, Abhay > Kulkarni, Madhan Neethiraj, Mehul Parikh, Nikhil P, Nitin Galave, Ramesh > Mani, Sailaja Polavarapu, and Velmurugan Periasamy. > > > Bugs: RANGER-2499 > https://issues.apache.org/jira/browse/RANGER-2499 > > > Repository: ranger > > > Description > ------- > > **Problem Statement:** Ranger takes an extremely long time to import bulk > policies. > **Current performance(Without patch):** > Time taken to import 500 policies : 27+ Minutes > Time taken to import 1000 policies : 81+ Minutes > > **Proposed Solution:** > in the bulk policy creation/deletion operation proposed patch shall call > EntityManager.flush() and EntityManager.clear() after processing a batch of > 10 records. > > **References: ** > https://en.wikibooks.org/wiki/Java_Persistence/Persisting > https://www.developerfusion.com/article/84945/flush-and-clear-or-mapping-antipatterns/ > > > Diffs > ----- > > > agents-common/src/main/java/org/apache/ranger/plugin/store/ServiceStore.java > d4879766f > security-admin/src/main/java/org/apache/ranger/biz/PolicyRefUpdater.java > e242d9092 > > security-admin/src/main/java/org/apache/ranger/biz/RangerPolicyRetriever.java > 4815f5bdb > security-admin/src/main/java/org/apache/ranger/biz/ServiceDBStore.java > ef22354de > security-admin/src/main/java/org/apache/ranger/biz/XUserMgr.java 037c591e8 > security-admin/src/main/java/org/apache/ranger/common/db/BaseDao.java > 4f339087d > > security-admin/src/main/java/org/apache/ranger/patch/PatchForNifiResourceUpdateExclude_J10011.java > 1c48eebcb > > security-admin/src/main/java/org/apache/ranger/patch/PatchMigration_J10002.java > 0af7a1da2 > security-admin/src/main/java/org/apache/ranger/rest/AssetREST.java > 66a57289f > security-admin/src/main/java/org/apache/ranger/rest/PublicAPIs.java > 7df957973 > security-admin/src/main/java/org/apache/ranger/rest/PublicAPIsv2.java > 2a4c53b48 > security-admin/src/main/java/org/apache/ranger/rest/ServiceREST.java > 348d072e6 > > security-admin/src/main/java/org/apache/ranger/service/RangerBaseModelService.java > de5d34307 > > security-admin/src/main/java/org/apache/ranger/service/RangerPolicyLabelsService.java > 1a1b56e4c > security-admin/src/test/java/org/apache/ranger/biz/TestServiceDBStore.java > b6f13f47e > security-admin/src/test/java/org/apache/ranger/rest/TestAssetREST.java > dce3b0b69 > security-admin/src/test/java/org/apache/ranger/rest/TestPublicAPIs.java > 2bf5ee6c9 > security-admin/src/test/java/org/apache/ranger/rest/TestPublicAPIsv2.java > f9ea26a31 > security-admin/src/test/java/org/apache/ranger/rest/TestServiceREST.java > a7e19bf03 > > > Diff: https://reviews.apache.org/r/71236/diff/1/ > > > Testing > ------- > > **Performance(With patch):** > > Time taken to import 500 policies : 84 Seconds > Time taken to import 1000 policies : 111 Seconds > > Time taken to delete 500 policies : 30 Seconds > > > Thanks, > > Pradeep Agrawal > >
