----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/61843/#review183824 -----------------------------------------------------------
hbase-agent/src/main/java/org/apache/ranger/authorization/hbase/RangerAuthorizationCoprocessor.java Line 333 (original), 337 (patched) <https://reviews.apache.org/r/61843/#comment259893> Cache key such as this is likely to perform incorrect authorization, in multiple cases: - consider "table=t1; columnFamily=c1" and "table="t1c; columnFamily=1". But will result in the same cacheKey - and can cause incorrect authorization - Ranger allows policy conditions to be included in policies, which can determine the result based on factors not included in the cacheKey - for example, time-of-access. Such conditions will not be honored with use of cache model such as this - HBases table can have large number of columns (in millions); this (and other factors like number of user, number of ip-addresses) can increase the memory footprint and can cause significant overhead. I would suggest you do performance runs on conditions that can add significant cache size and share your findings. - With support for tag-based authorization, the cache needs to be invalidated when the plugin receives updated tag information. You would need to look into this as well. Overall, I think it will be helpful to look into the cause for the 10% overhead you notice and try to improve policy execution, where feasible, instead of going for caching the results. - Madhan Neethiraj On Aug. 23, 2017, 6:39 p.m., Qiang Zhang wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/61843/ > ----------------------------------------------------------- > > (Updated Aug. 23, 2017, 6:39 p.m.) > > > Review request for ranger, Alok Lal, Ankita Sinha, Don Bosco Durai, Colm O > hEigeartaigh, Gautam Borad, Madhan Neethiraj, Ramesh Mani, Selvamohan > Neethiraj, and Velmurugan Periasamy. > > > Bugs: RANGER-1729 > https://issues.apache.org/jira/browse/RANGER-1729 > > > Repository: ranger > > > Description > ------- > > The hbase's performance will be significantly reduced after used Ranger to > control and configure Hbase data security. I used following environment to > verify issue. > Test tools: ycsb-0.1.4 > Test environment: > Node number: 4 > Node configuration detail is as following: > Node1,Node2: > CPU:32 core?Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz > memory: 128GB > > Node3,Node4 > CPU:48 core: Intel(R) Xeon(R) CPU E5-2670 v3 @ 2.30GHz > memory: 128GB > > The test result: > 2. Add one policy and not write audit log, HBase's performance decreased > 10.10%; > 3. Add one policy and write audit log, HBase's performance decreased 12.90%; > Please refer to result_detail.png. > > This decline is unacceptable in a real environment. It will seriously affect > the user to use Ranger. > > The idea of improvement is as follows: > 1. The enhancement doesn't effect the previous function. > 2. Using columns store to save frequently used access, and add frequently > access rights to the secondary cache. > 3. Synchronize the secondary cache when the policy changes > > The patch doesn't effect the previous function. But it significantly improves > the performance of the hbase's performance. The result is as following: > 1. Add one policy and not write audit log. The HBase's performance decreased > 4.18% when the user set supper user before used the patch. In the same > configuration case the HBase's performance decreased -3.20% after used the > patch; > 1. Add one policy and not write audit log. The HBase's performance decreased > 10.10% when the user set common user before used the patch. In the same > configuration case the HBase's performance decreased 2.74% after used the > patch; > Please refer to verify_patch_result_detail.png. > > > Diffs > ----- > > > agents-common/src/main/java/org/apache/ranger/plugin/service/RangerBasePlugin.java > 4d3731b > > agents-common/src/main/java/org/apache/ranger/plugin/util/PolicyRefresher.java > b5b4f16 > > hbase-agent/src/main/java/org/apache/ranger/authorization/hbase/HbaseAccessCache.java > PRE-CREATION > > hbase-agent/src/main/java/org/apache/ranger/authorization/hbase/RangerAuthorizationCoprocessor.java > da73c67 > > > Diff: https://reviews.apache.org/r/61843/diff/1/ > > > Testing > ------- > > > Thanks, > > Qiang Zhang > >
