> On 八月 25, 2017, 6:31 a.m., Madhan Neethiraj wrote: > > hbase-agent/src/main/java/org/apache/ranger/authorization/hbase/RangerAuthorizationCoprocessor.java > > Line 333 (original), 337 (patched) > > <https://reviews.apache.org/r/61843/diff/1/?file=1802180#file1802180line349> > > > > Cache key such as this is likely to perform incorrect authorization, in > > multiple cases: > > - consider "table=t1; columnFamily=c1" and "table="t1c; > > columnFamily=1". But will result in the same cacheKey - and can cause > > incorrect authorization > > - Ranger allows policy conditions to be included in policies, which > > can determine the result based on factors not included in the cacheKey - > > for example, time-of-access. Such conditions will not be honored with use > > of cache model such as this > > - HBases table can have large number of columns (in millions); this > > (and other factors like number of user, number of ip-addresses) can > > increase the memory footprint and can cause significant overhead. I would > > suggest you do performance runs on conditions that can add significant > > cache size and share your findings. > > - With support for tag-based authorization, the cache needs to be > > invalidated when the plugin receives updated tag information. You would > > need to look into this as well. > > > > > > Overall, I think it will be helpful to look into the cause for the 10% > > overhead you notice and try to improve policy execution, where feasible, > > instead of going for caching the results. > > Madhan Neethiraj wrote: > Few typos in my earlier comment; I apologize. Here is the corrected > version: > > Cache key such as this is likely to perform incorrect authorization, in > multiple cases: > - consider "table=t1; columnFamily=c1" and "table="t1c; columnFamily=1". > Both will result in the same cacheKey, and can cause incorrect authorization > - Ranger allows conditions to be included in policies, which can > determine the result based on factors not included in the cacheKey - for > example, time-of-access. Such conditions may not be honored with cacheing of > results > - With support for tag-based authorization, either cache-key should > include tags associated or the cache needs to be invalidated when the plugin > receives updated tag information > > HBases table can have large number of columns (in millions); this (and > other factors like number of users, number of ip-addresses) can increase the > memory footprint and can cause significant overhead. I would suggest you do > performance runs on conditions that can add significant cache size and share > your findings. > > > Overall, I think it will be helpful to look into the cause for the 10% > overhead you notice and try to improve policy execution, where feasible, > instead of going for caching the results.
Ok. Thanks for your valuable advice. I will further analyze the issue. - Qiang ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/61843/#review183824 ----------------------------------------------------------- On 八月 23, 2017, 6:39 p.m., Qiang Zhang wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/61843/ > ----------------------------------------------------------- > > (Updated 八月 23, 2017, 6:39 p.m.) > > > Review request for ranger, Alok Lal, Ankita Sinha, Don Bosco Durai, Colm O > hEigeartaigh, Gautam Borad, Madhan Neethiraj, Ramesh Mani, Selvamohan > Neethiraj, and Velmurugan Periasamy. > > > Bugs: RANGER-1729 > https://issues.apache.org/jira/browse/RANGER-1729 > > > Repository: ranger > > > Description > ------- > > The hbase's performance will be significantly reduced after used Ranger to > control and configure Hbase data security. I used following environment to > verify issue. > Test tools: ycsb-0.1.4 > Test environment: > Node number: 4 > Node configuration detail is as following: > Node1,Node2: > CPU:32 core?Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz > memory: 128GB > > Node3,Node4 > CPU:48 core: Intel(R) Xeon(R) CPU E5-2670 v3 @ 2.30GHz > memory: 128GB > > The test result: > 2. Add one policy and not write audit log, HBase's performance decreased > 10.10%; > 3. Add one policy and write audit log, HBase's performance decreased 12.90%; > Please refer to result_detail.png. > > This decline is unacceptable in a real environment. It will seriously affect > the user to use Ranger. > > The idea of improvement is as follows: > 1. The enhancement doesn't effect the previous function. > 2. Using columns store to save frequently used access, and add frequently > access rights to the secondary cache. > 3. Synchronize the secondary cache when the policy changes > > The patch doesn't effect the previous function. But it significantly improves > the performance of the hbase's performance. The result is as following: > 1. Add one policy and not write audit log. The HBase's performance decreased > 4.18% when the user set supper user before used the patch. In the same > configuration case the HBase's performance decreased -3.20% after used the > patch; > 1. Add one policy and not write audit log. The HBase's performance decreased > 10.10% when the user set common user before used the patch. In the same > configuration case the HBase's performance decreased 2.74% after used the > patch; > Please refer to verify_patch_result_detail.png. > > > Diffs > ----- > > > agents-common/src/main/java/org/apache/ranger/plugin/service/RangerBasePlugin.java > 4d3731b > > agents-common/src/main/java/org/apache/ranger/plugin/util/PolicyRefresher.java > b5b4f16 > > hbase-agent/src/main/java/org/apache/ranger/authorization/hbase/HbaseAccessCache.java > PRE-CREATION > > hbase-agent/src/main/java/org/apache/ranger/authorization/hbase/RangerAuthorizationCoprocessor.java > da73c67 > > > Diff: https://reviews.apache.org/r/61843/diff/1/ > > > Testing > ------- > > > Thanks, > > Qiang Zhang > >
