[jira] [Commented] (RANGER-1729) The hbase's performance will be significantly reduced after used Ranger to control and configure Hbase data security
[ https://issues.apache.org/jira/browse/RANGER-1729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16141302#comment-16141302 ] Madhan Neethiraj commented on RANGER-1729: -- Adding my comments from the review board here, for easier reference: Cache key such as this is likely to perform incorrect authorization, in multiple cases: - consider "table=t1; columnFamily=c1" and "table="t1c; columnFamily=1". Both will result in the same cacheKey, and can cause incorrect authorization - Ranger allows conditions to be included in policies, which can determine the result based on factors not included in the cacheKey - for example, time-of-access Such conditions may not be honored with cacheing of results - With support for tag-based authorization, either cache-key should include tags associated or the cache needs to be invalidated when the plugin receives updated tag information HBases table can have large number of columns (in millions); this (and other factors like number of users, number of ip-addresses) can increase the memory footprint and can cause significant overhead. I would suggest you do performance runs on conditions that can add significant cache size and share your findings. Overall, I think it will be helpful to look into the cause for the 10% overhead you notice and try to improve policy execution, where feasible, instead of going for caching the results. > The hbase's performance will be significantly reduced after used Ranger to > control and configure Hbase data security > > > Key: RANGER-1729 > URL: https://issues.apache.org/jira/browse/RANGER-1729 > Project: Ranger > Issue Type: Bug > Components: Ranger >Reporter: Qiang Zhang >Assignee: Qiang Zhang > Labels: patch > Attachments: > 0001-RANGER-1729-The-performance-of-hbase-will-be-signifi.patch, > Polling_interval_result_detail.png, result_detail.png, > verify_patch_result_detail.png > > > The hbase's performance will be significantly reduced after used Ranger to > control and configure Hbase data security. I used following environment to > verify issue. > Test tools：ycsb-0.1.4 > Test environment： > Node number：4 > Node configuration detail is as following: > Node1、Node2: > CPU:32 core，Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz > memory：128GB > Node3、Node4 > CPU:48 core，Intel(R) Xeon(R) CPU E5-2670 v3 @ 2.30GHz > memory：128GB > The test result: > 2. Add one policy and not write audit log, HBase's performance decreased > 10.10%; > 3. Add one policy and write audit log, HBase's performance decreased 12.90%; > Please refer to result_detail.png. > This decline is unacceptable in a real environment. It will seriously affect > the user to use Ranger. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (RANGER-1729) The hbase's performance will be significantly reduced after used Ranger to control and configure Hbase data security
[ https://issues.apache.org/jira/browse/RANGER-1729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16126814#comment-16126814 ] Qiang Zhang commented on RANGER-1729: - Hi [~bosco], You are justified, but the results are unacceptable in real big project, because the performance decline is too large, more than 10%. I'm trying to optimize this performance. If there is a problem, I will talk with you further. > The hbase's performance will be significantly reduced after used Ranger to > control and configure Hbase data security > > > Key: RANGER-1729 > URL: https://issues.apache.org/jira/browse/RANGER-1729 > Project: Ranger > Issue Type: Bug > Components: Ranger >Reporter: Qiang Zhang >Assignee: Qiang Zhang > Labels: patch > Attachments: Polling_interval_result_detail.png, result_detail.png > > > The hbase's performance will be significantly reduced after used Ranger to > control and configure Hbase data security. I used following environment to > verify issue. > Test tools：ycsb-0.1.4 > Test environment： > Node number：4 > Node configuration detail is as following: > Node1、Node2: > CPU:32 core，Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz > memory：128GB > Node3、Node4 > CPU:48 core，Intel(R) Xeon(R) CPU E5-2670 v3 @ 2.30GHz > memory：128GB > The test result: > 2. Add one policy and not write audit log, HBase's performance decreased > 10.10%; > 3. Add one policy and write audit log, HBase's performance decreased 12.90%; > Please refer to result_detail.png. > This decline is unacceptable in a real environment. It will seriously affect > the user to use Ranger. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (RANGER-1729) The hbase's performance will be significantly reduced after used Ranger to control and configure Hbase data security
[ https://issues.apache.org/jira/browse/RANGER-1729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16126704#comment-16126704 ] Don Bosco Durai commented on RANGER-1729: - Hi [~zhangqiang2], when you are doing benchmarking with HBase there are few things you need to be aware of: 1. When you do load/performance testing (without Ranger), HBase will use all available CPU 2. HBase also does a lot of IO (for put operation) 3. When Ranger is enabled, a slice of processing is consumed by Ranger. But since every operation calls Ranger, there is a visible performance difference. 4. Considering enabling Audit is only affected the performance by 2.8% is pretty good. Consider that every read in HBase that could translate to write (for audit). Since HBase is super optimized for READ, audit can never outperform HBase itself. However, Ranger does optimization by consolidating/summarizing the audits and minimizing writes. I would suggest few things: 1. Come up with what would be an acceptable overhead for security. In the case of HBase and Kafka, it will be never free, because these service are super optimized for performance. So you need to be reasonable here. 2. Enable native HBase access control and see what is the difference wrt when Ranger is enabled 3. Modify the Ranger code to return "true" when the co-processor is called. This will help in understanding what are the overhead of co-processor and other non-Ranger code. Once we have this number, we can look into Ranger code. 4. Increase the time interval for consolidating/batch Ranger Audits. The default is 5 seconds. You can try changing this property in HBase xasecure.audit.provider.summary.interval.ms to 3 > The hbase's performance will be significantly reduced after used Ranger to > control and configure Hbase data security > > > Key: RANGER-1729 > URL: https://issues.apache.org/jira/browse/RANGER-1729 > Project: Ranger > Issue Type: Bug > Components: Ranger >Reporter: Qiang Zhang >Assignee: Qiang Zhang > Labels: patch > Attachments: Polling_interval_result_detail.png, result_detail.png > > > The hbase's performance will be significantly reduced after used Ranger to > control and configure Hbase data security. I used following environment to > verify issue. > Test tools：ycsb-0.1.4 > Test environment： > Node number：4 > Node configuration detail is as following: > Node1、Node2: > CPU:32 core，Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz > memory：128GB > Node3、Node4 > CPU:48 core，Intel(R) Xeon(R) CPU E5-2670 v3 @ 2.30GHz > memory：128GB > The test result: > 2. Add one policy and not write audit log, HBase's performance decreased > 10.10%; > 3. Add one policy and write audit log, HBase's performance decreased 12.90%; > Please refer to result_detail.png. > This decline is unacceptable in a real environment. It will seriously affect > the user to use Ranger. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (RANGER-1729) The hbase's performance will be significantly reduced after used Ranger to control and configure Hbase data security
[ https://issues.apache.org/jira/browse/RANGER-1729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16122937#comment-16122937 ] Qiang Zhang commented on RANGER-1729: - Hi [~coheigea], I have tested the case. The result shows that the change for Polling Interval has little effect on the hbase's performance. 1% of the change can be seen as the impact of the environment. Please refer to Polling_interval_result_detail.png > The hbase's performance will be significantly reduced after used Ranger to > control and configure Hbase data security > > > Key: RANGER-1729 > URL: https://issues.apache.org/jira/browse/RANGER-1729 > Project: Ranger > Issue Type: Bug > Components: Ranger >Reporter: Qiang Zhang >Assignee: Qiang Zhang > Labels: patch > Attachments: Polling_interval_result_detail.png, result_detail.png > > > The hbase's performance will be significantly reduced after used Ranger to > control and configure Hbase data security. I used following environment to > verify issue. > Test tools：ycsb-0.1.4 > Test environment： > Node number：4节点 > Node configuration detail is as following: > Node1、Node2: > CPU:32 core，Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz > memory：128GB > Node3、Node4 > CPU:48 core，Intel(R) Xeon(R) CPU E5-2670 v3 @ 2.30GHz > memory：128GB > The test result: > 2. Add one policy and not write audit log, HBase's performance decreased > 10.10%; > 3. Add one policy and write audit log, HBase's performance decreased 12.90%; > Please refer to result_detail.png. > This decline is unacceptable in a real environment. It will seriously affect > the user to use Ranger. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (RANGER-1729) The hbase's performance will be significantly reduced after used Ranger to control and configure Hbase data security
[ https://issues.apache.org/jira/browse/RANGER-1729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16121474#comment-16121474 ] Qiang Zhang commented on RANGER-1729: - Hi [~coheigea], Ok. I will test this case. > The hbase's performance will be significantly reduced after used Ranger to > control and configure Hbase data security > > > Key: RANGER-1729 > URL: https://issues.apache.org/jira/browse/RANGER-1729 > Project: Ranger > Issue Type: Bug > Components: Ranger >Reporter: Qiang Zhang >Assignee: Qiang Zhang > Labels: patch > Attachments: result_detail.png > > > The hbase's performance will be significantly reduced after used Ranger to > control and configure Hbase data security. I used following environment to > verify issue. > Test tools：ycsb-0.1.4 > Test environment： > Node number：4节点 > Node configuration detail is as following: > Node1、Node2: > CPU:32 core，Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz > memory：128GB > Node3、Node4 > CPU:48 core，Intel(R) Xeon(R) CPU E5-2670 v3 @ 2.30GHz > memory：128GB > The test result: > 2. Add one policy and not write audit log, HBase's performance decreased > 10.10%; > 3. Add one policy and write audit log, HBase's performance decreased 12.90%; > Please refer to result_detail.png. > This decline is unacceptable in a real environment. It will seriously affect > the user to use Ranger. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (RANGER-1729) The hbase's performance will be significantly reduced after used Ranger to control and configure Hbase data security
[ https://issues.apache.org/jira/browse/RANGER-1729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16121441#comment-16121441 ] Colm O hEigeartaigh commented on RANGER-1729: - I wonder how much the slowdown could have been caused by policy retrieval - if you change the default value for policy retrieval from 30 seconds to say 1 seconds, what would be the slowdown? > The hbase's performance will be significantly reduced after used Ranger to > control and configure Hbase data security > > > Key: RANGER-1729 > URL: https://issues.apache.org/jira/browse/RANGER-1729 > Project: Ranger > Issue Type: Bug > Components: Ranger >Reporter: Qiang Zhang >Assignee: Qiang Zhang > Labels: patch > Attachments: result_detail.png > > > The hbase's performance will be significantly reduced after used Ranger to > control and configure Hbase data security. I used following environment to > verify issue. > Test tools：ycsb-0.1.4 > Test environment： > Node number：4节点 > Node configuration detail is as following: > Node1、Node2: > CPU:32 core，Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz > memory：128GB > Node3、Node4 > CPU:48 core，Intel(R) Xeon(R) CPU E5-2670 v3 @ 2.30GHz > memory：128GB > The test result: > 2. Add one policy and not write audit log, HBase's performance decreased > 10.10%; > 3. Add one policy and write audit log, HBase's performance decreased 12.90%; > Please refer to result_detail.png. > This decline is unacceptable in a real environment. It will seriously affect > the user to use Ranger. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
