[jira] [Commented] (RANGER-2149) REST search queries make Ranger incredibly slow
[ https://issues.apache.org/jira/browse/RANGER-2149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1654#comment-1654 ] Alexander Posledov commented on RANGER-2149: I'll check if it is a mysql 5.1 issue. Let me try the same with mysql 5.6, I'll get back with the results during the next week. > REST search queries make Ranger incredibly slow > > > Key: RANGER-2149 > URL: https://issues.apache.org/jira/browse/RANGER-2149 > Project: Ranger > Issue Type: Bug > Components: Ranger >Affects Versions: 0.7.0 >Reporter: Alexander Posledov >Assignee: Pradeep Agrawal >Priority: Major > Attachments: bulkpolicytest.png, totalPolicy.png > > > If amount of Ranger policies is relatively big (3000-4000), then REST search > queries (e.g. search policy by name) make Ranger very slow. Example: > curl -u admin:admin -X GET > http://host:6080/service/plugins/policies?policyName=somePolicyName > 2-3 such queries (if we have ~3k policies) can cause too long delays even for > simple requests, while these searches aren't completed. In some cases, when > we ran several search queries simultaneously, this Ranger behavior caused 9 > min delay for simple adding a new policy. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (RANGER-1935) Upgrade Ranger to support Apache Hadoop 3.0.0
[ https://issues.apache.org/jira/browse/RANGER-1935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16539735#comment-16539735 ] Colm O hEigeartaigh commented on RANGER-1935: - Yes, working on it at the moment. > Upgrade Ranger to support Apache Hadoop 3.0.0 > - > > Key: RANGER-1935 > URL: https://issues.apache.org/jira/browse/RANGER-1935 > Project: Ranger > Issue Type: Improvement > Components: plugins >Reporter: Colm O hEigeartaigh >Assignee: Colm O hEigeartaigh >Priority: Major > Fix For: 2.0.0 > > Attachments: > 0001-RANGER-1935-Upgrade-Ranger-to-support-Apache-Hadoop-.patch > > > This task is to upgrade Ranger to support Apache Hadoop 3.0.0. Here are some > notes about the upgrade: > a) The Hive plugin needs the Hadoop 3.0.0 jars to run the tests properly, as > Hive only supports the older Hadoop version, so an exclusion and some > additional 3.0.0 dependencies need to be added. > b) The Storm plugin bundles the hadoop-auth jars in storm-core (although they > really should be renamed here). Therefore, we have no option but to package > Storm with the Hadoop 2.7.x jars, until such time that Storm upgrades the > Hadoop dependency. > This is an initial patch to get some feedback. If there is broad agreement on > the upgrade I will test the distributions properly. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (RANGER-2149) REST search queries make Ranger incredibly slow
[ https://issues.apache.org/jira/browse/RANGER-2149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16539694#comment-16539694 ] Pradeep Agrawal commented on RANGER-2149: - [~aposledov] Ranger had total 8000 policies and that particular service had 4000 policies. I used the same REST that you have provided earlier in the post. Response time through UI and curl is almost same. > REST search queries make Ranger incredibly slow > > > Key: RANGER-2149 > URL: https://issues.apache.org/jira/browse/RANGER-2149 > Project: Ranger > Issue Type: Bug > Components: Ranger >Affects Versions: 0.7.0 >Reporter: Alexander Posledov >Assignee: Pradeep Agrawal >Priority: Major > Attachments: bulkpolicytest.png, totalPolicy.png > > > If amount of Ranger policies is relatively big (3000-4000), then REST search > queries (e.g. search policy by name) make Ranger very slow. Example: > curl -u admin:admin -X GET > http://host:6080/service/plugins/policies?policyName=somePolicyName > 2-3 such queries (if we have ~3k policies) can cause too long delays even for > simple requests, while these searches aren't completed. In some cases, when > we ran several search queries simultaneously, this Ranger behavior caused 9 > min delay for simple adding a new policy. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
