[jira] [Commented] (RANGER-2149) REST search queries make Ranger incredibly slow
[ https://issues.apache.org/jira/browse/RANGER-2149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16533282#comment-16533282 ] Don Bosco Durai commented on RANGER-2149: - [~pradeep.agrawal] do we have index in the DB table for policy name? In the DB world, 3000 rows is not a lot of data. Even full table scan should complete in seconds and not minutes. > REST search queries make Ranger incredibly slow > > > Key: RANGER-2149 > URL: https://issues.apache.org/jira/browse/RANGER-2149 > Project: Ranger > Issue Type: Bug > Components: Ranger >Affects Versions: 0.7.0 >Reporter: Alexander Posledov >Assignee: Pradeep Agrawal >Priority: Major > > If amount of Ranger policies is relatively big (3000-4000), then REST search > queries (e.g. search policy by name) make Ranger very slow. Example: > curl -u admin:admin -X GET > http://host:6080/service/plugins/policies?policyName=somePolicyName > 2-3 such queries (if we have ~3k policies) can cause too long delays even for > simple requests, while these searches aren't completed. In some cases, when > we ran several search queries simultaneously, this Ranger behavior caused 9 > min delay for simple adding a new policy. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (RANGER-2149) REST search queries make Ranger incredibly slow
[ https://issues.apache.org/jira/browse/RANGER-2149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16532675#comment-16532675 ] Alexander Posledov commented on RANGER-2149: DB: MySQL OS: CentOS 6 [root@qerpx02-vm1 log]# mysql --version mysql Ver 14.14 Distrib 5.1.73, for redhat-linux-gnu (x86_64) using readline 5.1 [root@qerpx02-vm1 log]# lscpu Architecture: x86_64 CPU op-mode(s): 32-bit, 64-bit Byte Order: Little Endian CPU(s): 6 On-line CPU(s) list: 0-5 Thread(s) per core: 1 Core(s) per socket: 1 Socket(s): 6 NUMA node(s): 1 Vendor ID: GenuineIntel CPU family: 6 Model: 94 Model name: Intel Core Processor (Skylake) Stepping: 3 CPU MHz: 2098.782 BogoMIPS: 4197.56 Hypervisor vendor: KVM Virtualization type: full L1d cache: 32K L1i cache: 32K L2 cache: 4096K NUMA node0 CPU(s): 0-5 > REST search queries make Ranger incredibly slow > > > Key: RANGER-2149 > URL: https://issues.apache.org/jira/browse/RANGER-2149 > Project: Ranger > Issue Type: Bug > Components: Ranger >Affects Versions: 0.7.0 >Reporter: Alexander Posledov >Assignee: Pradeep Agrawal >Priority: Major > > If amount of Ranger policies is relatively big (3000-4000), then REST search > queries (e.g. search policy by name) make Ranger very slow. Example: > curl -u admin:admin -X GET > http://host:6080/service/plugins/policies?policyName=somePolicyName > 2-3 such queries (if we have ~3k policies) can cause too long delays even for > simple requests, while these searches aren't completed. In some cases, when > we ran several search queries simultaneously, this Ranger behavior caused 9 > min delay for simple adding a new policy. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (RANGER-2149) REST search queries make Ranger incredibly slow
[ https://issues.apache.org/jira/browse/RANGER-2149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16532652#comment-16532652 ] Pradeep Agrawal commented on RANGER-2149: - [~aposledov] : Can you provide configuration of machine used. Please provide DB version details also. > REST search queries make Ranger incredibly slow > > > Key: RANGER-2149 > URL: https://issues.apache.org/jira/browse/RANGER-2149 > Project: Ranger > Issue Type: Bug > Components: Ranger >Affects Versions: 0.7.0 >Reporter: Alexander Posledov >Assignee: Pradeep Agrawal >Priority: Major > > If amount of Ranger policies is relatively big (3000-4000), then REST search > queries (e.g. search policy by name) make Ranger very slow. Example: > curl -u admin:admin -X GET > http://host:6080/service/plugins/policies?policyName=somePolicyName > 2-3 such queries (if we have ~3k policies) can cause too long delays even for > simple requests, while these searches aren't completed. In some cases, when > we ran several search queries simultaneously, this Ranger behavior caused 9 > min delay for simple adding a new policy. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (RANGER-2149) REST search queries make Ranger incredibly slow
[ https://issues.apache.org/jira/browse/RANGER-2149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pradeep Agrawal reassigned RANGER-2149: --- Assignee: Pradeep Agrawal > REST search queries make Ranger incredibly slow > > > Key: RANGER-2149 > URL: https://issues.apache.org/jira/browse/RANGER-2149 > Project: Ranger > Issue Type: Bug > Components: Ranger >Affects Versions: 0.7.0 >Reporter: Alexander Posledov >Assignee: Pradeep Agrawal >Priority: Major > > If amount of Ranger policies is relatively big (3000-4000), then REST search > queries (e.g. search policy by name) make Ranger very slow. Example: > curl -u admin:admin -X GET > http://host:6080/service/plugins/policies?policyName=somePolicyName > 2-3 such queries (if we have ~3k policies) can cause too long delays even for > simple requests, while these searches aren't completed. In some cases, when > we ran several search queries simultaneously, this Ranger behavior caused 9 > min delay for simple adding a new policy. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (RANGER-2149) REST search queries make Ranger incredibly slow
Alexander Posledov created RANGER-2149: -- Summary: REST search queries make Ranger incredibly slow Key: RANGER-2149 URL: https://issues.apache.org/jira/browse/RANGER-2149 Project: Ranger Issue Type: Bug Components: Ranger Affects Versions: 0.7.0 Reporter: Alexander Posledov If amount of Ranger policies is relatively big (3000-4000), then REST search queries (e.g. search policy by name) make Ranger very slow. Example: curl -u admin:admin -X GET http://host:6080/service/plugins/policies?policyName=somePolicyName 2-3 such queries (if we have ~3k policies) can cause too long delays even for simple requests, while these searches aren't completed. In some cases, when we ran several search queries simultaneously, this Ranger behavior caused 9 min delay for simple adding a new policy. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (RANGER-2148) Update Ranger Hive dependency version to 3.0
[ https://issues.apache.org/jira/browse/RANGER-2148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16532398#comment-16532398 ] Colm O hEigeartaigh commented on RANGER-2148: - Is this targeted for 2.0.0? I have some work done on this already - in conjunction with the upgrade to Hadoop 3.x > Update Ranger Hive dependency version to 3.0 > - > > Key: RANGER-2148 > URL: https://issues.apache.org/jira/browse/RANGER-2148 > Project: Ranger > Issue Type: Bug > Components: Ranger >Affects Versions: master >Reporter: Ramesh Mani >Assignee: Ramesh Mani >Priority: Critical > Fix For: master > > > Update Ranger Hive dependency version to 3.0 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
