[jira] [Updated] (SENTRY-2555) Slow list_sentry_privileges_for_provider request in heavy load
[ https://issues.apache.org/jira/browse/SENTRY-2555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihua Deng updated SENTRY-2555: Description: Hive uses list_sentry_privileges_for_provider to retrieve privileges of the current user. ORM will cause poor performance on the cluster with heavy load. (was: Hive uses list_sentry_privileges_for_provider to retrieve privileges of the current user. A transaction on the method will cause poor performance on the cluster with heavy load.) > Slow list_sentry_privileges_for_provider request in heavy load > -- > > Key: SENTRY-2555 > URL: https://issues.apache.org/jira/browse/SENTRY-2555 > Project: Sentry > Issue Type: Improvement > Components: sentrystore >Affects Versions: 1.7.1 > Environment: Hive 1.2.1 >Reporter: Zhihua Deng >Priority: Major > > Hive uses list_sentry_privileges_for_provider to retrieve privileges of the > current user. ORM will cause poor performance on the cluster with heavy > load. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (SENTRY-2555) Slow list_sentry_privileges_for_provider request in heavy load
[ https://issues.apache.org/jira/browse/SENTRY-2555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17087407#comment-17087407 ] Zhihua Deng edited comment on SENTRY-2555 at 4/20/20, 9:22 AM: --- Can we use direct sql to retrieve the privileges? Some tests show that we can gain about 3.0X faster if we do like this. was (Author: dengzh): Can we use direct sql to retrieve the privileges? Some tests show that we can gain about 3.0X faster if we do like this. !image-2020-04-20-17-17-54-251.png! > Slow list_sentry_privileges_for_provider request in heavy load > -- > > Key: SENTRY-2555 > URL: https://issues.apache.org/jira/browse/SENTRY-2555 > Project: Sentry > Issue Type: Improvement > Components: sentrystore >Affects Versions: 1.7.1 > Environment: Hive 1.2.1 >Reporter: Zhihua Deng >Priority: Major > Attachments: image-2020-04-20-17-17-54-251.png > > > Hive uses list_sentry_privileges_for_provider to retrieve privileges of the > current user. ORM will cause poor performance on the cluster with heavy > load. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (SENTRY-2555) Slow list_sentry_privileges_for_provider request in heavy load
[ https://issues.apache.org/jira/browse/SENTRY-2555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17087407#comment-17087407 ] Zhihua Deng edited comment on SENTRY-2555 at 4/20/20, 9:17 AM: --- Can we use direct sql to retrieve the privileges? Some tests show that we can gain about 3.0X faster if we do like this. !image-2020-04-20-17-17-54-251.png! was (Author: dengzh): Can we use direct sql to retrieve the privileges? Some tests show that we can gain about 3.0X faster if we do like this. > Slow list_sentry_privileges_for_provider request in heavy load > -- > > Key: SENTRY-2555 > URL: https://issues.apache.org/jira/browse/SENTRY-2555 > Project: Sentry > Issue Type: Improvement > Components: sentrystore >Affects Versions: 1.7.1 > Environment: Hive 1.2.1 >Reporter: Zhihua Deng >Priority: Major > Attachments: image-2020-04-20-17-17-54-251.png > > > Hive uses list_sentry_privileges_for_provider to retrieve privileges of the > current user. ORM will cause poor performance on the cluster with heavy > load. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (SENTRY-2555) Slow list_sentry_privileges_for_provider request in heavy load
[ https://issues.apache.org/jira/browse/SENTRY-2555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17087407#comment-17087407 ] Zhihua Deng edited comment on SENTRY-2555 at 4/20/20, 9:16 AM: --- Can we use direct sql to retrieve the privileges? Some tests show that we can gain about 3.0X faster if we do like this. was (Author: dengzh): Can we remove the transaction or provide someway to ignore it? Some tests show that we can gain about 3.0X faster if we remove the transaction on our hive cluster. > Slow list_sentry_privileges_for_provider request in heavy load > -- > > Key: SENTRY-2555 > URL: https://issues.apache.org/jira/browse/SENTRY-2555 > Project: Sentry > Issue Type: Improvement > Components: sentrystore >Affects Versions: 1.7.1 > Environment: Hive 1.2.1 >Reporter: Zhihua Deng >Priority: Major > > Hive uses list_sentry_privileges_for_provider to retrieve privileges of the > current user. ORM will cause poor performance on the cluster with heavy > load. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (SENTRY-2555) Slow list_sentry_privileges_for_provider request in heavy load
[ https://issues.apache.org/jira/browse/SENTRY-2555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17087407#comment-17087407 ] Zhihua Deng commented on SENTRY-2555: - Can we remove the transaction or provide someway to ignore it? Some tests show that we can gain about 3.0X faster if we remove the transaction on our hive cluster. > Slow list_sentry_privileges_for_provider request in heavy load > -- > > Key: SENTRY-2555 > URL: https://issues.apache.org/jira/browse/SENTRY-2555 > Project: Sentry > Issue Type: Improvement > Components: sentrystore >Affects Versions: 1.7.1 > Environment: Hive 1.2.1 >Reporter: Zhihua Deng >Priority: Major > > Hive uses list_sentry_privileges_for_provider to retrieve privileges of the > current user. A transaction on the method will cause poor performance on > the cluster with heavy load. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (SENTRY-2555) Slow list_sentry_privileges_for_provider request in heavy load
Zhihua Deng created SENTRY-2555: --- Summary: Slow list_sentry_privileges_for_provider request in heavy load Key: SENTRY-2555 URL: https://issues.apache.org/jira/browse/SENTRY-2555 Project: Sentry Issue Type: Improvement Components: sentrystore Affects Versions: 1.7.1 Environment: Hive 1.2.1 Reporter: Zhihua Deng Hive uses list_sentry_privileges_for_provider to retrieve privileges of the current user. A transaction on the method will cause poor performance on the cluster with heavy load. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (SENTRY-2560) SentryService emits open connections metrics
[ https://issues.apache.org/jira/browse/SENTRY-2560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihua Deng updated SENTRY-2560: Attachment: SENTRY-2560.001.patch > SentryService emits open connections metrics > > > Key: SENTRY-2560 > URL: https://issues.apache.org/jira/browse/SENTRY-2560 > Project: Sentry > Issue Type: Improvement >Reporter: Zhihua Deng >Priority: Major > Attachments: SENTRY-2560.001.patch > > > SentryService use TThreadPoolServer to process the incoming requests. A tcp > connection will occupies a thread in TThreadPoolServer until being closed. > It's better to emit the connection metrics to give a internal view of the > TThreadPoolServer. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (SENTRY-2561) Fix SimpleSemanticAnalyzer regex pattern
Zhihua Deng created SENTRY-2561: --- Summary: Fix SimpleSemanticAnalyzer regex pattern Key: SENTRY-2561 URL: https://issues.apache.org/jira/browse/SENTRY-2561 Project: Sentry Issue Type: Bug Reporter: Zhihua Deng Right now SimpleSemanticAnalyzer is unable to extract the input/output from a ddl query like this: desc `abc`. This should be fixed before it can be cleanly removed. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (SENTRY-2562) Add connection and session timeout for zookeeper
Zhihua Deng created SENTRY-2562: --- Summary: Add connection and session timeout for zookeeper Key: SENTRY-2562 URL: https://issues.apache.org/jira/browse/SENTRY-2562 Project: Sentry Issue Type: Improvement Reporter: Zhihua Deng Right now there is no other way to configure the zookeeper session or connection timeout. This options should be better to provide to allow for fast failing of zookeeper in case. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (SENTRY-2563) Fix NPE on numPrivs > 0 in SentryStore
[ https://issues.apache.org/jira/browse/SENTRY-2563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihua Deng updated SENTRY-2563: Attachment: SENTRY-2563.001.patch > Fix NPE on numPrivs > 0 in SentryStore > -- > > Key: SENTRY-2563 > URL: https://issues.apache.org/jira/browse/SENTRY-2563 > Project: Sentry > Issue Type: Bug >Reporter: Zhihua Deng >Priority: Major > Attachments: SENTRY-2563.001.patch > > > if numPrivs is null, comparing it to zero will throw npe. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (SENTRY-2563) Fix NPE on numPrivs > 0 in SentryStore
Zhihua Deng created SENTRY-2563: --- Summary: Fix NPE on numPrivs > 0 in SentryStore Key: SENTRY-2563 URL: https://issues.apache.org/jira/browse/SENTRY-2563 Project: Sentry Issue Type: Bug Reporter: Zhihua Deng if numPrivs is null, comparing it to zero will throw npe. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (SENTRY-2564) Clean up SentryAuthorizerUtil
[ https://issues.apache.org/jira/browse/SENTRY-2564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihua Deng updated SENTRY-2564: Status: Patch Available (was: Open) > Clean up SentryAuthorizerUtil > - > > Key: SENTRY-2564 > URL: https://issues.apache.org/jira/browse/SENTRY-2564 > Project: Sentry > Issue Type: Improvement >Reporter: Zhihua Deng >Priority: Minor > Attachments: SENTRY-2564.001.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (SENTRY-2563) Fix NPE on numPrivs > 0 in SentryStore
[ https://issues.apache.org/jira/browse/SENTRY-2563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihua Deng updated SENTRY-2563: Status: Patch Available (was: Open) > Fix NPE on numPrivs > 0 in SentryStore > -- > > Key: SENTRY-2563 > URL: https://issues.apache.org/jira/browse/SENTRY-2563 > Project: Sentry > Issue Type: Bug >Reporter: Zhihua Deng >Priority: Major > Attachments: SENTRY-2563.001.patch > > > if numPrivs is null, comparing it to zero will throw npe. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (SENTRY-2562) Add connection and session timeout for zookeeper
[ https://issues.apache.org/jira/browse/SENTRY-2562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihua Deng updated SENTRY-2562: Status: Patch Available (was: Open) > Add connection and session timeout for zookeeper > - > > Key: SENTRY-2562 > URL: https://issues.apache.org/jira/browse/SENTRY-2562 > Project: Sentry > Issue Type: Improvement >Reporter: Zhihua Deng >Priority: Major > Attachments: SENTRY-2562.001.patch > > > Right now there is no other way to configure the zookeeper session or > connection timeout. This options should be better to provide to allow for > fast failing of zookeeper in case. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (SENTRY-2564) Clean up SentryAuthorizerUtil.java
[ https://issues.apache.org/jira/browse/SENTRY-2564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihua Deng updated SENTRY-2564: Summary: Clean up SentryAuthorizerUtil.java (was: Clean up SentryAuthorizerUtil) > Clean up SentryAuthorizerUtil.java > -- > > Key: SENTRY-2564 > URL: https://issues.apache.org/jira/browse/SENTRY-2564 > Project: Sentry > Issue Type: Improvement >Reporter: Zhihua Deng >Priority: Minor > Attachments: SENTRY-2564.001.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (SENTRY-2561) Fix SimpleSemanticAnalyzer regex pattern
[ https://issues.apache.org/jira/browse/SENTRY-2561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihua Deng updated SENTRY-2561: Status: Patch Available (was: Open) > Fix SimpleSemanticAnalyzer regex pattern > > > Key: SENTRY-2561 > URL: https://issues.apache.org/jira/browse/SENTRY-2561 > Project: Sentry > Issue Type: Bug >Reporter: Zhihua Deng >Priority: Major > Attachments: SENTRY-2561.001.patch > > > Right now SimpleSemanticAnalyzer is unable to extract the input/output from a > ddl query like this: desc `abc`. This should be fixed before it can be > cleanly removed. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (SENTRY-2559) DefaultSentryValidator: Use session's specified classloader to load the udf class
[ https://issues.apache.org/jira/browse/SENTRY-2559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17217526#comment-17217526 ] Zhihua Deng edited comment on SENTRY-2559 at 10/20/20, 11:55 AM: - [~sean_impala_9b93] could you please table a look at the changes? thanks! was (Author: dengzh): [~mackrorysd] could you table a look at the changes? thank you! > DefaultSentryValidator: Use session's specified classloader to load the udf > class > - > > Key: SENTRY-2559 > URL: https://issues.apache.org/jira/browse/SENTRY-2559 > Project: Sentry > Issue Type: Bug > Components: Hive V2 >Affects Versions: 1.7.1 >Reporter: Zhihua Deng >Priority: Major > Attachments: SENTRY-2559.001.patch > > > When hive creating functions, a ClassNotFoundException may be thrown if the > corresponding external jar is added to the session: > > Caused by: > org.apache.hadoop.hive.ql.security.authorization.plugin.HiveAuthzPluginException: > Error retrieving udf class > at > org.apache.sentry.binding.hive.v2.authorizer.DefaultSentryValidator.addExtendHierarchy(DefaultSentryValidator.java:321) > at > org.apache.sentry.binding.hive.v2.authorizer.DefaultSentryValidator.checkPrivileges(DefaultSentryValidator.java:181) > ... 23 more > Caused by: java.lang.ClassNotFoundException: com.xxx.IP2Address > at java.net.URLClassLoader.findClass(URLClassLoader.java:381) > at java.lang.ClassLoader.loadClass(ClassLoader.java:424) > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331) > at java.lang.ClassLoader.loadClass(ClassLoader.java:357) > at java.lang.Class.forName0(Native Method) > at java.lang.Class.forName(Class.java:264) > at > org.apache.sentry.binding.hive.v2.authorizer.DefaultSentryValidator.addExtendHierarchy(DefaultSentryValidator.java:305) > > Use session's specified classloader to load the udf class instead -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (SENTRY-2560) SentryService emits open connections metrics
Zhihua Deng created SENTRY-2560: --- Summary: SentryService emits open connections metrics Key: SENTRY-2560 URL: https://issues.apache.org/jira/browse/SENTRY-2560 Project: Sentry Issue Type: Improvement Reporter: Zhihua Deng SentryService use TThreadPoolServer to process the incoming requests. A tcp connection will occupies a thread in TThreadPoolServer until being closed. It's better to emit the connection metrics to give a internal view of the TThreadPoolServer. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (SENTRY-2562) Add connection and session timeout for zookeeper
[ https://issues.apache.org/jira/browse/SENTRY-2562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihua Deng updated SENTRY-2562: Attachment: SENTRY-2562.001.patch > Add connection and session timeout for zookeeper > - > > Key: SENTRY-2562 > URL: https://issues.apache.org/jira/browse/SENTRY-2562 > Project: Sentry > Issue Type: Improvement >Reporter: Zhihua Deng >Priority: Major > Attachments: SENTRY-2562.001.patch > > > Right now there is no other way to configure the zookeeper session or > connection timeout. This options should be better to provide to allow for > fast failing of zookeeper in case. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (SENTRY-2564) Clean up SentryAuthorizerUtil
[ https://issues.apache.org/jira/browse/SENTRY-2564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihua Deng updated SENTRY-2564: Attachment: SENTRY-2564.001.patch > Clean up SentryAuthorizerUtil > - > > Key: SENTRY-2564 > URL: https://issues.apache.org/jira/browse/SENTRY-2564 > Project: Sentry > Issue Type: Improvement >Reporter: Zhihua Deng >Priority: Minor > Attachments: SENTRY-2564.001.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (SENTRY-2560) SentryService emits open connections metrics
[ https://issues.apache.org/jira/browse/SENTRY-2560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihua Deng updated SENTRY-2560: Status: Patch Available (was: Open) > SentryService emits open connections metrics > > > Key: SENTRY-2560 > URL: https://issues.apache.org/jira/browse/SENTRY-2560 > Project: Sentry > Issue Type: Improvement >Reporter: Zhihua Deng >Priority: Major > Attachments: SENTRY-2560.001.patch > > > SentryService use TThreadPoolServer to process the incoming requests. A tcp > connection will occupies a thread in TThreadPoolServer until being closed. > It's better to emit the connection metrics to give a internal view of the > TThreadPoolServer. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (SENTRY-2564) Clean up SentryAuthorizerUtil
Zhihua Deng created SENTRY-2564: --- Summary: Clean up SentryAuthorizerUtil Key: SENTRY-2564 URL: https://issues.apache.org/jira/browse/SENTRY-2564 Project: Sentry Issue Type: Improvement Reporter: Zhihua Deng -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (SENTRY-2559) DefaultSentryValidator: Use session's specified classloader to load the udf class
[ https://issues.apache.org/jira/browse/SENTRY-2559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17217526#comment-17217526 ] Zhihua Deng commented on SENTRY-2559: - [~mackrorysd] could you table a look at the changes? thank you! > DefaultSentryValidator: Use session's specified classloader to load the udf > class > - > > Key: SENTRY-2559 > URL: https://issues.apache.org/jira/browse/SENTRY-2559 > Project: Sentry > Issue Type: Bug > Components: Hive V2 >Affects Versions: 1.7.1 >Reporter: Zhihua Deng >Priority: Major > Attachments: SENTRY-2559.001.patch > > > When hive creating functions, a ClassNotFoundException may be thrown if the > corresponding external jar is added to the session: > > Caused by: > org.apache.hadoop.hive.ql.security.authorization.plugin.HiveAuthzPluginException: > Error retrieving udf class > at > org.apache.sentry.binding.hive.v2.authorizer.DefaultSentryValidator.addExtendHierarchy(DefaultSentryValidator.java:321) > at > org.apache.sentry.binding.hive.v2.authorizer.DefaultSentryValidator.checkPrivileges(DefaultSentryValidator.java:181) > ... 23 more > Caused by: java.lang.ClassNotFoundException: com.xxx.IP2Address > at java.net.URLClassLoader.findClass(URLClassLoader.java:381) > at java.lang.ClassLoader.loadClass(ClassLoader.java:424) > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331) > at java.lang.ClassLoader.loadClass(ClassLoader.java:357) > at java.lang.Class.forName0(Native Method) > at java.lang.Class.forName(Class.java:264) > at > org.apache.sentry.binding.hive.v2.authorizer.DefaultSentryValidator.addExtendHierarchy(DefaultSentryValidator.java:305) > > Use session's specified classloader to load the udf class instead -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (SENTRY-2561) Fix SimpleSemanticAnalyzer regex pattern
[ https://issues.apache.org/jira/browse/SENTRY-2561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihua Deng updated SENTRY-2561: Attachment: SENTRY-2561.001.patch > Fix SimpleSemanticAnalyzer regex pattern > > > Key: SENTRY-2561 > URL: https://issues.apache.org/jira/browse/SENTRY-2561 > Project: Sentry > Issue Type: Bug >Reporter: Zhihua Deng >Priority: Major > Attachments: SENTRY-2561.001.patch > > > Right now SimpleSemanticAnalyzer is unable to extract the input/output from a > ddl query like this: desc `abc`. This should be fixed before it can be > cleanly removed. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (SENTRY-2559) DefaultSentryValidator: Use session's specified classloader to load the udf class
[ https://issues.apache.org/jira/browse/SENTRY-2559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17203833#comment-17203833 ] Zhihua Deng commented on SENTRY-2559: - [~linaataustin] could you please take a look at this? > DefaultSentryValidator: Use session's specified classloader to load the udf > class > - > > Key: SENTRY-2559 > URL: https://issues.apache.org/jira/browse/SENTRY-2559 > Project: Sentry > Issue Type: Bug > Components: Hive V2 >Affects Versions: 1.7.1 >Reporter: Zhihua Deng >Priority: Major > Attachments: SENTRY-2559.001.patch > > > When hive creating functions, a ClassNotFoundException may be thrown if the > corresponding external jar is added to the session: > > Caused by: > org.apache.hadoop.hive.ql.security.authorization.plugin.HiveAuthzPluginException: > Error retrieving udf class > at > org.apache.sentry.binding.hive.v2.authorizer.DefaultSentryValidator.addExtendHierarchy(DefaultSentryValidator.java:321) > at > org.apache.sentry.binding.hive.v2.authorizer.DefaultSentryValidator.checkPrivileges(DefaultSentryValidator.java:181) > ... 23 more > Caused by: java.lang.ClassNotFoundException: com.xxx.IP2Address > at java.net.URLClassLoader.findClass(URLClassLoader.java:381) > at java.lang.ClassLoader.loadClass(ClassLoader.java:424) > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331) > at java.lang.ClassLoader.loadClass(ClassLoader.java:357) > at java.lang.Class.forName0(Native Method) > at java.lang.Class.forName(Class.java:264) > at > org.apache.sentry.binding.hive.v2.authorizer.DefaultSentryValidator.addExtendHierarchy(DefaultSentryValidator.java:305) > > Use session's specified classloader to load the udf class instead -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (SENTRY-2559) DefaultSentryValidator: Use session's specified classloader to load the udf class instead
Zhihua Deng created SENTRY-2559: --- Summary: DefaultSentryValidator: Use session's specified classloader to load the udf class instead Key: SENTRY-2559 URL: https://issues.apache.org/jira/browse/SENTRY-2559 Project: Sentry Issue Type: Improvement Components: Hive V2 Affects Versions: 1.7.1 Reporter: Zhihua Deng When hive creating functions, a ClassNotFoundException may be thrown if the corresponding external jar is added to the session: Caused by: org.apache.hadoop.hive.ql.security.authorization.plugin.HiveAuthzPluginException: Error retrieving udf class at org.apache.sentry.binding.hive.v2.authorizer.DefaultSentryValidator.addExtendHierarchy(DefaultSentryValidator.java:321) at org.apache.sentry.binding.hive.v2.authorizer.DefaultSentryValidator.checkPrivileges(DefaultSentryValidator.java:181) ... 23 more Caused by: java.lang.ClassNotFoundException: com.xxx.IP2Address at java.net.URLClassLoader.findClass(URLClassLoader.java:381) at java.lang.ClassLoader.loadClass(ClassLoader.java:424) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.sentry.binding.hive.v2.authorizer.DefaultSentryValidator.addExtendHierarchy(DefaultSentryValidator.java:305) Use session's specified classloader to load the udf class instead -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (SENTRY-2559) DefaultSentryValidator: Use session's specified classloader to load the udf class
[ https://issues.apache.org/jira/browse/SENTRY-2559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihua Deng updated SENTRY-2559: Summary: DefaultSentryValidator: Use session's specified classloader to load the udf class (was: DefaultSentryValidator: Use session's specified classloader to load the udf class instead) > DefaultSentryValidator: Use session's specified classloader to load the udf > class > - > > Key: SENTRY-2559 > URL: https://issues.apache.org/jira/browse/SENTRY-2559 > Project: Sentry > Issue Type: Improvement > Components: Hive V2 >Affects Versions: 1.7.1 >Reporter: Zhihua Deng >Priority: Major > Attachments: SENTRY-2559.001.patch > > > When hive creating functions, a ClassNotFoundException may be thrown if the > corresponding external jar is added to the session: > > Caused by: > org.apache.hadoop.hive.ql.security.authorization.plugin.HiveAuthzPluginException: > Error retrieving udf class > at > org.apache.sentry.binding.hive.v2.authorizer.DefaultSentryValidator.addExtendHierarchy(DefaultSentryValidator.java:321) > at > org.apache.sentry.binding.hive.v2.authorizer.DefaultSentryValidator.checkPrivileges(DefaultSentryValidator.java:181) > ... 23 more > Caused by: java.lang.ClassNotFoundException: com.xxx.IP2Address > at java.net.URLClassLoader.findClass(URLClassLoader.java:381) > at java.lang.ClassLoader.loadClass(ClassLoader.java:424) > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331) > at java.lang.ClassLoader.loadClass(ClassLoader.java:357) > at java.lang.Class.forName0(Native Method) > at java.lang.Class.forName(Class.java:264) > at > org.apache.sentry.binding.hive.v2.authorizer.DefaultSentryValidator.addExtendHierarchy(DefaultSentryValidator.java:305) > > Use session's specified classloader to load the udf class instead -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (SENTRY-2559) DefaultSentryValidator: Use session's specified classloader to load the udf class
[ https://issues.apache.org/jira/browse/SENTRY-2559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihua Deng updated SENTRY-2559: Issue Type: Bug (was: Improvement) > DefaultSentryValidator: Use session's specified classloader to load the udf > class > - > > Key: SENTRY-2559 > URL: https://issues.apache.org/jira/browse/SENTRY-2559 > Project: Sentry > Issue Type: Bug > Components: Hive V2 >Affects Versions: 1.7.1 >Reporter: Zhihua Deng >Priority: Major > Attachments: SENTRY-2559.001.patch > > > When hive creating functions, a ClassNotFoundException may be thrown if the > corresponding external jar is added to the session: > > Caused by: > org.apache.hadoop.hive.ql.security.authorization.plugin.HiveAuthzPluginException: > Error retrieving udf class > at > org.apache.sentry.binding.hive.v2.authorizer.DefaultSentryValidator.addExtendHierarchy(DefaultSentryValidator.java:321) > at > org.apache.sentry.binding.hive.v2.authorizer.DefaultSentryValidator.checkPrivileges(DefaultSentryValidator.java:181) > ... 23 more > Caused by: java.lang.ClassNotFoundException: com.xxx.IP2Address > at java.net.URLClassLoader.findClass(URLClassLoader.java:381) > at java.lang.ClassLoader.loadClass(ClassLoader.java:424) > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331) > at java.lang.ClassLoader.loadClass(ClassLoader.java:357) > at java.lang.Class.forName0(Native Method) > at java.lang.Class.forName(Class.java:264) > at > org.apache.sentry.binding.hive.v2.authorizer.DefaultSentryValidator.addExtendHierarchy(DefaultSentryValidator.java:305) > > Use session's specified classloader to load the udf class instead -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (SENTRY-2559) DefaultSentryValidator: Use session's specified classloader to load the udf class instead
[ https://issues.apache.org/jira/browse/SENTRY-2559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihua Deng updated SENTRY-2559: Attachment: SENTRY-2559.001.patch > DefaultSentryValidator: Use session's specified classloader to load the udf > class instead > - > > Key: SENTRY-2559 > URL: https://issues.apache.org/jira/browse/SENTRY-2559 > Project: Sentry > Issue Type: Improvement > Components: Hive V2 >Affects Versions: 1.7.1 >Reporter: Zhihua Deng >Priority: Major > Attachments: SENTRY-2559.001.patch > > > When hive creating functions, a ClassNotFoundException may be thrown if the > corresponding external jar is added to the session: > > Caused by: > org.apache.hadoop.hive.ql.security.authorization.plugin.HiveAuthzPluginException: > Error retrieving udf class > at > org.apache.sentry.binding.hive.v2.authorizer.DefaultSentryValidator.addExtendHierarchy(DefaultSentryValidator.java:321) > at > org.apache.sentry.binding.hive.v2.authorizer.DefaultSentryValidator.checkPrivileges(DefaultSentryValidator.java:181) > ... 23 more > Caused by: java.lang.ClassNotFoundException: com.xxx.IP2Address > at java.net.URLClassLoader.findClass(URLClassLoader.java:381) > at java.lang.ClassLoader.loadClass(ClassLoader.java:424) > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331) > at java.lang.ClassLoader.loadClass(ClassLoader.java:357) > at java.lang.Class.forName0(Native Method) > at java.lang.Class.forName(Class.java:264) > at > org.apache.sentry.binding.hive.v2.authorizer.DefaultSentryValidator.addExtendHierarchy(DefaultSentryValidator.java:305) > > Use session's specified classloader to load the udf class instead -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (SENTRY-2559) DefaultSentryValidator: Use session's specified classloader to load the udf class
[ https://issues.apache.org/jira/browse/SENTRY-2559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihua Deng updated SENTRY-2559: Attachment: SENTRY-2559.001.patch Status: Patch Available (was: Open) > DefaultSentryValidator: Use session's specified classloader to load the udf > class > - > > Key: SENTRY-2559 > URL: https://issues.apache.org/jira/browse/SENTRY-2559 > Project: Sentry > Issue Type: Bug > Components: Hive V2 >Affects Versions: 1.7.1 >Reporter: Zhihua Deng >Priority: Major > Attachments: SENTRY-2559.001.patch > > > When hive creating functions, a ClassNotFoundException may be thrown if the > corresponding external jar is added to the session: > > Caused by: > org.apache.hadoop.hive.ql.security.authorization.plugin.HiveAuthzPluginException: > Error retrieving udf class > at > org.apache.sentry.binding.hive.v2.authorizer.DefaultSentryValidator.addExtendHierarchy(DefaultSentryValidator.java:321) > at > org.apache.sentry.binding.hive.v2.authorizer.DefaultSentryValidator.checkPrivileges(DefaultSentryValidator.java:181) > ... 23 more > Caused by: java.lang.ClassNotFoundException: com.xxx.IP2Address > at java.net.URLClassLoader.findClass(URLClassLoader.java:381) > at java.lang.ClassLoader.loadClass(ClassLoader.java:424) > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331) > at java.lang.ClassLoader.loadClass(ClassLoader.java:357) > at java.lang.Class.forName0(Native Method) > at java.lang.Class.forName(Class.java:264) > at > org.apache.sentry.binding.hive.v2.authorizer.DefaultSentryValidator.addExtendHierarchy(DefaultSentryValidator.java:305) > > Use session's specified classloader to load the udf class instead -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (SENTRY-2559) DefaultSentryValidator: Use session's specified classloader to load the udf class
[ https://issues.apache.org/jira/browse/SENTRY-2559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihua Deng updated SENTRY-2559: Attachment: (was: SENTRY-2559.001.patch) > DefaultSentryValidator: Use session's specified classloader to load the udf > class > - > > Key: SENTRY-2559 > URL: https://issues.apache.org/jira/browse/SENTRY-2559 > Project: Sentry > Issue Type: Bug > Components: Hive V2 >Affects Versions: 1.7.1 >Reporter: Zhihua Deng >Priority: Major > Attachments: SENTRY-2559.001.patch > > > When hive creating functions, a ClassNotFoundException may be thrown if the > corresponding external jar is added to the session: > > Caused by: > org.apache.hadoop.hive.ql.security.authorization.plugin.HiveAuthzPluginException: > Error retrieving udf class > at > org.apache.sentry.binding.hive.v2.authorizer.DefaultSentryValidator.addExtendHierarchy(DefaultSentryValidator.java:321) > at > org.apache.sentry.binding.hive.v2.authorizer.DefaultSentryValidator.checkPrivileges(DefaultSentryValidator.java:181) > ... 23 more > Caused by: java.lang.ClassNotFoundException: com.xxx.IP2Address > at java.net.URLClassLoader.findClass(URLClassLoader.java:381) > at java.lang.ClassLoader.loadClass(ClassLoader.java:424) > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331) > at java.lang.ClassLoader.loadClass(ClassLoader.java:357) > at java.lang.Class.forName0(Native Method) > at java.lang.Class.forName(Class.java:264) > at > org.apache.sentry.binding.hive.v2.authorizer.DefaultSentryValidator.addExtendHierarchy(DefaultSentryValidator.java:305) > > Use session's specified classloader to load the udf class instead -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (SENTRY-2565) Optimize the authorization of AuthorizingObjectStore
Zhihua Deng created SENTRY-2565: --- Summary: Optimize the authorization of AuthorizingObjectStore Key: SENTRY-2565 URL: https://issues.apache.org/jira/browse/SENTRY-2565 Project: Sentry Issue Type: Improvement Components: Hive Binding Reporter: Zhihua Deng A simple call of HiveMetaStoreClient can result to multiple calls to ObjectStore, like drop database for example, there are 1 call to getDatabase, multiple calls in order to get all tables and one call per each table to get table details, etc. Each call will result a rpc to sentry for privileges. This can result to pool performance and more burden on sentry. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (SENTRY-2565) Optimize the authorization of AuthorizingObjectStore
[ https://issues.apache.org/jira/browse/SENTRY-2565?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihua Deng updated SENTRY-2565: Status: Patch Available (was: Open) > Optimize the authorization of AuthorizingObjectStore > > > Key: SENTRY-2565 > URL: https://issues.apache.org/jira/browse/SENTRY-2565 > Project: Sentry > Issue Type: Improvement > Components: Hive Binding >Reporter: Zhihua Deng >Priority: Major > Attachments: SENTRY-2565.patch > > > A simple call of HiveMetaStoreClient can result to multiple calls to > ObjectStore, like drop database for example, there are 1 call to > getDatabase, multiple calls in order to get all tables and one call per each > table to get table details, etc. Each call will result a rpc to sentry for > privileges. This can result to pool performance and more burden on sentry. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (SENTRY-2565) Optimize the authorization of AuthorizingObjectStore
[ https://issues.apache.org/jira/browse/SENTRY-2565?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihua Deng updated SENTRY-2565: Attachment: SENTRY-2565.patch > Optimize the authorization of AuthorizingObjectStore > > > Key: SENTRY-2565 > URL: https://issues.apache.org/jira/browse/SENTRY-2565 > Project: Sentry > Issue Type: Improvement > Components: Hive Binding >Reporter: Zhihua Deng >Priority: Major > Attachments: SENTRY-2565.patch > > > A simple call of HiveMetaStoreClient can result to multiple calls to > ObjectStore, like drop database for example, there are 1 call to > getDatabase, multiple calls in order to get all tables and one call per each > table to get table details, etc. Each call will result a rpc to sentry for > privileges. This can result to pool performance and more burden on sentry. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (SENTRY-2555) Slow list_sentry_privileges_for_provider request in heavy load
[ https://issues.apache.org/jira/browse/SENTRY-2555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihua Deng updated SENTRY-2555: Attachment: SENTRY-2555_branch-1.7.0.001.patch > Slow list_sentry_privileges_for_provider request in heavy load > -- > > Key: SENTRY-2555 > URL: https://issues.apache.org/jira/browse/SENTRY-2555 > Project: Sentry > Issue Type: Improvement > Components: sentrystore >Affects Versions: 1.7.1 > Environment: Hive 1.2.1 >Reporter: Zhihua Deng >Priority: Major > Attachments: SENTRY-2555_branch-1.7.0.001.patch, > image-2020-04-20-17-17-54-251.png > > > Hive uses list_sentry_privileges_for_provider to retrieve privileges of the > current user. ORM will cause poor performance on the cluster with heavy > load. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (SENTRY-2555) Slow list_sentry_privileges_for_provider request in heavy load
[ https://issues.apache.org/jira/browse/SENTRY-2555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17243189#comment-17243189 ] Zhihua Deng commented on SENTRY-2555: - Apologies for the delay, attach a patch based on branch 1.7.0 to show the improvement. > Slow list_sentry_privileges_for_provider request in heavy load > -- > > Key: SENTRY-2555 > URL: https://issues.apache.org/jira/browse/SENTRY-2555 > Project: Sentry > Issue Type: Improvement > Components: sentrystore >Affects Versions: 1.7.1 > Environment: Hive 1.2.1 >Reporter: Zhihua Deng >Priority: Major > Attachments: SENTRY-2555_branch-1.7.0.001.patch, > image-2020-04-20-17-17-54-251.png > > > Hive uses list_sentry_privileges_for_provider to retrieve privileges of the > current user. ORM will cause poor performance on the cluster with heavy > load. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (SENTRY-2555) Slow list_sentry_privileges_for_provider request in heavy load
[ https://issues.apache.org/jira/browse/SENTRY-2555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17242144#comment-17242144 ] Zhihua Deng commented on SENTRY-2555: - The count means the total method calls of the listPrivilegesForProvider, we use SQL directly in our production to fetch the privileges from MySQL. > Slow list_sentry_privileges_for_provider request in heavy load > -- > > Key: SENTRY-2555 > URL: https://issues.apache.org/jira/browse/SENTRY-2555 > Project: Sentry > Issue Type: Improvement > Components: sentrystore >Affects Versions: 1.7.1 > Environment: Hive 1.2.1 >Reporter: Zhihua Deng >Priority: Major > Attachments: image-2020-04-20-17-17-54-251.png > > > Hive uses list_sentry_privileges_for_provider to retrieve privileges of the > current user. ORM will cause poor performance on the cluster with heavy > load. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (SENTRY-2555) Slow list_sentry_privileges_for_provider request in heavy load
[ https://issues.apache.org/jira/browse/SENTRY-2555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17242176#comment-17242176 ] Zhihua Deng commented on SENTRY-2555: - In our case, there are 99k privileges in table SENTRY_DB_PRIVILEGE, and about 19k public privileges. I will file a patch to this Jira latter today. > Slow list_sentry_privileges_for_provider request in heavy load > -- > > Key: SENTRY-2555 > URL: https://issues.apache.org/jira/browse/SENTRY-2555 > Project: Sentry > Issue Type: Improvement > Components: sentrystore >Affects Versions: 1.7.1 > Environment: Hive 1.2.1 >Reporter: Zhihua Deng >Priority: Major > Attachments: image-2020-04-20-17-17-54-251.png > > > Hive uses list_sentry_privileges_for_provider to retrieve privileges of the > current user. ORM will cause poor performance on the cluster with heavy > load. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (SENTRY-2566) Output failed privileges when HiveAuthzBinding throws AuthorizationException
[
https://issues.apache.org/jira/browse/SENTRY-2566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17236553#comment-17236553
]
Zhihua Deng commented on SENTRY-2566:
-
[~kalyan] could you please take a look? thanks in advance!
> Output failed privileges when HiveAuthzBinding throws AuthorizationException
>
>
> Key: SENTRY-2566
> URL: https://issues.apache.org/jira/browse/SENTRY-2566
> Project: Sentry
> Issue Type: Improvement
>Reporter: Zhihua Deng
>Priority: Major
> Attachments: SENTRY-2566.001.patch
>
>
> When using MetastoreAuthzBindingBase to authorize meta changes like creating
> tables, if user does not have the right privileges, the hive client gets the
> error message like this:
> {noformat}
> User u_dengzhihua does not have privileges for CREATETABLE{noformat}
> It's not easy for the user to figure out what privileges are missing. We
> should output the failed message like the DefaultSentryValidator does, the
> message would be like this:
> {noformat}
> User u_dengzhihua does not have privileges
> Server=server1->Db=test->action=create for CREATETABLE{noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (SENTRY-2566) Output failed privileges when HiveAuthzBinding throws AuthorizationException
Zhihua Deng created SENTRY-2566:
---
Summary: Output failed privileges when HiveAuthzBinding throws
AuthorizationException
Key: SENTRY-2566
URL: https://issues.apache.org/jira/browse/SENTRY-2566
Project: Sentry
Issue Type: Improvement
Reporter: Zhihua Deng
When using MetastoreAuthzBindingBase to authorize meta changes like creating
tables, if user does not have the right privileges, the hive client gets the
error message like this:
{noformat}
User u_dengzhihua does not have privileges for CREATETABLE{noformat}
It's not easy for the user to figure out what privileges are missing. We
should output the failed message like the DefaultSentryValidator does, the
message would be like this:
{noformat}
User u_dengzhihua does not have privileges
Server=server1->Db=test->action=create for CREATETABLE{noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (SENTRY-2566) Output failed privileges when HiveAuthzBinding throws AuthorizationException
[
https://issues.apache.org/jira/browse/SENTRY-2566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhihua Deng updated SENTRY-2566:
Attachment: (was: SENTRY-2566.patch)
> Output failed privileges when HiveAuthzBinding throws AuthorizationException
>
>
> Key: SENTRY-2566
> URL: https://issues.apache.org/jira/browse/SENTRY-2566
> Project: Sentry
> Issue Type: Improvement
>Reporter: Zhihua Deng
>Priority: Major
> Attachments: SENTRY-2566.001.patch
>
>
> When using MetastoreAuthzBindingBase to authorize meta changes like creating
> tables, if user does not have the right privileges, the hive client gets the
> error message like this:
> {noformat}
> User u_dengzhihua does not have privileges for CREATETABLE{noformat}
> It's not easy for the user to figure out what privileges are missing. We
> should output the failed message like the DefaultSentryValidator does, the
> message would be like this:
> {noformat}
> User u_dengzhihua does not have privileges
> Server=server1->Db=test->action=create for CREATETABLE{noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (SENTRY-2566) Output failed privileges when HiveAuthzBinding throws AuthorizationException
[
https://issues.apache.org/jira/browse/SENTRY-2566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhihua Deng updated SENTRY-2566:
Attachment: SENTRY-2566.001.patch
> Output failed privileges when HiveAuthzBinding throws AuthorizationException
>
>
> Key: SENTRY-2566
> URL: https://issues.apache.org/jira/browse/SENTRY-2566
> Project: Sentry
> Issue Type: Improvement
>Reporter: Zhihua Deng
>Priority: Major
> Attachments: SENTRY-2566.001.patch
>
>
> When using MetastoreAuthzBindingBase to authorize meta changes like creating
> tables, if user does not have the right privileges, the hive client gets the
> error message like this:
> {noformat}
> User u_dengzhihua does not have privileges for CREATETABLE{noformat}
> It's not easy for the user to figure out what privileges are missing. We
> should output the failed message like the DefaultSentryValidator does, the
> message would be like this:
> {noformat}
> User u_dengzhihua does not have privileges
> Server=server1->Db=test->action=create for CREATETABLE{noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (SENTRY-2566) Output failed privileges when HiveAuthzBinding throws AuthorizationException
[
https://issues.apache.org/jira/browse/SENTRY-2566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhihua Deng updated SENTRY-2566:
Status: Patch Available (was: Open)
> Output failed privileges when HiveAuthzBinding throws AuthorizationException
>
>
> Key: SENTRY-2566
> URL: https://issues.apache.org/jira/browse/SENTRY-2566
> Project: Sentry
> Issue Type: Improvement
>Reporter: Zhihua Deng
>Priority: Major
> Attachments: SENTRY-2566.patch
>
>
> When using MetastoreAuthzBindingBase to authorize meta changes like creating
> tables, if user does not have the right privileges, the hive client gets the
> error message like this:
> {noformat}
> User u_dengzhihua does not have privileges for CREATETABLE{noformat}
> It's not easy for the user to figure out what privileges are missing. We
> should output the failed message like the DefaultSentryValidator does, the
> message would be like this:
> {noformat}
> User u_dengzhihua does not have privileges
> Server=server1->Db=test->action=create for CREATETABLE{noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (SENTRY-2566) Output failed privileges when HiveAuthzBinding throws AuthorizationException
[
https://issues.apache.org/jira/browse/SENTRY-2566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhihua Deng updated SENTRY-2566:
Attachment: SENTRY-2566.patch
> Output failed privileges when HiveAuthzBinding throws AuthorizationException
>
>
> Key: SENTRY-2566
> URL: https://issues.apache.org/jira/browse/SENTRY-2566
> Project: Sentry
> Issue Type: Improvement
>Reporter: Zhihua Deng
>Priority: Major
> Attachments: SENTRY-2566.patch
>
>
> When using MetastoreAuthzBindingBase to authorize meta changes like creating
> tables, if user does not have the right privileges, the hive client gets the
> error message like this:
> {noformat}
> User u_dengzhihua does not have privileges for CREATETABLE{noformat}
> It's not easy for the user to figure out what privileges are missing. We
> should output the failed message like the DefaultSentryValidator does, the
> message would be like this:
> {noformat}
> User u_dengzhihua does not have privileges
> Server=server1->Db=test->action=create for CREATETABLE{noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (SENTRY-2565) Optimize the authorization of AuthorizingObjectStore
[ https://issues.apache.org/jira/browse/SENTRY-2565?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihua Deng updated SENTRY-2565: Attachment: SENTRY-2565.patch > Optimize the authorization of AuthorizingObjectStore > > > Key: SENTRY-2565 > URL: https://issues.apache.org/jira/browse/SENTRY-2565 > Project: Sentry > Issue Type: Improvement > Components: Hive Binding >Reporter: Zhihua Deng >Priority: Major > Attachments: SENTRY-2565.patch > > > A simple call of HiveMetaStoreClient can result to multiple calls to > ObjectStore, like drop database for example, there are 1 call to > getDatabase, multiple calls in order to get all tables and one call per each > table to get table details, etc. Each call will result a rpc to sentry for > privileges. This can result to pool performance and more burden on sentry. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (SENTRY-2565) Optimize the authorization of AuthorizingObjectStore
[ https://issues.apache.org/jira/browse/SENTRY-2565?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihua Deng updated SENTRY-2565: Attachment: (was: SENTRY-2565.patch) > Optimize the authorization of AuthorizingObjectStore > > > Key: SENTRY-2565 > URL: https://issues.apache.org/jira/browse/SENTRY-2565 > Project: Sentry > Issue Type: Improvement > Components: Hive Binding >Reporter: Zhihua Deng >Priority: Major > > A simple call of HiveMetaStoreClient can result to multiple calls to > ObjectStore, like drop database for example, there are 1 call to > getDatabase, multiple calls in order to get all tables and one call per each > table to get table details, etc. Each call will result a rpc to sentry for > privileges. This can result to pool performance and more burden on sentry. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (SENTRY-2565) Optimize the authorization of AuthorizingObjectStore
[ https://issues.apache.org/jira/browse/SENTRY-2565?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihua Deng updated SENTRY-2565: Attachment: (was: SENTRY-2565.patch) > Optimize the authorization of AuthorizingObjectStore > > > Key: SENTRY-2565 > URL: https://issues.apache.org/jira/browse/SENTRY-2565 > Project: Sentry > Issue Type: Improvement > Components: Hive Binding >Reporter: Zhihua Deng >Priority: Major > Attachments: SENTRY-2565.patch > > > A simple call of HiveMetaStoreClient can result to multiple calls to > ObjectStore, like drop database for example, there are 1 call to > getDatabase, multiple calls in order to get all tables and one call per each > table to get table details, etc. Each call will result a rpc to sentry for > privileges. This can result to pool performance and more burden on sentry. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (SENTRY-2565) Optimize the authorization of AuthorizingObjectStore
[ https://issues.apache.org/jira/browse/SENTRY-2565?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihua Deng updated SENTRY-2565: Attachment: SENTRY-2565.patch > Optimize the authorization of AuthorizingObjectStore > > > Key: SENTRY-2565 > URL: https://issues.apache.org/jira/browse/SENTRY-2565 > Project: Sentry > Issue Type: Improvement > Components: Hive Binding >Reporter: Zhihua Deng >Priority: Major > Attachments: SENTRY-2565.patch > > > A simple call of HiveMetaStoreClient can result to multiple calls to > ObjectStore, like drop database for example, there are 1 call to > getDatabase, multiple calls in order to get all tables and one call per each > table to get table details, etc. Each call will result a rpc to sentry for > privileges. This can result to pool performance and more burden on sentry. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (SENTRY-2565) Optimize the authorization of AuthorizingObjectStore
[ https://issues.apache.org/jira/browse/SENTRY-2565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17233602#comment-17233602 ] Zhihua Deng commented on SENTRY-2565: - [~linaataustin] [~sean_impala_9b93] would you please take a look? thanks in advance! > Optimize the authorization of AuthorizingObjectStore > > > Key: SENTRY-2565 > URL: https://issues.apache.org/jira/browse/SENTRY-2565 > Project: Sentry > Issue Type: Improvement > Components: Hive Binding >Reporter: Zhihua Deng >Priority: Major > Attachments: SENTRY-2565.patch > > > A simple call of HiveMetaStoreClient can result to multiple calls to > ObjectStore, like drop database for example, there are 1 call to > getDatabase, multiple calls in order to get all tables and one call per each > table to get table details, etc. Each call will result a rpc to sentry for > privileges. This can result to pool performance and more burden on sentry. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (SENTRY-2562) Add connection and session timeout for zookeeper
[ https://issues.apache.org/jira/browse/SENTRY-2562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17218025#comment-17218025 ] Zhihua Deng commented on SENTRY-2562: - Thanks for reviewing this, [~mackrorysd]! Change the session timeout from 2min to 10s, and connection timeout from 15s to 3s. > Add connection and session timeout for zookeeper > - > > Key: SENTRY-2562 > URL: https://issues.apache.org/jira/browse/SENTRY-2562 > Project: Sentry > Issue Type: Improvement >Reporter: Zhihua Deng >Priority: Major > Attachments: SENTRY-2562.001.patch, SENTRY-2562.002.patch > > > Right now there is no other way to configure the zookeeper session or > connection timeout. This options should be better to provide to allow for > fast failing of zookeeper in case. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (SENTRY-2563) Fix NPE on numPrivs > 0 in SentryStore
[ https://issues.apache.org/jira/browse/SENTRY-2563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17218033#comment-17218033 ] Zhihua Deng commented on SENTRY-2563: - Thanks much for pointing it out. I removed the ternary operators and ran the tests successfully on my local machine. > Fix NPE on numPrivs > 0 in SentryStore > -- > > Key: SENTRY-2563 > URL: https://issues.apache.org/jira/browse/SENTRY-2563 > Project: Sentry > Issue Type: Bug >Reporter: Zhihua Deng >Priority: Major > Attachments: SENTRY-2563.001.patch, SENTRY-2563.002.patch > > > if numPrivs is null, comparing it to zero will throw npe. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (SENTRY-2563) Fix NPE on numPrivs > 0 in SentryStore
[ https://issues.apache.org/jira/browse/SENTRY-2563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihua Deng updated SENTRY-2563: Attachment: SENTRY-2563.002.patch > Fix NPE on numPrivs > 0 in SentryStore > -- > > Key: SENTRY-2563 > URL: https://issues.apache.org/jira/browse/SENTRY-2563 > Project: Sentry > Issue Type: Bug >Reporter: Zhihua Deng >Priority: Major > Attachments: SENTRY-2563.001.patch, SENTRY-2563.002.patch > > > if numPrivs is null, comparing it to zero will throw npe. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (SENTRY-2562) Add connection and session timeout for zookeeper
[ https://issues.apache.org/jira/browse/SENTRY-2562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihua Deng updated SENTRY-2562: Attachment: SENTRY-2562.002.patch > Add connection and session timeout for zookeeper > - > > Key: SENTRY-2562 > URL: https://issues.apache.org/jira/browse/SENTRY-2562 > Project: Sentry > Issue Type: Improvement >Reporter: Zhihua Deng >Priority: Major > Attachments: SENTRY-2562.001.patch, SENTRY-2562.002.patch > > > Right now there is no other way to configure the zookeeper session or > connection timeout. This options should be better to provide to allow for > fast failing of zookeeper in case. -- This message was sent by Atlassian Jira (v8.3.4#803005)
