[
https://issues.apache.org/jira/browse/IMPALA-9195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16981473#comment-16981473
]
Quanlong Huang commented on IMPALA-9195:
----------------------------------------
[~xuzhou] Could you share the scale of the number of privileges and roles in
your environment?
IMPALA-9002 also targets to address this problem. Most of the privilege checks
are redundant in practise. So it adds a flag to skip some checks. However, it's
a pity that it only focus on SHOW TABLES/GET_TABLES(), not SHOW
DATABASES/GET_SCHEMAS().
For Ranger, I've done some tests it and don't find this performance issue. Some
Sentry codes need to be optimized.
MT_DOP is used in the backend execution. It's a good idea to reuse it in SHOW
TABLES. Contribution is always welcome!
> Using multithreaded execution to accelerate ‘show tables/databases’
> -------------------------------------------------------------------
>
> Key: IMPALA-9195
> URL: https://issues.apache.org/jira/browse/IMPALA-9195
> Project: IMPALA
> Issue Type: Improvement
> Components: Frontend
> Reporter: xuzhou
> Priority: Critical
>
> Impala version: 2.12
> Using sentry for authentication
> While users with multi group-policies(group-policy may be nested) executing
> 'show tables/databases'，it seems to be awful with a long latency. In my case,
> the database has 910 tables, the user waiting 65.886 seconds to get 160
> tables.
> I study the code and find that while executing Frontend.getTableNames:
> for table in tables:
> for action in actions(all actions defined in DBModelAction):
> ResourceAuthorizationProvider.hasAccess
> It seems that 'hasAccess' is responsable for bad performance while checking
> users with complex group-policies.
> I tried to use 16 threads in getTablesNames and it costs 4.752 seconds in my
> case.
> The code seems to be the same while using sentry service in the latest
> impala. I'm not sure that if any promotion has been done in the latest sentry
> service as I failed to migrate file-based sentry authentication to the sentry
> service. I see that ranger is supported in the latest impala, does ranger
> have the similar problem?
> It seems 'show tables/databases' can benefit from multithreaded execution
> while using sentry , is it reasonable to support such operations in query
> option MT_DOP?
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
