[
https://issues.apache.org/jira/browse/IMPALA-9195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zoltán Borók-Nagy reassigned IMPALA-9195:
-----------------------------------------
Assignee: xuzhou
> Using multithreaded execution to accelerate ‘show tables/databases’
> -------------------------------------------------------------------
>
> Key: IMPALA-9195
> URL: https://issues.apache.org/jira/browse/IMPALA-9195
> Project: IMPALA
> Issue Type: Improvement
> Components: Frontend
> Reporter: xuzhou
> Assignee: xuzhou
> Priority: Critical
>
> Impala version: 2.12
> Using sentry for authentication
> While users with multi group-policies(group-policy may be nested) executing
> 'show tables/databases'，it seems to be awful with a long latency. In my case,
> the database has 910 tables, the user waiting 65.886 seconds to get 160
> tables.
> I study the code and find that while executing Frontend.getTableNames:
> for table in tables:
> for action in actions(all actions defined in DBModelAction):
> ResourceAuthorizationProvider.hasAccess
> It seems that 'hasAccess' is responsable for bad performance while checking
> users with complex group-policies.
> I tried to use 16 threads in getTablesNames and it costs 4.752 seconds in my
> case.
> The code seems to be the same while using sentry service in the latest
> impala. I'm not sure that if any promotion has been done in the latest sentry
> service as I failed to migrate file-based sentry authentication to the sentry
> service. I see that ranger is supported in the latest impala, does ranger
> have the similar problem?
> It seems 'show tables/databases' can benefit from multithreaded execution
> while using sentry , is it reasonable to support such operations in query
> option MT_DOP?
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
