[
https://issues.apache.org/jira/browse/RANGER-4741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Madhan Neethiraj updated RANGER-4741:
-------------------------------------
Attachment: RANGER-4741.patch
> Hive plugin optimization to avoid excessive metastore API calls
> ---------------------------------------------------------------
>
> Key: RANGER-4741
> URL: https://issues.apache.org/jira/browse/RANGER-4741
> Project: Ranger
> Issue Type: Improvement
> Components: plugins
> Reporter: Madhan Neethiraj
> Assignee: Madhan Neethiraj
> Priority: Major
> Fix For: 3.0.0
>
> Attachments: RANGER-4741.patch
>
>
> Authorizing access to tables with large number of columns can take a long
> time, as shown below. Time taken to for a table with 400 columns takes about
> 100 seconds.
> {noformat}
> 0: jdbc:hive2://localhost:10000> SELECT * FROM large_tbl_4000;
> ...
> No rows selected (98.674 seconds)
> 0: jdbc:hive2://localhost:10000> SELECT * FROM large_tbl_1000;
> ...
> No rows selected (10.4 seconds)
> {noformat}
>
> For each column referenced in the query, Ranger Hive authorizer calls
> metastore API to obtain owner of the table. Optimizing to call the metastore
> API once per table can significantly reduce the time taken to authorize
> queries.
> Here is the time taken to query the same tables with the Ranger Hive
> authorizer optimized to call metastore API only once per table referenced in
> the query:
> {noformat}
> 0: jdbc:hive2://localhost:10000> SELECT * FROM large_tbl_4000;
> ...
> No rows selected (1.328 seconds)
> 0: jdbc:hive2://localhost:10000> SELECT * FROM large_tbl_1000;
> ...
> No rows selected (0.194 seconds)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
