Naveen Gangam created HIVE-27595:
------------------------------------
Summary: Improve efficiency in the filtering hooks
Key: HIVE-27595
URL: https://issues.apache.org/jira/browse/HIVE-27595
Project: Hive
Issue Type: Improvement
Components: HiveServer2
Affects Versions: 4.0.0-alpha-2
Reporter: Naveen Gangam
Assignee: Henri Biestro
https://github.com/apache/hive/blob/a406d6d4417277e45b93f1733bed5201afdee29b/ql/src/java/org/apache/hadoop/hive/ql/security/authorization/plugin/metastore/HiveMetaStoreAuthorizer.java#L353-L377
In case where the tableList has large amounts of tables (tested with 200k in my
case), the hivePrivilegedObjects could just as big. So both these lists are
200k.
Essentially. the code is trying to return a subset of tableList collection that
matches the objects returned in hivePrivilegedObjects. This results in a N*N
iteration that causes bad performance. (in my case, the HMS client timeout
expired and show tables failed).
This code needs to be optimized for performance.
we have a similar problem in this code as well.
ql/src/java/org/apache/hadoop/hive/ql/security/authorization/plugin/AuthorizationMetaStoreFilterHook.java
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
