Naveen Gangam created HIVE-27595: ------------------------------------ Summary: Improve efficiency in the filtering hooks Key: HIVE-27595 URL: https://issues.apache.org/jira/browse/HIVE-27595 Project: Hive Issue Type: Improvement Components: HiveServer2 Affects Versions: 4.0.0-alpha-2 Reporter: Naveen Gangam Assignee: Henri Biestro
https://github.com/apache/hive/blob/a406d6d4417277e45b93f1733bed5201afdee29b/ql/src/java/org/apache/hadoop/hive/ql/security/authorization/plugin/metastore/HiveMetaStoreAuthorizer.java#L353-L377 In case where the tableList has large amounts of tables (tested with 200k in my case), the hivePrivilegedObjects could just as big. So both these lists are 200k. Essentially. the code is trying to return a subset of tableList collection that matches the objects returned in hivePrivilegedObjects. This results in a N*N iteration that causes bad performance. (in my case, the HMS client timeout expired and show tables failed). This code needs to be optimized for performance. we have a similar problem in this code as well. ql/src/java/org/apache/hadoop/hive/ql/security/authorization/plugin/AuthorizationMetaStoreFilterHook.java -- This message was sent by Atlassian Jira (v8.20.10#820010)