Kurt Deschler has posted comments on this change. ( http://gerrit.cloudera.org:8080/23436 )
Change subject: IMPALA-14447: Parallelize table loading in getMissingTables() ...................................................................... Patch Set 11: (4 comments) http://gerrit.cloudera.org:8080/#/c/23436/11//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/23436/11//COMMIT_MSG@23 PS11, Line 23: Testing: Any performance resutls to share here? http://gerrit.cloudera.org:8080/#/c/23436/11/fe/src/main/java/org/apache/impala/analysis/StmtMetadataLoader.java File fe/src/main/java/org/apache/impala/analysis/StmtMetadataLoader.java: http://gerrit.cloudera.org:8080/#/c/23436/11/fe/src/main/java/org/apache/impala/analysis/StmtMetadataLoader.java@403 PS11, Line 403: } finally { Should catch RejectedExecutionException and fall through to original logic. Should also use the original logic if LoaderThreads<=1 to avoid requiring extra threads and allow disabling the new code. http://gerrit.cloudera.org:8080/#/c/23436/11/fe/src/main/java/org/apache/impala/catalog/local/LocalCatalog.java File fe/src/main/java/org/apache/impala/catalog/local/LocalCatalog.java: http://gerrit.cloudera.org:8080/#/c/23436/11/fe/src/main/java/org/apache/impala/catalog/local/LocalCatalog.java@118 PS11, Line 118: Map<String, FeDb> dbs = new ConcurrentHashMap<>(); This probably does not need to be ConcurrentHashMap since loading is synchronized. Could use unmodifiableMap or ImmutableMap for safety. http://gerrit.cloudera.org:8080/#/c/23436/11/fe/src/main/java/org/apache/impala/catalog/local/LocalDb.java File fe/src/main/java/org/apache/impala/catalog/local/LocalDb.java: http://gerrit.cloudera.org:8080/#/c/23436/11/fe/src/main/java/org/apache/impala/catalog/local/LocalDb.java@202 PS11, Line 202: Map<String, FeTable> newMap = new ConcurrentHashMap<>(); This probably does not need to be ConcurrentHashMap since loading is synchronized. Could use unmodifiableMap or ImmutableMap for safety. -- To view, visit http://gerrit.cloudera.org:8080/23436 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I97a5165844ae846b28338d62e93a20121488d79f Gerrit-Change-Number: 23436 Gerrit-PatchSet: 11 Gerrit-Owner: Riza Suminto <[email protected]> Gerrit-Reviewer: Fang-Yu Rao <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Kurt Deschler <[email protected]> Gerrit-Reviewer: Michael Smith <[email protected]> Gerrit-Reviewer: Quanlong Huang <[email protected]> Gerrit-Reviewer: Riza Suminto <[email protected]> Gerrit-Reviewer: Steve Carlin <[email protected]> Gerrit-Comment-Date: Mon, 06 Oct 2025 19:32:24 +0000 Gerrit-HasComments: Yes
