Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/14846 )
Change subject: IMPALA-9195: Using multithreaded execution to accelerate 'show tables/databases' ...................................................................... Patch Set 7: (1 comment) http://gerrit.cloudera.org:8080/#/c/14846/7/fe/src/main/java/org/apache/impala/service/Frontend.java File fe/src/main/java/org/apache/impala/service/Frontend.java: http://gerrit.cloudera.org:8080/#/c/14846/7/fe/src/main/java/org/apache/impala/service/Frontend.java@859 PS7, Line 859: submit > You are right. All the check tasks created concurrently by the users when e I am ok with the static thread pool if we ensure that we do not regress too much for the default case. This could be done by Guava's MoreExecutors.sameThreadExecutor(), Impala already uses it: https://github.com/apache/impala/blob/f2f348c0f93208a0f34c33b6a4dc82f4d9d4b290/fe/src/main/java/org/apache/impala/catalog/ParallelFileMetadataLoader.java#L141 This would mean that by default the tasks will be executed on the current thread, which should add minimal overhead to the current solution. Meanwhile bumping num_check_access_threads could help some users with SHOW DATABASES/TABLES slowness if the number of parallel calls is not too large. If there are lot of parallel SHOW DATABASES/TABLES and they are also slow, then I think that there is not much we can do, the user should add more Coordinator nodes. -- To view, visit http://gerrit.cloudera.org:8080/14846 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I860e0d18afa0421665f8b3b1c5561d6bdacc5e96 Gerrit-Change-Number: 14846 Gerrit-PatchSet: 7 Gerrit-Owner: Zhou Xu <[email protected]> Gerrit-Reviewer: Csaba Ringhofer <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Quanlong Huang <[email protected]> Gerrit-Reviewer: Tim Armstrong <[email protected]> Gerrit-Reviewer: Vihang Karajgaonkar <[email protected]> Gerrit-Reviewer: Zhou Xu <[email protected]> Gerrit-Comment-Date: Tue, 17 Dec 2019 13:51:05 +0000 Gerrit-HasComments: Yes
