Vihang Karajgaonkar has posted comments on this change. ( http://gerrit.cloudera.org:8080/15263 )
Change subject: IMPALA-9369: Make createInsertEvents() async. ...................................................................... Patch Set 2: (1 comment) http://gerrit.cloudera.org:8080/#/c/15263/2/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java File fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java: http://gerrit.cloudera.org:8080/#/c/15263/2/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java@4393 PS2, Line 4393: fireInsertEventsAsync(table.getDb().getName(), This implementation will spin up a singleThreadExecutor for each partition which was inserted which might be expensive. Can we collect all the insert events first and then call the async method so all the insert events are processed by same single thread? May be something like: CompletableFuture.runAsync(() -> { try (MetaStoreClient metaStoreClient = catalog_.getMetaStoreClient()) { for (InsertEventInfo info : insertEventInfos) { MetaStoreUtil.fireInsertEvent(info); } catch (Exception e) { LOG.error(...); }}, Executors.newSingleThreadExecutor()).thenRun(()->fireInsertEventThread.shutdown) }) -- To view, visit http://gerrit.cloudera.org:8080/15263 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I97802a5c03abc067fccf9e3a9d0047324626706e Gerrit-Change-Number: 15263 Gerrit-PatchSet: 2 Gerrit-Owner: Anurag Mantripragada <[email protected]> Gerrit-Reviewer: Anurag Mantripragada <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Quanlong Huang <[email protected]> Gerrit-Reviewer: Vihang Karajgaonkar <[email protected]> Gerrit-Comment-Date: Tue, 03 Mar 2020 00:52:01 +0000 Gerrit-HasComments: Yes
