Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/15263 )
Change subject: IMPALA-9369: Make createInsertEvents() async. ...................................................................... IMPALA-9369: Make createInsertEvents() async. This patch makes the createInsertEvents() method async to avoid blocking the insert code path for long periods for tables with large number of partitions and files. Currently the createInsertEvents() method fires the HMS insert event one partition at a time. This makes insert statements with thousands of new files significantly slower. This change makes the createInsertEvent() call asynchronous by making it run in a separate thread. Testing: - Ran MetastoreEventsProcessorTest#testInsertEvents. - Ran test_events_processing::test_insert_events. Change-Id: I97802a5c03abc067fccf9e3a9d0047324626706e Reviewed-on: http://gerrit.cloudera.org:8080/15263 Reviewed-by: Impala Public Jenkins <[email protected]> Tested-by: Impala Public Jenkins <[email protected]> --- M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java M fe/src/main/java/org/apache/impala/util/MetaStoreUtil.java M fe/src/test/java/org/apache/impala/catalog/events/MetastoreEventsProcessorTest.java 3 files changed, 84 insertions(+), 33 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/15263 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I97802a5c03abc067fccf9e3a9d0047324626706e Gerrit-Change-Number: 15263 Gerrit-PatchSet: 5 Gerrit-Owner: Anurag Mantripragada <[email protected]> Gerrit-Reviewer: Anurag Mantripragada <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Quanlong Huang <[email protected]> Gerrit-Reviewer: Vihang Karajgaonkar <[email protected]>
