Anurag Mantripragada has uploaded a new patch set (#3). ( http://gerrit.cloudera.org:8080/15263 )
Change subject: IMPALA-9369: Make createInsertEvents() async. ...................................................................... IMPALA-9369: Make createInsertEvents() async. This patch makes the createInsertEvents() method async to avoid blocking the insert code path for long periods for tables with large number of partitions and files. Currently the createInsertEvents() method fires the HMS insert event one partition at a time. This makes insert statements with thousands of new files significantly slower. This change makes the createInsertEvent() call asynchronous by making it run in a separate thread. Testing: - Ran MetastoreEventsProcessorTest#testInsertEvents. - Ran test_events_processing::test_insert_events. Change-Id: I97802a5c03abc067fccf9e3a9d0047324626706e --- M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java M fe/src/main/java/org/apache/impala/util/MetaStoreUtil.java M fe/src/test/java/org/apache/impala/catalog/events/MetastoreEventsProcessorTest.java 3 files changed, 84 insertions(+), 33 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/63/15263/3 -- To view, visit http://gerrit.cloudera.org:8080/15263 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I97802a5c03abc067fccf9e3a9d0047324626706e Gerrit-Change-Number: 15263 Gerrit-PatchSet: 3 Gerrit-Owner: Anurag Mantripragada <[email protected]> Gerrit-Reviewer: Anurag Mantripragada <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Quanlong Huang <[email protected]> Gerrit-Reviewer: Vihang Karajgaonkar <[email protected]>
