Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/23159 )
Change subject: IMPALA-14082: Support batch processing of RELOAD events on same table ...................................................................... IMPALA-14082: Support batch processing of RELOAD events on same table Currently, RELOAD events of partitioned table are processed one after the other. Processing them one by one acquires the table lock multiple times to load individual partitions in sequence. This also keeps the table version changing which impacts performance of coordinators in local-catalog mode - query planning needs retry to handle InconsistentMetadataFetchException due to table version changes. This patch handles the batch processing logic RELOAD events on same table by reusing the exisiting logic of BatchPartitionEvent. This implementation adds four new methods canBeBatched(),addToBatchEvents(), getPartitionForBatching(), getBatchEventType()(pre-requisites to reuse batching logic) to the RELOAD event class. Testing: - Added an end-to-end to verify the batching. Change-Id: Ie3e9a99b666a1c928ac2a136bded1e5420f77dab Reviewed-on: http://gerrit.cloudera.org:8080/23159 Reviewed-by: Impala Public Jenkins <[email protected]> Tested-by: Impala Public Jenkins <[email protected]> --- M fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java M tests/custom_cluster/test_events_custom_configs.py 3 files changed, 108 insertions(+), 10 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/23159 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Ie3e9a99b666a1c928ac2a136bded1e5420f77dab Gerrit-Change-Number: 23159 Gerrit-PatchSet: 14 Gerrit-Owner: Sai Hemanth Gantasala <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Quanlong Huang <[email protected]> Gerrit-Reviewer: Riza Suminto <[email protected]> Gerrit-Reviewer: Sai Hemanth Gantasala <[email protected]>
