Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/22197 )
Change subject: IMPALA-13599: Reduce the number of interactions with alter_partition() HMS API ...................................................................... IMPALA-13599: Reduce the number of interactions with alter_partition() HMS API Drop incremental stats and set/unset cached operations in Impala are calling alter_partition() HMS API as many times as the number of partitions in the table. This patch reduces these unnecessary interactions by calling alter_partitions() HMS API to #partitions/500 times regardless of number of partitions, since we have a limit of 500 partitions per HMS RPC in bulkAlterPartitions(). Note: This doesn't change the number of ALTER_PARTITION events generated by HMS. Once HIVE-27746 is included in the impala build, this patch further benefits by generating #partitions/500 ALTER_PARTITIONS events. Testing: - Added an end-to-end test to verify that HMS API is called only once. Change-Id: I2f2f1d9637e8be9c931da0415a17dd0839637e4c Reviewed-on: http://gerrit.cloudera.org:8080/22197 Reviewed-by: Impala Public Jenkins <[email protected]> Tested-by: Impala Public Jenkins <[email protected]> --- M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java M tests/custom_cluster/test_events_custom_configs.py 2 files changed, 38 insertions(+), 13 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/22197 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I2f2f1d9637e8be9c931da0415a17dd0839637e4c Gerrit-Change-Number: 22197 Gerrit-PatchSet: 7 Gerrit-Owner: Sai Hemanth Gantasala <[email protected]> Gerrit-Reviewer: Anonymous Coward <[email protected]> Gerrit-Reviewer: Csaba Ringhofer <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Quanlong Huang <[email protected]> Gerrit-Reviewer: Sai Hemanth Gantasala <[email protected]>
