Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/16392 )
Change subject: IMPALA-10075: Reuse unchanged partition instances ...................................................................... IMPALA-10075: Reuse unchanged partition instances Currently, we always update the partition instance when we reload a partition. If a partition remains the same after reloading, we should reuse the old partition instance. So we won't send redundant updates on these partitions. This reduces the size of the catalog topic update. When a huge table is REFRESHed, catalogd only propagates the changed partitions. Tests: - Add tests to verify that partition instances are reused after some DDL/DMLs. Change-Id: I2dd645c260d271291021e52fdac4b74924df1170 Reviewed-on: http://gerrit.cloudera.org:8080/16392 Reviewed-by: Impala Public Jenkins <[email protected]> Tested-by: Impala Public Jenkins <[email protected]> --- M fe/src/main/java/org/apache/impala/catalog/FileMetadataLoader.java M fe/src/main/java/org/apache/impala/catalog/HdfsPartition.java M fe/src/main/java/org/apache/impala/catalog/HdfsTable.java M fe/src/main/java/org/apache/impala/catalog/ParallelFileMetadataLoader.java A tests/metadata/test_reuse_partitions.py 5 files changed, 198 insertions(+), 10 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/16392 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I2dd645c260d271291021e52fdac4b74924df1170 Gerrit-Change-Number: 16392 Gerrit-PatchSet: 13 Gerrit-Owner: Quanlong Huang <[email protected]> Gerrit-Reviewer: Csaba Ringhofer <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Qifan Chen <[email protected]> Gerrit-Reviewer: Quanlong Huang <[email protected]> Gerrit-Reviewer: Vihang Karajgaonkar <[email protected]>
