Hello Qifan Chen, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/16392
to look at the new patch set (#3).
Change subject: [WIP] MPALA-10075: Reuse unchanged partition instances
......................................................................
[WIP] MPALA-10075: Reuse unchanged partition instances
Currently, we always update the partition instance when we reload a
partition. If a partition remains the same after reloading, we should
reuse the old partition instance. So we won't send redudant updates on
these partitions. This reduces the size of the catalog topic update.
When a huge table is REFRESHed, catalogd only propagates the changes
partitions.
Tests:
- Manually verified that repeate REFERSHs won't trigger catalog updates
on partitions.
- Still adding more tests
Change-Id: I2dd645c260d271291021e52fdac4b74924df1170
---
M fe/src/main/java/org/apache/impala/catalog/FileMetadataLoader.java
M fe/src/main/java/org/apache/impala/catalog/HdfsPartition.java
M fe/src/main/java/org/apache/impala/catalog/HdfsTable.java
M fe/src/main/java/org/apache/impala/catalog/ParallelFileMetadataLoader.java
A tests/metadata/test_reuse_partitions.py
5 files changed, 158 insertions(+), 6 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/92/16392/3
--
To view, visit http://gerrit.cloudera.org:8080/16392
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I2dd645c260d271291021e52fdac4b74924df1170
Gerrit-Change-Number: 16392
Gerrit-PatchSet: 3
Gerrit-Owner: Quanlong Huang <huangquanl...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com>
Gerrit-Reviewer: Qifan Chen <qc...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <huangquanl...@gmail.com>
