Hello Quanlong Huang, Kurt Deschler, Vihang Karajgaonkar, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/16516
to look at the new patch set (#6).
Change subject: IMPALA-10189: don't reload partitions for drop stats
......................................................................
IMPALA-10189: don't reload partitions for drop stats
This patch avoids reloading partition metadata from the
HMS or filesystems when partition stats are changed,
by a DROP STATS command.
bulkAlterPartitions() is modified to allow callers to
control whether partitions are marked dirty (and therefore
reloaded).
DDLs that do not mark partitions dirty need to correctly update
the in-memory state of the Partition object so that it will
be the same as it would be after a reload from the HMS.
In this case, numRows_ was not previously updated, but now
needs to be updated.
Testing:
* Ran exhaustive tests
* Manually tested some scenarios with modifications to partitions
do in Hive outside of Impala. The previous behaviour is preserved
(i.e. cached partition state in Impala overwrites the Hive
modifictions).
* Manually tested dropping and computing regular and incremental
stats and inspected the results in both Impala and Hive. We
already have automated test coverage for these scenarios as well.
Change-Id: I91009d6e9fdf25b3a3341fd1d29219519eb39a3d
---
M fe/src/main/java/org/apache/impala/catalog/HdfsPartition.java
M fe/src/main/java/org/apache/impala/catalog/HdfsTable.java
M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java
3 files changed, 79 insertions(+), 37 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/16/16516/6
--
To view, visit http://gerrit.cloudera.org:8080/16516
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I91009d6e9fdf25b3a3341fd1d29219519eb39a3d
Gerrit-Change-Number: 16516
Gerrit-PatchSet: 6
Gerrit-Owner: Tim Armstrong <tarmstr...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com>
Gerrit-Reviewer: Kurt Deschler <kdesc...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <huangquanl...@gmail.com>
Gerrit-Reviewer: Tim Armstrong <tarmstr...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vih...@cloudera.com>
