[
https://issues.apache.org/jira/browse/IMPALA-10189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17209569#comment-17209569
]
ASF subversion and git services commented on IMPALA-10189:
----------------------------------------------------------
Commit d6f2dcab79bc80448f3241ddf24d3259aa3919cd in impala's branch
refs/heads/master from Tim Armstrong
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=d6f2dca ]
IMPALA-10189: don't reload partitions for drop stats
This patch avoids reloading partition metadata from the
HMS or filesystems when partition stats are changed,
by a DROP STATS command.
bulkAlterPartitions() is modified to allow callers to
control whether partitions are marked dirty (and therefore
reloaded).
DDLs that do not mark partitions dirty need to correctly update
the in-memory state of the Partition object so that it will
be the same as it would be after a reload from the HMS.
In this case, numRows_ was not previously updated, but now
needs to be updated.
Testing:
* Ran exhaustive tests
* Manually tested some scenarios with modifications to partitions
do in Hive outside of Impala. The previous behaviour is preserved
(i.e. cached partition state in Impala overwrites the Hive
modifictions).
* Manually tested dropping and computing regular and incremental
stats and inspected the results in both Impala and Hive. We
already have automated test coverage for these scenarios as well.
Change-Id: I91009d6e9fdf25b3a3341fd1d29219519eb39a3d
Reviewed-on: http://gerrit.cloudera.org:8080/16516
Reviewed-by: Impala Public Jenkins <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> Avoid unnecessarily loading metadata for drop stats DDL
> -------------------------------------------------------
>
> Key: IMPALA-10189
> URL: https://issues.apache.org/jira/browse/IMPALA-10189
> Project: IMPALA
> Issue Type: Sub-task
> Components: Catalog
> Affects Versions: Impala 2.10.0, Impala 2.11.0, Impala 3.0, Impala 2.12.0,
> Impala 3.1.0, Impala 3.2.0, Impala 3.3.0, Impala 3.4.0
> Reporter: Tim Armstrong
> Assignee: Tim Armstrong
> Priority: Critical
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
