Hello Lars Volker, Zoltan Borok-Nagy,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/10116
to look at the new patch set (#3).
Change subject: IMPALA-6131: Track time of last statistics update in metadata
......................................................................
IMPALA-6131: Track time of last statistics update in metadata
The timestamp of the last COMPUTE STATS operation is saved to
table property "impala.lastComputeStatsTime". The format is
the same as in "transient_lastDdlTime", so the two can be
compared to check if the schema has changed since computing
statistics.
Other changes:
- handling of "transient_lastDdlTime" is simplified to use
HMS's default mechanism by removing this property before
calling alter_table()
- computing / setting stats on HDFS tables no longer increases
"transient_lastDdlTime" - for Kudu behavior, see notes
Notes:
- Kudu tables work a bit differently, because they overwrite
their HMS representation whenever they are reloaded from
Kudu. This means that "transient_lastDdlTime" is increased
during several operations, for example any operation that
loads the table after INVALIDATE METADATA. I did not change
this behavior. Kudu will be able to sync its tables to HMS
in the near future (see KUDU-2191), so the Kudu metadata
handling in Impala may need to be redesigned.
Testing:
tests/metadata/test_last_ddl_time_update.py is extended by
- also checking "impala.lastComputeStatsTime"
- testing more SQL statements
- tests for Kudu tables
Note that test_last_ddl_time_update.py is ran only in
exhaustive testing.
Change-Id: I59a671ac29d352bd92ce40d5cb6662bb23f146b5
---
M fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java
M fe/src/main/java/org/apache/impala/catalog/KuduTable.java
M fe/src/main/java/org/apache/impala/catalog/Table.java
M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java
M tests/metadata/test_last_ddl_time_update.py
5 files changed, 105 insertions(+), 116 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/16/10116/3
--
To view, visit http://gerrit.cloudera.org:8080/10116
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I59a671ac29d352bd92ce40d5cb6662bb23f146b5
Gerrit-Change-Number: 10116
Gerrit-PatchSet: 3
Gerrit-Owner: Csaba Ringhofer <[email protected]>
Gerrit-Reviewer: Lars Volker <[email protected]>
Gerrit-Reviewer: Zoltan Borok-Nagy <[email protected]>
