Csaba Ringhofer has uploaded a new patch set (#2). ( http://gerrit.cloudera.org:8080/10116 )
Change subject: IMPALA-6131: Track time of last statistics update in metadata ...................................................................... IMPALA-6131: Track time of last statistics update in metadata The timestamp of the last COMPUTE STATS operation is saved to table property "impala.lastComputeStatsTime". The format is the same as in "transient_lastDdlTime", so the two can be compared to check if the schema has changed since computing statistics. Other changes: - handling of "transient_lastDdlTime" is simplified to use HMS's default mechanism by removing this property before calling alter_table() - computing / setting stats on HDFS tables no longer increases "transient_lastDdlTime" - for Kudu behavior, see notes Notes: - Kudu tables work a bit differently, because they overwrite their HMS representation whenever they are reloaded from Kudu. This means that "transient_lastDdlTime" is increased during several operations, for example any operation that loads the table after INVALIDATE METADATA. I did not change this behavior. Kudu will be ablo sync its tables to HMS in the near future (see KUDU-2191), so the Kudu metadata handling in Impala may need to redesigned. Testing: tests/metadata/test_last_ddl_time_update.py is extended by - also checking "impala.lastComputeStatsTime" - testing more SQL statements - tests for Kudu tables Note that test_last_ddl_time_update.py is ran only in exhaustive testing. Change-Id: I59a671ac29d352bd92ce40d5cb6662bb23f146b5 --- M fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java M fe/src/main/java/org/apache/impala/catalog/KuduTable.java M fe/src/main/java/org/apache/impala/catalog/Table.java M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java M tests/metadata/test_last_ddl_time_update.py 5 files changed, 105 insertions(+), 116 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/16/10116/2 -- To view, visit http://gerrit.cloudera.org:8080/10116 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I59a671ac29d352bd92ce40d5cb6662bb23f146b5 Gerrit-Change-Number: 10116 Gerrit-PatchSet: 2 Gerrit-Owner: Csaba Ringhofer <[email protected]>
