Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/14032 )
Change subject: WIP: IMPALA-8842: Change column stat HMS calls to accommodate HIVE-22046. ...................................................................... Patch Set 1: (1 comment) http://gerrit.cloudera.org:8080/#/c/14032/1/tests/metadata/test_hms_integration.py File tests/metadata/test_hms_integration.py: http://gerrit.cloudera.org:8080/#/c/14032/1/tests/metadata/test_hms_integration.py@902 PS1, Line 902: # Impala doesn't read column stats written by Hive. : self.client.execute('invalidate metadata %s' % table_name) : impala_stats = self.impala_all_column_stats(table_name) : assert '-1' == impala_stats['x']['#nulls'] : assert '-1' == impala_stats['x']['ndv'] > It would be a change in design, but wouldn't it make sense to use stats wri Agreed, I think this is important to think about, either to provide documentation or a solution in the code. We can't avoid breaking some workflows, e.g. if a user depends on one engine's refresh/analyze to keep stats fresh for all engines, but there are also stale stats for the other engine. I think mostly we depend on users running Impala's compute stats on the tables they use with Impala - I believe they get column stats missing warnings if they don't do this since Hive's analyze (I think) doesn't fill in all the column stats. But yeah, we should consider what mix of documentation and code we use to solve this. -- To view, visit http://gerrit.cloudera.org:8080/14032 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia782bda3891dee1281cfcd5cf8817aa1226217ad Gerrit-Change-Number: 14032 Gerrit-PatchSet: 1 Gerrit-Owner: Attila Jeges <[email protected]> Gerrit-Reviewer: Csaba Ringhofer <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Tim Armstrong <[email protected]> Gerrit-Comment-Date: Thu, 08 Aug 2019 15:00:07 +0000 Gerrit-HasComments: Yes
