Attila Jeges has posted comments on this change. ( http://gerrit.cloudera.org:8080/14032 )
Change subject: WIP: IMPALA-8842: Change column stat HMS calls to accommodate HIVE-22046. ...................................................................... Patch Set 2: (3 comments) http://gerrit.cloudera.org:8080/#/c/14032/1//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/14032/1//COMMIT_MSG@7 PS1, Line 7: WIP: > Can you add information about why is this in WIP state? The change seems co Done http://gerrit.cloudera.org:8080/#/c/14032/1/tests/metadata/test_hms_integration.py File tests/metadata/test_hms_integration.py: http://gerrit.cloudera.org:8080/#/c/14032/1/tests/metadata/test_hms_integration.py@893 PS1, Line 893: '(x int)' > Can you also test what happens with partitioned tables? Hive stores column stats for partitioned tables in PART_COL_STATS while Impala uses TAB_COL_STATS (as described in L884-886), so in case of partition tables the Impala/Hive column statistics were properly separated even before this patch. I can still add a partition-table test if you think it is necessary. http://gerrit.cloudera.org:8080/#/c/14032/1/tests/metadata/test_hms_integration.py@902 PS1, Line 902: # Impala doesn't read column stats written by Hive. : self.client.execute('invalidate metadata %s' % table_name) : impala_stats = self.impala_all_column_stats(table_name) : assert '-1' == impala_stats['x']['#nulls'] : assert '-1' == impala_stats['x']['ndv'] > Agreed, I think this is important to think about, either to provide documen As discussed with the other stakeholders for now we will keep the stats separately for simplicity/transparency reasons. We can revisit this behavior later. -- To view, visit http://gerrit.cloudera.org:8080/14032 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia782bda3891dee1281cfcd5cf8817aa1226217ad Gerrit-Change-Number: 14032 Gerrit-PatchSet: 2 Gerrit-Owner: Attila Jeges <[email protected]> Gerrit-Reviewer: Attila Jeges <[email protected]> Gerrit-Reviewer: Bharath Vissapragada <[email protected]> Gerrit-Reviewer: Csaba Ringhofer <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Tim Armstrong <[email protected]> Gerrit-Comment-Date: Thu, 08 Aug 2019 15:53:34 +0000 Gerrit-HasComments: Yes
