Quanlong Huang has uploaded this change for review. ( http://gerrit.cloudera.org:8080/21445
Change subject: IMPALA-13102: Normalize invalid column stats in HMS ...................................................................... IMPALA-13102: Normalize invalid column stats in HMS Column stats like numDVs, numNulls in HMS could have arbitrary values. Impala expects them to be non-negative or -1 for unknown. So loading tables with invalid stats values (<-1) will fail. This patch adds logic to normalize the stats values. If the value < -1, use -1 for it and add corresponding warning logs. Also refactor some redundant codes in ColumnStats. Tests: - Add e2e test Change-Id: If6216e3d6e73a529a9b3a8c0ea9d22727ab43f1a --- M fe/src/main/java/org/apache/impala/analysis/AlterTableSetColumnStats.java M fe/src/main/java/org/apache/impala/catalog/Column.java M fe/src/main/java/org/apache/impala/catalog/ColumnStats.java M fe/src/main/java/org/apache/impala/catalog/FeCatalogUtils.java M tests/metadata/test_compute_stats.py 5 files changed, 147 insertions(+), 73 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/45/21445/1 -- To view, visit http://gerrit.cloudera.org:8080/21445 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: If6216e3d6e73a529a9b3a8c0ea9d22727ab43f1a Gerrit-Change-Number: 21445 Gerrit-PatchSet: 1 Gerrit-Owner: Quanlong Huang <[email protected]>
