Internal Jenkins has submitted this change and it was merged. Change subject: IMPALA-4363: Add Parquet timestamp validation ......................................................................
IMPALA-4363: Add Parquet timestamp validation Before this patch, we would simply read the INT96 Parquet timestamp representation and assume that it's valid. However, not all bit permutations represent a valid timestamp. One of the boost functions raised an exception (that we didn't catch) when passed an invalid boost date object, which resulted in a crash. This patch fixes problem by validating that the date falls into 1400..9999 year range as we are scanning Parquet. Change-Id: Ieaab5d33e6f0df831d0e67e1d318e5416ffb90ac Reviewed-on: http://gerrit.cloudera.org:8080/5343 Reviewed-by: Taras Bobrovytsky <tbobrovyt...@cloudera.com> Tested-by: Internal Jenkins --- M be/src/exec/parquet-column-readers.cc M be/src/exec/parquet-column-readers.h M be/src/runtime/timestamp-value.h M common/thrift/generate_error_codes.py M testdata/data/README A testdata/data/out_of_range_timestamp.parquet M testdata/workloads/functional-query/queries/QueryTest/grant_revoke.test A testdata/workloads/functional-query/queries/QueryTest/out-of-range-timestamp-abort-on-error.test A testdata/workloads/functional-query/queries/QueryTest/out-of-range-timestamp-continue-on-error.test M testdata/workloads/functional-query/queries/QueryTest/parquet-resolution-by-name.test M testdata/workloads/functional-query/queries/QueryTest/parquet.test M tests/common/impala_test_suite.py M tests/query_test/test_scanners.py 13 files changed, 141 insertions(+), 36 deletions(-) Approvals: Taras Bobrovytsky: Looks good to me, approved Internal Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/5343 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: merged Gerrit-Change-Id: Ieaab5d33e6f0df831d0e67e1d318e5416ffb90ac Gerrit-PatchSet: 3 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Taras Bobrovytsky <tbobrovyt...@cloudera.com> Gerrit-Reviewer: Alex Behm <alex.b...@cloudera.com> Gerrit-Reviewer: Dan Hecht <dhe...@cloudera.com> Gerrit-Reviewer: Internal Jenkins Gerrit-Reviewer: Taras Bobrovytsky <tbobrovyt...@cloudera.com> Gerrit-Reviewer: Tim Armstrong <tarmstr...@cloudera.com>