Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/15959 )
Change subject: IMPALA-9707: fix Parquet stat filtering when min/max values are cast to NULL ...................................................................... IMPALA-9707: fix Parquet stat filtering when min/max values are cast to NULL The min/max stat predicate is allowed when the left side is not a slot but an implicit cast of a slot. This could lead to incorrectly dropping a row group or page when min/max values were not castable to the type, e.g. it is string with a pre 1400 date and we want to cast it to a timestamp. The change should only affect timestamps, as dates return an error on failed cast from a string, and numeric types won't be cast implicitly from string. The fix is simply to accept NULL result for the min/max predicate in the backend. Note that the alternative solution of casting the right (const) side of the predicate instead of the left side would be tricky, as more than one string can mean the same timestamp, e.g. "1970-01-01" and "1970-01-01 00:00:00". Testing: - added an EE regression test and ran it Change-Id: I35f66e1dfc4523624c249073004f9d5eddd07bb6 Reviewed-on: http://gerrit.cloudera.org:8080/15959 Reviewed-by: Impala Public Jenkins <[email protected]> Tested-by: Impala Public Jenkins <[email protected]> --- M be/src/exec/parquet/hdfs-parquet-scanner.cc M be/src/exprs/scalar-expr-evaluator.h M testdata/workloads/functional-query/queries/QueryTest/parquet-stats.test 3 files changed, 22 insertions(+), 2 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/15959 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I35f66e1dfc4523624c249073004f9d5eddd07bb6 Gerrit-Change-Number: 15959 Gerrit-PatchSet: 3 Gerrit-Owner: Csaba Ringhofer <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Tim Armstrong <[email protected]>
