Internal Jenkins has submitted this change and it was merged. Change subject: IMPALA-4387: validate decimal type in Avro file schema ......................................................................
IMPALA-4387: validate decimal type in Avro file schema This patch prevents an invalid decimal type in an Avro file schema from crashing Impala. Most invalid Avro schemas are caught by the frontend, but file schemas still need to be validated by the backend. After this patch files with bad schemas are skipped. Testing: This was hit very rarely by the scanner fuzzing. Added a regression test that scans a file with a bad schema. Change-Id: I25a326ee2220bc14d3b5f887dc288b4adf859cfc Reviewed-on: http://gerrit.cloudera.org:8080/4876 Reviewed-by: Tim Armstrong <[email protected]> Tested-by: Internal Jenkins --- M be/src/exec/hdfs-avro-scanner.cc M be/src/runtime/decimal-test.cc M be/src/runtime/types.h M be/src/util/avro-util.cc M be/src/util/avro-util.h M common/thrift/generate_error_codes.py M testdata/bad_avro_snap/README A testdata/bad_avro_snap/invalid_decimal_schema.avro M testdata/datasets/functional/functional_schema_template.sql M testdata/datasets/functional/schema_constraints.csv M testdata/workloads/functional-query/queries/DataErrorsTest/avro-errors.test 11 files changed, 104 insertions(+), 27 deletions(-) Approvals: Internal Jenkins: Verified Tim Armstrong: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/4876 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: merged Gerrit-Change-Id: I25a326ee2220bc14d3b5f887dc288b4adf859cfc Gerrit-PatchSet: 5 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Tim Armstrong <[email protected]> Gerrit-Reviewer: Dan Hecht <[email protected]> Gerrit-Reviewer: Internal Jenkins Gerrit-Reviewer: Lars Volker <[email protected]> Gerrit-Reviewer: Tim Armstrong <[email protected]> Gerrit-Reviewer: Zoltan Ivanfi <[email protected]>
