Impala Public Jenkins has submitted this change and it was merged. (
http://gerrit.cloudera.org:8080/18224 )
Change subject: IMPALA-10948: Default scale and DecimalType
......................................................................
IMPALA-10948: Default scale and DecimalType
Added default 0 for scale if it is not set to comply with parquet spec.
Wrapped reading scale and precision in a function to support reading
LogicalType.DecimalType if it is set, falling back to old ones if it is
not, for backward compatibility.
Regenerated bad_parquet_decimals table with filled DecimalType, moved
missing scale test, as it is no longer a bad table.
Added no_scale.parquet table to test reading table without set scale.
Checked it with parquet-tools:
message schema {
optional fixed_len_byte_array(2) d1 (DECIMAL(4,0));
}
Change-Id: I003220b6e2ef39d25d1c33df62c8432803fdc6eb
Reviewed-on: http://gerrit.cloudera.org:8080/18224
Reviewed-by: Zoltan Borok-Nagy <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
---
M be/src/exec/parquet/parquet-data-converter.h
M be/src/exec/parquet/parquet-metadata-utils.cc
M testdata/bad_parquet_data/README
M testdata/bad_parquet_data/illegal_decimals.parq
M testdata/data/README
A testdata/data/no_scale.parquet
A testdata/workloads/functional-query/queries/QueryTest/default-scale.test
M
testdata/workloads/functional-query/queries/QueryTest/parquet-abort-on-error.test
M
testdata/workloads/functional-query/queries/QueryTest/parquet-continue-on-error.test
M tests/query_test/test_scanners.py
10 files changed, 126 insertions(+), 48 deletions(-)
Approvals:
Zoltan Borok-Nagy: Looks good to me, approved
Impala Public Jenkins: Verified
--
To view, visit http://gerrit.cloudera.org:8080/18224
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I003220b6e2ef39d25d1c33df62c8432803fdc6eb
Gerrit-Change-Number: 18224
Gerrit-PatchSet: 7
Gerrit-Owner: Gergely Fürnstáhl <[email protected]>
Gerrit-Reviewer: Gergely Fürnstáhl <[email protected]>
Gerrit-Reviewer: Impala Public Jenkins <[email protected]>
Gerrit-Reviewer: Zoltan Borok-Nagy <[email protected]>
