Sahil Takiar has uploaded this change for review. ( http://gerrit.cloudera.org:8080/12163
Change subject: IMPALA-7087: Read Parquet decimal columns with lower precision/scale ...................................................................... IMPALA-7087: Read Parquet decimal columns with lower precision/scale Allows Impala to read Parquet files that have been written with a lower precision / scale compared to the precision / scale specified during table creation. Currently, if the precision / scale in the table does not exactly match the precision / scale in the underlying Parquet files, Impala will throw an error during any attempt to read data from the table. This patch only allows reading Parquet files with a *lower* precision / scale compared to the table metadata. Users still get an error if they try to read higher precision / scale data into a lower precision / scale. The ability to read lower precision / scale data is allowed by the SQL Standard and several other engines, such as Hive, MySQL, and Postgres, allow for this behavior. If the underlying Parquet files contain rows that cannot be converted to the higher scale without overflow, an error is thrown and the query fails. This is different from other engines such as Hive which set the row to NULL and allow the query to run to completion without error. Testing: * Ran core tests * Added new tests in test_scanners.py Change-Id: Iafc8efd12379a39756e3e70f022a81a636dadb61 --- M be/src/exec/parquet/parquet-column-readers.cc M be/src/exec/parquet/parquet-metadata-utils.cc M common/thrift/generate_error_codes.py M testdata/data/README A testdata/data/binary_decimal_precision_and_scale_widening.parquet A testdata/workloads/functional-query/queries/QueryTest/parquet-decimal-precision-and-scale-widening.test M tests/query_test/test_scanners.py 7 files changed, 357 insertions(+), 9 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/63/12163/1 -- To view, visit http://gerrit.cloudera.org:8080/12163 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Iafc8efd12379a39756e3e70f022a81a636dadb61 Gerrit-Change-Number: 12163 Gerrit-PatchSet: 1 Gerrit-Owner: Sahil Takiar <stak...@cloudera.com>