Hello Daniel Vanko, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/24059
to look at the new patch set (#4).
Change subject: IMPALA-14700: Add support for Parquet LZ4_RAW compression
......................................................................
IMPALA-14700: Add support for Parquet LZ4_RAW compression
This change requires a Parquet Version higher than the current
one (1.12.3), because of the LZ4_RAW Thrift enum value.
For that reason, APACHE_PARQUET_VERSION is increased to 1.15.2 in this
patch, and is used instead of CDP_PARQUET_VERSION, until
CDP_PARQUET_VERSION gets to a high enough version.
Parquet deprecated LZ4 compression, and added a new one, LZ4_RAW.
This patch adds support for LZ4_RAW. It uses Lz4Compressor
(Corresponding to THdfsCompression::LZ4).
Testing:
-Added a small test file using lz4_raw compression, from the
parquet-testing repository.
-Added a test case to test_scanners.py to check we can read the file.
Change-Id: I22ee4e5bf9abec37be941c1dca8019a563343d34
---
M be/src/exec/parquet/parquet-common.cc
M be/src/exec/parquet/parquet-metadata-utils.cc
M be/src/util/codec.cc
M be/src/util/codec.h
M bin/impala-config.sh
M common/thrift/parquet.thrift
A testdata/data/lz4_raw_compressed.parquet
A
testdata/workloads/functional-query/queries/QueryTest/parquet-lz4-raw-compression.test
M tests/query_test/test_scanners.py
9 files changed, 39 insertions(+), 14 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/59/24059/4
--
To view, visit http://gerrit.cloudera.org:8080/24059
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I22ee4e5bf9abec37be941c1dca8019a563343d34
Gerrit-Change-Number: 24059
Gerrit-PatchSet: 4
Gerrit-Owner: Balazs Hevele <[email protected]>
Gerrit-Reviewer: Balazs Hevele <[email protected]>
Gerrit-Reviewer: Daniel Vanko <[email protected]>
Gerrit-Reviewer: Impala Public Jenkins <[email protected]>
