Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/11949 )
Change subject: IMPALA-7869: break up parquet-column-readers.cc ...................................................................... IMPALA-7869: break up parquet-column-readers.cc Move parquet classes into exec/parquet. Move CollectionColumnReader and ParquetLevelDecoder into separate files. Remove unnecessary 'encoding_' field from ParquetLevelDecoder. Switch BOOLEAN decoding to use composition instead of inheritance. This lets the boolean decoding use the faster batched implementations in ScalarColumnReader and avoids some confusing aspects of the class hierarchy, like the ReadValueBatch() implementation on the base class that was shared between BoolColumnReader and CollectionColumnReader. Improve compile times by instantiating BitPacking templates in a separate file (this looks to give a 30s+ speedup for compiling parquet-column-readers.cc). Testing: Ran exhaustive tests. Change-Id: I0efd5c50b781fe9e3c022b33c66c06cfb529c0b8 Reviewed-on: http://gerrit.cloudera.org:8080/11949 Reviewed-by: Impala Public Jenkins <[email protected]> Tested-by: Impala Public Jenkins <[email protected]> --- M be/CMakeLists.txt M be/src/benchmarks/bit-packing-benchmark.cc M be/src/benchmarks/bswap-benchmark.cc M be/src/codegen/impala-ir.cc M be/src/exec/CMakeLists.txt M be/src/exec/data-source-scan-node.cc M be/src/exec/hdfs-scan-node-base.cc M be/src/exec/hdfs-table-sink.cc A be/src/exec/parquet/CMakeLists.txt R be/src/exec/parquet/hdfs-parquet-scanner-ir.cc R be/src/exec/parquet/hdfs-parquet-scanner-test.cc R be/src/exec/parquet/hdfs-parquet-scanner.cc R be/src/exec/parquet/hdfs-parquet-scanner.h R be/src/exec/parquet/hdfs-parquet-table-writer.cc R be/src/exec/parquet/hdfs-parquet-table-writer.h A be/src/exec/parquet/parquet-bool-decoder.cc A be/src/exec/parquet/parquet-bool-decoder.h A be/src/exec/parquet/parquet-collection-column-reader.cc A be/src/exec/parquet/parquet-collection-column-reader.h R be/src/exec/parquet/parquet-column-readers.cc R be/src/exec/parquet/parquet-column-readers.h R be/src/exec/parquet/parquet-column-stats.cc R be/src/exec/parquet/parquet-column-stats.h R be/src/exec/parquet/parquet-column-stats.inline.h R be/src/exec/parquet/parquet-common.cc R be/src/exec/parquet/parquet-common.h A be/src/exec/parquet/parquet-level-decoder.cc A be/src/exec/parquet/parquet-level-decoder.h R be/src/exec/parquet/parquet-metadata-utils.cc R be/src/exec/parquet/parquet-metadata-utils.h R be/src/exec/parquet/parquet-plain-test.cc R be/src/exec/parquet/parquet-scratch-tuple-batch.h R be/src/exec/parquet/parquet-version-test.cc M be/src/util/CMakeLists.txt M be/src/util/bit-packing-test.cc A be/src/util/bit-packing.cc M be/src/util/bit-packing.h M be/src/util/bit-packing.inline.h M be/src/util/bit-stream-utils.inline.h M be/src/util/dict-encoding.h M be/src/util/dict-test.cc M be/src/util/parquet-reader.cc M be/src/util/rle-test.cc M common/thrift/generate_error_codes.py M testdata/workloads/functional-query/queries/QueryTest/parquet-num-values-def-levels-mismatch.test 45 files changed, 1,171 insertions(+), 785 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/11949 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I0efd5c50b781fe9e3c022b33c66c06cfb529c0b8 Gerrit-Change-Number: 11949 Gerrit-PatchSet: 12 Gerrit-Owner: Tim Armstrong <[email protected]> Gerrit-Reviewer: Csaba Ringhofer <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Tim Armstrong <[email protected]> Gerrit-Reviewer: Zoltan Borok-Nagy <[email protected]>
