Alex Behm has submitted this change and it was merged. Change subject: IMPALA-3845: Split up hdfs-parquet-scanner.cc into more files/components. ......................................................................
IMPALA-3845: Split up hdfs-parquet-scanner.cc into more files/components. This patch refactors hdfs-parquet-scanner.cc into several files. The new responsibilities of each file/component are roughly as follows: hdfs-parquet-scanner.h/cc - Creates column readers and uses them to materializes row batches. - Evaluates runtime filters and conjuncts, populates row batch queue. parquet-metadata-utils.h/cc - Contains utilities for validating Parquet file metadata. - Parses the schema of a Parquet file into our internal schema representation. - Resolves SchemaPaths (e.g. from a table descriptor) against the internal representation of the Parquet file schema. parquet-column-readers.h/cc - Contains the per-column data reading, parsing and value materialization logic. Testing: A private core/hdfs run passed. Change-Id: I4c5fd46f9c1a0ff2a4c30ea5a712fbae17c68f92 Reviewed-on: http://gerrit.cloudera.org:8080/3596 Tested-by: Internal Jenkins Reviewed-by: Alex Behm <[email protected]> --- M be/src/exec/CMakeLists.txt M be/src/exec/base-sequence-scanner.cc M be/src/exec/hdfs-parquet-scanner.cc M be/src/exec/hdfs-parquet-scanner.h M be/src/exec/hdfs-rcfile-scanner.cc M be/src/exec/hdfs-scanner.cc M be/src/exec/hdfs-scanner.h M be/src/exec/hdfs-text-scanner.cc A be/src/exec/parquet-column-readers.cc A be/src/exec/parquet-column-readers.h A be/src/exec/parquet-metadata-utils.cc A be/src/exec/parquet-metadata-utils.h A be/src/exec/parquet-scratch-tuple-batch.h M be/src/exec/parquet-version-test.cc M be/src/exprs/expr-value.h M be/src/runtime/runtime-state.cc M be/src/runtime/runtime-state.h M be/src/util/debug-util.cc M be/src/util/debug-util.h 19 files changed, 2,687 insertions(+), 2,465 deletions(-) Approvals: Internal Jenkins: Verified Alex Behm: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/3596 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: merged Gerrit-Change-Id: I4c5fd46f9c1a0ff2a4c30ea5a712fbae17c68f92 Gerrit-PatchSet: 8 Gerrit-Project: Impala Gerrit-Branch: cdh5-trunk Gerrit-Owner: Alex Behm <[email protected]> Gerrit-Reviewer: Alex Behm <[email protected]> Gerrit-Reviewer: Internal Jenkins Gerrit-Reviewer: Marcel Kornacker <[email protected]> Gerrit-Reviewer: Tim Armstrong <[email protected]>
