Internal Jenkins has submitted this change and it was merged. Change subject: IMPALA-3629: Codegen TransferScratchTuples() in hdfs-parquet-scanner ......................................................................
IMPALA-3629: Codegen TransferScratchTuples() in hdfs-parquet-scanner We currently don't do any codegen in the hdfs-parquet scanner, which limits performance. This patch creates a new function, ProcessScratchBatch, which contains the inner loop of TransferScratchTuples, allowing us to cross-compile only this performance critical part. It also uses CodegenEvalConjuncts to replace the call to EvalConjuncts in ProcessScratchBatch with a codegen'd version. Additionally, it modifies the Codegen functions in hdfs-avro/text/sequence-scanner to take an output parameter for the codegen'd function and return a Status in order to improve logging around codegen failures. Change-Id: Ic327e437c7cd2b3f92cdb11c1e907bfee2d44ee8 Reviewed-on: http://gerrit.cloudera.org:8080/4093 Reviewed-by: Thomas Tauber-Marshall <[email protected]> Reviewed-by: Matthew Jacobs <[email protected]> Tested-by: Internal Jenkins --- M be/src/codegen/gen_ir_descriptions.py M be/src/codegen/impala-ir.cc M be/src/exec/CMakeLists.txt M be/src/exec/hdfs-avro-scanner.cc M be/src/exec/hdfs-avro-scanner.h A be/src/exec/hdfs-parquet-scanner-ir.cc M be/src/exec/hdfs-parquet-scanner.cc M be/src/exec/hdfs-parquet-scanner.h M be/src/exec/hdfs-scan-node.cc M be/src/exec/hdfs-scanner.cc M be/src/exec/hdfs-scanner.h M be/src/exec/hdfs-sequence-scanner.cc M be/src/exec/hdfs-sequence-scanner.h M be/src/exec/hdfs-text-scanner.cc M be/src/exec/hdfs-text-scanner.h 15 files changed, 276 insertions(+), 128 deletions(-) Approvals: Matthew Jacobs: Looks good to me, approved Internal Jenkins: Verified Thomas Tauber-Marshall: Looks good to me, but someone else must approve -- To view, visit http://gerrit.cloudera.org:8080/4093 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: merged Gerrit-Change-Id: Ic327e437c7cd2b3f92cdb11c1e907bfee2d44ee8 Gerrit-PatchSet: 2 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Thomas Tauber-Marshall <[email protected]> Gerrit-Reviewer: Internal Jenkins Gerrit-Reviewer: Matthew Jacobs <[email protected]> Gerrit-Reviewer: Thomas Tauber-Marshall <[email protected]>
