Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/8684 )
Change subject: IMPALA-3804: Re-enable per-scan filtering for sequence-based scanners ...................................................................... IMPALA-3804: Re-enable per-scan filtering for sequence-based scanners IMPALA-3798 disabled per-scan filtering for sequence- based scanners due to a race between runtime filter arrival and header splits processing. This commit enables per-scan filtering again for the sequence based files. In HdfsScanNode::ProcessSplit() we check if the current range is the header of a sequence file. If so, and the filters reject the file, the whole file skipped. If it is not a sequence header, but the filters reject the partition, we call RangeComplete() on the current scan range. Change-Id: I4b38c26bcbe67f83efcc65a1723d766626ae3d3e Reviewed-on: http://gerrit.cloudera.org:8080/8684 Reviewed-by: Tim Armstrong <[email protected]> Tested-by: Impala Public Jenkins --- M be/src/exec/base-sequence-scanner.cc M be/src/exec/hdfs-scan-node-base.cc M be/src/exec/hdfs-scan-node-base.h M be/src/exec/hdfs-scan-node.cc M be/src/exec/hdfs-scanner.cc M tests/custom_cluster/test_always_false_filter.py 6 files changed, 51 insertions(+), 33 deletions(-) Approvals: Tim Armstrong: Looks good to me, approved Impala Public Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/8684 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I4b38c26bcbe67f83efcc65a1723d766626ae3d3e Gerrit-Change-Number: 8684 Gerrit-PatchSet: 7 Gerrit-Owner: Zoltan Borok-Nagy <[email protected]> Gerrit-Reviewer: Dan Hecht <[email protected]> Gerrit-Reviewer: Henry Robinson <[email protected]> Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong <[email protected]> Gerrit-Reviewer: Zoltan Borok-Nagy <[email protected]>
