Tim Armstrong has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/7245 )
Change subject: IMPALA-4863/IMPALA-5311: Correctly account the file type and compression codec ...................................................................... IMPALA-4863/IMPALA-5311: Correctly account the file type and compression codec If a scan range is skipped at runtime the scan node skips reading the range and never figures out the underlying compression codec used to compress the files. In such a scenario we default the compression codec to NONE which can be misleading. This change marks these files as filtered in the scan node profile e.g. - File Formats: TEXT/NONE:364 TEXT/NONE(Skipped):1460 Change-Id: I797916505f62e568f4159e07099481b8ff571da2 Reviewed-on: http://gerrit.cloudera.org:8080/7245 Reviewed-by: Tim Armstrong <[email protected]> Tested-by: Tim Armstrong <[email protected]> --- M be/src/exec/hdfs-parquet-scanner.cc M be/src/exec/hdfs-scan-node-base.cc M be/src/exec/hdfs-scan-node-base.h M be/src/exec/hdfs-scan-node.cc M be/src/exec/hdfs-scan-node.h M testdata/workloads/functional-query/queries/QueryTest/hdfs_scanner_profile.test M tests/query_test/test_scanners.py 7 files changed, 71 insertions(+), 20 deletions(-) Approvals: Tim Armstrong: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/7245 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I797916505f62e568f4159e07099481b8ff571da2 Gerrit-Change-Number: 7245 Gerrit-PatchSet: 12 Gerrit-Owner: anujphadke <[email protected]> Gerrit-Reviewer: Dan Hecht <[email protected]> Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong <[email protected]> Gerrit-Reviewer: anujphadke <[email protected]>
