Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/20482 )
Change subject: IMPALA-12431: Support reading compressed JSON file ...................................................................... IMPALA-12431: Support reading compressed JSON file This patch adds the functionality to read compressed JSON files for the JSON scanner. Because the decompression code can largely be reused from HdfsTextScanner, this patch moves that part of the code from HdfsTextScanner to HdfsScanner so that HdfsJsonScanner can also call it. As it reuses the relevant code from the TEXT scanner, the compression formats supported by the Json scanner are the same as those supported by the TEXT scanner. Tests - Most of the existing end-to-end JSON format tests can run on compressed JSON format too. Change-Id: I2471855d97d4cdd51363b321055e6b06aa6d81e8 Reviewed-on: http://gerrit.cloudera.org:8080/20482 Reviewed-by: Impala Public Jenkins <[email protected]> Tested-by: Impala Public Jenkins <[email protected]> --- M be/src/exec/hdfs-scanner.cc M be/src/exec/hdfs-scanner.h M be/src/exec/json/hdfs-json-scanner.cc M be/src/exec/json/hdfs-json-scanner.h M be/src/exec/text/hdfs-text-scanner.cc M be/src/exec/text/hdfs-text-scanner.h M fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java M testdata/workloads/functional-query/functional-query_exhaustive.csv M testdata/workloads/tpcds/tpcds_core.csv M testdata/workloads/tpcds/tpcds_exhaustive.csv M testdata/workloads/tpcds/tpcds_pairwise.csv M testdata/workloads/tpch/tpch_core.csv M testdata/workloads/tpch/tpch_exhaustive.csv M testdata/workloads/tpch/tpch_pairwise.csv M tests/query_test/test_compressed_formats.py 15 files changed, 259 insertions(+), 218 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/20482 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I2471855d97d4cdd51363b321055e6b06aa6d81e8 Gerrit-Change-Number: 20482 Gerrit-PatchSet: 15 Gerrit-Owner: Zihao Ye <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Joe McDonnell <[email protected]> Gerrit-Reviewer: Quanlong Huang <[email protected]> Gerrit-Reviewer: Zihao Ye <[email protected]>
