This is an automated email from the ASF dual-hosted git repository.
tarmstrong pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/impala.git
The following commit(s) were added to refs/heads/master by this push:
new e61f485 IMPALA-9446: Fix bug that impala failed to read zstd file on
s3
e61f485 is described below
commit e61f485a374ccefb346ba012aee6cbfc647c2928
Author: xiaomeng <[email protected]>
AuthorDate: Mon Mar 9 11:51:44 2020 -0700
IMPALA-9446: Fix bug that impala failed to read zstd file on s3
S3 filesystem doesn't have a block concept, so scheduler splits each
file into a smaller scan ranges. Adding ZSTD as a case in the switch
statement to add support for zstd.
Testing done:
In mini-cluster, read external table located on s3 zstd file.
Run end to end test successfully on s3.
Change-Id: I13f8837fda7454ddb4bb47d20a675d6315a3462d
Reviewed-on: http://gerrit.cloudera.org:8080/15391
Reviewed-by: Impala Public Jenkins <[email protected]>
Tested-by: Tim Armstrong <[email protected]>
---
be/src/util/flat_buffer.cc | 3 +++
1 file changed, 3 insertions(+)
diff --git a/be/src/util/flat_buffer.cc b/be/src/util/flat_buffer.cc
index 46448e9..d0e90e2 100644
--- a/be/src/util/flat_buffer.cc
+++ b/be/src/util/flat_buffer.cc
@@ -58,6 +58,9 @@ Status FromFbCompression(
case FbCompression_ZLIB:
*thrift_compression = THdfsCompression::ZLIB;
break;
+ case FbCompression_ZSTD:
+ *thrift_compression = THdfsCompression::ZSTD;
+ break;
default:
return Status(strings::Substitute(
"Invalid file descriptor compression code: $0", fb_compression));
