Joe McDonnell created IMPALA-10630:
--------------------------------------
Summary: Allow specifying negative compression_levels for ZSTD
Key: IMPALA-10630
URL: https://issues.apache.org/jira/browse/IMPALA-10630
Project: IMPALA
Issue Type: Improvement
Affects Versions: Impala 4.0
Reporter: Joe McDonnell
ParseCompressionCodec prohibits the compression_level from being below 1:
{noformat}
if (status != StringParser::PARSE_SUCCESS || compression_level < 1
|| compression_level > ZSTD_maxCLevel()) {
return Status(Substitute("Invalid ZSTD compression level '$0'."
" Valid values are in [1,$1]",
clevel, ZSTD_maxCLevel()));
}{noformat}
[https://github.com/apache/impala/blob/ebbe52b4bed944d3012e3679dc984827ce11d5a8/be/src/util/parse-util.cc#L142-L147]
ZSTD now supports negative compression levels that speed up compression and
decompression at the expense of reduced compression ratio. See the "--fast"
options on [https://github.com/facebook/zstd]
We should add support for those negative compression levels, as they seem very
competitive with Snappy.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
