[
https://issues.apache.org/jira/browse/AVRO-4262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Lynch updated AVRO-4262:
---------------------------------
Summary: avro-tools recodec defaults to level -1 which is an incorrect
level for xz and not default for zstd (was: avro-tools recodec defaults to
level -1 which is an incorrect level for xz and zstd)
> avro-tools recodec defaults to level -1 which is an incorrect level for xz
> and not default for zstd
> ---------------------------------------------------------------------------------------------------
>
> Key: AVRO-4262
> URL: https://issues.apache.org/jira/browse/AVRO-4262
> Project: Apache Avro
> Issue Type: Bug
> Components: tools
> Affects Versions: 1.12.1
> Reporter: Jonathan Lynch
> Priority: Major
> Labels: pull-request-available
> Fix For: 1.13.0
>
> Attachments:
> 0001-use-default-codec-compression-level-for-gzip-xz-and-.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> avro-tools recodec should explicitly set the default codec compression level
> for the gzip, xz and zstd codecs when the level parameter is not provided.
> The level defaults to -1 which is fine for gzip but is invalid for xz and
> this -1 value converts to level 1 for zstd silently which was the most
> frustrating.
> ```
> {code:java}
> java -jar avro-tools-1.12.1.jar recodec --codec xz input.avro
> output_conv_xz.avro
> Exception in thread "main" org.tukaani.xz.UnsupportedOptionsException:
> Unsupported preset: -1
> at org.tukaani.xz.LZMA2Options.setPreset(LZMA2Options.java:196)
> at org.tukaani.xz.LZMA2Options.<init>(LZMA2Options.java:157)
> at
> org.apache.commons.compress.compressors.xz.XZCompressorOutputStream.<init>(XZCompressorOutputStream.java:148)
> at org.apache.avro.file.XZCodec.compress(XZCodec.java:62)
> at
> org.apache.avro.file.DataFileStream$DataBlock.compressUsing(DataFileStream.java:384)
> at org.apache.avro.file.DataFileWriter.appendAllFrom(DataFileWriter.java:403)
> at org.apache.avro.tool.RecodecTool.run(RecodecTool.java:80)
> at org.apache.avro.tool.Main.run(Main.java:67)
> at org.apache.avro.tool.Main.main(Main.java:56)
>
> Working with patch
> java -jar avro-tools-1.13.0-SNAPSHOT.jar recodec --codec xz input.avro
> output_conv_xz.avro
> java -jar avro-tools-1.13.0-SNAPSHOT.jar getmeta output_conv_xz.avro | grep
> codec
> avro.codec xz
> {code}
> ```
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
