Jonathan Lynch created AVRO-4262:
------------------------------------
Summary: avro-tools recodec defaults to -1 incorrect levels for xz
and zstd
Key: AVRO-4262
URL: https://issues.apache.org/jira/browse/AVRO-4262
Project: Apache Avro
Issue Type: Bug
Components: tools
Affects Versions: 1.12.1
Reporter: Jonathan Lynch
Fix For: 1.13.0
Attachments:
0001-use-default-codec-compression-level-for-gzip-xz-and-.patch
avro-tools recodec should explicitly set the default codec compression level
for the gzip, xz and zstd codecs when the level parameter is not provided. The
level defaults to -1 which is fine for gzip but is invalid for xz and this -1
value converts to level 1 for zstd silently which was the most frustrating.
```
{code:java}
java -jar avro-tools-1.12.1.jar recodec --codec xz input.avro
output_conv_xz.avro
Exception in thread "main" org.tukaani.xz.UnsupportedOptionsException:
Unsupported preset: -1
at org.tukaani.xz.LZMA2Options.setPreset(LZMA2Options.java:196)
at org.tukaani.xz.LZMA2Options.<init>(LZMA2Options.java:157)
at
org.apache.commons.compress.compressors.xz.XZCompressorOutputStream.<init>(XZCompressorOutputStream.java:148)
at org.apache.avro.file.XZCodec.compress(XZCodec.java:62)
at
org.apache.avro.file.DataFileStream$DataBlock.compressUsing(DataFileStream.java:384)
at org.apache.avro.file.DataFileWriter.appendAllFrom(DataFileWriter.java:403)
at org.apache.avro.tool.RecodecTool.run(RecodecTool.java:80)
at org.apache.avro.tool.Main.run(Main.java:67)
at org.apache.avro.tool.Main.main(Main.java:56)
Working with patch
java -jar avro-tools-1.13.0-SNAPSHOT.jar recodec --codec xz input.avro
output_conv_xz.avro
java -jar avro-tools-1.13.0-SNAPSHOT.jar getmeta output_conv_xz.avro | grep
codec
avro.codec xz
{code}
```
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
