Nirdosh Kumar Yadav created HBASE-30538:
-------------------------------------------
Summary: Bump snappy-java to 1.1.10.12 — 1.1.10.9 breaks writes
through Hadoop SnappyCodec
Key: HBASE-30538
URL: https://issues.apache.org/jira/browse/HBASE-30538
Project: HBase
Issue Type: Bug
Affects Versions: 2.5.17, 2.6.8, 3.0.1, 2.7.0, 3.1.0, 4.0.0-alpha-1
Reporter: Nirdosh Kumar Yadav
HBASE-30508 bumped snappy-java to 1.1.10.9. That release added a bounds check
(xerial/snappy-java#739) in Snappy.compress(ByteBuffer, ByteBuffer) that
validates
the output space against compressed.remaining():
{code}
checkOutputSpace(compressed.remaining(), 0, maxCompressedLength(uLen));
{code}
Hadoop's org.apache.hadoop.io.compress.snappy.SnappyCompressor sets
compressedDirectBuf.limit(0) before calling compressDirectBuf(), so every
compress
through Hadoop's SnappyCodec fails:
{noformat}
java.lang.IllegalArgumentException: not enough space for output: need N bytes,
but only 0 remaining
at org.xerial.snappy.Snappy.checkOutputSpace(Snappy.java:622)
at org.xerial.snappy.Snappy.compress(Snappy.java:160)
at
org.apache.hadoop.io.compress.snappy.SnappyCompressor.compressDirectBuf(SnappyCompressor.java:282)
at
org.apache.hadoop.io.compress.snappy.SnappyCompressor.compress(SnappyCompressor.java:210)
{noformat}
Impact on HBase: the default codec for Compression.Algorithm.SNAPPY is
org.apache.hadoop.io.compress.SnappyCodec (hbase.io.compress.snappy.codec), so
flushes and compactions of SNAPPY column families fail, and a failed flush
aborts the
RegionServer. MapReduce jobs using Snappy map-output compression are also
affected.
The xerial codec in hbase-compression-snappy is not affected because it clears
the
output buffer before compressing, which is likely why the existing tests did not
catch this.
Upstream issue: https://github.com/xerial/snappy-java/issues/765
Fixed in 1.1.10.11 (#766: checks against capacity() from the current position).
1.1.10.12 also fixes #767 (SnappyHadoopCompatibleOutputStream block header
size).
Both keep the CVE fixes from 1.1.10.9. Hadoop trunk and Spark (SPARK-60121) are
already on 1.1.10.12.
Proposal: bump snappy-java 1.1.10.9 -> 1.1.10.12 on all branches that received
HBASE-30508.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
