[
https://issues.apache.org/jira/browse/HBASE-30538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nirdosh Kumar Yadav reassigned HBASE-30538:
-------------------------------------------
Assignee: Nirdosh Kumar Yadav
> Bump snappy-java to 1.1.10.12 — 1.1.10.9 breaks writes through Hadoop
> SnappyCodec
> ---------------------------------------------------------------------------------
>
> Key: HBASE-30538
> URL: https://issues.apache.org/jira/browse/HBASE-30538
> Project: HBase
> Issue Type: Bug
> Affects Versions: 4.0.0-alpha-1, 2.7.0, 3.1.0, 3.0.1, 2.6.8, 2.5.17
> Reporter: Nirdosh Kumar Yadav
> Assignee: Nirdosh Kumar Yadav
> Priority: Major
>
> HBASE-30508 bumped snappy-java to 1.1.10.9. That release added a bounds check
> (xerial/snappy-java#739) in Snappy.compress(ByteBuffer, ByteBuffer) that
> validates
> the output space against compressed.remaining():
> {code}
> checkOutputSpace(compressed.remaining(), 0, maxCompressedLength(uLen));
> {code}
> Hadoop's org.apache.hadoop.io.compress.snappy.SnappyCompressor sets
> compressedDirectBuf.limit(0) before calling compressDirectBuf(), so every
> compress
> through Hadoop's SnappyCodec fails:
> {noformat}
> java.lang.IllegalArgumentException: not enough space for output: need N
> bytes, but only 0 remaining
> at org.xerial.snappy.Snappy.checkOutputSpace(Snappy.java:622)
> at org.xerial.snappy.Snappy.compress(Snappy.java:160)
> at
> org.apache.hadoop.io.compress.snappy.SnappyCompressor.compressDirectBuf(SnappyCompressor.java:282)
> at
> org.apache.hadoop.io.compress.snappy.SnappyCompressor.compress(SnappyCompressor.java:210)
> {noformat}
> Impact on HBase: the default codec for Compression.Algorithm.SNAPPY is
> org.apache.hadoop.io.compress.SnappyCodec (hbase.io.compress.snappy.codec), so
> flushes and compactions of SNAPPY column families fail, and a failed flush
> aborts the
> RegionServer. MapReduce jobs using Snappy map-output compression are also
> affected.
> The xerial codec in hbase-compression-snappy is not affected because it
> clears the
> output buffer before compressing, which is likely why the existing tests did
> not
> catch this.
> Upstream issue: https://github.com/xerial/snappy-java/issues/765
> Fixed in 1.1.10.11 (#766: checks against capacity() from the current
> position).
> 1.1.10.12 also fixes #767 (SnappyHadoopCompatibleOutputStream block header
> size).
> Both keep the CVE fixes from 1.1.10.9. Hadoop trunk and Spark (SPARK-60121)
> are
> already on 1.1.10.12.
>
> Proposal: bump snappy-java 1.1.10.9 -> 1.1.10.12 on all branches that received
> HBASE-30508.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
