+1 same here. gzip/lzo in the past, Snappy or zstd now. On Tue, Apr 2, 2024 at 7:50 PM 张铎(Duo Zhang) <palomino...@gmail.com> wrote:
> For me I've never seen people actually use the xz compression. > > For size, usually people will choose gzip, and for speed, in the past > people will choose lzo and now they choose snappy or zstd. > > So for me I prefer we just deprecated the xz compression immediately > and remove it 2.6.0. > > Thanks. > > Andrew Purtell <apurt...@apache.org> 于2024年4月2日周二 08:02写道： > > > > Red Hat filed CVE-2024-3094 late last week on 2024-03-29. This implicates > > recent releases of the native liblzma library as a vector for malicious > > code. > > > > This is not the pure Java version that we depend upon for HBase's support > > for the LZMA algorithm ( > > > https://github.com/apache/hbase/tree/master/hbase-compression/hbase-compression-xz > ). > > We depend on version 1.9 of xz-java, which was published in 2021, well > > before maintenance changes in the project and the involvement of a person > > who is now believed to be a malicious actor. Projects like HBase that > > depend on xz-java have no reason to be concerned about the issues > affecting > > the native xz library. > > > > How the backdoor was introduced calls into question the trustworthiness > and > > viability of the XZ project. GitHub has disabled all repositories related > > to XZ and liblzma, even xz-java. The webpage for XZ and xz-java is down. > > The open source software community is responding vigorously. > CVE-2024-3094 > > has a CVSS score 10, the highest possible score. Your security team may > > become interested in HBase because of hbase-compression-xz's dependency > on > > xz-java. It is likely any discovered dependency on any LZMA > implementation > > will at least raise questions. > > > > For now xz-java remains available in Maven central. (See > > https://central.sonatype.com/artifact/org.tukaani/xz/versions) We may > have > > no choice but to immediately remove hbase-compression-xz if Maven blocks > or > > drops xz-java too, because that will break our builds. > > > > There is no immediate cause for concern. Still, we believe XZ compression > > provides little to no value over more modern alternatives, like > ZStandard, > > that can also achieve similar compression ratios. XZ, and alternatives > like > > ZStandard with the compression level set to a high value, are also > suitable > > only for archival use cases and unsuitable for compression of flush files > > or for use in minor compactions. Given how niche any use of XZ > > compression could > > be, we are wondering if there are actually any users of it. > > > > If we have no users of hbase-compression-xz, then it provides little to > no > > value and continued maintenance of hbase-compression-xz given the issues > > with its dependency does not make sense. > > > > Do you use XZ compression, or are you planning to? > > > > If we deprecate XZ compression immediately and then remove it in 2.6, > would > > this present a problem? In a private discussion we reached consensus on > > this approach, but, of course, that is not yet a plan, and something that > > could easily change based on feedback. > > > > From https://nvd.nist.gov/vuln/detail/CVE-2024-3094: > > "Malicious code was discovered in the upstream tarballs of xz, starting > > with version 5.6.0. Through a series of complex obfuscations, the liblzma > > build process extracts a prebuilt object file from a disguised test file > > existing in the source code, which is then used to modify specific > > functions in the liblzma code. This results in a modified liblzma library > > that can be used by any software linked against this library, > intercepting > > and modifying the data interaction with this library." > > > > -- > > Best regards, > > Andrew >
