To clarify, I was referring to "It’s weird to have netty dependency versions diverging." Which I do agree with, but was explaining my understanding of the rationale.
Otherwise, I agree with everything else. I have the PR ready for updating netty in hbase-thirdparty. I can also vote and do the bump of hbase-thirdparty.version in hbase once the release is made. On Thu, Jan 4, 2024 at 3:15 PM Andrew Purtell <[email protected]> wrote: > We should not ship a netty with known CVEs in hbase-thirdparty. Likewise > for the other components shipped in hbase-thirdparty. As with all things > this is a guideline, not a rule, because each situation is different and > people do not always share the same opinion. > > I am of the opinion that moderate to high scoring CVEs in a dependency -- > and it doesn't matter if direct, transitive, or shaded -- is going to be a > problem for many users or potential users simply if they exist in our bill > of materials. At my employer we need to juggle the "cleanliness" of our > software bill of materials among priorities and I do not think we are > exceptional in any way. We do it in a fork but I have been meaning to do a > pass over the public open source project's dependency set, if this would > have some value for the project (which I believe there is). > > I can do a thirdparty release now if everyone else is busy. > > On Thu, Jan 4, 2024 at 10:50 AM Bryan Beaudreault <[email protected] > > > wrote: > > > I think we explicitly don't want our hbase-thirdparty netty version to be > > locked to the one for transitive dependencies. That's sort of why we have > > it in thirdparty/shaded at all, right? > > > > I created https://issues.apache.org/jira/browse/HBASE-28291 to update > > pom.xml in hbase-thirdparty. I will handle that. We will need to do an > > hbase-thirdparty release, which I'm not sure I'll have time for given I'm > > already behind on the 2.6.0 release > > > > On Thu, Jan 4, 2024 at 11:58 AM Andrew Purtell <[email protected] > > > > wrote: > > > > > We should do that bump to hbase-thirdparty and spin another release to > > > keep our house in order. It isn’t urgent but would be good to address > > this > > > in the normal release cadence. That has been about once per fiscal > > quarter > > > recently. It’s weird to have netty dependency versions diverging. > > > > > > I will make a note as RM to look at hbase-thirdparty versions with > > respect > > > to the base POM and known security issues, using snyk probably, and > > update > > > it ahead of 2.5.8. As well as direct dependencies in the base POM. > > > Unfortunately I can’t promise to do anything about transitive issues > > > imported from something that impacts operational compatibility. Those > > must > > > be weighed case by case. > > > > > > > On Jan 4, 2024, at 8:31 AM, Bryan Beaudreault < > [email protected] > > > > > > wrote: > > > > > > > > It looks like that CVE only affects io.netty:netty-codec-http2. > Since > > > our > > > > hbase-shaded-netty depends on netty-all, that module is included. > > > However, > > > > I don't think we use anything from netty-codec-http2. So I don't > think > > > the > > > > CVE is a risk for this usage, unless you are building an app using > the > > > > org.apache.hbase.thirdparty.io.netty classes. This would not be > > advised. > > > > > > > > That said, we could try to bump hbase-thirdparty to 4.1.100+ and > > include > > > > that in the upcoming 2.6.0 or 2.5.8 when that happens. If the CVE > were > > > > critical we could rush out another minor release, but I don't think > > it's > > > > necessary here? I also wonder if we should update hbase-shaded-netty > to > > > > only pull in the netty modules we actually use. > > > > > > > >> On Thu, Jan 4, 2024 at 11:14 AM Dan Huff > <[email protected] > > > > > > >> wrote: > > > >> > > > >> Thanks Bryan. That does help explain things. I have been looking at > > > >> > > https://github.com/netty/netty/security/advisories/GHSA-xpw8-rcwv-8f8p > > > and > > > >> have been trying to determine if hbase is vulnerable to this attack > > > vector > > > >> or not. I got excited when I saw 4.1.100.Final in 2.5.7 but it > sounds > > > like > > > >> that excitement was misplaced :) > > > >> > > > >> Dan > > > >> > > > >> On Tue, Jan 2, 2024 at 12:54 PM Bryan Beaudreault < > > > [email protected] > > > >>> > > > >> wrote: > > > >> > > > >>> Hello, > > > >>> > > > >>> As the comment above the netty version change says, this only > affects > > > the > > > >>> transitive netty dependencies from thirdparty dependencies like > > > zookeeper > > > >>> and hadoop. HBase's internal netty usage (i.e. for HBase's RPC > > > protocol) > > > >>> uses the shaded netty provided by hbase-thirdparty. > > > >>> > > > >>> While you're generally correct that in maven you'd expect a version > > > >> defined > > > >>> in dependencyManagement to affect all transitive dependencies, that > > is > > > >> not > > > >>> the case for hbase-thirdparty due to the shading we do there. At > the > > > time > > > >>> of building hbase-thirdparty, the defined netty version there is > > pulled > > > >> in > > > >>> and relocated to org.apache.hbase.thirdparty.io.netty and published > > as > > > a > > > >>> new maven module named hbase-shaded-netty. As such, the > > > >>> dependencyManagement has no effect on it. > > > >>> > > > >>> I hope this helps > > > >>> > > > >>> On Tue, Jan 2, 2024 at 2:40 PM Dan Huff > <[email protected] > > > > > > >>> wrote: > > > >>> > > > >>>> Hello there Hbase Devs-- > > > >>>> > > > >>>> I have been investigating taking an update to Hbase 2.5.7 after > the > > > >>> release > > > >>>> last week and have what I hope is a quick question about commit > > > 7639345 > > > >>>> < > > > >>>> > > > >>> > > > >> > > > > > > https://github.com/apache/hbase/commit/7639345a970636e7a9eb7adf6d84dadd6f3fccb9 > > > >>>>> > > > >>>> in > > > >>>> branch-2.5. > > > >>>> > > > >>>> Am I correct in believing that the direct inclusion of netty > > > >>> 4.1.100.Final > > > >>>> in Hbase's pom.xml will override the 4.1.97.Final version that is > > > >>>> specified in hbase-thirdparty > > > >>>> < > > > >> > > https://github.com/apache/hbase-thirdparty/blob/rel/4.1.5/pom.xml#L137 > > > >>>> ? > > > >>>> I > > > >>>> see 4.1.100.Final listed on > > > >>>> https://hbase.apache.org/dependency-management.html which to me > > > >> suggests > > > >>>> that I am understanding this correctly that issues flagged against > > > >>>> 4.1.97.Final can be ignored since Hbase will now just use > > > >> 4.1.100.Final. > > > >>>> > > > >>>> Thanks so much for your time, > > > >>>> > > > >>>> Dan Huff > > > >>>> > > > >>> > > > >> > > > > > > > > -- > Best regards, > Andrew > > Unrest, ignorance distilled, nihilistic imbeciles - > It's what we’ve earned > Welcome, apocalypse, what’s taken you so long? > Bring us the fitting end that we’ve been counting on > - A23, Welcome, Apocalypse >
