We don't have a complete set of shaded artefacts -so it's not fair to point at the downstream users and say it's "your own fault". We need to do more here ourselves.
Here: it is a CVE, they should upgrade anyway. Guava is special because it has been so brittle cross versions and so widely used by so many applications. On Tue, Oct 22, 2019 at 6:15 PM Wei-Chiu Chuang <weic...@apache.org> wrote: > Hi Sean, > Thanks for the valuable feedback. > Good point on not using dependency classes in public API parameters. One > example is HADOOP-15502 > <https://issues.apache.org/jira/browse/HADOOP-15502> (blame > me for breaking the API) > > From what I know, the biggest risk is that downstreamers include > dependencies from Hadoop implicitly. Therefore, if Hadoop updates a > dependency that has a breaking change, the downstream application breaks, > sometimes during compile time, sometimes at runtime. > > Jetty is probably not the best example. But take Guava as an example, when > we updated Guava from 11.0 to 27.0, it breaks downstreamers like crazy -- > Hive, Tez, Pheonix, Oozie all have to make changes. Probably not Hadoop's > responsibility that they don't use shaded Hadoop client artifacts, but > maybe we can add release note and state potential breaking changes. > > On Tue, Oct 22, 2019 at 7:43 AM Sean Busbey <bus...@cloudera.com.invalid> > wrote: > > > speaking with my HBase hat on instead of my Hadoop hat, when the > > Hadoop project publishes that there's a CVE but does not include a > > maintenance release that mitigates it for a given minor release line, > > we assume that means the Hadoop project is saying that release line is > > EOM and should be abandoned. > > > > I don't know if that's an accurate interpretation in all cases. > > > > With my Hadoop hat on, I think downstream projects should use the > > interfaces we say are safe to use and those interfaces should not > > include dependencies where practical. I don't know how often a CVE > > comes along for things like our logging API dependency, for example. > > But downstream folks should definitely not rely on dependencies we use > > for internal service, so I'm surprised that a version change for Jetty > > would impact downstream. > > > > > > On Mon, Oct 21, 2019 at 12:33 PM Wei-Chiu Chuang <weic...@apache.org> > > wrote: > > > > > > Hi Hadoop developers, > > > > > > I've always had this question and I don't know the answer. > > > > > > For the last few months I finally spent time to deal with the > > vulnerability > > > reports from our internal dependency check tools. > > > > > > Say in HADOOP-16152 < > https://issues.apache.org/jira/browse/HADOOP-16152> > > > we update Jetty from 9.3.27 to 9.4.20 because of CVE-2019-16869, > should I > > > cherrypick the fix into all lower releases? > > > This is not a trivial change, and it breaks downstreams like Tez. On > the > > > other hand, it doesn't seem reasonable if I put this fix only in trunk, > > and > > > left older releases vulnerable. What's the expectation of downstream > > > applications w.r.t breaking compatibility vs fixing security issues? > > > > > > Thoughts? > > > > > > > > -- > > busbey > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org > > For additional commands, e-mail: common-dev-h...@hadoop.apache.org > > > > >
