Sounds good. Let me put up a PR for the Hadoop upgrade, we can re-run OWASP, or whatever CVE checker after that.
On Sat, Sep 7, 2024 at 1:28 AM Viraj Jasani <vjas...@apache.org> wrote: > Sounds good, I agree with this solution. We can keep them in profile and > disable profile once Hadoop resolves CVEs. > I feel we should resolve Avro and Jettison CVEs and rollout 5.2.1 unless > newer released version of Hadoop already has them resolved. WDYT? > > > On Thu, Sep 5, 2024 at 9:46 PM Istvan Toth <st...@cloudera.com.invalid> > wrote: > > > Thanks for your response, Viraj. > > > > Sure, managing transitive dependencies and using a newer Hadoop are not > > mutually exclusive. > > > > For old HBase versions like 2.1 even the newest officially supported > Hadoop > > is going to be old. > > > > What do you think about my proposals on how to handle the transitive > > dependency management ? > > I quite like the idea of putting those into a profile (which is enabled > by > > default) so that they can be disabled if a newer hadoop is used for > > building. > > > > Istvan > > > > On Thu, Sep 5, 2024 at 5:37 PM Viraj Jasani <vjas...@apache.org> wrote: > > > > > I agree with upgrading Hadoop version but it is helpful only if the > CVEs > > > from indirect dependencies have been resolved by Hadoop. > > > The release frequency is too slow for Hadoop (similar to us) so some > CVE > > > resolution might be WIP and take considerable time to get released. In > > such > > > cases, maybe we can also try to manage dependency within Phoenix pom. > > WDYT? > > > > > > > > > On Wed, Sep 4, 2024 at 1:02 AM Istvan Toth <st...@apache.org> wrote: > > > > > > > Hi! > > > > > > > > In light of the recent discussions with the Trino team, we should > think > > > > about improving our CVE stance. > > > > > > > > We are doing a good job of updating our direct dependencies, but we > > don't > > > > have a good solution for our indirect dependencies, which are > included > > in > > > > our official shaded binary artifacts, and show up in CVE scans. > > > > > > > > These are basically all coming from Hadoop, as HBase is also good at > > > > keeping its direct dependencies in order. > > > > > > > > The main problem is that in order to maximize compatibility, we build > > our > > > > binaries with the oldest supported Hadoop major.minor version, which > is > > > > often long EOL. > > > > > > > > I don't have experience with mixing Hadoop server and client > versions, > > > but > > > > I recall that Hadoop major versions are supposed to be backwards wire > > > > compatible. > > > > > > > > The only binaries that include the Hadoop (and HBase) code are the > > > > phoenix-client-embedded and phoenix-client-lite uberjars, which are > > > > unfortunately the most widely used artifacts. > > > > > > > > My first proposal is to use the latest supported Hadoop version > instead > > > of > > > > the oldest one for each HBase profile. This should remove many of the > > > > existing CVEs from the binaries. > > > > > > > > Do you think this would cause backwards compatibility issues ? > > > > > > > > If not, do you think that we should do this for 5.2.1 ? > > > > > > > > My second discussion point is dependencyManage-ing transitive > > dependency > > > > versions for the CVEs that present in whatever Hadoop version we > build > > > > with. > > > > This is a slippery slope, and I personally would like to minimize and > > > > compartmentalize it as much as possible. > > > > > > > > What is your opinion on this ? > > > > Is this worth doing ? > > > > Should we do it in the main code, or only for the uberjars (which > would > > > be > > > > big a testing problem) > > > > Should we put those dependencyManage entries in a profile, so that > they > > > can > > > > be turned off when building with newer Hadoop versions ? > > > > > > > > looking forward to hearing your thoughts on this > > > > > > > > Istvan > > > > > > > > > > > > > -- > > *István Tóth* | Sr. Staff Software Engineer > > *Email*: st...@cloudera.com > > cloudera.com <https://www.cloudera.com> > > [image: Cloudera] <https://www.cloudera.com/> > > [image: Cloudera on Twitter] <https://twitter.com/cloudera> [image: > > Cloudera on Facebook] <https://www.facebook.com/cloudera> [image: > Cloudera > > on LinkedIn] <https://www.linkedin.com/company/cloudera> > > ------------------------------ > > ------------------------------ > > > -- *István Tóth* | Sr. Staff Software Engineer *Email*: st...@cloudera.com cloudera.com <https://www.cloudera.com> [image: Cloudera] <https://www.cloudera.com/> [image: Cloudera on Twitter] <https://twitter.com/cloudera> [image: Cloudera on Facebook] <https://www.facebook.com/cloudera> [image: Cloudera on LinkedIn] <https://www.linkedin.com/company/cloudera> ------------------------------ ------------------------------
