https://issues.apache.org/jira/browse/OMID-231 is the ticket.
On Mon, Aug 29, 2022 at 10:28 AM Istvan Toth <[email protected]> wrote: > I had a go at this, and it seems to be much easier than I thought. > > I'm going to open a ticket and PR for this soon. > > Istvan > > On Fri, Aug 26, 2022 at 5:15 PM Andrew Purtell <[email protected]> > wrote: > >> I agree with Geoffrey that OMID should align with Phoenix on default >> dependency versions but based on this you don’t have an immediate >> integration problem. I don’t believe the Configuration API has changed in a >> long time. >> >> The security posture of Hadoop 2 in general is a problem, though. I know >> Hadoop released 2.10.2 to address some CVE worthy problems but it is >> unclear if 2.10.2 addresses all known issues, unlike 3.3.4. Also as you >> know Hadoop 2 has unpatchable dependencies on org.codehaus versions of >> Jackson and Jetty, which themselves have high scoring CVEs that will never >> be fixed because they are EOL, and other similar issues. Hadoop 3 doesn’t >> completely solve such problems but is the only realistic place we can hope >> they can be addressed as required. For organizations that implement or >> require a top to bottom security audit of their software bill of materials, >> it seems possible to avoid some user pain here by aligning build defaults. >> >> I also see Geoffrey started a discussion on dev@hbase about HBase >> producing Hadoop 3 variant builds that could be consumed by downstreamers. >> I have responded on that thread today to express my intention to sponsor >> that effort, for what it’s worth. >> >> >> > On Aug 25, 2022, at 10:41 PM, Istvan Toth <[email protected]> >> wrote: >> > >> > The only non-hbase Hadoop APIs referred in Omid are >> > org.apache.hadoop.conf.Configuration, and a few classes from of >> > org.apache.hadoop.security >> > (the latter is not referenced directly from the coprocessors) . >> > >> > Also, the coprocessor JARs don't have any Hbase or Hadoop code shaded >> in. >> > >> > >> > >> >> On Fri, Aug 26, 2022 at 5:45 AM Andrew Purtell < >> [email protected]> >> >> wrote: >> >> >> >> Is any OMID coprocessor code using Hadoop APIs? >> >> >> >>> On Aug 25, 2022, at 9:41 AM, Istvan Toth <[email protected]> >> >> wrote: >> >>> >> >>> We dependencyManage every Hadoop component in Phoenix, so going to >> >> Hadoop3 >> >>> in Omid is not strictly necessary. >> >>> Hadoop is also added to the TSO libraries in the assembly, but AFAIK >> >>> Hadoop2 and Hadoop3 are wire compatible >> >>> (and I'm not sure if Omid even calls Hadoop endpoints directly), so >> that >> >> is >> >>> not a show stopper either. >> >>> >> >>> If we go with the assumption that there is no significant usage of >> Omid >> >>> outside HBase, then updating to Hadoop3 is a no-brainer, if only to >> avoid >> >>> juggling >> >>> the Hadoop2 and Hadoop3 compiled HBases when developing. >> >>> Unfortunately it's quite a bit of work with all the dependency >> >> management, >> >>> build docs, and updating the CI pipelines to rebuild HBase >> >>> like we do in Phoenix. >> >>> >> >>> Istvan >> >>> >> >>> >> >>>> On Wed, Aug 24, 2022 at 9:24 PM Geoffrey Jacoby <[email protected]> >> >> wrote: >> >>>> >> >>>> Thanks for volunteering to be RM for the next Omid release. >> >>>> >> >>>> I agree we should have a release soon (I think we'll want Phoenix >> 5.2 to >> >>>> use the new Omid). In addition to OMID-226, I suggest we also upgrade >> >>>> Omid's internal Hadoop version to align with whatever default Hadoop >> we >> >>>> choose for Phoenix 5.2, to keep Phoenix's transitive dependencies (a >> >>>> little) simpler. Right now Omid appears to depend on Hadoop 2.10, >> which >> >>>> since Phoenix 5 is Hadoop 3-based, seems incorrect. >> >>>> >> >>>> Geoffrey >> >>>> >> >>>>> On Wed, Aug 24, 2022 at 12:46 AM Istvan Toth <[email protected]> >> wrote: >> >>>>> >> >>>>> Hi! >> >>>>> >> >>>>> Most of the planned OMID changes for Phoenix 5.2 have landed. >> >>>>> The only outstanding ticket that I'm aware of is OMID-226 which I >> also >> >>>>> expect to land soon. >> >>>>> >> >>>>> Unless someone has more changes targeted for the next release, I >> >> propose >> >>>>> that we release the next Omid version soon after OMID-226. >> >>>>> >> >>>>> I also propose bumping the version to 1.1.0, though because of the >> >> HBase >> >>>>> 1.x compatibility removal, and maven artifact changes we could also >> >> argue >> >>>>> for 2.0.0 >> >>>>> >> >>>>> If there are no other volunteers, I also volunteer to be the RM for >> the >> >>>>> release. >> >>>>> >> >>>>> regards >> >>>>> Istvan >> >>>>> >> >>>> >> >>> >> >>> >> >>> -- >> >>> *István Tóth* | Staff Software Engineer >> >>> [email protected] <https://www.cloudera.com> >> >>> [image: Cloudera] <https://www.cloudera.com/> >> >>> [image: Cloudera on Twitter] <https://twitter.com/cloudera> [image: >> >>> Cloudera on Facebook] <https://www.facebook.com/cloudera> [image: >> >> Cloudera >> >>> on LinkedIn] <https://www.linkedin.com/company/cloudera> >> >>> <https://www.cloudera.com/> >> >>> ------------------------------ >> >> >> > >> > >> > -- >> > *István Tóth* | Staff Software Engineer >> > [email protected] <https://www.cloudera.com> >> > [image: Cloudera] <https://www.cloudera.com/> >> > [image: Cloudera on Twitter] <https://twitter.com/cloudera> [image: >> > Cloudera on Facebook] <https://www.facebook.com/cloudera> [image: >> Cloudera >> > on LinkedIn] <https://www.linkedin.com/company/cloudera> >> > <https://www.cloudera.com/> >> > ------------------------------ >> > > > -- > *István Tóth* | Staff Software Engineer > [email protected] <https://www.cloudera.com> > [image: Cloudera] <https://www.cloudera.com/> > [image: Cloudera on Twitter] <https://twitter.com/cloudera> [image: > Cloudera on Facebook] <https://www.facebook.com/cloudera> [image: > Cloudera on LinkedIn] <https://www.linkedin.com/company/cloudera> > <https://www.cloudera.com/> > ------------------------------ > -- *István Tóth* | Staff Software Engineer [email protected] <https://www.cloudera.com> [image: Cloudera] <https://www.cloudera.com/> [image: Cloudera on Twitter] <https://twitter.com/cloudera> [image: Cloudera on Facebook] <https://www.facebook.com/cloudera> [image: Cloudera on LinkedIn] <https://www.linkedin.com/company/cloudera> <https://www.cloudera.com/> ------------------------------
