correction: many of which use unallocated protobuf,
I meant unrelocated. On Mon, Oct 7, 2024 at 9:37 AM Istvan Toth <st...@cloudera.com> wrote: > Unfortunately Protobuf has made the decision of breaking binary > compatibility between the frequently changing major versions, > creating a Guava-like problem. > > Since Omid (or rather HBase/Phoenix) is frequently used together with > other Hadoop-adjacent software depending on Protobuf, many of which use > unallocated protobuf, > we need to solve this somehow. > > To avoid adding another repo, we could add protobuf to the existing > phoenix-thirdparty repo, since Omid is a Phoenix subproject now. > > Istvan > > On Wed, Oct 2, 2024 at 1:51 PM Villő Szűcs <szucsvi...@cloudera.com.invalid> > wrote: > >> Hey all, >> I've been working on OMID-284 and OMID-276, specifically attempting to >> upgrade Protobuf in OMID, and I've encountered several challenges that I >> want to bring to the community for discussion. >> >> I tried upgrading Protobuf to version 4.28.2: see PR #170 >> <https://github.com/apache/phoenix-omid/pull/170>. The primary motivation >> for using this version was to align with hbase-thirdparty, which also uses >> Protobuf 4. However, during testing, I encountered the following failure: >> >> java.lang.NoSuchMethodError:org.apache.hadoop.hdfs.server.namenode.FsImageProto$FileSummary.makeExtensionsImmutable()V >> This error suggests that Hadoop is still using an older version of >> Protobuf, and this is causing compatibility issues. >> I also tried using Protobuf version 3.25.5, and it worked without any test >> failures. However, this is not aligned with the Protobuf version used in >> hbase-thirdparty, and if we try to fully switch to hbase-thirdparty's >> shaded Protobuf, we are likely to run into the same issues that I faced >> with Protobuf 4. >> >> To resolve this, my current suggestion is to create a new Omid-thirdparty >> repository and shade Protobuf there. This would help us isolate OMID's >> dependencies and avoid any conflicts with Hadoop's older Protobuf version. >> >> Do you think creating an Omid-thirdparty repository for shading Protobuf >> is >> a viable solution? >> Are there any other options that might avoid introducing a new repository >> while still addressing the compatibility issues? >> >> Thanks in advance for your input! >> Best, >> Villő >> > > > -- > *István Tóth* | Sr. Staff Software Engineer > *Email*: st...@cloudera.com > cloudera.com <https://www.cloudera.com> > [image: Cloudera] <https://www.cloudera.com/> > [image: Cloudera on Twitter] <https://twitter.com/cloudera> [image: > Cloudera on Facebook] <https://www.facebook.com/cloudera> [image: > Cloudera on LinkedIn] <https://www.linkedin.com/company/cloudera> > ------------------------------ > ------------------------------ > -- *István Tóth* | Sr. Staff Software Engineer *Email*: st...@cloudera.com cloudera.com <https://www.cloudera.com> [image: Cloudera] <https://www.cloudera.com/> [image: Cloudera on Twitter] <https://twitter.com/cloudera> [image: Cloudera on Facebook] <https://www.facebook.com/cloudera> [image: Cloudera on LinkedIn] <https://www.linkedin.com/company/cloudera> ------------------------------ ------------------------------
