Unfortunately Protobuf has made the decision of breaking binary compatibility between the frequently changing major versions, creating a Guava-like problem.
Since Omid (or rather HBase/Phoenix) is frequently used together with other Hadoop-adjacent software depending on Protobuf, many of which use unallocated protobuf, we need to solve this somehow. To avoid adding another repo, we could add protobuf to the existing phoenix-thirdparty repo, since Omid is a Phoenix subproject now. Istvan On Wed, Oct 2, 2024 at 1:51 PM Villő Szűcs <szucsvi...@cloudera.com.invalid> wrote: > Hey all, > I've been working on OMID-284 and OMID-276, specifically attempting to > upgrade Protobuf in OMID, and I've encountered several challenges that I > want to bring to the community for discussion. > > I tried upgrading Protobuf to version 4.28.2: see PR #170 > <https://github.com/apache/phoenix-omid/pull/170>. The primary motivation > for using this version was to align with hbase-thirdparty, which also uses > Protobuf 4. However, during testing, I encountered the following failure: > > java.lang.NoSuchMethodError:org.apache.hadoop.hdfs.server.namenode.FsImageProto$FileSummary.makeExtensionsImmutable()V > This error suggests that Hadoop is still using an older version of > Protobuf, and this is causing compatibility issues. > I also tried using Protobuf version 3.25.5, and it worked without any test > failures. However, this is not aligned with the Protobuf version used in > hbase-thirdparty, and if we try to fully switch to hbase-thirdparty's > shaded Protobuf, we are likely to run into the same issues that I faced > with Protobuf 4. > > To resolve this, my current suggestion is to create a new Omid-thirdparty > repository and shade Protobuf there. This would help us isolate OMID's > dependencies and avoid any conflicts with Hadoop's older Protobuf version. > > Do you think creating an Omid-thirdparty repository for shading Protobuf is > a viable solution? > Are there any other options that might avoid introducing a new repository > while still addressing the compatibility issues? > > Thanks in advance for your input! > Best, > Villő > -- *István Tóth* | Sr. Staff Software Engineer *Email*: st...@cloudera.com cloudera.com <https://www.cloudera.com> [image: Cloudera] <https://www.cloudera.com/> [image: Cloudera on Twitter] <https://twitter.com/cloudera> [image: Cloudera on Facebook] <https://www.facebook.com/cloudera> [image: Cloudera on LinkedIn] <https://www.linkedin.com/company/cloudera> ------------------------------ ------------------------------
