> ... Filed https://issues.apache.org/jira/browse/HDDS-12750
Submitted a pull request [3] and compared the before and the after. The change is - wire/binary compatible and - protobuf text format [4] compatible but - Java API incompatible. The wire/binary format and the protobuf text format indeed remain unchanged. The Java API incompatibility seems okay since (i) users should use StorageType instead of the proto Java class, and (ii) this is a major version -- we do allow incompatible change. Anyone directly using the proto Java class does have to update the import. It is a simple fix. Please review the PR [3]. Thank you in advance. Tsz-Wo [3] https://github.com/apache/ozone/pull/8208 [4] https://protobuf.dev/reference/protobuf/textformat-spec/ On Mon, Mar 31, 2025 at 4:13 PM Tsz-Wo Nicholas Sze <szets...@gmail.com> wrote: > Hi dev, > > Since this is a now-or-next-year situation, let me split the change that > requires updating protolock file from HDDS-11351. Filed > https://issues.apache.org/jira/browse/HDDS-12750 > > Tsz-Wo > > > On Sun, Mar 30, 2025 at 2:04 PM Tsz-Wo Nicholas Sze <szets...@gmail.com> > wrote: > >> Hi Wei-Chiu, >> >> (1) A reason for not yet being able to merge the PR is the protolock >> file. We are not sure how and when to update it for such a change. The >> "Build and commit the proto.lock change" section in our release guideline >> [2] does not mention it. >> >> (2) For future incompatible changes, we must wait for the next major >> version. Anyway, this PR definitely is a good improvement. >> >> (3) If we are not doing it now, I guess it may be hard to get it in a >> maintenance release such as 2.0.1. This kind of change is usually >> unwelcome in a maintenance release. >> >> XiChen, not sure if you could work on HDDS-11351 in a timely manner. If >> not, I could continue the work. Since we already know what to do, we >> should be able to merge the change within a week. >> >> Tsz-Wo >> [2] >> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=173085374#OzoneReleaseGuideline-Buildandcommittheproto.lockchange >> >> >> >> On Sun, Mar 30, 2025 at 1:12 PM Wei-Chiu Chuang <weic...@apache.org> >> wrote: >> >>> Thanks for bringing it up! >>> >>> I'm not against including it but >>> >>> (1) the PR has stalled for a few months now. Do we think it can be done >>> soon? >>> (2) there's no guarantee the PR will be the final version. What if later >>> on >>> we realize we need to change protobuf again? >>> (3) if it's a compatible change, it doesn't need to be in 2.0.0. >>> >>> On Sat, Mar 29, 2025 at 12:23 PM Tsz-Wo Nicholas Sze <szets...@gmail.com >>> > >>> wrote: >>> >>> > More info on HDDS-11351: >>> > >>> > TLDR: The change is wire compatible but requires updating protolock. >>> > >>> > XiChen pointed out that both hdds.proto and >>> > ScmServerDatanodeHeartbeatProtocol.proto have the same proto package >>> > "hadoop.hdds". So, we could safely move StorageTypeProto from >>> > ScmServerDatanodeHeartbeatProtocol.proto to hdds.proto. The only >>> difference >>> > is the java_outer_classname. Fortunately, >>> > ScmServerDatanodeHeartbeatProtocol.proto is a non-user facing internal >>> > protocol and the change is wire compatible. (It is API incompatible >>> but it >>> > is fine since the protocol is not a public API.) So there are no >>> > compatibility issues. >>> > >>> > A problem is that we need to update the protolock file. If we are >>> going to >>> > do it, let's also rename ScmServerDatanodeHeartbeatProtocol.proto to >>> > StorageContainerDatanodeProtocol.proto, i.e. make it consistent with >>> its >>> > java_outer_classname. >>> > >>> > Tsz-Wo >>> > [1] https://github.com/apache/ozone/pull/7109#discussion_r2008750162 >>> > >>> > >>> > On Sat, Mar 29, 2025 at 10:04 AM Tsz-Wo Nicholas Sze < >>> szets...@gmail.com> >>> > wrote: >>> > >>> > > Hi Ozone dev, >>> > > >>> > > HDDS-11351 is unifying the protobuf definition of StorageType. How >>> about >>> > > we get it in 2.0.0? >>> > > >>> > > Sorry that I came with this idea late. >>> > > >>> > > Tsz-Wo >>> > > >>> > > >>> > > >>> > > On Wed, Mar 26, 2025 at 11:05 PM Wei-Chiu Chuang <weic...@apache.org >>> > >>> > > wrote: >>> > > >>> > >> By the way, build environment: >>> > >> >>> > >> x86, Amazon Linux, OpenJDK8, Maven 3.9.9, gcc 11 >>> > >> >>> > >> build parameters: >>> > >> mvn clean install -Dmaven.javadoc.skip=true -DskipTests >>> -Psign,dist,src >>> > >> -Dtar -Dgpg.keyname="$CODESIGNINGKEY" -Drocks_tools_native >>> > >> >>> > >> # cat /etc/amazon-linux-release >>> > >> Amazon Linux release 2023.6.20250317 (Amazon Linux) >>> > >> >>> > >> # java -version >>> > >> openjdk version "1.8.0_442" >>> > >> OpenJDK Runtime Environment Corretto-8.442.06.1 (build >>> 1.8.0_442-b06) >>> > >> OpenJDK 64-Bit Server VM Corretto-8.442.06.1 (build 25.442-b06, >>> mixed >>> > >> mode) >>> > >> >>> > >> # mvn -v >>> > >> Apache Maven 3.9.9 (8e8579a9e76f7d015ee5ec7bfcdc97d260186937) >>> > >> Maven home: /root/apache-maven-3.9.9 >>> > >> Java version: 1.8.0_442, vendor: Amazon.com Inc., runtime: >>> > >> /usr/lib/jvm/java-1.8.0-amazon-corretto.x86_64/jre >>> > >> Default locale: en, platform encoding: UTF-8 >>> > >> OS name: "linux", version: "6.1.130-139.222.amzn2023.x86_64", arch: >>> > >> "amd64", family: "unix" >>> > >> >>> > >> # g++ -v >>> > >> Using built-in specs. >>> > >> COLLECT_GCC=g++ >>> > >> >>> COLLECT_LTO_WRAPPER=/usr/libexec/gcc/x86_64-amazon-linux/11/lto-wrapper >>> > >> OFFLOAD_TARGET_NAMES=nvptx-none >>> > >> OFFLOAD_TARGET_DEFAULT=1 >>> > >> Target: x86_64-amazon-linux >>> > >> Configured with: ../configure --enable-bootstrap --enable-host-pie >>> > >> --enable-host-bind-now --enable-languages=c,c++,fortran,lto >>> > --prefix=/usr >>> > >> --mandir=/usr/share/man --infodir=/usr/share/info --with-bugurl= >>> > >> https://github.com/amazonlinux/amazon-linux-2022 --enable-shared >>> > >> --enable-threads=posix --enable-checking=release --with-system-zlib >>> > >> --enable-__cxa_atexit --disable-libunwind-exceptions >>> > >> --enable-gnu-unique-object --enable-linker-build-id >>> > >> --with-gcc-major-version-only --enable-plugin >>> --enable-initfini-array >>> > >> >>> > >> >>> > >>> --with-isl=/builddir/build/BUILD/gcc-11.5.0-20240719/obj-x86_64-amazon-linux/isl-install >>> > >> --enable-multilib --with-linker-hash-style=gnu >>> > >> --enable-offload-targets=nvptx-none --without-cuda-driver >>> > >> --enable-gnu-indirect-function --enable-cet --with-tune=generic >>> > >> --with-arch_64=x86-64-v2 --with-arch_32=x86-64 >>> > --build=x86_64-amazon-linux >>> > >> --with-build-config=bootstrap-lto --enable-link-serialization=1 >>> > >> Thread model: posix >>> > >> Supported LTO compression algorithms: zlib zstd >>> > >> gcc version 11.5.0 20240719 (Red Hat 11.5.0-5) (GCC) >>> > >> >>> > >> >>> > >> >>> > >> >>> > >> On Wed, Mar 26, 2025 at 10:58 PM Wei-Chiu Chuang < >>> weic...@apache.org> >>> > >> wrote: >>> > >> >>> > >> > Here's RC1 vote: >>> > >> > >>> > >> > Git tag: >>> https://github.com/apache/ozone/releases/tag/ozone-2.0.0-RC1 >>> > >> > A total of 1699 resolved jiras: >>> > >> > >>> > >> >>> > >>> https://issues.apache.org/jira/issues/?jql=project%20%3D%20HDDS%20AND%20status%20in%20(Resolved%2C%20Closed)%20AND%20fixVersion%20%3D%202.0.0 >>> > >> > >>> > >> > Source code and binary tarball: >>> > >> > https://dist.apache.org/repos/dist/dev/ozone/2.0.0-rc1/ >>> > >> > Maven artifacts: >>> > >> > >>> > >>> https://repository.apache.org/content/repositories/orgapacheozone-1030/ >>> > >> > PGP key: https://dist.apache.org/repos/dist/dev/ozone/KEYS >>> > >> > Fingerprint: 3ED23305D7631918 >>> > >> > >>> > >> > RC0 and RC1 diff: >>> > >> > >>> > >> >>> > >>> https://github.com/apache/ozone/compare/ozone-2.0.0-RC0...ozone-2.0.0-RC1 >>> > >> > 7 commits: >>> > >> > >>> > >> > HDDS-12327. HDDS-12668. Fix HSync upgrade test failure in non-HA >>> > upgrade >>> > >> > test (#8171) >>> > >> > HDDS-12684. Update NOTICE and LICENSE file (#8160) >>> > >> > HDDS-12327. Restore non-HA (to HA) upgrade test (#7880) >>> > >> > HDDS-12662. Rename upgrade callback directory 1.5.0 to 2.0.0 >>> (#8131) >>> > >> > HDDS-12668. HSync upgrade test failure (#8137) >>> > >> > HDDS-12671. Include .editorconfig and .run in source tarball >>> (#8139) >>> > >> > HDDS-12666. Remove -SNAPSHOT from OZONE_CURRENT_VERSION in upgrade >>> > test >>> > >> > (#8136) >>> > >> > >>> > >> > Please also take a look at the release page draft: >>> > >> > https://jojochuang.github.io/release/2.0.0/ >>> > >> > Full CHANGELOGS >>> > >> > https://jojochuang.github.io/release/2.0.0/changelog.2.0.0/ >>> > >> > Full ReleaseNotes: >>> > >> > https://jojochuang.github.io/release/2.0.0/releasenotes.2.0.0/ >>> > >> > >>> > >> > changelogs and releasenotes are generated by Apache Yetus >>> > >> releasedocmaker: >>> > >> > ./releasedocmaker --project HDDS --version 2.0.0 --projecttitle >>> > "Apache >>> > >> > Ozone" --fileversions --dirversions --usetoday >>> > >> > >>> > >> > On Fri, Mar 21, 2025 at 3:42 PM Wei-Chiu Chuang < >>> weic...@apache.org> >>> > >> > wrote: >>> > >> > >>> > >> >> Hi Ozone community, >>> > >> >> >>> > >> >> Please try out and cast your vote for the Ozone 2.0.0 release >>> > >> candidate 0. >>> > >> >> >>> > >> >> This is a huge release, containing 1691 resolved jiras, numerous >>> > >> features >>> > >> >> and stability improvements. >>> > >> >> >>> > >> >> Release process: >>> > >> >> >>> > >> >>> > >>> https://ozone-site-v2.staged.apache.org/docs/developer-guide/project/release-guide/ >>> > >> >> Git tag: >>> > https://github.com/apache/ozone/releases/tag/ozone-2.0.0-RC0 >>> > >> >> All resolved jiras: >>> > >> >> >>> > >> >>> > >>> https://issues.apache.org/jira/issues/?jql=project%20%3D%20HDDS%20AND%20status%20in%20(Resolved%2C%20Closed)%20AND%20fixVersion%20%3D%202.0.0 >>> > >> >> Source code and binary tarball: >>> > >> >> https://dist.apache.org/repos/dist/release/ozone/2.0.0-rc0/ >>> > >> >> The Maven artifacts are staged at: >>> > >> >> >>> > >> >>> https://repository.apache.org/content/repositories/orgapacheozone-1029/ >>> > >> >> PGP key: https://dist.apache.org/repos/dist/dev/ozone/KEYS >>> > >> >> Fingerprint: 3ED23305D7631918 >>> > >> >> >>> > >> >> Per Apache policy, this release candidate vote will open for 7 >>> days >>> > >> until >>> > >> >> the end of March 28th 2025. PMC members can cast binding votes >>> while >>> > >> >> committers and community contributors are welcomed to cast >>> > non-binding >>> > >> >> votes. >>> > >> >> >>> > >> >> Best regards, >>> > >> >> Weichiu >>> > >> >> >>> > >> > >>> > >> >>> > > >>> > >>> >>
