I am seeing several commits in ozone-2.0 after ozone-2.0.0-RC1. Anything else critical that we must resolve for 2.0.0? I plan to make a RC2 for vote later tonight.
ea8762e509 (origin/ozone-2.0) HDDS-12750. Move StorageTypeProto from ScmServerDatanodeHeartbeatProtocol.proto to hdds.proto (#8208) 24060a598c HDDS-12751. native build fails with CMake 4 (#8215) 18f8e3fc0c HDDS-12703. Close pipeline command should display error on failure (#8169) 847f9eb7a1 HDDS-12736. Bump hadoop-thirdparty to 1.4.0 (#8193) af3e6c1525 HDDS-12724. hdds-rocks-native build fails if JAVA_HOME not set (#8183) 85363a2ecb HDDS-12733. Bump axios to 0.30.0 (#8189) aa2fa15215 (HEAD) HDDS-12327. HDDS-12668. Fix HSync upgrade test failure in non-HA upgrade test (#8171) On Thu, Apr 3, 2025 at 11:23 AM Tsz-Wo Nicholas Sze <szets...@gmail.com> wrote: > The PR [3] is merged. Thanks everyone for helping out! > > > ... let's also rename ScmServerDatanodeHeartbeatProtocol.proto to > StorageContainerDatanodeProtocol.proto, i.e. make it consistent with its > java_outer_classname. > > BTW, we have not done the rename due to the 1-1-1 Best Practice [5]. We > probably should follow it. > > Tsz-Wo > [3] https://github.com/apache/ozone/pull/8208 > [5] https://protobuf.dev/best-practices/1-1-1/ > > On Thu, Apr 3, 2025 at 10:43 AM Wei-Chiu Chuang <weic...@apache.org> > wrote: > > > I reviewed and +1 the PR. Looks good to me. > > > > Is there a design doc for the overall Ozone Storage Policy support? Under > > HDDS-11233 <https://issues.apache.org/jira/browse/HDDS-11233> I see no > > design docs attached, and there's none in > > https://ozone.apache.org/docs/edge/design.html > > > > There are several similar proposal HDDS-7711 > > <https://issues.apache.org/jira/browse/HDDS-7711> HDDS-3755 > > <https://issues.apache.org/jira/browse/HDDS-3755> > > they have design doc attached to them. Should I assume the HDDS-11233 > will > > be following one of the designs? > > > > On Tue, Apr 1, 2025 at 3:55 PM Tsz-Wo Nicholas Sze <szets...@gmail.com> > > wrote: > > > > > > ... Filed https://issues.apache.org/jira/browse/HDDS-12750 > > > > > > Submitted a pull request [3] and compared the before and the after. > The > > > change is > > > - wire/binary compatible and > > > - protobuf text format [4] compatible but > > > - Java API incompatible. > > > > > > The wire/binary format and the protobuf text format indeed remain > > > unchanged. > > > > > > The Java API incompatibility seems okay since > > > (i) users should use StorageType instead of the proto Java class, and > > > (ii) this is a major version -- we do allow incompatible change. > > > Anyone directly using the proto Java class does have to update the > > import. > > > It is a simple fix. > > > > > > Please review the PR [3]. Thank you in advance. > > > > > > Tsz-Wo > > > [3] https://github.com/apache/ozone/pull/8208 > > > [4] https://protobuf.dev/reference/protobuf/textformat-spec/ > > > > > > On Mon, Mar 31, 2025 at 4:13 PM Tsz-Wo Nicholas Sze < > szets...@gmail.com> > > > wrote: > > > > > > > Hi dev, > > > > > > > > Since this is a now-or-next-year situation, let me split the change > > that > > > > requires updating protolock file from HDDS-11351. Filed > > > > https://issues.apache.org/jira/browse/HDDS-12750 > > > > > > > > Tsz-Wo > > > > > > > > > > > > On Sun, Mar 30, 2025 at 2:04 PM Tsz-Wo Nicholas Sze < > > szets...@gmail.com> > > > > wrote: > > > > > > > >> Hi Wei-Chiu, > > > >> > > > >> (1) A reason for not yet being able to merge the PR is the protolock > > > >> file. We are not sure how and when to update it for such a change. > > The > > > >> "Build and commit the proto.lock change" section in our release > > > guideline > > > >> [2] does not mention it. > > > >> > > > >> (2) For future incompatible changes, we must wait for the next major > > > >> version. Anyway, this PR definitely is a good improvement. > > > >> > > > >> (3) If we are not doing it now, I guess it may be hard to get it in > a > > > >> maintenance release such as 2.0.1. This kind of change is usually > > > >> unwelcome in a maintenance release. > > > >> > > > >> XiChen, not sure if you could work on HDDS-11351 in a timely manner. > > > If > > > >> not, I could continue the work. Since we already know what to do, > we > > > >> should be able to merge the change within a week. > > > >> > > > >> Tsz-Wo > > > >> [2] > > > >> > > > > > > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=173085374#OzoneReleaseGuideline-Buildandcommittheproto.lockchange > > > >> > > > >> > > > >> > > > >> On Sun, Mar 30, 2025 at 1:12 PM Wei-Chiu Chuang <weic...@apache.org > > > > > >> wrote: > > > >> > > > >>> Thanks for bringing it up! > > > >>> > > > >>> I'm not against including it but > > > >>> > > > >>> (1) the PR has stalled for a few months now. Do we think it can be > > done > > > >>> soon? > > > >>> (2) there's no guarantee the PR will be the final version. What if > > > later > > > >>> on > > > >>> we realize we need to change protobuf again? > > > >>> (3) if it's a compatible change, it doesn't need to be in 2.0.0. > > > >>> > > > >>> On Sat, Mar 29, 2025 at 12:23 PM Tsz-Wo Nicholas Sze < > > > szets...@gmail.com > > > >>> > > > > >>> wrote: > > > >>> > > > >>> > More info on HDDS-11351: > > > >>> > > > > >>> > TLDR: The change is wire compatible but requires updating > > protolock. > > > >>> > > > > >>> > XiChen pointed out that both hdds.proto and > > > >>> > ScmServerDatanodeHeartbeatProtocol.proto have the same proto > > package > > > >>> > "hadoop.hdds". So, we could safely move StorageTypeProto from > > > >>> > ScmServerDatanodeHeartbeatProtocol.proto to hdds.proto. The only > > > >>> difference > > > >>> > is the java_outer_classname. Fortunately, > > > >>> > ScmServerDatanodeHeartbeatProtocol.proto is a non-user facing > > > internal > > > >>> > protocol and the change is wire compatible. (It is API > > incompatible > > > >>> but it > > > >>> > is fine since the protocol is not a public API.) So there are no > > > >>> > compatibility issues. > > > >>> > > > > >>> > A problem is that we need to update the protolock file. If we > are > > > >>> going to > > > >>> > do it, let's also rename ScmServerDatanodeHeartbeatProtocol.proto > > to > > > >>> > StorageContainerDatanodeProtocol.proto, i.e. make it consistent > > with > > > >>> its > > > >>> > java_outer_classname. > > > >>> > > > > >>> > Tsz-Wo > > > >>> > [1] > > https://github.com/apache/ozone/pull/7109#discussion_r2008750162 > > > >>> > > > > >>> > > > > >>> > On Sat, Mar 29, 2025 at 10:04 AM Tsz-Wo Nicholas Sze < > > > >>> szets...@gmail.com> > > > >>> > wrote: > > > >>> > > > > >>> > > Hi Ozone dev, > > > >>> > > > > > >>> > > HDDS-11351 is unifying the protobuf definition of StorageType. > > How > > > >>> about > > > >>> > > we get it in 2.0.0? > > > >>> > > > > > >>> > > Sorry that I came with this idea late. > > > >>> > > > > > >>> > > Tsz-Wo > > > >>> > > > > > >>> > > > > > >>> > > > > > >>> > > On Wed, Mar 26, 2025 at 11:05 PM Wei-Chiu Chuang < > > > weic...@apache.org > > > >>> > > > > >>> > > wrote: > > > >>> > > > > > >>> > >> By the way, build environment: > > > >>> > >> > > > >>> > >> x86, Amazon Linux, OpenJDK8, Maven 3.9.9, gcc 11 > > > >>> > >> > > > >>> > >> build parameters: > > > >>> > >> mvn clean install -Dmaven.javadoc.skip=true -DskipTests > > > >>> -Psign,dist,src > > > >>> > >> -Dtar -Dgpg.keyname="$CODESIGNINGKEY" -Drocks_tools_native > > > >>> > >> > > > >>> > >> # cat /etc/amazon-linux-release > > > >>> > >> Amazon Linux release 2023.6.20250317 (Amazon Linux) > > > >>> > >> > > > >>> > >> # java -version > > > >>> > >> openjdk version "1.8.0_442" > > > >>> > >> OpenJDK Runtime Environment Corretto-8.442.06.1 (build > > > >>> 1.8.0_442-b06) > > > >>> > >> OpenJDK 64-Bit Server VM Corretto-8.442.06.1 (build > 25.442-b06, > > > >>> mixed > > > >>> > >> mode) > > > >>> > >> > > > >>> > >> # mvn -v > > > >>> > >> Apache Maven 3.9.9 (8e8579a9e76f7d015ee5ec7bfcdc97d260186937) > > > >>> > >> Maven home: /root/apache-maven-3.9.9 > > > >>> > >> Java version: 1.8.0_442, vendor: Amazon.com Inc., runtime: > > > >>> > >> /usr/lib/jvm/java-1.8.0-amazon-corretto.x86_64/jre > > > >>> > >> Default locale: en, platform encoding: UTF-8 > > > >>> > >> OS name: "linux", version: "6.1.130-139.222.amzn2023.x86_64", > > > arch: > > > >>> > >> "amd64", family: "unix" > > > >>> > >> > > > >>> > >> # g++ -v > > > >>> > >> Using built-in specs. > > > >>> > >> COLLECT_GCC=g++ > > > >>> > >> > > > >>> > > COLLECT_LTO_WRAPPER=/usr/libexec/gcc/x86_64-amazon-linux/11/lto-wrapper > > > >>> > >> OFFLOAD_TARGET_NAMES=nvptx-none > > > >>> > >> OFFLOAD_TARGET_DEFAULT=1 > > > >>> > >> Target: x86_64-amazon-linux > > > >>> > >> Configured with: ../configure --enable-bootstrap > > --enable-host-pie > > > >>> > >> --enable-host-bind-now --enable-languages=c,c++,fortran,lto > > > >>> > --prefix=/usr > > > >>> > >> --mandir=/usr/share/man --infodir=/usr/share/info > --with-bugurl= > > > >>> > >> https://github.com/amazonlinux/amazon-linux-2022 > > --enable-shared > > > >>> > >> --enable-threads=posix --enable-checking=release > > > --with-system-zlib > > > >>> > >> --enable-__cxa_atexit --disable-libunwind-exceptions > > > >>> > >> --enable-gnu-unique-object --enable-linker-build-id > > > >>> > >> --with-gcc-major-version-only --enable-plugin > > > >>> --enable-initfini-array > > > >>> > >> > > > >>> > >> > > > >>> > > > > >>> > > > > > > --with-isl=/builddir/build/BUILD/gcc-11.5.0-20240719/obj-x86_64-amazon-linux/isl-install > > > >>> > >> --enable-multilib --with-linker-hash-style=gnu > > > >>> > >> --enable-offload-targets=nvptx-none --without-cuda-driver > > > >>> > >> --enable-gnu-indirect-function --enable-cet > --with-tune=generic > > > >>> > >> --with-arch_64=x86-64-v2 --with-arch_32=x86-64 > > > >>> > --build=x86_64-amazon-linux > > > >>> > >> --with-build-config=bootstrap-lto > --enable-link-serialization=1 > > > >>> > >> Thread model: posix > > > >>> > >> Supported LTO compression algorithms: zlib zstd > > > >>> > >> gcc version 11.5.0 20240719 (Red Hat 11.5.0-5) (GCC) > > > >>> > >> > > > >>> > >> > > > >>> > >> > > > >>> > >> > > > >>> > >> On Wed, Mar 26, 2025 at 10:58 PM Wei-Chiu Chuang < > > > >>> weic...@apache.org> > > > >>> > >> wrote: > > > >>> > >> > > > >>> > >> > Here's RC1 vote: > > > >>> > >> > > > > >>> > >> > Git tag: > > > >>> https://github.com/apache/ozone/releases/tag/ozone-2.0.0-RC1 > > > >>> > >> > A total of 1699 resolved jiras: > > > >>> > >> > > > > >>> > >> > > > >>> > > > > >>> > > > > > > https://issues.apache.org/jira/issues/?jql=project%20%3D%20HDDS%20AND%20status%20in%20(Resolved%2C%20Closed)%20AND%20fixVersion%20%3D%202.0.0 > > > >>> > >> > > > > >>> > >> > Source code and binary tarball: > > > >>> > >> > https://dist.apache.org/repos/dist/dev/ozone/2.0.0-rc1/ > > > >>> > >> > Maven artifacts: > > > >>> > >> > > > > >>> > > > > >>> > > > > https://repository.apache.org/content/repositories/orgapacheozone-1030/ > > > >>> > >> > PGP key: https://dist.apache.org/repos/dist/dev/ozone/KEYS > > > >>> > >> > Fingerprint: 3ED23305D7631918 > > > >>> > >> > > > > >>> > >> > RC0 and RC1 diff: > > > >>> > >> > > > > >>> > >> > > > >>> > > > > >>> > > > > > > https://github.com/apache/ozone/compare/ozone-2.0.0-RC0...ozone-2.0.0-RC1 > > > >>> > >> > 7 commits: > > > >>> > >> > > > > >>> > >> > HDDS-12327. HDDS-12668. Fix HSync upgrade test failure in > > non-HA > > > >>> > upgrade > > > >>> > >> > test (#8171) > > > >>> > >> > HDDS-12684. Update NOTICE and LICENSE file (#8160) > > > >>> > >> > HDDS-12327. Restore non-HA (to HA) upgrade test (#7880) > > > >>> > >> > HDDS-12662. Rename upgrade callback directory 1.5.0 to 2.0.0 > > > >>> (#8131) > > > >>> > >> > HDDS-12668. HSync upgrade test failure (#8137) > > > >>> > >> > HDDS-12671. Include .editorconfig and .run in source tarball > > > >>> (#8139) > > > >>> > >> > HDDS-12666. Remove -SNAPSHOT from OZONE_CURRENT_VERSION in > > > upgrade > > > >>> > test > > > >>> > >> > (#8136) > > > >>> > >> > > > > >>> > >> > Please also take a look at the release page draft: > > > >>> > >> > https://jojochuang.github.io/release/2.0.0/ > > > >>> > >> > Full CHANGELOGS > > > >>> > >> > https://jojochuang.github.io/release/2.0.0/changelog.2.0.0/ > > > >>> > >> > Full ReleaseNotes: > > > >>> > >> > > > https://jojochuang.github.io/release/2.0.0/releasenotes.2.0.0/ > > > >>> > >> > > > > >>> > >> > changelogs and releasenotes are generated by Apache Yetus > > > >>> > >> releasedocmaker: > > > >>> > >> > ./releasedocmaker --project HDDS --version 2.0.0 > > --projecttitle > > > >>> > "Apache > > > >>> > >> > Ozone" --fileversions --dirversions --usetoday > > > >>> > >> > > > > >>> > >> > On Fri, Mar 21, 2025 at 3:42 PM Wei-Chiu Chuang < > > > >>> weic...@apache.org> > > > >>> > >> > wrote: > > > >>> > >> > > > > >>> > >> >> Hi Ozone community, > > > >>> > >> >> > > > >>> > >> >> Please try out and cast your vote for the Ozone 2.0.0 > release > > > >>> > >> candidate 0. > > > >>> > >> >> > > > >>> > >> >> This is a huge release, containing 1691 resolved jiras, > > > numerous > > > >>> > >> features > > > >>> > >> >> and stability improvements. > > > >>> > >> >> > > > >>> > >> >> Release process: > > > >>> > >> >> > > > >>> > >> > > > >>> > > > > >>> > > > > > > https://ozone-site-v2.staged.apache.org/docs/developer-guide/project/release-guide/ > > > >>> > >> >> Git tag: > > > >>> > https://github.com/apache/ozone/releases/tag/ozone-2.0.0-RC0 > > > >>> > >> >> All resolved jiras: > > > >>> > >> >> > > > >>> > >> > > > >>> > > > > >>> > > > > > > https://issues.apache.org/jira/issues/?jql=project%20%3D%20HDDS%20AND%20status%20in%20(Resolved%2C%20Closed)%20AND%20fixVersion%20%3D%202.0.0 > > > >>> > >> >> Source code and binary tarball: > > > >>> > >> >> > https://dist.apache.org/repos/dist/release/ozone/2.0.0-rc0/ > > > >>> > >> >> The Maven artifacts are staged at: > > > >>> > >> >> > > > >>> > >> > > > >>> > > > > https://repository.apache.org/content/repositories/orgapacheozone-1029/ > > > >>> > >> >> PGP key: https://dist.apache.org/repos/dist/dev/ozone/KEYS > > > >>> > >> >> Fingerprint: 3ED23305D7631918 > > > >>> > >> >> > > > >>> > >> >> Per Apache policy, this release candidate vote will open > for > > 7 > > > >>> days > > > >>> > >> until > > > >>> > >> >> the end of March 28th 2025. PMC members can cast binding > > votes > > > >>> while > > > >>> > >> >> committers and community contributors are welcomed to cast > > > >>> > non-binding > > > >>> > >> >> votes. > > > >>> > >> >> > > > >>> > >> >> Best regards, > > > >>> > >> >> Weichiu > > > >>> > >> >> > > > >>> > >> > > > > >>> > >> > > > >>> > > > > > >>> > > > > >>> > > > >> > > > > > >
