Re: [VOTE] Release Apache Hadoop 3.4.0 (RC1)
We will backport all the PRs from the branch-3.4 branch to branch-3.4.0, and use this to release hadoop-3.4.0-RC2. It should include the s3a checksum and region stuff. Best Regards, Shilun Fan. On Mon, Feb 12, 2024 at 7:10 PM Steve Loughran wrote: > so it'll have the s3a checksum and region stuff? that'd be wonderful! I've > been assuming i'd have to push out a 3.4.1 release for that alone. > > On Sat, 10 Feb 2024 at 23:36, slfan1989 wrote: > >> We will end the voting for hadoop-3.4.0-RC1 and will open the voting for >> hadoop-3.4.0-RC2 in 1-2 days. >> >> * hadoop-3.4.0-RC2 will contain all commits in branch-3.4, and newly >> submitted(in branch-3.4) commits will be cherry-picked to branch-3.4.0. >> * hadoop-3.4.0-RC2 will use hadoop-thirdparty-1.2.0. >> >> Best Regards, >> Shilun Fan. >> >> On Mon, Jan 29, 2024 at 10:33 PM slfan1989 wrote: >> >> > Apache Hadoop 3.4.0 >> > >> > Xiaoqiao He and I have put together a release candidate (RC1) for Hadoop >> > 3.4.0. >> > >> > What we would like is for anyone who can to verify the tarballs, >> especially >> > anyone who can try the arm64 binaries as we want to include them too. >> > >> > The RC is available at: >> > https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.4.0-RC1/ >> > >> > The git tag is release-3.4.0-RC1, commit 7e2edd8c5d1 >> > >> > The maven artifacts are staged at >> > >> https://repository.apache.org/content/repositories/orgapachehadoop-1395/ >> > >> > You can find my public key at: >> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS >> > >> > Change log >> > >> https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.4.0-RC1/CHANGELOG.md >> > >> > Release notes >> > >> > >> https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.4.0-RC1/RELEASENOTES.md >> > >> > This is off branch-3.4.0 and is the first big release since 3.3.6. >> > >> > Key changes include >> > >> > * S3A: Upgrade AWS SDK to V2 >> > * HDFS DataNode Split one FsDatasetImpl lock to volume grain locks >> > * YARN Federation improvements >> > * YARN Capacity Scheduler improvements >> > * HDFS RBF: Code Enhancements, New Features, and Bug Fixes >> > * HDFS EC: Code Enhancements and Bug Fixes >> > * Transitive CVE fixes >> > >> > Differences from RC0 >> > >> > * We've improved Hadoop 3.4.0 Highlight big features and improvements. >> > * Confirmed the JIRA status of Hadoop, HDFS, YARN, and MAPREDUCE >> modules. >> > * Use validate-hadoop-client-artifacts[1] for packaging and >> verification. >> > >> > Note, because the arm64 binaries are built separately on a different >> > platform and JVM, their jar files may not match those of the x86 >> > release -and therefore the maven artifacts. I don't think this is >> > an issue (the ASF actually releases source tarballs, the binaries are >> > there for help only, though with the maven repo that's a bit blurred). >> > >> > The only way to be consistent would actually untar the x86.tar.gz, >> > overwrite its binaries with the arm stuff, retar, sign and push out >> > for the vote. Even automating that would be risky. >> > >> > [1] validate-hadoop-client-artifacts: >> > https://github.com/steveloughran/validate-hadoop-client-artifacts >> > Thanks to steve for providing validate-hadoop-client-artifacts. >> > >> > Best Regards, >> > Shilun Fan. >> > >> >
Re: [VOTE] Release Apache Hadoop 3.4.0 (RC1)
so it'll have the s3a checksum and region stuff? that'd be wonderful! I've been assuming i'd have to push out a 3.4.1 release for that alone. On Sat, 10 Feb 2024 at 23:36, slfan1989 wrote: > We will end the voting for hadoop-3.4.0-RC1 and will open the voting for > hadoop-3.4.0-RC2 in 1-2 days. > > * hadoop-3.4.0-RC2 will contain all commits in branch-3.4, and newly > submitted(in branch-3.4) commits will be cherry-picked to branch-3.4.0. > * hadoop-3.4.0-RC2 will use hadoop-thirdparty-1.2.0. > > Best Regards, > Shilun Fan. > > On Mon, Jan 29, 2024 at 10:33 PM slfan1989 wrote: > > > Apache Hadoop 3.4.0 > > > > Xiaoqiao He and I have put together a release candidate (RC1) for Hadoop > > 3.4.0. > > > > What we would like is for anyone who can to verify the tarballs, > especially > > anyone who can try the arm64 binaries as we want to include them too. > > > > The RC is available at: > > https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.4.0-RC1/ > > > > The git tag is release-3.4.0-RC1, commit 7e2edd8c5d1 > > > > The maven artifacts are staged at > > https://repository.apache.org/content/repositories/orgapachehadoop-1395/ > > > > You can find my public key at: > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS > > > > Change log > > > https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.4.0-RC1/CHANGELOG.md > > > > Release notes > > > > > https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.4.0-RC1/RELEASENOTES.md > > > > This is off branch-3.4.0 and is the first big release since 3.3.6. > > > > Key changes include > > > > * S3A: Upgrade AWS SDK to V2 > > * HDFS DataNode Split one FsDatasetImpl lock to volume grain locks > > * YARN Federation improvements > > * YARN Capacity Scheduler improvements > > * HDFS RBF: Code Enhancements, New Features, and Bug Fixes > > * HDFS EC: Code Enhancements and Bug Fixes > > * Transitive CVE fixes > > > > Differences from RC0 > > > > * We've improved Hadoop 3.4.0 Highlight big features and improvements. > > * Confirmed the JIRA status of Hadoop, HDFS, YARN, and MAPREDUCE modules. > > * Use validate-hadoop-client-artifacts[1] for packaging and verification. > > > > Note, because the arm64 binaries are built separately on a different > > platform and JVM, their jar files may not match those of the x86 > > release -and therefore the maven artifacts. I don't think this is > > an issue (the ASF actually releases source tarballs, the binaries are > > there for help only, though with the maven repo that's a bit blurred). > > > > The only way to be consistent would actually untar the x86.tar.gz, > > overwrite its binaries with the arm stuff, retar, sign and push out > > for the vote. Even automating that would be risky. > > > > [1] validate-hadoop-client-artifacts: > > https://github.com/steveloughran/validate-hadoop-client-artifacts > > Thanks to steve for providing validate-hadoop-client-artifacts. > > > > Best Regards, > > Shilun Fan. > > >
Re: [VOTE] Release Apache Hadoop 3.4.0 (RC1)
We will end the voting for hadoop-3.4.0-RC1 and will open the voting for hadoop-3.4.0-RC2 in 1-2 days. * hadoop-3.4.0-RC2 will contain all commits in branch-3.4, and newly submitted(in branch-3.4) commits will be cherry-picked to branch-3.4.0. * hadoop-3.4.0-RC2 will use hadoop-thirdparty-1.2.0. Best Regards, Shilun Fan. On Mon, Jan 29, 2024 at 10:33 PM slfan1989 wrote: > Apache Hadoop 3.4.0 > > Xiaoqiao He and I have put together a release candidate (RC1) for Hadoop > 3.4.0. > > What we would like is for anyone who can to verify the tarballs, especially > anyone who can try the arm64 binaries as we want to include them too. > > The RC is available at: > https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.4.0-RC1/ > > The git tag is release-3.4.0-RC1, commit 7e2edd8c5d1 > > The maven artifacts are staged at > https://repository.apache.org/content/repositories/orgapachehadoop-1395/ > > You can find my public key at: > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS > > Change log > https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.4.0-RC1/CHANGELOG.md > > Release notes > > https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.4.0-RC1/RELEASENOTES.md > > This is off branch-3.4.0 and is the first big release since 3.3.6. > > Key changes include > > * S3A: Upgrade AWS SDK to V2 > * HDFS DataNode Split one FsDatasetImpl lock to volume grain locks > * YARN Federation improvements > * YARN Capacity Scheduler improvements > * HDFS RBF: Code Enhancements, New Features, and Bug Fixes > * HDFS EC: Code Enhancements and Bug Fixes > * Transitive CVE fixes > > Differences from RC0 > > * We've improved Hadoop 3.4.0 Highlight big features and improvements. > * Confirmed the JIRA status of Hadoop, HDFS, YARN, and MAPREDUCE modules. > * Use validate-hadoop-client-artifacts[1] for packaging and verification. > > Note, because the arm64 binaries are built separately on a different > platform and JVM, their jar files may not match those of the x86 > release -and therefore the maven artifacts. I don't think this is > an issue (the ASF actually releases source tarballs, the binaries are > there for help only, though with the maven repo that's a bit blurred). > > The only way to be consistent would actually untar the x86.tar.gz, > overwrite its binaries with the arm stuff, retar, sign and push out > for the vote. Even automating that would be risky. > > [1] validate-hadoop-client-artifacts: > https://github.com/steveloughran/validate-hadoop-client-artifacts > Thanks to steve for providing validate-hadoop-client-artifacts. > > Best Regards, > Shilun Fan. >
