+1 (binding)
Thanks for the great work, Shilun Fan.
+ verified checksums and signatures of src and site tarballs.
+ built from source tarball with native profile enabled on Rocky Linux 8
and Ubuntu 22.04 (x86_64).
+ deployed pseudo cluster with and without kerberos enabled.
+ ran some example jobs on encryption zone.
+ ran some simple query against httpfs.
+ built site documentation and skimmed the contents.
+ skimmed the contents of pre-built site documentation.
- downstream like HBase can not be built against 3.4.0 without addressing
changes of transitive dependencies.
- OpenSSL feature of libhadoop.so does not work on the platform without
SM4 support (e.g. Red Hat) while it falls back to Java implementation.
I want HADOOP-17609 be in the next release (or OpenSSL 3 support considering
this)::
$ bin/hdfs dfs -put README.txt /zone1/
2024-03-08 15:57:42,685 WARN crypto.OpensslCipher: Failed to load OpenSSL
Cipher.
java.lang.UnsatisfiedLinkError: Cannot find AES-CTR/SM4-CTR support, is
your version of Openssl new enough?
at org.apache.hadoop.crypto.OpensslCipher.initIDs(Native Method)
Masatake Iwasaki
On 2024/03/05 7:07, slfan1989 wrote:
Hi folks,
Xiaoqiao He and I have put together a release candidate (RC3) for Hadoop
3.4.0.
What we would like is for anyone who can to verify the tarballs, especially
anyone who can try the arm64 binaries as we want to include them too.
The RC is available at:
https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.4.0-RC3/
The git tag is release-3.4.0-RC3, commit bd8b77f398f
The maven artifacts are staged at
https://repository.apache.org/content/repositories/orgapachehadoop-1408
You can find my public key at:
https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
Change log
https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.4.0-RC3/CHANGELOG.md
Release notes
https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.4.0-RC3/RELEASENOTES.md
This is off branch-3.4.0 and is the first big release since 3.3.6.
Key changes include
* S3A: Upgrade AWS SDK to V2
* HDFS DataNode Split one FsDatasetImpl lock to volume grain locks
* YARN Federation improvements
* YARN Capacity Scheduler improvements
* HDFS RBF: Code Enhancements, New Features, and Bug Fixes
* HDFS EC: Code Enhancements and Bug Fixes
* Transitive CVE fixes
Differences from Hadoop-3.4.0-RC2
* From branch-3.4 to branch-3.4.0 backport 2 Prs
* HADOOP-18088: Replacing log4j 1.x with reload4j. (ad8b6541117b)
* HADOOP-19084: Pruning hadoop-common transitive dependencies.
(80b4bb68159c)
* Use hadoop-release-support[1] for packaging and verification.
* Add protobuf compatibility issue description
Note, because the arm64 binaries are built separately on a different
platform and JVM, their jar files may not match those of the x86
release -and therefore the maven artifacts. I don't think this is
an issue (the ASF actually releases source tarballs, the binaries are
there for help only, though with the maven repo that's a bit blurred).
The only way to be consistent would actually untar the x86.tar.gz,
overwrite its binaries with the arm stuff, retar, sign and push out
for the vote. Even automating that would be risky.
[1] hadoop-release-support: https://github.com/apache/hadoop-release-support
Thanks to steve for providing hadoop-release-support.
Best Regards,
Shilun Fan.
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
