Hi folks, The voting for hadoop-3.4.0-RC3 has ended, and we have received a total of 8 affirmative votes, with 6 binding and 2 non-binding.
In the order of the voting timeline: binding (+6) Masatake Iwasaki Xiaoqiao He Mukund Madhav Thakur Steve Loughran Takanobu Asanuma Shilun Fan non-binding(+2) Suhail, Ahmar Dongjoon Hyun Thank you very much for helping with the review and validation! We have received 6 binding PMC affirmative votes. hadoop-3.4.0 will be officially released in the next 1-2 days. Best Regards, Shilun Fan. On Fri, Mar 15, 2024 at 8:32 AM slfan1989 <slfan1...@apache.org> wrote: > +1 (binding). > > * Verified the signature and checksum of all tarballs. > * Built from the source tarball on macOS 10.14 and OpenJDK 8. > * Execute basic HDFS/YARN commands and run the WordCount MapReduce job. > * Check the Yarn federation page and basic functionalities. > * Read the basic help documentation. > > Best Regards, > Shilun Fan. > > On Thu, Mar 14, 2024 at 4:40 PM Takanobu Asanuma <tasan...@apache.org> > wrote: > >> +1 (binding). >> >> Thanks for the great work, Shilun Fan. >> >> * Verified signatures and checksums >> * Successfully built from source with native code >> * Deployed a distributed cluster (on K8s) >> * Successfully ran some Erasure Coding operations with ISA-L codec >> * Successfully ran some HDFS RBF operations >> >> Regards, >> - Takanobu Asanuma >> >> 2024年3月14日(木) 15:19 Xiaoqiao He <hexiaoq...@apache.org>: >> >>> Thanks Ayush for highlighting this information. Absolutely true, we >>> should >>> count RM's vote when explicit +1 here. >>> >>> Best Regards, >>> - He Xiaoqiao >>> >>> On Thu, Mar 14, 2024 at 3:55 AM Ayush Saxena <ayush...@gmail.com> wrote: >>> >>> > > Counter should be with yourself vote, where the current summary >>> > is 5 +1 binding and 1 +1 non-binding. Let's re-count when deadline. >>> > >>> > Just on the process: The release manager needs to "explicitly" vote >>> like >>> > any other before counting their own vote, there has been a lot of >>> > discussions around that at multiple places & the official apache doc >>> has >>> > been updated as well [1], the last paragraph reads: >>> > >>> > "Note that there is no implicit +1 from the release manager, or from >>> > anyone in any ASF vote. Only explicit votes are valid. The release >>> manager >>> > is encouraged to vote on releases, like any reviewer would do." >>> > >>> > So, do put an explicit +1, before you count yourself. Good Luck!!! >>> > >>> > -Ayush >>> > >>> > [1] https://www.apache.org/foundation/voting.html#ReleaseVotes >>> > >>> > On Tue, 12 Mar 2024 at 17:27, Steve Loughran >>> <ste...@cloudera.com.invalid> >>> > wrote: >>> > >>> >> followup: overnight work happy too. >>> >> >>> >> one interesting pain point is that on a raspberry pi 64 os checknative >>> >> complains that libcrypto is missing >>> >> >>> >> > bin/hadoop checknative >>> >> >>> >> 2024-03-12 11:50:24,359 INFO bzip2.Bzip2Factory: Successfully loaded & >>> >> initialized native-bzip2 library system-native >>> >> 2024-03-12 11:50:24,363 INFO zlib.ZlibFactory: Successfully loaded & >>> >> initialized native-zlib library >>> >> 2024-03-12 11:50:24,370 WARN erasurecode.ErasureCodeNative: ISA-L >>> support >>> >> is not available in your platform... using builtin-java codec where >>> >> applicable >>> >> 2024-03-12 11:50:24,429 INFO nativeio.NativeIO: The native code was >>> built >>> >> without PMDK support. >>> >> 2024-03-12 11:50:24,431 WARN crypto.OpensslCipher: Failed to load >>> OpenSSL >>> >> Cipher. >>> >> java.lang.UnsatisfiedLinkError: Cannot load libcrypto.so >>> (libcrypto.so: >>> >> cannot open shared object file: No such file or directory)! >>> >> at org.apache.hadoop.crypto.OpensslCipher.initIDs(Native >>> Method) >>> >> at >>> >> org.apache.hadoop.crypto.OpensslCipher.<clinit>(OpensslCipher.java:90) >>> >> at >>> >> >>> >> >>> org.apache.hadoop.util.NativeLibraryChecker.main(NativeLibraryChecker.java:111) >>> >> Native library checking: >>> >> hadoop: true >>> >> >>> >> >>> /home/stevel/Projects/hadoop-release-support/target/arm-untar/hadoop-3.4.0/lib/native/libhadoop.so.1.0.0 >>> >> zlib: true /lib/aarch64-linux-gnu/libz.so.1 >>> >> zstd : true /lib/aarch64-linux-gnu/libzstd.so.1 >>> >> bzip2: true /lib/aarch64-linux-gnu/libbz2.so.1 >>> >> openssl: false Cannot load libcrypto.so (libcrypto.so: cannot open >>> shared >>> >> object file: No such file or directory)! >>> >> ISA-L: false libhadoop was built without ISA-L support >>> >> PMDK: false The native code was built without PMDK support. >>> >> >>> >> which happens because its not in /lib/aarch64-linux-gnu but instead in >>> >> /usr/lib/aarch64-linux-gnu/l >>> >> ls -l /usr/lib/aarch64-linux-gnu/libcrypto* >>> >> -rw-r--r-- 1 root root 2739952 Sep 19 13:09 >>> >> /usr/lib/aarch64-linux-gnu/libcrypto.so.1.1 >>> >> -rw-r--r-- 1 root root 4466856 Oct 27 13:40 >>> >> /usr/lib/aarch64-linux-gnu/libcrypto.so.3 >>> >> >>> >> Anyone got any insights on how I should set up this (debian-based) OS >>> >> here? >>> >> I know it's only a small box but with arm64 VMs becoming available in >>> >> cloud >>> >> infras, it'd be good to know if they are similar. >>> >> >>> >> Note: checknative itself is happy; but checknative -a will fail >>> because of >>> >> this -though it's an OS setup issue, nothing related to the hadoop >>> >> binaries. >>> >> >>> >> steve >>> >> >>> >> On Tue, 12 Mar 2024 at 02:26, Xiaoqiao He <hexiaoq...@apache.org> >>> wrote: >>> >> >>> >> > Hi Shilun, Counter should be with yourself vote, where the current >>> >> summary >>> >> > is 5 +1 binding and 1 +1 non-binding. Let's re-count when deadline. >>> >> > Thanks again. >>> >> > >>> >> > Best Regards, >>> >> > - He Xiaoqiao >>> >> > >>> >> > On Tue, Mar 12, 2024 at 9:00 AM slfan1989 <slfan1...@apache.org> >>> wrote: >>> >> > >>> >> > > As of now, we have collected 5 affirmative votes, with 4 votes >>> binding >>> >> > and >>> >> > > 1 vote non-binding. >>> >> > > >>> >> > > Thank you very much for voting and verifying! >>> >> > > >>> >> > > This voting will continue until March 15th, this Friday. >>> >> > > >>> >> > > Best Regards, >>> >> > > Shilun Fan. >>> >> > > >>> >> > > On Tue, Mar 12, 2024 at 4:29 AM Steve Loughran >>> >> > <ste...@cloudera.com.invalid >>> >> > > > >>> >> > > wrote: >>> >> > > >>> >> > > > +1 binding >>> >> > > > >>> >> > > > (sorry, this had ended in the yarn-dev folder, otherwise I'd >>> have >>> >> seen >>> >> > it >>> >> > > > earlier. been testing it this afternoon: >>> >> > > > >>> >> > > > pulled the latest version of >>> >> > > > https://github.com/apache/hadoop-release-support >>> >> > > > (note, this module is commit-then-review; whoever is working >>> >> > > on/validating >>> >> > > > a release can commit as they go along. This is not production >>> >> code...) >>> >> > > > >>> >> > > > * went through the "validating a release" step, validating maven >>> >> > > artifacts >>> >> > > > * building the same downstream modules which built for me last >>> time >>> >> > (avro >>> >> > > > too complex; hboss not aws v2 in apache yet) >>> >> > > > >>> >> > > > spark build is still ongoing, but I'm not going to wait. It is >>> >> > building, >>> >> > > > which is key. >>> >> > > > >>> >> > > > The core changes I needed in are at the dependency level and >>> I've >>> >> > > > verified they are good. >>> >> > > > >>> >> > > > Oh, and I've also got my raspberry p5 doing the download of the >>> arm >>> >> > > > stuff for its checknative; not expecting problems. >>> >> > > > >>> >> > > > So: i've got some stuff still ongoing, but the core changes to >>> >> > packaging >>> >> > > > are in and the rest I'm not worried about -they shouldn't block >>> the >>> >> > > release >>> >> > > > as I already validated them on RC2 >>> >> > > > >>> >> > > > >>> >> > > > >>> >> > > > >>> >> > > > >>> >> > > > On Mon, 4 Mar 2024 at 22:08, slfan1989 <slfan1...@apache.org> >>> >> wrote: >>> >> > > > >>> >> > > > > Hi folks, >>> >> > > > > >>> >> > > > > Xiaoqiao He and I have put together a release candidate (RC3) >>> for >>> >> > > Hadoop >>> >> > > > > 3.4.0. >>> >> > > > > >>> >> > > > > What we would like is for anyone who can to verify the >>> tarballs, >>> >> > > > especially >>> >> > > > > anyone who can try the arm64 binaries as we want to include >>> them >>> >> too. >>> >> > > > > >>> >> > > > > The RC is available at: >>> >> > > > > >>> https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.4.0-RC3/ >>> >> > > > > >>> >> > > > > The git tag is release-3.4.0-RC3, commit bd8b77f398f >>> >> > > > > >>> >> > > > > The maven artifacts are staged at >>> >> > > > > >>> >> > > >>> >> >>> https://repository.apache.org/content/repositories/orgapachehadoop-1408 >>> >> > > > > >>> >> > > > > You can find my public key at: >>> >> > > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS >>> >> > > > > >>> >> > > > > Change log >>> >> > > > > >>> >> > > > >>> >> > > >>> >> > >>> >> >>> https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.4.0-RC3/CHANGELOG.md >>> >> > > > > >>> >> > > > > Release notes >>> >> > > > > >>> >> > > > > >>> >> > > > >>> >> > > >>> >> > >>> >> >>> https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.4.0-RC3/RELEASENOTES.md >>> >> > > > > >>> >> > > > > This is off branch-3.4.0 and is the first big release since >>> 3.3.6. >>> >> > > > > >>> >> > > > > Key changes include >>> >> > > > > >>> >> > > > > * S3A: Upgrade AWS SDK to V2 >>> >> > > > > * HDFS DataNode Split one FsDatasetImpl lock to volume grain >>> locks >>> >> > > > > * YARN Federation improvements >>> >> > > > > * YARN Capacity Scheduler improvements >>> >> > > > > * HDFS RBF: Code Enhancements, New Features, and Bug Fixes >>> >> > > > > * HDFS EC: Code Enhancements and Bug Fixes >>> >> > > > > * Transitive CVE fixes >>> >> > > > > >>> >> > > > > Differences from Hadoop-3.4.0-RC2 >>> >> > > > > >>> >> > > > > * From branch-3.4 to branch-3.4.0 backport 2 Prs >>> >> > > > > * HADOOP-18088: Replacing log4j 1.x with reload4j. >>> >> (ad8b6541117b) >>> >> > > > > * HADOOP-19084: Pruning hadoop-common transitive >>> dependencies. >>> >> > > > > (80b4bb68159c) >>> >> > > > > * Use hadoop-release-support[1] for packaging and >>> verification. >>> >> > > > > * Add protobuf compatibility issue description >>> >> > > > > >>> >> > > > > Note, because the arm64 binaries are built separately on a >>> >> different >>> >> > > > > platform and JVM, their jar files may not match those of the >>> x86 >>> >> > > > > release -and therefore the maven artifacts. I don't think >>> this is >>> >> > > > > an issue (the ASF actually releases source tarballs, the >>> binaries >>> >> are >>> >> > > > > there for help only, though with the maven repo that's a bit >>> >> > blurred). >>> >> > > > > >>> >> > > > > The only way to be consistent would actually untar the >>> x86.tar.gz, >>> >> > > > > overwrite its binaries with the arm stuff, retar, sign and >>> push >>> >> out >>> >> > > > > for the vote. Even automating that would be risky. >>> >> > > > > >>> >> > > > > [1] hadoop-release-support: >>> >> > > > > https://github.com/apache/hadoop-release-support >>> >> > > > > Thanks to steve for providing hadoop-release-support. >>> >> > > > > >>> >> > > > > Best Regards, >>> >> > > > > Shilun Fan. >>> >> > > > > >>> >> > > > > >>> >> > > > >>> >> > > >>> >> > >>> >> >>> > >>> >>
