Hi, is there any schedule to resume Apache Hadoop 3.4.1 release? Dongjoon.
On 2024/08/16 15:17:28 Steve Loughran wrote: > Afraid I have to say -1 to this iteration, but I promise I'll help address > the issues > > First, I've cherrypicked a few final changes from branch-3.4 in, including > this major one > HADOOP-19153. hadoop-common exports logback as a transitive dependency > (#6999) > This broke parquet hadoop-test runs without an explicit exclusion > downstream -it'll help others upgrade. > > Second, the cherrypick branch test run showed parquet JDK incompatibilities > > https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6997/1/testReport/ > > java.lang.NoSuchMethodError: > java.nio.ByteBuffer.position(I)Ljava/nio/ByteBuffer; > at > org.apache.hadoop.thirdparty.protobuf.IterableByteBufferInputStream.read(IterableByteBufferInputStream.java:143) > at > org.apache.hadoop.thirdparty.protobuf.CodedInputStream$StreamDecoder.read(CodedInputStream.java:2080) > at > org.apache.hadoop.thirdparty.protobuf.CodedInputStream$StreamDecoder.tryRefillBuffer(CodedInputStream.java:2831) > at > org.apache.hadoop.thirdparty.protobuf.CodedInputStream$StreamDecoder.refillBuffer(CodedInputStream.java:2777) > at > org.apache.hadoop.thirdparty.protobuf.CodedInputStream$StreamDecoder.readRawByte(CodedInputStream.java:2859) > at > org.apache.hadoop.thirdparty.protobuf.CodedInputStream$StreamDecoder.readRawVarint64SlowPath(CodedInputStream.java:2648) > at > org.apache.hadoop.thirdparty.protobuf.CodedInputStream$StreamDecoder.readRawVarint64(CodedInputStream.java:2641) > at > org.apache.hadoop.thirdparty.protobuf.CodedInputStream$StreamDecoder.readSInt64(CodedInputStream.java:2497) > at > org.apache.hadoop.hdfs.protocol.BlockListAsLongs$BufferDecoder$1.next(BlockListAsLongs.java:419) > at > org.apache.hadoop.hdfs.protocol.BlockListAsLongs$BufferDecoder$1.next(BlockListAsLongs.java:397) > at > org.apache.hadoop.hdfs.protocol.BlockListAsLongs$BufferDecoder.getBlockListAsLongs(BlockListAsLongs.java:375) > at > org.apache.hadoop.hdfs.protocol.TestBlockListAsLongs.checkReport(TestBlockListAsLongs.java:156) > at > org.apache.hadoop.hdfs.protocol.TestBlockListAsLongs.testFuzz(TestBlockListAsLongs.java:139) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) > at > org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) > at > org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) > at > org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) > at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) > at > org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) > > This is addressed by "HADOOP-19163. Use protobuf-java 3.25.3", which bumped > up the shaded version > then modified hadoop dependencies to match. > > > I'm going to release thirdparty jar 1.3.0 with the relevant updates, > upgrade 3.4.1+ to use it once that's out (I'll have the pending PRs up) > > HADOOP-19252. Release Hadoop Third-Party 1.3.0 > https://issues.apache.org/jira/browse/HADOOP-19252 > > The other thing we should all look at is making sure we are current with > all dependency updates in trunk *without doing any last minute update of > jar versions entirely* > > I've got a PR of the kafka update: > https://github.com/apache/hadoop/pull/7000 ; will merge if yetus doesn't > complain > > > > > On Thu, 8 Aug 2024 at 19:06, Mukund Madhav Thakur > <mtha...@cloudera.com.invalid> wrote: > > > Apache Hadoop 3.4.1 > > > > > > > > I with help of Steve have put together a release candidate (RC1) for Hadoop > > 3.4.1. > > > > > > > > What we would like is for anyone who can to verify the tarballs, especially > > > > anyone who can try the arm64 binaries as we want to include them too. > > > > > > > > The RC is available at: > > > > https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.4.1-RC1/ > > > > > > > > The git tag is release-3.4.1-RC1, commit > > 247daf0f827adc96a3847bb40e0fec3fc85f33bd > > > > > > > > The maven artifacts are staged at > > > > https://repository.apache.org/content/repositories/orgapachehadoop-1417 > > > > > > > > You can find my public key at: > > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS > > > > > > > > Change log > > > > https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.4.1-RC1/CHANGELOG.md > > > > > > > > Release notes > > > > > > https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.4.1-RC1/RELEASENOTES.md > > > > > > > > This is off branch-3.4. > > > > > > > > Key changes include > > > > > > > > * Bulk Delete API. https://issues.apache.org/jira/browse/HADOOP-18679 > > > > * Fixes and enhancements in Vectored IO API. > > > > * Improvements in Hadoop Azure connector. > > > > * Fixes and improvements post upgrade to AWS V2 SDK in S3AConnector. > > > > * This release includes Arm64 binaries. Please can anyone with > > > > compatible systems validate these. > > > > > > > > Note, because the arm64 binaries are built separately on a different > > > > platform and JVM, their jar files may not match those of the x86 > > > > release -and therefore the maven artifacts. I don't think this is > > > > an issue (the ASF actually releases source tarballs, the binaries are > > > > there for help only, though with the maven repo that's a bit blurred). > > > > > > > > The only way to be consistent would actually untar the x86.tar.gz, > > > > overwrite its binaries with the arm stuff, retar, sign and push out > > > > for the vote. Even automating that would be risky. > > > > > > > > As this is just a first try to get this out, there might be issues, Please > > try the release and let me know. Also let me know if you would like to add > > somethings in 3.4.1 > > > > > > Also found two issues in hadoop-yarn-ui while building the arm binaries > > > > https://issues.apache.org/jira/browse/YARN-11712 > > > > https://issues.apache.org/jira/browse/YARN-11713 > > > > > > > > Thanks, > > > > Mukund > > > --------------------------------------------------------------------- To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
