Apache Hadoop qbt Report: trunk+JDK11 on Linux/x86_64
For more details, see https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java11-linux-x86_64/630/ [Feb 13, 2024, 10:46:36 AM] (github) HADOOP-19057. S3A: Landsat bucket used in tests no longer accessible (#6515) [Feb 13, 2024, 4:33:51 PM] (github) HADOOP-18088. Replace log4j 1.x with reload4j. (#4052) [Feb 14, 2024, 12:24:23 AM] (github) HDFS-17354:Delay invoke clearStaleNamespacesInRouterStateIdContext during router start up (#6498) - To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86_64
For more details, see https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/1502/ [Feb 13, 2024, 4:33:51 PM] (github) HADOOP-18088. Replace log4j 1.x with reload4j. (#4052) [Feb 14, 2024, 12:24:23 AM] (github) HDFS-17354:Delay invoke clearStaleNamespacesInRouterStateIdContext during router start up (#6498) - To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
Re: [VOTE] Release Apache Hadoop 3.4.0 (RC2)
Thank you for helping review hadoop-3.4.0-RC2. Compared to RC1, we have made two improvements: 1. Merged some patches from the branch-3.4 branch to branch-3.4.0. 2. Upgrade the version of hadoop-thirdparty to 1.2.0. Best Regards, Shilun Fan. On Fri, Feb 16, 2024 at 4:41 AM Mukund Madhav Thakur wrote: > Thanks, Shilun for putting this together. > > Tried the below things and everything worked for me. > > validated checksum and gpg signature. > compiled from source. > Ran AWS integration tests. > untar the binaries and able to access objects in S3 via hadoop fs commands. > compiled gcs-connector successfully using the 3.4.0 version. > > qq: what is the difference between RC1 and RC2? apart from some extra > patches. > > > > On Thu, Feb 15, 2024 at 10:58 AM slfan1989 wrote: > >> Thank you for explaining this part! >> >> hadoop-3.4.0-RC2 used the validate-hadoop-client-artifacts tool to >> generate >> the ARM tar package, which should meet expectations. >> >> We also look forward to other members helping to verify. >> >> Best Regards, >> Shilun Fan. >> >> On Fri, Feb 16, 2024 at 12:22 AM Steve Loughran >> wrote: >> >> > >> > >> > On Mon, 12 Feb 2024 at 15:32, slfan1989 wrote: >> > >> >> >> >> >> >> Note, because the arm64 binaries are built separately on a different >> >> platform and JVM, their jar files may not match those of the x86 >> >> release -and therefore the maven artifacts. I don't think this is >> >> an issue (the ASF actually releases source tarballs, the binaries are >> >> there for help only, though with the maven repo that's a bit blurred). >> >> >> >> The only way to be consistent would actually untar the x86.tar.gz, >> >> overwrite its binaries with the arm stuff, retar, sign and push out >> >> for the vote. >> > >> > >> > >> > that's exactly what the "arm.release" target in my client-validator >> does. >> > builds an arm tar with the x86 binaries but the arm native libs, signs >> it. >> > >> > >> > >> >> Even automating that would be risky. >> >> >> >> >> > automating is the *only* way to do it; apache ant has everything needed >> > for this including the ability to run gpg. >> > >> > we did this on the relevant 3.3.x releases and nobody has yet >> complained... >> > >> >
Re: [VOTE] Release Apache Hadoop 3.4.0 (RC2)
Thanks, Shilun for putting this together. Tried the below things and everything worked for me. validated checksum and gpg signature. compiled from source. Ran AWS integration tests. untar the binaries and able to access objects in S3 via hadoop fs commands. compiled gcs-connector successfully using the 3.4.0 version. qq: what is the difference between RC1 and RC2? apart from some extra patches. On Thu, Feb 15, 2024 at 10:58 AM slfan1989 wrote: > Thank you for explaining this part! > > hadoop-3.4.0-RC2 used the validate-hadoop-client-artifacts tool to generate > the ARM tar package, which should meet expectations. > > We also look forward to other members helping to verify. > > Best Regards, > Shilun Fan. > > On Fri, Feb 16, 2024 at 12:22 AM Steve Loughran > wrote: > > > > > > > On Mon, 12 Feb 2024 at 15:32, slfan1989 wrote: > > > >> > >> > >> Note, because the arm64 binaries are built separately on a different > >> platform and JVM, their jar files may not match those of the x86 > >> release -and therefore the maven artifacts. I don't think this is > >> an issue (the ASF actually releases source tarballs, the binaries are > >> there for help only, though with the maven repo that's a bit blurred). > >> > >> The only way to be consistent would actually untar the x86.tar.gz, > >> overwrite its binaries with the arm stuff, retar, sign and push out > >> for the vote. > > > > > > > > that's exactly what the "arm.release" target in my client-validator does. > > builds an arm tar with the x86 binaries but the arm native libs, signs > it. > > > > > > > >> Even automating that would be risky. > >> > >> > > automating is the *only* way to do it; apache ant has everything needed > > for this including the ability to run gpg. > > > > we did this on the relevant 3.3.x releases and nobody has yet > complained... > > >
Re: [VOTE] Release Apache Hadoop 3.4.0 (RC2)
Thank you for explaining this part! hadoop-3.4.0-RC2 used the validate-hadoop-client-artifacts tool to generate the ARM tar package, which should meet expectations. We also look forward to other members helping to verify. Best Regards, Shilun Fan. On Fri, Feb 16, 2024 at 12:22 AM Steve Loughran wrote: > > > On Mon, 12 Feb 2024 at 15:32, slfan1989 wrote: > >> >> >> Note, because the arm64 binaries are built separately on a different >> platform and JVM, their jar files may not match those of the x86 >> release -and therefore the maven artifacts. I don't think this is >> an issue (the ASF actually releases source tarballs, the binaries are >> there for help only, though with the maven repo that's a bit blurred). >> >> The only way to be consistent would actually untar the x86.tar.gz, >> overwrite its binaries with the arm stuff, retar, sign and push out >> for the vote. > > > > that's exactly what the "arm.release" target in my client-validator does. > builds an arm tar with the x86 binaries but the arm native libs, signs it. > > > >> Even automating that would be risky. >> >> > automating is the *only* way to do it; apache ant has everything needed > for this including the ability to run gpg. > > we did this on the relevant 3.3.x releases and nobody has yet complained... >
Re: [VOTE] Release Apache Hadoop 3.4.0 (RC2)
On Mon, 12 Feb 2024 at 15:32, slfan1989 wrote: > > > Note, because the arm64 binaries are built separately on a different > platform and JVM, their jar files may not match those of the x86 > release -and therefore the maven artifacts. I don't think this is > an issue (the ASF actually releases source tarballs, the binaries are > there for help only, though with the maven repo that's a bit blurred). > > The only way to be consistent would actually untar the x86.tar.gz, > overwrite its binaries with the arm stuff, retar, sign and push out > for the vote. that's exactly what the "arm.release" target in my client-validator does. builds an arm tar with the x86 binaries but the arm native libs, signs it. > Even automating that would be risky. > > automating is the *only* way to do it; apache ant has everything needed for this including the ability to run gpg. we did this on the relevant 3.3.x releases and nobody has yet complained...
Apache Hadoop qbt Report: branch-2.10+JDK7 on Linux/x86_64
For more details, see https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/1303/ No changes -1 overall The following subsystems voted -1: asflicense hadolint mvnsite pathlen unit The following subsystems voted -1 but were configured to be filtered/ignored: cc checkstyle javac javadoc pylint shellcheck whitespace The following subsystems are considered long running: (runtime bigger than 1h 0m 0s) unit Specific tests: Failed junit tests : hadoop.fs.TestFileUtil hadoop.contrib.bkjournal.TestBookKeeperHACheckpoints hadoop.hdfs.TestLeaseRecovery2 hadoop.hdfs.server.blockmanagement.TestReplicationPolicyWithUpgradeDomain hadoop.hdfs.server.namenode.snapshot.TestSnapshotDeletion hadoop.hdfs.TestFileLengthOnClusterRestart hadoop.hdfs.server.namenode.ha.TestPipelinesFailover hadoop.hdfs.TestDFSInotifyEventInputStream hadoop.hdfs.server.namenode.snapshot.TestSnapshotBlocksMap hadoop.hdfs.qjournal.server.TestJournalNodeRespectsBindHostKeys hadoop.fs.viewfs.TestViewFileSystemHdfs hadoop.hdfs.server.federation.router.TestRouterQuota hadoop.hdfs.server.federation.router.TestRouterNamenodeHeartbeat hadoop.hdfs.server.federation.resolver.order.TestLocalResolver hadoop.hdfs.server.federation.resolver.TestMultipleDestinationResolver hadoop.contrib.bkjournal.TestBookKeeperHACheckpoints hadoop.mapreduce.lib.input.TestLineRecordReader hadoop.mapred.TestLineRecordReader hadoop.mapreduce.jobhistory.TestHistoryViewerPrinter hadoop.resourceestimator.service.TestResourceEstimatorService hadoop.resourceestimator.solver.impl.TestLpSolver hadoop.yarn.sls.TestSLSRunner hadoop.yarn.server.nodemanager.containermanager.linux.resources.TestNumaResourceAllocator hadoop.yarn.server.nodemanager.containermanager.linux.resources.TestNumaResourceHandlerImpl hadoop.yarn.server.resourcemanager.TestClientRMService hadoop.yarn.server.resourcemanager.recovery.TestFSRMStateStore hadoop.yarn.server.resourcemanager.monitor.invariants.TestMetricsInvariantChecker cc: https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/1303/artifact/out/diff-compile-cc-root.txt [4.0K] javac: https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/1303/artifact/out/diff-compile-javac-root.txt [488K] checkstyle: https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/1303/artifact/out/diff-checkstyle-root.txt [14M] hadolint: https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/1303/artifact/out/diff-patch-hadolint.txt [4.0K] mvnsite: https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/1303/artifact/out/patch-mvnsite-root.txt [572K] pathlen: https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/1303/artifact/out/pathlen.txt [12K] pylint: https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/1303/artifact/out/diff-patch-pylint.txt [20K] shellcheck: https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/1303/artifact/out/diff-patch-shellcheck.txt [72K] whitespace: https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/1303/artifact/out/whitespace-eol.txt [12M] https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/1303/artifact/out/whitespace-tabs.txt [1.3M] javadoc: https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/1303/artifact/out/patch-javadoc-root.txt [36K] unit: https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/1303/artifact/out/patch-unit-hadoop-common-project_hadoop-common.txt [220K] https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/1303/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt [1.8M] https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/1303/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs-rbf.txt [36K] https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/1303/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs_src_contrib_bkjournal.txt [16K] https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/1303/artifact/out/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core.txt [104K] https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/1303/artifact/out/patch-unit-hadoop-tools_hadoop-azure.txt [20K]
