Hi Shilun, Thanks for your great work!
Compare branch-3.4 and branch-3.4.0, I found that commit [1] is only checked in branch-3.4.0 but not branch-3.4. Please double check if we need to backport branch-3.4. Moreover, before 3.4.0 release we should cherry-pick from trunk to branch-3.4 first if necessary, then to branch-3.4.0. Thanks again. Best Regards, - He Xiaoqiao [1] https://github.com/apache/hadoop/commit/5e30d28d7e524dd3eed179d81c38e2eb82ae4673 On Wed, Jan 17, 2024 at 9:19 AM slfan1989 <slfan1...@apache.org> wrote: > Hi, all > > Since the hadoop-3.4.0-RC0 vote, I have received valuable feedback. I > encountered some issues during the preparation of hadoop-3.4.0-RC0, and I > will address these issues in hadoop-3.4.0-RC1. > > The voting for RC0 will be closed. After the release of RC1, I will invite > members of the community to review and vote once again. > > Thank you all once again for your support! > > Best Regards, > Shilun Fan. > > On Wed, Jan 17, 2024 at 9:03 AM slfan1989 <slfan1...@apache.org> wrote: > > > Thank you very much for the response! > > > > The content is very comprehensive and valuable. > > > > I will prepare hadoop-3.4.0-RC1 according to the instructions provided by > > you, and after RC1 is packaged, I will use > validate-hadoop-client-artifacts > > for validation. > > > > Best Regards, > > Shilun Fan. > > > > On Tue, Jan 16, 2024 at 12:34 AM Steve Loughran > > <ste...@cloudera.com.invalid> wrote: > > > >> -1 I'm afraid, just due to staging/packaging issues. > >> > >> This took me a few goes to get right myself, so nothing unusual. > >> > >> Note I used my validator project which is set to retrieve binaries, > check > >> signatures, run maven builds against staged artifacts *and clean up any > >> local copies first*and more. > >> > >> This uses apache ant to manage all this: > >> > >> https://github.com/steveloughran/validate-hadoop-client-artifacts > >> > >> Here's the initial build.properties:file I used to try and manage this > >> > >> ###### build.properties: > >> hadoop.version=3.4.0 > >> rc=RC0 > >> amd.src.dir=https://home.apache.org/~slfan1989/hadoop-3.4.0-RC0-amd64/ > >> http.source=https://home.apache.org/~slfan1989/hadoop-3.4.0-RC0-amd64 > >> < > https://home.apache.org/~slfan1989/hadoop-3.4.0-RC0-amd64/http.source=https://home.apache.org/~slfan1989/hadoop-3.4.0-RC0-amd64 > > > >> > >> release=hadoop-${hadoop.version}-RC0 > >> rc.dirname=${release} > >> release.native.binaries=false > >> git.commit.id=cdb8af4f22ec > >> nexus.staging.url= > >> > https://repository.apache.org/content/repositories/orgapachehadoop-1391/ > >> hadoop.source.dir=${local.dev.dir}/hadoop-trunk > >> ###### > >> > >> When I did my own builds, all the artifacts created were without the RC0 > >> suffix. It is critical this happens because the .sha512 checksums > include > >> that in their paths > >> > >> > cat hadoop-3.4.0-RC0.tar.gz.sha512 > >> SHA512 (hadoop-3.4.0-RC0.tar.gz) = > >> > >> > e50e68aecb36867c610db8309ccd3aae812184da21354b50d2a461b29c73f21d097fb27372c73c150e1c035003bb99a61c64db26c090fe0fb9e7ed6041722eab > >> > >> > >> Maven artifacts: staging problems > >> > >> Couldn't build with a -Pstaging profile as the staging repository wasn't > >> yet closed -I tried to do that myself. > >> > >> This failed with some rule problem > >> > >> Event: Failed: Checksum Validation > >> Monday, January 15, 2024 14:37:13 GMT (GMT+0000) > >> typeId checksum-staging > >> failureMessage INVALID SHA-1: > >> > >> > '/org/apache/hadoop/hadoop-mapreduce-client-jobclient/3.4.0/hadoop-mapreduce-client-jobclient-3.4.0-tests.jar.sha1' > >> failureMessage Requires one-of SHA-1: > >> > >> > /org/apache/hadoop/hadoop-mapreduce-client-jobclient/3.4.0/hadoop-mapreduce-client-jobclient-3.4.0-tests.jar.sha1, > >> SHA-256: > >> > >> > /org/apache/hadoop/hadoop-mapreduce-client-jobclient/3.4.0/hadoop-mapreduce-client-jobclient-3.4.0-tests.jar.sha256, > >> SHA-512: > >> > >> > /org/apache/hadoop/hadoop-mapreduce-client-jobclient/3.4.0/hadoop-mapreduce-client-jobclient-3.4.0-tests.jar.sha512 > >> > >> I don't know precisely what this means...my guess is that the upload > >> didn't > >> include everything. > >> > >> Note my client-validator module can check this; just run its maven test > >> commands > >> > >> mvn clean test -U -P3.4 -Pstaging > >> > >> GPG signing: all good. > >> > >> Picked your key up from the site ( ant gpg.keys ) ... first validation > >> with > >> ant gpg.verify was unhappy as your key wasn't trusted. I've signed it > and > >> pushed that signature up, so people who trust me get some reassurance > >> about > >> you. > >> > >> My build then failed as the gpg code couldn't find the > >> hadoop-3.4.0-aarch64.tar.gz.asc > >> > >> The problem here is that although we want separate arm and x86 tar > files, > >> we don't really want separate binaries as it only creates different jars > >> in > >> the wild. > >> > >> The way I addressed that was after creating that x86 release on an ec2 > vm > >> and downloading it, I then did a local arm64 build and then created an > arm > >> .tar.gz file, copied it into the same dir as the amd66 binaries but with > >> the arm64 .tar.gz filename, .asc and .sha512 checksum files all renamed > >> (checksum file patches to match the name). > >> > >> > >> > https://github.com/steveloughran/validate-hadoop-client-artifacts?tab=readme-ov-file#arm64-binaries > >> > > >
