just discovering as I do the third party compatibility testing of the latest sdk that the fs.s3a.create.checksum.algorithm stuff is a bit broken -nothing to worry about though as its a new feature so not a regression
On Wed, 20 Aug 2025 at 16:13, Suhail, Ahmar <ahma...@amazon.co.uk.invalid> wrote: > Hey all, > > I've fixed the following issues: > 1) Issues with changelog, release notes and landing page. Created a new > build so they now include all the JIRA's that were missing before. > 2) Create the lean tar on ubuntu so you won't get the tar: Ignoring > unknown extended header keyword 'LIBARCHIVE.xattr.com.apple.quarantine' > message when untarring > 3) Previous issues with sha512 and tar not being gzipped. > > RC3 for Hadoop 3.4.2 is now available for voting. > > The RC is available at: > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC3/ > > The git tag is release-3.4.2-RC3, commit > 84e8b89ee2ebe6923691205b9e171badde7a495c > > The maven artifacts are staged at: > > https://repository.apache.org/content/repositories/orgapachehadoop-1443 > > You can find my public key (02085AFB652F796A3B01D11FD737A6F52281FA98) at: > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS > > > This release has been created off of branch-3.4. > > Thanks, > Ahmar > > On 20/08/2025, 09:25, "Masatake Iwasaki" <iwasak...@oss.nttdata.com > <mailto:iwasak...@oss.nttdata.com>> wrote: > > > CAUTION: This email originated from outside of the organization. Do not > click links or open attachments unless you can confirm the sender and know > the content is safe. > > > > > > > > Since I have to regenerate the site and changelog, do I need to do the > whole build again or is there an easier way to do that? (just by > regenerating the site manually and updating the artifacts in SVN maybe?) > > > I think just running the create-release[1] script could be an easy way > while it does not provide options for building only site or source > tarballs (now). > Since the binary tarballs are containing affected CHANGELOG and > RELEASENOTES, > it would be nice to update the binary tarballs. > # Swapping the CHANGELOG and RELEASENOTES of existing binary tarballs > # could be ok if only documentation is updated. > # Signing and checksums must be cared in that case. > > > In addition, I got following warnings when I extracted the binary tarballs > (hadoop-3.4.2-lean.tar.gz and hadoop-3.4.2-aarch64-lean.tar.gz) on > Ubuntu.:: > > > tar: Ignoring unknown extended header keyword > 'LIBARCHIVE.xattr.com.apple.provenance' > tar: Ignoring unknown extended header keyword > 'LIBARCHIVE.xattr.com.apple.quarantine' > tar: Ignoring unknown extended header keyword > 'LIBARCHIVE.xattr.com.apple.provenance' > tar: Ignoring unknown extended header keyword > 'LIBARCHIVE.xattr.com.apple.quarantine' > > > I guess the cause is modifying the tar files on OSX environment. > It would be nice to fix too. > > > [1] > https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release > < > https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release > > > > > Masatake Iwasaki > > > On 8/19/2025 10:28 PM, Suhail, Ahmar wrote: > > Thanks Masatake and Steve, > > > > Here's the PR for updating the top page of the site > https://github.com/apache/hadoop/pull/7887 < > https://github.com/apache/hadoop/pull/7887> , if there's anything else we > think needs to go in there, let me know, otherwise I will merge by tomorrow. > > > > Since I have to regenerate the site and changelog, do I need to do the > whole build again or is there an easier way to do that? (just by > regenerating the site manually and updating the artifacts in SVN maybe?) > > > > I will update the artifacts for the hadoop-3.4.2-lean.tar.gz and > hadoop-3.4.2-aarch64-lean.tar.sha512 now. > > > > On 19/08/2025, 08:26, "Masatake Iwasaki" <iwasak...@oss.nttdata.com > <mailto:iwasak...@oss.nttdata.com> <mailto:iwasak...@oss.nttdata.com > <mailto:iwasak...@oss.nttdata.com>>> wrote: > > > > > > CAUTION: This email originated from outside of the organization. Do not > click links or open attachments unless you can confirm the sender and know > the content is safe. > > > > > > > > > > > > > > Thanks for the update, Ahmar > > > > > > The binary artifacts basically looks good. > > I have some comments about inconsistencies. > > > > > > - Some JIRAs of additional commits did not have 3.4.2 as "Fix > Version/s". I updated them. > > > > > > Jira/Git commit message diff starting: > ############################################## > > Jira not present with version: 3.4.2. Commit: > e1c0dee881820a4d834ec4a4d2c70d0d953bb933 HADOOP-19627. S3A: > testIfMatchOverwriteWithOutdatedEtag() fails when not using SSE-KMS #7816 > > ... > > Jira not present with version: 3.4.2. Commit: > 6a4ea9b1155aefb7732b700d8d7154f977141e39 HADOOP-19626. S3A: Updates AAL > version to 1.2.1 (#7807) (#7815) > > Jira not present with version: 3.4.2. Commit: > 9677363829c21adeb2ce41f93059ab65fc115862 HADOOP-19554. LocalDirAllocator > still doesn't always recover from directory deletion (#7683) (#7685) > > ... > > Jira not present with version: 3.4.2. Commit: > 57d821e6aad23b604833272071c8481567363e63 HADOOP-19143. Upgrade commons-cli > to 1.9.0. (#7126) (#7146) Contributed by Min Yan. > > > > > > - The top page of site documentation was not updated for 3.4.2. > > > https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm > < > https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm> > < > https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm> > < > https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm> > ;> > > > > > > - Some release artifact are inconsistent as Steve pointed out. > > > > > > - `hadoop-3.4.2-lean.tar.gz` is not gzipped. > > > > > > - `hadoop-3.4.2-aarch64-lean.tar.sha512` should be > `hadoop-3.4.2-aarch64-lean.tar.gz.sha512` > > > > > > Masatake Iwasaki > > > > > > On 8/14/2025 6:12 PM, Ahmar Suhail wrote: > >> Hey all, > >> > >> Just a reminder about this. Would be great to get some votes in asap and > >> hopefully finalise 3.4.2. > >> > >> Thanks, > >> Ahmar > >> > >> On Thu, Aug 7, 2025 at 9:35 PM Ahmar Suhail <ah...@apache.org <mailto: > ah...@apache.org> <mailto:ah...@apache.org <mailto:ah...@apache.org>>> > wrote: > >> > >>> Hey all, > >>> > >>> RC2 for Hadoop 3.4.2 is now available for voting. > >>> > >>> There are a couple of things to note: > >>> > >>> 1/ We have ARM artifacts again! Thanks to Masatake and Wei-Chiu Chuang > for > >>> working on the fix. > >>> > >>> 2/ Relevant for anyone testing S3A: We've removed the AWS SDK bundle > >>> from hadoop-3.4.2.tar.gz. This is because the SDK bundle is now ~600MB, > >>> which makes the size of tar > 1GB, and it can no longer be uploaded to > SVN. > >>> For S3A, download SDK bundle v2.29.52 from: > >>> > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52 < > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52> > <https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52> > < > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52> > ;>, > >>> and drop it into /share/hadoop/common/lib. Release notes will be > updated > >>> with these instructions. > >>> > >>> > >>> The RC is available at: > >>> > >>> https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/ < > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> < > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> < > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>> > >>> > >>> The git tag is release-3.4.2-RC2, commit > >>> e1c0dee881820a4d834ec4a4d2c70d0d953bb933 > >>> > >>> The maven artifacts are staged at: > >>> > >>> > https://repository.apache.org/content/repositories/orgapachehadoop-1441 < > https://repository.apache.org/content/repositories/orgapachehadoop-1441> < > https://repository.apache.org/content/repositories/orgapachehadoop-1441> < > https://repository.apache.org/content/repositories/orgapachehadoop-1441> > ;> > >>> > >>> > >>> You can find my public key (02085AFB652F796A3B01D11FD737A6F52281FA98) > at: > >>> > >>> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS < > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> < > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> < > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>> > >>> > >>> > >>> This release has been created off of branch-3.4. > >>> > >>> Thanks, > >>> Ahmar > >>> > >> > > > > > > > > > > > > > > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org > For additional commands, e-mail: common-dev-h...@hadoop.apache.org >
