Hi, Ahmar Here are the PRs for the fix raised for ABFS close() causing NPE raised on RC. PR on branch-3.4.2: https://github.com/apache/hadoop/pull/7888 PR on branch-3.4: https://github.com/apache/hadoop/pull/7889
Feel free to merge them as per the release timelines. Thanks On Wed, Aug 20, 2025 at 11:24 PM Steve Loughran <ste...@cloudera.com.invalid> wrote: > just discovering as I do the third party compatibility testing of the > latest sdk that the fs.s3a.create.checksum.algorithm stuff is a bit broken > -nothing to worry about though as its a new feature so not a regression > > On Wed, 20 Aug 2025 at 16:13, Suhail, Ahmar <ahma...@amazon.co.uk.invalid> > wrote: > > > Hey all, > > > > I've fixed the following issues: > > 1) Issues with changelog, release notes and landing page. Created a new > > build so they now include all the JIRA's that were missing before. > > 2) Create the lean tar on ubuntu so you won't get the tar: Ignoring > > unknown extended header keyword 'LIBARCHIVE.xattr.com.apple.quarantine' > > message when untarring > > 3) Previous issues with sha512 and tar not being gzipped. > > > > RC3 for Hadoop 3.4.2 is now available for voting. > > > > The RC is available at: > > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC3/ > > > > The git tag is release-3.4.2-RC3, commit > > 84e8b89ee2ebe6923691205b9e171badde7a495c > > > > The maven artifacts are staged at: > > > > https://repository.apache.org/content/repositories/orgapachehadoop-1443 > > > > You can find my public key (02085AFB652F796A3B01D11FD737A6F52281FA98) at: > > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS > > > > > > This release has been created off of branch-3.4. > > > > Thanks, > > Ahmar > > > > On 20/08/2025, 09:25, "Masatake Iwasaki" <iwasak...@oss.nttdata.com > > <mailto:iwasak...@oss.nttdata.com>> wrote: > > > > > > CAUTION: This email originated from outside of the organization. Do not > > click links or open attachments unless you can confirm the sender and > know > > the content is safe. > > > > > > > > > > > > > > > Since I have to regenerate the site and changelog, do I need to do the > > whole build again or is there an easier way to do that? (just by > > regenerating the site manually and updating the artifacts in SVN maybe?) > > > > > > I think just running the create-release[1] script could be an easy way > > while it does not provide options for building only site or source > > tarballs (now). > > Since the binary tarballs are containing affected CHANGELOG and > > RELEASENOTES, > > it would be nice to update the binary tarballs. > > # Swapping the CHANGELOG and RELEASENOTES of existing binary tarballs > > # could be ok if only documentation is updated. > > # Signing and checksums must be cared in that case. > > > > > > In addition, I got following warnings when I extracted the binary > tarballs > > (hadoop-3.4.2-lean.tar.gz and hadoop-3.4.2-aarch64-lean.tar.gz) on > > Ubuntu.:: > > > > > > tar: Ignoring unknown extended header keyword > > 'LIBARCHIVE.xattr.com.apple.provenance' > > tar: Ignoring unknown extended header keyword > > 'LIBARCHIVE.xattr.com.apple.quarantine' > > tar: Ignoring unknown extended header keyword > > 'LIBARCHIVE.xattr.com.apple.provenance' > > tar: Ignoring unknown extended header keyword > > 'LIBARCHIVE.xattr.com.apple.quarantine' > > > > > > I guess the cause is modifying the tar files on OSX environment. > > It would be nice to fix too. > > > > > > [1] > > > https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release > > < > > > https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release > > > > > > > > > Masatake Iwasaki > > > > > > On 8/19/2025 10:28 PM, Suhail, Ahmar wrote: > > > Thanks Masatake and Steve, > > > > > > Here's the PR for updating the top page of the site > > https://github.com/apache/hadoop/pull/7887 < > > https://github.com/apache/hadoop/pull/7887> , if there's anything else > we > > think needs to go in there, let me know, otherwise I will merge by > tomorrow. > > > > > > Since I have to regenerate the site and changelog, do I need to do the > > whole build again or is there an easier way to do that? (just by > > regenerating the site manually and updating the artifacts in SVN maybe?) > > > > > > I will update the artifacts for the hadoop-3.4.2-lean.tar.gz and > > hadoop-3.4.2-aarch64-lean.tar.sha512 now. > > > > > > On 19/08/2025, 08:26, "Masatake Iwasaki" <iwasak...@oss.nttdata.com > > <mailto:iwasak...@oss.nttdata.com> <mailto:iwasak...@oss.nttdata.com > > <mailto:iwasak...@oss.nttdata.com>>> wrote: > > > > > > > > > CAUTION: This email originated from outside of the organization. Do not > > click links or open attachments unless you can confirm the sender and > know > > the content is safe. > > > > > > > > > > > > > > > > > > > > > Thanks for the update, Ahmar > > > > > > > > > The binary artifacts basically looks good. > > > I have some comments about inconsistencies. > > > > > > > > > - Some JIRAs of additional commits did not have 3.4.2 as "Fix > > Version/s". I updated them. > > > > > > > > > Jira/Git commit message diff starting: > > ############################################## > > > Jira not present with version: 3.4.2. Commit: > > e1c0dee881820a4d834ec4a4d2c70d0d953bb933 HADOOP-19627. S3A: > > testIfMatchOverwriteWithOutdatedEtag() fails when not using SSE-KMS #7816 > > > ... > > > Jira not present with version: 3.4.2. Commit: > > 6a4ea9b1155aefb7732b700d8d7154f977141e39 HADOOP-19626. S3A: Updates AAL > > version to 1.2.1 (#7807) (#7815) > > > Jira not present with version: 3.4.2. Commit: > > 9677363829c21adeb2ce41f93059ab65fc115862 HADOOP-19554. LocalDirAllocator > > still doesn't always recover from directory deletion (#7683) (#7685) > > > ... > > > Jira not present with version: 3.4.2. Commit: > > 57d821e6aad23b604833272071c8481567363e63 HADOOP-19143. Upgrade > commons-cli > > to 1.9.0. (#7126) (#7146) Contributed by Min Yan. > > > > > > > > > - The top page of site documentation was not updated for 3.4.2. > > > > > > https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm > > < > > > https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm > > > > < > > > https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm > > > > < > > > https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm> > > ;> > > > > > > > > > - Some release artifact are inconsistent as Steve pointed out. > > > > > > > > > - `hadoop-3.4.2-lean.tar.gz` is not gzipped. > > > > > > > > > - `hadoop-3.4.2-aarch64-lean.tar.sha512` should be > > `hadoop-3.4.2-aarch64-lean.tar.gz.sha512` > > > > > > > > > Masatake Iwasaki > > > > > > > > > On 8/14/2025 6:12 PM, Ahmar Suhail wrote: > > >> Hey all, > > >> > > >> Just a reminder about this. Would be great to get some votes in asap > and > > >> hopefully finalise 3.4.2. > > >> > > >> Thanks, > > >> Ahmar > > >> > > >> On Thu, Aug 7, 2025 at 9:35 PM Ahmar Suhail <ah...@apache.org > <mailto: > > ah...@apache.org> <mailto:ah...@apache.org <mailto:ah...@apache.org>>> > > wrote: > > >> > > >>> Hey all, > > >>> > > >>> RC2 for Hadoop 3.4.2 is now available for voting. > > >>> > > >>> There are a couple of things to note: > > >>> > > >>> 1/ We have ARM artifacts again! Thanks to Masatake and Wei-Chiu > Chuang > > for > > >>> working on the fix. > > >>> > > >>> 2/ Relevant for anyone testing S3A: We've removed the AWS SDK bundle > > >>> from hadoop-3.4.2.tar.gz. This is because the SDK bundle is now > ~600MB, > > >>> which makes the size of tar > 1GB, and it can no longer be uploaded > to > > SVN. > > >>> For S3A, download SDK bundle v2.29.52 from: > > >>> > > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52 > < > > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52 > > > > < > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52> > > < > > > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52> > > ;>, > > >>> and drop it into /share/hadoop/common/lib. Release notes will be > > updated > > >>> with these instructions. > > >>> > > >>> > > >>> The RC is available at: > > >>> > > >>> https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/ < > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> < > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> < > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>> > > >>> > > >>> The git tag is release-3.4.2-RC2, commit > > >>> e1c0dee881820a4d834ec4a4d2c70d0d953bb933 > > >>> > > >>> The maven artifacts are staged at: > > >>> > > >>> > > https://repository.apache.org/content/repositories/orgapachehadoop-1441 > < > > https://repository.apache.org/content/repositories/orgapachehadoop-1441> > < > > https://repository.apache.org/content/repositories/orgapachehadoop-1441> > < > > > https://repository.apache.org/content/repositories/orgapachehadoop-1441> > > ;> > > >>> > > >>> > > >>> You can find my public key (02085AFB652F796A3B01D11FD737A6F52281FA98) > > at: > > >>> > > >>> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS < > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> < > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> < > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>> > > >>> > > >>> > > >>> This release has been created off of branch-3.4. > > >>> > > >>> Thanks, > > >>> Ahmar > > >>> > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org > > For additional commands, e-mail: common-dev-h...@hadoop.apache.org > > >
