Thanks Anuj, I've merged. Since I cut RC-3 yesterday, this won't be included in the release yet, if the RC fails the vote, will include it in the next one.
On 21/08/2025, 05:08, "Anuj Modi" <anujmodi2...@gmail.com <mailto:anujmodi2...@gmail.com>> wrote: CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe. Hi, Ahmar Here are the PRs for the fix raised for ABFS close() causing NPE raised on RC. PR on branch-3.4.2: https://github.com/apache/hadoop/pull/7888 <https://github.com/apache/hadoop/pull/7888> PR on branch-3.4: https://github.com/apache/hadoop/pull/7889 <https://github.com/apache/hadoop/pull/7889> Feel free to merge them as per the release timelines. Thanks On Wed, Aug 20, 2025 at 11:24 PM Steve Loughran <ste...@cloudera.com.inva <mailto:ste...@cloudera.com.inva>lid> wrote: > just discovering as I do the third party compatibility testing of the > latest sdk that the fs.s3a.create.checksum.algorithm stuff is a bit broken > -nothing to worry about though as its a new feature so not a regression > > On Wed, 20 Aug 2025 at 16:13, Suhail, Ahmar <ahma...@amazon.co.uk.inva > <mailto:ahma...@amazon.co.uk.inva>lid> > wrote: > > > Hey all, > > > > I've fixed the following issues: > > 1) Issues with changelog, release notes and landing page. Created a new > > build so they now include all the JIRA's that were missing before. > > 2) Create the lean tar on ubuntu so you won't get the tar: Ignoring > > unknown extended header keyword 'LIBARCHIVE.xattr.com.apple.quarantine' > > message when untarring > > 3) Previous issues with sha512 and tar not being gzipped. > > > > RC3 for Hadoop 3.4.2 is now available for voting. > > > > The RC is available at: > > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC3/ > > <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC3/> > > > > The git tag is release-3.4.2-RC3, commit > > 84e8b89ee2ebe6923691205b9e171badde7a495c > > > > The maven artifacts are staged at: > > > > https://repository.apache.org/content/repositories/orgapachehadoop-1443 > > <https://repository.apache.org/content/repositories/orgapachehadoop-1443> > > > > You can find my public key (02085AFB652F796A3B01D11FD737A6F52281FA98) at: > > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS > > <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> > > > > > > This release has been created off of branch-3.4. > > > > Thanks, > > Ahmar > > > > On 20/08/2025, 09:25, "Masatake Iwasaki" <iwasak...@oss.nttdata.com > > <mailto:iwasak...@oss.nttdata.com> > > <mailto:iwasak...@oss.nttdata.com <mailto:iwasak...@oss.nttdata.com>>> > > wrote: > > > > > > CAUTION: This email originated from outside of the organization. Do not > > click links or open attachments unless you can confirm the sender and > know > > the content is safe. > > > > > > > > > > > > > > > Since I have to regenerate the site and changelog, do I need to do the > > whole build again or is there an easier way to do that? (just by > > regenerating the site manually and updating the artifacts in SVN maybe?) > > > > > > I think just running the create-release[1] script could be an easy way > > while it does not provide options for building only site or source > > tarballs (now). > > Since the binary tarballs are containing affected CHANGELOG and > > RELEASENOTES, > > it would be nice to update the binary tarballs. > > # Swapping the CHANGELOG and RELEASENOTES of existing binary tarballs > > # could be ok if only documentation is updated. > > # Signing and checksums must be cared in that case. > > > > > > In addition, I got following warnings when I extracted the binary > tarballs > > (hadoop-3.4.2-lean.tar.gz and hadoop-3.4.2-aarch64-lean.tar.gz) on > > Ubuntu.:: > > > > > > tar: Ignoring unknown extended header keyword > > 'LIBARCHIVE.xattr.com.apple.provenance' > > tar: Ignoring unknown extended header keyword > > 'LIBARCHIVE.xattr.com.apple.quarantine' > > tar: Ignoring unknown extended header keyword > > 'LIBARCHIVE.xattr.com.apple.provenance' > > tar: Ignoring unknown extended header keyword > > 'LIBARCHIVE.xattr.com.apple.quarantine' > > > > > > I guess the cause is modifying the tar files on OSX environment. > > It would be nice to fix too. > > > > > > [1] > > > https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release > > <https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release> > > < > > > https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release > > <https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release> > > > > > > > > > Masatake Iwasaki > > > > > > On 8/19/2025 10:28 PM, Suhail, Ahmar wrote: > > > Thanks Masatake and Steve, > > > > > > Here's the PR for updating the top page of the site > > https://github.com/apache/hadoop/pull/7887 > > <https://github.com/apache/hadoop/pull/7887> < > > https://github.com/apache/hadoop/pull/7887> > > <https://github.com/apache/hadoop/pull/7887>> , if there's anything else > we > > think needs to go in there, let me know, otherwise I will merge by > tomorrow. > > > > > > Since I have to regenerate the site and changelog, do I need to do the > > whole build again or is there an easier way to do that? (just by > > regenerating the site manually and updating the artifacts in SVN maybe?) > > > > > > I will update the artifacts for the hadoop-3.4.2-lean.tar.gz and > > hadoop-3.4.2-aarch64-lean.tar.sha512 now. > > > > > > On 19/08/2025, 08:26, "Masatake Iwasaki" <iwasak...@oss.nttdata.com > > > <mailto:iwasak...@oss.nttdata.com> > > <mailto:iwasak...@oss.nttdata.com <mailto:iwasak...@oss.nttdata.com>> > > <mailto:iwasak...@oss.nttdata.com <mailto:iwasak...@oss.nttdata.com> > > <mailto:iwasak...@oss.nttdata.com <mailto:iwasak...@oss.nttdata.com>>>> > > wrote: > > > > > > > > > CAUTION: This email originated from outside of the organization. Do not > > click links or open attachments unless you can confirm the sender and > know > > the content is safe. > > > > > > > > > > > > > > > > > > > > > Thanks for the update, Ahmar > > > > > > > > > The binary artifacts basically looks good. > > > I have some comments about inconsistencies. > > > > > > > > > - Some JIRAs of additional commits did not have 3.4.2 as "Fix > > Version/s". I updated them. > > > > > > > > > Jira/Git commit message diff starting: > > ############################################## > > > Jira not present with version: 3.4.2. Commit: > > e1c0dee881820a4d834ec4a4d2c70d0d953bb933 HADOOP-19627. S3A: > > testIfMatchOverwriteWithOutdatedEtag() fails when not using SSE-KMS #7816 > > > ... > > > Jira not present with version: 3.4.2. Commit: > > 6a4ea9b1155aefb7732b700d8d7154f977141e39 HADOOP-19626. S3A: Updates AAL > > version to 1.2.1 (#7807) (#7815) > > > Jira not present with version: 3.4.2. Commit: > > 9677363829c21adeb2ce41f93059ab65fc115862 HADOOP-19554. LocalDirAllocator > > still doesn't always recover from directory deletion (#7683) (#7685) > > > ... > > > Jira not present with version: 3.4.2. Commit: > > 57d821e6aad23b604833272071c8481567363e63 HADOOP-19143. Upgrade > commons-cli > > to 1.9.0. (#7126) (#7146) Contributed by Min Yan. > > > > > > > > > - The top page of site documentation was not updated for 3.4.2. > > > > > > https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm > > <https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm> > > < > > > https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm > > <https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm> > > > > < > > > https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm > > <https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm> > > > > < > > > https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm> > > <https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm&gt> > > ;> > > > > > > > > > - Some release artifact are inconsistent as Steve pointed out. > > > > > > > > > - `hadoop-3.4.2-lean.tar.gz` is not gzipped. > > > > > > > > > - `hadoop-3.4.2-aarch64-lean.tar.sha512` should be > > `hadoop-3.4.2-aarch64-lean.tar.gz.sha512` > > > > > > > > > Masatake Iwasaki > > > > > > > > > On 8/14/2025 6:12 PM, Ahmar Suhail wrote: > > >> Hey all, > > >> > > >> Just a reminder about this. Would be great to get some votes in asap > and > > >> hopefully finalise 3.4.2. > > >> > > >> Thanks, > > >> Ahmar > > >> > > >> On Thu, Aug 7, 2025 at 9:35 PM Ahmar Suhail <ah...@apache.org > > >> <mailto:ah...@apache.org> > <mailto: > > ah...@apache.org <mailto:ah...@apache.org>> <mailto:ah...@apache.org > > <mailto:ah...@apache.org> <mailto:ah...@apache.org > > <mailto:ah...@apache.org>>>> > > wrote: > > >> > > >>> Hey all, > > >>> > > >>> RC2 for Hadoop 3.4.2 is now available for voting. > > >>> > > >>> There are a couple of things to note: > > >>> > > >>> 1/ We have ARM artifacts again! Thanks to Masatake and Wei-Chiu > Chuang > > for > > >>> working on the fix. > > >>> > > >>> 2/ Relevant for anyone testing S3A: We've removed the AWS SDK bundle > > >>> from hadoop-3.4.2.tar.gz. This is because the SDK bundle is now > ~600MB, > > >>> which makes the size of tar > 1GB, and it can no longer be uploaded > to > > SVN. > > >>> For S3A, download SDK bundle v2.29.52 from: > > >>> > > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52 > > <https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52> > < > > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52 > > <https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52> > > > > < > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52> > <https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>> > > < > > > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52> > <https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52&gt> > > ;>, > > >>> and drop it into /share/hadoop/common/lib. Release notes will be > > updated > > >>> with these instructions. > > >>> > > >>> > > >>> The RC is available at: > > >>> > > >>> https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/ > > >>> <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> < > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> > > <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>> < > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> > > <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>> < > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>> > > <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/&gt;>> > > >>> > > >>> The git tag is release-3.4.2-RC2, commit > > >>> e1c0dee881820a4d834ec4a4d2c70d0d953bb933 > > >>> > > >>> The maven artifacts are staged at: > > >>> > > >>> > > https://repository.apache.org/content/repositories/orgapachehadoop-1441 > > <https://repository.apache.org/content/repositories/orgapachehadoop-1441> > < > > https://repository.apache.org/content/repositories/orgapachehadoop-1441> > > <https://repository.apache.org/content/repositories/orgapachehadoop-1441>> > < > > https://repository.apache.org/content/repositories/orgapachehadoop-1441> > > <https://repository.apache.org/content/repositories/orgapachehadoop-1441>> > < > > > https://repository.apache.org/content/repositories/orgapachehadoop-1441> > <https://repository.apache.org/content/repositories/orgapachehadoop-1441&gt> > > ;> > > >>> > > >>> > > >>> You can find my public key (02085AFB652F796A3B01D11FD737A6F52281FA98) > > at: > > >>> > > >>> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS > > >>> <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> < > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> > > <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>> < > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> > > <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>> < > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>> > > <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS&gt;>> > > >>> > > >>> > > >>> This release has been created off of branch-3.4. > > >>> > > >>> Thanks, > > >>> Ahmar > > >>> > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org > > <mailto:common-dev-unsubscr...@hadoop.apache.org> > > For additional commands, e-mail: common-dev-h...@hadoop.apache.org > > <mailto:common-dev-h...@hadoop.apache.org> > > > --------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
