Unfortunately, it turns out to be a regression in addition to a breaking change.
In short, HADOOP-19098 (or more) makes Hadoop 3.4.1 fails even when users give disjoint ranges. I filed a Hadoop JIRA issue and a PR. Please take a look at that. - HADOOP-19291. `CombinedFileRange.merge` should not convert disjoint ranges into overlapped ones - https://github.com/apache/hadoop/pull/7079 I believe this is a Hadoop release blocker for both Apache ORC and Apache Parquet project perspective. Dongjoon. On 2024/09/29 03:16:18 Dongjoon Hyun wrote: > Thank you for 3.4.1 RC2. > > HADOOP-19098 (Vector IO: consistent specified rejection of overlapping > ranges) seems to be a hard breaking change at 3.4.1. > > Do you think we can have an option to handle the overlapping ranges in Hadoop > layer instead of introducing a breaking change to the users at the > maintenance release? > > Dongjoon. > > On 2024/09/25 20:13:48 Mukund Madhav Thakur wrote: > > Apache Hadoop 3.4.1 > > > > > > With help from Steve I have put together a release candidate (RC2) for > > Hadoop 3.4.1. > > > > > > What we would like is for anyone who can to verify the tarballs, especially > > > > anyone who can try the arm64 binaries as we want to include them too. > > > > > > The RC is available at: > > > > https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.4.1-RC2/ > > > > > > The git tag is release-3.4.1-RC2, commit > > b3a4b582eeb729a0f48eca77121dd5e2983b2004 > > > > > > The maven artifacts are staged at > > > > https://repository.apache.org/content/repositories/orgapachehadoop-1426 > > > > > > You can find my public key at: > > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS > > > > > > Change log > > > > https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.4.1-RC2/CHANGELOG.md > > > > > > Release notes > > > > https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.4.1-RC2/RELEASENOTES.md > > > > > > This is off branch-3.4. > > > > > > Key changes include > > > > > > * Bulk Delete API. https://issues.apache.org/jira/browse/HADOOP-18679 > > > > * Fixes and enhancements in Vectored IO API. > > > > * Improvements in Hadoop Azure connector. > > > > * Fixes and improvements post upgrade to AWS V2 SDK in S3AConnector. > > > > * This release includes Arm64 binaries. Please can anyone with > > > > compatible systems validate these. > > > > > > Note, because the arm64 binaries are built separately on a different > > > > platform and JVM, their jar files may not match those of the x86 > > > > release -and therefore the maven artifacts. I don't think this is > > > > an issue (the ASF actually releases source tarballs, the binaries are > > > > there for help only, though with the maven repo that's a bit blurred). > > > > > > The only way to be consistent would actually untar the x86.tar.gz, > > > > overwrite its binaries with the arm stuff, retar, sign and push out > > > > for the vote. Even automating that would be risky. > > > > > > Please try the release and vote. The vote will run for 5 days. > > > > > > > > Thanks, > > > > Mukund > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org > For additional commands, e-mail: common-dev-h...@hadoop.apache.org > > --------------------------------------------------------------------- To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
