Author: mthakur Date: Tue Aug 6 22:38:00 2024 New Revision: 70732 Log: adding hadoop 3.4.1-rc0
Added: dev/hadoop/hadoop-3.4.1-RC1/ dev/hadoop/hadoop-3.4.1-RC1/CHANGELOG.md dev/hadoop/hadoop-3.4.1-RC1/CHANGELOG.md.asc dev/hadoop/hadoop-3.4.1-RC1/CHANGELOG.md.sha512 dev/hadoop/hadoop-3.4.1-RC1/RELEASENOTES.md dev/hadoop/hadoop-3.4.1-RC1/RELEASENOTES.md.asc dev/hadoop/hadoop-3.4.1-RC1/RELEASENOTES.md.sha512 dev/hadoop/hadoop-3.4.1-RC1/hadoop-3.4.1-rat.txt dev/hadoop/hadoop-3.4.1-RC1/hadoop-3.4.1-rat.txt.asc dev/hadoop/hadoop-3.4.1-RC1/hadoop-3.4.1-rat.txt.sha512 dev/hadoop/hadoop-3.4.1-RC1/hadoop-3.4.1-site.tar.gz (with props) dev/hadoop/hadoop-3.4.1-RC1/hadoop-3.4.1-site.tar.gz.asc dev/hadoop/hadoop-3.4.1-RC1/hadoop-3.4.1-site.tar.gz.sha512 dev/hadoop/hadoop-3.4.1-RC1/hadoop-3.4.1-src.tar.gz (with props) dev/hadoop/hadoop-3.4.1-RC1/hadoop-3.4.1-src.tar.gz.asc dev/hadoop/hadoop-3.4.1-RC1/hadoop-3.4.1-src.tar.gz.sha512 dev/hadoop/hadoop-3.4.1-RC1/hadoop-3.4.1.tar.gz (with props) dev/hadoop/hadoop-3.4.1-RC1/hadoop-3.4.1.tar.gz.asc dev/hadoop/hadoop-3.4.1-RC1/hadoop-3.4.1.tar.gz.sha512 Added: dev/hadoop/hadoop-3.4.1-RC1/CHANGELOG.md ============================================================================== --- dev/hadoop/hadoop-3.4.1-RC1/CHANGELOG.md (added) +++ dev/hadoop/hadoop-3.4.1-RC1/CHANGELOG.md Tue Aug 6 22:38:00 2024 @@ -0,0 +1,154 @@ + +<!--- +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +--> +# Apache Hadoop Changelog + +## Release 3.4.1 - 2024-08-02 + + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HDFS-17359](https://issues.apache.org/jira/browse/HDFS-17359) | EC: recheck failed streamers should only after flushing all packets. | Minor | ec | farmmamba | farmmamba | +| [HADOOP-18987](https://issues.apache.org/jira/browse/HADOOP-18987) | Corrections to Hadoop FileSystem API Definition | Minor | documentation | Dieter De Paepe | Dieter De Paepe | +| [HADOOP-18993](https://issues.apache.org/jira/browse/HADOOP-18993) | S3A: Add option fs.s3a.classloader.isolation (#6301) | Minor | fs/s3 | Antonio Murgia | Antonio Murgia | +| [HADOOP-19059](https://issues.apache.org/jira/browse/HADOOP-19059) | S3A: update AWS SDK to 2.23.19 to support S3 Access Grants | Minor | build, fs/s3 | Jason Han | Jason Han | +| [HADOOP-19065](https://issues.apache.org/jira/browse/HADOOP-19065) | Update Protocol Buffers installation to 3.21.12 | Major | build | Zhaobo Huang | Zhaobo Huang | +| [HADOOP-19082](https://issues.apache.org/jira/browse/HADOOP-19082) | S3A: Update AWS SDK V2 to 2.24.6 | Major | fs/s3 | Harshit Gupta | Harshit Gupta | +| [YARN-11657](https://issues.apache.org/jira/browse/YARN-11657) | Remove protobuf-2.5 as dependency of hadoop-yarn-api | Major | api | Steve Loughran | Steve Loughran | +| [HDFS-17404](https://issues.apache.org/jira/browse/HDFS-17404) | Add Namenode info to log message when setting block keys from active nn | Trivial | . | Joseph Dell'Aringa | | +| [HADOOP-19090](https://issues.apache.org/jira/browse/HADOOP-19090) | Update Protocol Buffers installation to 3.23.4 | Major | build | PJ Fanning | PJ Fanning | +| [HDFS-17431](https://issues.apache.org/jira/browse/HDFS-17431) | Fix log format for BlockRecoveryWorker#recoverBlocks | Major | . | Haiyang Hu | Haiyang Hu | +| [MAPREDUCE-7469](https://issues.apache.org/jira/browse/MAPREDUCE-7469) | NNBench createControlFiles should use thread pool to improve performance. | Minor | mapreduce-client | liuguanghua | | +| [HADOOP-19052](https://issues.apache.org/jira/browse/HADOOP-19052) | Hadoop use Shell command to get the count of the hard link which takes a lot of time | Major | fs | liang yu | | +| [HADOOP-19047](https://issues.apache.org/jira/browse/HADOOP-19047) | Support InMemory Tracking Of S3A Magic Commits | Major | fs/s3 | Syed Shameerur Rahman | Syed Shameerur Rahman | +| [HDFS-17429](https://issues.apache.org/jira/browse/HDFS-17429) | Datatransfer sender.java LOG variable uses interface's, causing log fileName mistake | Trivial | . | Zhongkun Wu | | +| [YARN-11663](https://issues.apache.org/jira/browse/YARN-11663) | [Federation] Add Cache Entity Nums Limit. | Major | federation, yarn | Yuan Luo | Shilun Fan | +| [HADOOP-19135](https://issues.apache.org/jira/browse/HADOOP-19135) | Remove Jcache 1.0-alpha | Major | common | Shilun Fan | Shilun Fan | +| [YARN-11444](https://issues.apache.org/jira/browse/YARN-11444) | Improve YARN md documentation format | Major | yarn | Shilun Fan | Shilun Fan | +| [HDFS-17367](https://issues.apache.org/jira/browse/HDFS-17367) | Add PercentUsed for Different StorageTypes in JMX | Major | metrics, namenode | Hualong Zhang | Hualong Zhang | +| [HADOOP-19159](https://issues.apache.org/jira/browse/HADOOP-19159) | Fix hadoop-aws document for fs.s3a.committer.abort.pending.uploads | Minor | documentation | Xi Chen | Xi Chen | +| [HADOOP-19146](https://issues.apache.org/jira/browse/HADOOP-19146) | noaa-cors-pds bucket access with global endpoint fails | Minor | fs/s3, test | Viraj Jasani | Viraj Jasani | +| [HADOOP-19160](https://issues.apache.org/jira/browse/HADOOP-19160) | hadoop-auth should not depend on kerb-simplekdc | Major | auth | Attila Doroszlai | Attila Doroszlai | +| [HADOOP-19172](https://issues.apache.org/jira/browse/HADOOP-19172) | Upgrade aws-java-sdk to 1.12.720 | Minor | build, fs/s3 | Steve Loughran | Steve Loughran | +| [YARN-11471](https://issues.apache.org/jira/browse/YARN-11471) | FederationStateStoreFacade Cache Support Caffeine | Major | federation | Shilun Fan | Shilun Fan | +| [YARN-11699](https://issues.apache.org/jira/browse/YARN-11699) | Diagnostics lacks userlimit info when user capacity has reached its maximum limit | Major | capacity scheduler | Jiandan Yang | Jiandan Yang | +| [HADOOP-19192](https://issues.apache.org/jira/browse/HADOOP-19192) | Log level is WARN when fail to load native hadoop libs | Minor | documentation | Cheng Pan | Cheng Pan | +| [HADOOP-18931](https://issues.apache.org/jira/browse/HADOOP-18931) | FileSystem.getFileSystemClass() to log at debug the jar the .class came from | Minor | fs | Steve Loughran | Viraj Jasani | +| [HADOOP-19203](https://issues.apache.org/jira/browse/HADOOP-19203) | WrappedIO BulkDelete API to raise IOEs as UncheckedIOExceptions | Major | fs | Steve Loughran | Steve Loughran | +| [HADOOP-19194](https://issues.apache.org/jira/browse/HADOOP-19194) | Add test to find unshaded dependencies in the aws sdk | Major | fs/s3 | Harshit Gupta | Harshit Gupta | +| [HADOOP-19195](https://issues.apache.org/jira/browse/HADOOP-19195) | Upgrade aws sdk v2 to 2.25.53 | Major | fs/s3 | Harshit Gupta | Harshit Gupta | +| [HADOOP-19218](https://issues.apache.org/jira/browse/HADOOP-19218) | Avoid DNS lookup while creating IPC Connection object | Major | ipc | Viraj Jasani | Viraj Jasani | +| [HADOOP-19161](https://issues.apache.org/jira/browse/HADOOP-19161) | S3A: option "fs.s3a.performance.flags" to take list of performance flags | Major | fs/s3 | Steve Loughran | Steve Loughran | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-19049](https://issues.apache.org/jira/browse/HADOOP-19049) | Class loader leak caused by StatisticsDataReferenceCleaner thread | Major | common | Jia Fan | Jia Fan | +| [HDFS-17299](https://issues.apache.org/jira/browse/HDFS-17299) | HDFS is not rack failure tolerant while creating a new file. | Critical | . | Rushabh Shah | Ritesh | +| [YARN-11660](https://issues.apache.org/jira/browse/YARN-11660) | SingleConstraintAppPlacementAllocator performance regression | Major | scheduler | Junfan Zhang | Junfan Zhang | +| [HDFS-17354](https://issues.apache.org/jira/browse/HDFS-17354) | Delay invoke clearStaleNamespacesInRouterStateIdContext during router start up | Major | . | lei w | lei w | +| [HADOOP-19116](https://issues.apache.org/jira/browse/HADOOP-19116) | update to zookeeper client 3.8.4 due to CVE-2024-23944 | Major | CVE | PJ Fanning | PJ Fanning | +| [YARN-11668](https://issues.apache.org/jira/browse/YARN-11668) | Potential concurrent modification exception for node attributes of node manager | Major | . | Junfan Zhang | Junfan Zhang | +| [HADOOP-19115](https://issues.apache.org/jira/browse/HADOOP-19115) | upgrade to nimbus-jose-jwt 9.37.2 due to CVE | Major | build, CVE | PJ Fanning | PJ Fanning | +| [HADOOP-19110](https://issues.apache.org/jira/browse/HADOOP-19110) | ITestExponentialRetryPolicy failing in branch-3.4 | Major | fs/azure | Mukund Thakur | Anuj Modi | +| [HADOOP-19106](https://issues.apache.org/jira/browse/HADOOP-19106) | [ABFS] All tests of. ITestAzureBlobFileSystemAuthorization fails with NPE | Major | fs/azure | Mukund Thakur | Anuj Modi | +| [YARN-11684](https://issues.apache.org/jira/browse/YARN-11684) | PriorityQueueComparator violates general contract | Major | capacityscheduler | Tamas Domok | Tamas Domok | +| [HADOOP-19170](https://issues.apache.org/jira/browse/HADOOP-19170) | Fixes compilation issues on Mac | Major | . | Chenyu Zheng | Chenyu Zheng | +| [HDFS-17520](https://issues.apache.org/jira/browse/HDFS-17520) | TestDFSAdmin.testAllDatanodesReconfig and TestDFSAdmin.testDecommissionDataNodesReconfig failed | Major | hdfs | ZanderXu | ZanderXu | +| [MAPREDUCE-7474](https://issues.apache.org/jira/browse/MAPREDUCE-7474) | [ABFS] Improve commit resilience and performance in Manifest Committer | Major | client | Steve Loughran | Steve Loughran | +| [MAPREDUCE-7475](https://issues.apache.org/jira/browse/MAPREDUCE-7475) | Fix non-idempotent unit tests | Minor | test | Kaiyao Ke | Kaiyao Ke | +| [HADOOP-19188](https://issues.apache.org/jira/browse/HADOOP-19188) | TestHarFileSystem and TestFilterFileSystem failing after bulk delete API added | Minor | fs, test | Steve Loughran | Mukund Thakur | +| [HADOOP-19114](https://issues.apache.org/jira/browse/HADOOP-19114) | upgrade to commons-compress 1.26.1 due to cves | Major | build, CVE | PJ Fanning | PJ Fanning | +| [HADOOP-19196](https://issues.apache.org/jira/browse/HADOOP-19196) | Bulk delete api doesn't take the path to delete as the base path | Minor | fs | Steve Loughran | Mukund Thakur | +| [YARN-11701](https://issues.apache.org/jira/browse/YARN-11701) | Enhance Federation Cache Clean Conditions | Major | federation | Shilun Fan | Shilun Fan | +| [HADOOP-19222](https://issues.apache.org/jira/browse/HADOOP-19222) | Switch yum repo baseurl due to CentOS 7 sunset | Major | build | Cheng Pan | Cheng Pan | +| [HADOOP-19238](https://issues.apache.org/jira/browse/HADOOP-19238) | Fix create-release script for arm64 based MacOS | Major | . | Mukund Thakur | Mukund Thakur | + + +### TESTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HDFS-17370](https://issues.apache.org/jira/browse/HDFS-17370) | Fix junit dependency for running parameterized tests in hadoop-hdfs-rbf | Major | . | Takanobu Asanuma | Takanobu Asanuma | +| [HDFS-17432](https://issues.apache.org/jira/browse/HDFS-17432) | Fix junit dependency to enable JUnit4 tests to run in hadoop-hdfs-rbf | Major | . | Takanobu Asanuma | Takanobu Asanuma | +| [HDFS-17435](https://issues.apache.org/jira/browse/HDFS-17435) | Fix TestRouterRpc failed | Major | . | Takanobu Asanuma | Takanobu Asanuma | +| [HDFS-17441](https://issues.apache.org/jira/browse/HDFS-17441) | Fix junit dependency by adding missing library in hadoop-hdfs-rbf | Major | . | Takanobu Asanuma | Takanobu Asanuma | +| [HADOOP-19190](https://issues.apache.org/jira/browse/HADOOP-19190) | Skip ITestS3AEncryptionWithDefaultS3Settings.testEncryptionFileAttributes when bucket not encrypted with sse-kms | Minor | fs/s3 | Mukund Thakur | Mukund Thakur | + + +### SUB-TASKS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-19004](https://issues.apache.org/jira/browse/HADOOP-19004) | S3A: Support Authentication through HttpSigner API | Major | fs/s3 | Steve Loughran | Harshit Gupta | +| [HADOOP-19027](https://issues.apache.org/jira/browse/HADOOP-19027) | S3A: S3AInputStream doesn't recover from HTTP/channel exceptions | Major | fs/s3 | Steve Loughran | Steve Loughran | +| [HADOOP-18883](https://issues.apache.org/jira/browse/HADOOP-18883) | Expect-100 JDK bug resolution: prevent multiple server calls | Major | fs/azure | Pranav Saxena | Pranav Saxena | +| [HADOOP-19015](https://issues.apache.org/jira/browse/HADOOP-19015) | Increase fs.s3a.connection.maximum to 500 to minimize risk of Timeout waiting for connection from pool | Major | fs/s3 | Mukund Thakur | Mukund Thakur | +| [HADOOP-18975](https://issues.apache.org/jira/browse/HADOOP-18975) | AWS SDK v2: extend support for FIPS endpoints | Major | fs/s3 | Steve Loughran | Steve Loughran | +| [HADOOP-19046](https://issues.apache.org/jira/browse/HADOOP-19046) | S3A: update AWS sdk versions to 2.23.5 and 1.12.599 | Major | build, fs/s3 | Steve Loughran | Steve Loughran | +| [HADOOP-18830](https://issues.apache.org/jira/browse/HADOOP-18830) | S3A: Cut S3 Select | Major | fs/s3 | Steve Loughran | Steve Loughran | +| [HADOOP-18980](https://issues.apache.org/jira/browse/HADOOP-18980) | S3A credential provider remapping: make extensible | Minor | fs/s3 | Steve Loughran | Viraj Jasani | +| [HADOOP-19044](https://issues.apache.org/jira/browse/HADOOP-19044) | AWS SDK V2 - Update S3A region logic | Major | fs/s3 | Ahmar Suhail | Viraj Jasani | +| [HADOOP-19045](https://issues.apache.org/jira/browse/HADOOP-19045) | HADOOP-19045. S3A: CreateSession Timeout after 10 seconds | Major | fs/s3 | Steve Loughran | Steve Loughran | +| [HADOOP-19057](https://issues.apache.org/jira/browse/HADOOP-19057) | S3 public test bucket landsat-pds unreadable -needs replacement | Critical | fs/s3, test | Steve Loughran | Steve Loughran | +| [HADOOP-19084](https://issues.apache.org/jira/browse/HADOOP-19084) | prune dependency exports of hadoop-\* modules | Blocker | build | Steve Loughran | Steve Loughran | +| [HADOOP-19097](https://issues.apache.org/jira/browse/HADOOP-19097) | core-default fs.s3a.connection.establish.timeout value too low -warning always printed | Minor | fs/s3 | Steve Loughran | Steve Loughran | +| [HADOOP-19066](https://issues.apache.org/jira/browse/HADOOP-19066) | AWS SDK V2 - Enabling FIPS should be allowed with central endpoint | Major | fs/s3 | Viraj Jasani | Viraj Jasani | +| [HADOOP-19119](https://issues.apache.org/jira/browse/HADOOP-19119) | spotbugs complaining about possible NPE in org.apache.hadoop.crypto.key.kms.ValueQueue.getSize() | Minor | crypto | Steve Loughran | Steve Loughran | +| [HADOOP-19089](https://issues.apache.org/jira/browse/HADOOP-19089) | [ABFS] Reverting Back Support of setXAttr() and getXAttr() on root path | Major | fs/azure | Anuj Modi | Anuj Modi | +| [HADOOP-19141](https://issues.apache.org/jira/browse/HADOOP-19141) | Update VectorIO default values consistently | Major | fs, fs/s3 | Dongjoon Hyun | Dongjoon Hyun | +| [HADOOP-19098](https://issues.apache.org/jira/browse/HADOOP-19098) | Vector IO: consistent specified rejection of overlapping ranges | Major | fs, fs/s3 | Steve Loughran | Steve Loughran | +| [HADOOP-19101](https://issues.apache.org/jira/browse/HADOOP-19101) | Vectored Read into off-heap buffer broken in fallback implementation | Blocker | fs, fs/azure | Steve Loughran | Steve Loughran | +| [HADOOP-19096](https://issues.apache.org/jira/browse/HADOOP-19096) | [ABFS] Enhancing Client-Side Throttling Metrics Updation Logic | Major | fs/azure | Anuj Modi | Anuj Modi | +| [HADOOP-19129](https://issues.apache.org/jira/browse/HADOOP-19129) | ABFS: Fixing Test Script Bug and Some Known test Failures in ABFS Test Suite | Major | fs/azure | Anuj Modi | Anuj Modi | +| [HADOOP-19102](https://issues.apache.org/jira/browse/HADOOP-19102) | [ABFS]: FooterReadBufferSize should not be greater than readBufferSize | Major | fs/azure | Pranav Saxena | Pranav Saxena | +| [HADOOP-19150](https://issues.apache.org/jira/browse/HADOOP-19150) | Test ITestAbfsRestOperationException#testAuthFailException is broken. | Major | . | Mukund Thakur | Anuj Modi | +| [HADOOP-19013](https://issues.apache.org/jira/browse/HADOOP-19013) | fs.getXattrs(path) for S3FS doesn't have x-amz-server-side-encryption-aws-kms-key-id header. | Major | fs/s3 | Mukund Thakur | Mukund Thakur | +| [HADOOP-18011](https://issues.apache.org/jira/browse/HADOOP-18011) | ABFS: Enable config control for default connection timeout | Major | fs/azure | Sneha Vijayarajan | Sneha Vijayarajan | +| [HADOOP-18759](https://issues.apache.org/jira/browse/HADOOP-18759) | [ABFS][Backoff-Optimization] Have a Static retry policy for connection timeout failures | Major | fs/azure | Anuj Modi | Anuj Modi | +| [HADOOP-19184](https://issues.apache.org/jira/browse/HADOOP-19184) | TestStagingCommitter.testJobCommitFailure failing | Critical | fs/s3 | Mukund Thakur | Mukund Thakur | +| [HADOOP-18679](https://issues.apache.org/jira/browse/HADOOP-18679) | Add API for bulk/paged delete of files and objects | Major | fs/s3 | Steve Loughran | Mukund Thakur | +| [HADOOP-19178](https://issues.apache.org/jira/browse/HADOOP-19178) | WASB Driver Deprecation and eventual removal | Major | fs/azure | Sneha Vijayarajan | Anuj Modi | +| [HADOOP-18516](https://issues.apache.org/jira/browse/HADOOP-18516) | [ABFS]: Support fixed SAS token config in addition to Custom SASTokenProvider Implementation | Minor | fs/azure | Sree Bhattacharyya | Anuj Modi | +| [HADOOP-19137](https://issues.apache.org/jira/browse/HADOOP-19137) | [ABFS]Prevent ABFS initialization for non-hierarchical-namespace account if Customer-provided-key configs given. | Major | fs/azure | Pranav Saxena | Pranav Saxena | +| [HADOOP-18508](https://issues.apache.org/jira/browse/HADOOP-18508) | support multiple s3a integration test runs on same bucket in parallel | Minor | fs/s3, test | Steve Loughran | Steve Loughran | +| [HADOOP-19204](https://issues.apache.org/jira/browse/HADOOP-19204) | VectorIO regression: empty ranges are now rejected | Major | fs | Steve Loughran | Steve Loughran | +| [HADOOP-19210](https://issues.apache.org/jira/browse/HADOOP-19210) | s3a: TestS3AAWSCredentialsProvider and TestS3AInputStreamRetry really slow | Major | fs/s3, test | Steve Loughran | Steve Loughran | +| [HADOOP-19205](https://issues.apache.org/jira/browse/HADOOP-19205) | S3A initialization/close slower than with v1 SDK | Major | fs/s3 | Steve Loughran | Steve Loughran | +| [HADOOP-19208](https://issues.apache.org/jira/browse/HADOOP-19208) | ABFS: Fixing logic to determine HNS nature of account to avoid extra getAcl() calls | Major | fs/azure | Anuj Modi | Anuj Modi | +| [HADOOP-19120](https://issues.apache.org/jira/browse/HADOOP-19120) | [ABFS]: ApacheHttpClient adaptation as network library | Major | fs/azure | Pranav Saxena | Pranav Saxena | +| [HADOOP-19245](https://issues.apache.org/jira/browse/HADOOP-19245) | S3ABlockOutputStream no longer sends progress events in close() | Critical | fs/s3 | Steve Loughran | Steve Loughran | + + +### OTHER: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HDFS-17362](https://issues.apache.org/jira/browse/HDFS-17362) | RBF: Implement RouterObserverReadConfiguredFailoverProxyProvider | Major | . | Takanobu Asanuma | Takanobu Asanuma | +| [HADOOP-19024](https://issues.apache.org/jira/browse/HADOOP-19024) | Use bouncycastle jdk18 1.77 | Major | . | PJ Fanning | PJ Fanning | +| [HDFS-17450](https://issues.apache.org/jira/browse/HDFS-17450) | Add explicit dependency on httpclient jar | Major | . | PJ Fanning | PJ Fanning | +| [HADOOP-19123](https://issues.apache.org/jira/browse/HADOOP-19123) | Update commons-configuration2 to 2.10.1 due to CVE | Major | . | PJ Fanning | PJ Fanning | +| [HADOOP-19079](https://issues.apache.org/jira/browse/HADOOP-19079) | HttpExceptionUtils to check that loaded class is really an exception before instantiation | Major | common, security | PJ Fanning | PJ Fanning | +| [HDFS-17591](https://issues.apache.org/jira/browse/HDFS-17591) | RBF: Router should follow X-FRAME-OPTIONS protection setting | Major | . | Takanobu Asanuma | Takanobu Asanuma | + + Added: dev/hadoop/hadoop-3.4.1-RC1/CHANGELOG.md.asc ============================================================================== --- dev/hadoop/hadoop-3.4.1-RC1/CHANGELOG.md.asc (added) +++ dev/hadoop/hadoop-3.4.1-RC1/CHANGELOG.md.asc Tue Aug 6 22:38:00 2024 @@ -0,0 +1,16 @@ +-----BEGIN PGP SIGNATURE----- + +iQIzBAABCgAdFiEEU5MdqnCCkUCZWL1HTSK7fTKIIgEFAmatOfAACgkQTSK7fTKI +IgGUnQ//WrUWAAlaotfCFeFS179jpLi3cAy+6e9QHa5L2Hv9vO8DVMDFlc8f932X +M0at1qDuYsQ0fBJvb1BCSJFFTjymVbNKVM1MsmsmkP9djDvkdZ3ibQSVS9gzpEFr +3XnjHDVItKHJH6T3iCOTOz2g1+DhCP4O4k7/lMLHrKT4NqvQLgLkThvoyXRMdsBV +4am7JhEjh4tw4BZPDII52C/1Wc9nZK4Veeb+tvXtg64mipvKeRjuYF+0FhiYfljH +DxQyIcELEPWr71fjWrlnARdmgh/D2x7WWmWSSeGKdgyy8+tPlZVr+RUgUrIX3bsO +/TJkRLUAquWtBwIrMBqvQ4igMO2pVbg4aUGt7pfZNWrixk5RCjsDYrjfx95lS0rQ +OvzuTKG+cnRdweB28bGJgqSia74dPjcT+sG2Hvu6QdKR1e+YU5erwXIzGmoXpHMB +9pe1rZjtix2wGrznkgKIzm/nJxi2DqL3kNOcmpoOQeYK8o+cTO6aCxoaSGiRLgGG +zPDWcc/82g9HE1L1k9rCfTe0oiNXB/XS5CvbI7H88FLw5jvtHUSkTpv6CofTYhyO +fq3WmlKi9v4WlKM6+lmQQjzxgNksCBS4XV8cEDGIuGleRFIU8xTYn+aoT7XUz9Q6 +HfYPG89N82fhsUYTjdXPngsQ3iKlwiQH9zH3VTj0/c+NPfVWzk0= +=YHr6 +-----END PGP SIGNATURE----- Added: dev/hadoop/hadoop-3.4.1-RC1/CHANGELOG.md.sha512 ============================================================================== --- dev/hadoop/hadoop-3.4.1-RC1/CHANGELOG.md.sha512 (added) +++ dev/hadoop/hadoop-3.4.1-RC1/CHANGELOG.md.sha512 Tue Aug 6 22:38:00 2024 @@ -0,0 +1 @@ +SHA512 (CHANGELOG.md) = 5ca9d12beb942b07f36c1a765e5713f883433d1a1b0da26a1bfd81f9a989713b7957aec39c21a2120ba917fc089c82e3509a21f9638ace3b0245fc85b83186f0 Added: dev/hadoop/hadoop-3.4.1-RC1/RELEASENOTES.md ============================================================================== --- dev/hadoop/hadoop-3.4.1-RC1/RELEASENOTES.md (added) +++ dev/hadoop/hadoop-3.4.1-RC1/RELEASENOTES.md Tue Aug 6 22:38:00 2024 @@ -0,0 +1,106 @@ + +<!--- +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +--> +# Apache Hadoop 3.4.1 Release Notes + +These release notes cover new developer and user-facing incompatibilities, important issues, features, and major improvements. + + +--- + +* [HADOOP-18830](https://issues.apache.org/jira/browse/HADOOP-18830) | *Major* | **S3A: Cut S3 Select** + +S3 Select is no longer supported through the S3A connector + + +--- + +* [HADOOP-18993](https://issues.apache.org/jira/browse/HADOOP-18993) | *Minor* | **S3A: Add option fs.s3a.classloader.isolation (#6301)** + +If the user wants to load custom implementations of AWS Credential Providers through user provided jars can set {{fs.s3a.extensions.isolated.classloader}} to {{false}}. + + +--- + +* [HADOOP-19084](https://issues.apache.org/jira/browse/HADOOP-19084) | *Blocker* | **prune dependency exports of hadoop-\* modules** + +maven/ivy imports of hadoop-common are less likely to end up with log4j versions on their classpath. + + +--- + +* [HADOOP-19101](https://issues.apache.org/jira/browse/HADOOP-19101) | *Blocker* | **Vectored Read into off-heap buffer broken in fallback implementation** + +PositionedReadable.readVectored() will read incorrect data when reading from hdfs, azure abfs and other stores when given a direct buffer allocator. + +For cross-version compatibility, use on-heap buffer allocators only + + +--- + +* [HADOOP-19110](https://issues.apache.org/jira/browse/HADOOP-19110) | *Major* | **ITestExponentialRetryPolicy failing in branch-3.4** + +https://github.com/apache/hadoop/pull/6676 + + +--- + +* [HADOOP-19106](https://issues.apache.org/jira/browse/HADOOP-19106) | *Major* | **[ABFS] All tests of. ITestAzureBlobFileSystemAuthorization fails with NPE** + +https://github.com/apache/hadoop/pull/6676 + + +--- + +* [HADOOP-19129](https://issues.apache.org/jira/browse/HADOOP-19129) | *Major* | **ABFS: Fixing Test Script Bug and Some Known test Failures in ABFS Test Suite** + +https://github.com/apache/hadoop/pull/6676 + + +--- + +* [HADOOP-18011](https://issues.apache.org/jira/browse/HADOOP-18011) | *Major* | **ABFS: Enable config control for default connection timeout** + +https://github.com/apache/hadoop/pull/5881 + + +--- + +* [HADOOP-18759](https://issues.apache.org/jira/browse/HADOOP-18759) | *Major* | **[ABFS][Backoff-Optimization] Have a Static retry policy for connection timeout failures** + +https://github.com/apache/hadoop/pull/5881 + + +--- + +* [HADOOP-19120](https://issues.apache.org/jira/browse/HADOOP-19120) | *Major* | **[ABFS]: ApacheHttpClient adaptation as network library** + +Apache httpclient 4.5.x is the new default implementation of http connections; this supports a large configurable pool of connections along withthe ability to limit their lifespan. + +The networking library can be chosen using the configuration +option fs.azure.networking.library + +The supported values are +- APACHE\_HTTP\_CLIENT : Use Apache HttpClient [Default] +- JDK\_HTTP\_URL\_CONNECTION : Use JDK networking library + +Important: unless the networking library is switched back to +the JDK, the apache httpcore and httpclient must be on the classpath + + + Added: dev/hadoop/hadoop-3.4.1-RC1/RELEASENOTES.md.asc ============================================================================== --- dev/hadoop/hadoop-3.4.1-RC1/RELEASENOTES.md.asc (added) +++ dev/hadoop/hadoop-3.4.1-RC1/RELEASENOTES.md.asc Tue Aug 6 22:38:00 2024 @@ -0,0 +1,16 @@ +-----BEGIN PGP SIGNATURE----- + +iQIzBAABCgAdFiEEU5MdqnCCkUCZWL1HTSK7fTKIIgEFAmatOfoACgkQTSK7fTKI +IgHM4hAAn7zTYGx5I96k4kxbkLubheRtZkAeGNQnpDET31hjPQ6BpjFAVZfk0cpj +ckD7ybfV1uD9oy0fZj053tM5qA8r2dda/7Bh70R8egOn7SGJujsO7IpC7gIUvT5Z +6bUwq1lRLgOWeEkPZF3CjZkLjXINqonv5FYiF46Un8rN8K7vCy++njmuZyqPYpaX +cp8uQG/X5Cw9WblMqJd2RRKui2YnXk9cKpPocrDUbJ90VO11ptBPCsBfhUgKbeZv +W6L7hwFXBTG3Pg/l2DfeCTY0hE1OkwO8G1GapsBe6Ewx7uI7nEiAZX/Y5/j6v77s +VR0bycKO0vuNfXCwv4oX1c0v1dvQpXwItUOn1hmc8vD0d1iM5u4psoFClsxIFizx +lYvvVMb3oAiDDF3pcmVA+DUow56urb8k38C7/QuimPZNWQ7+5JXaZEulXj8EYIkH +9wsNiGsZsp2w1bebmfPZY0to6UoOp/Rgepb06uoF2/Nj/e51WviesavCwgA93skp +TRrAQ48nMwpQnCCEwgnIUCd963zfrpi2n2vBKASBGwXQTbozosl046VwLqFLiXGg +IvDPewUxAV6RdmNM16k1GxP7mdLp9Qdjo989eDeK+WYEps8gRPLSkSz48nCj2TyT +t+zm5REYwCAwUu9gOvGdMq5DY1GemU4ZGtydYiNi9Nm6BfbtjO0= +=uv9V +-----END PGP SIGNATURE----- Added: dev/hadoop/hadoop-3.4.1-RC1/RELEASENOTES.md.sha512 ============================================================================== --- dev/hadoop/hadoop-3.4.1-RC1/RELEASENOTES.md.sha512 (added) +++ dev/hadoop/hadoop-3.4.1-RC1/RELEASENOTES.md.sha512 Tue Aug 6 22:38:00 2024 @@ -0,0 +1 @@ +SHA512 (RELEASENOTES.md) = d797b69f42dd24ace57d2e2456478fac498788036975a13e87d988f6624a705ee435668d273e86735ba55251873243a9669041fbee7b7c3b6d09eaa5f75523e2 --------------------------------------------------------------------- To unsubscribe, e-mail: common-commits-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-commits-h...@hadoop.apache.org