This is an automated email from the ASF dual-hosted git repository.
mthakur pushed a commit to branch branch-3.4.1
in repository https://gitbox.apache.org/repos/asf/hadoop.git
The following commit(s) were added to refs/heads/branch-3.4.1 by this push:
new c1b57df87ed HADOOP-19279. ABFS: Disabling Apache Http Client as
Default Http Client for ABFS Driver(#7055)
c1b57df87ed is described below
commit c1b57df87ed0d186a95db0923eab8ac73db42e7c
Author: manika137 <80244229+manika...@users.noreply.github.com>
AuthorDate: Sat Sep 21 03:08:56 2024 +0530
HADOOP-19279. ABFS: Disabling Apache Http Client as Default Http Client for
ABFS Driver(#7055)
As part of work done under HADOOP-19120 [ABFS]: ApacheHttpClient adaptation
as network library - ASF JIRA
Apache HTTP Client was introduced as an alternative Network Library that
can be used with ABFS Driver. Earlier JDK Http Client was the only supported
network library.
Apache HTTP Client was found to be more helpful in terms of controls and
knobs it provides to manage the Network aspects of the driver better. Hence,
the default Network Client was made to be used with the ABFS Driver.
Recently while running scale workloads, we observed a regression where some
unexpected wait time was observed while establishing connections. A possible
fix has been identified and we are working on getting it fixed.
There was also a possible NPE scenario was identified on the new network
client code.
Until we are done with the code fixes and revalidated the whole Apache
client flow, we would like to make JDK Client as default client again. The
support will still be there, but it will be disabled behind a config.
Contributed by: manika137
---
.../apache/hadoop/fs/azurebfs/constants/FileSystemConfigurations.java | 2 +-
hadoop-tools/hadoop-azure/src/site/markdown/abfs.md | 4 ++--
2 files changed, 3 insertions(+), 3 deletions(-)
diff --git
a/hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/constants/FileSystemConfigurations.java
b/hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/constants/FileSystemConfigurations.java
index eab43d320ff..cef6994f8d2 100644
---
a/hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/constants/FileSystemConfigurations.java
+++
b/hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/constants/FileSystemConfigurations.java
@@ -166,7 +166,7 @@ public final class FileSystemConfigurations {
public static final long THOUSAND = 1000L;
public static final HttpOperationType DEFAULT_NETWORKING_LIBRARY
- = HttpOperationType.APACHE_HTTP_CLIENT;
+ = HttpOperationType.JDK_HTTP_URL_CONNECTION;
public static final int DEFAULT_APACHE_HTTP_CLIENT_MAX_IO_EXCEPTION_RETRIES
= 3;
diff --git a/hadoop-tools/hadoop-azure/src/site/markdown/abfs.md
b/hadoop-tools/hadoop-azure/src/site/markdown/abfs.md
index 312c2a041bb..36d909623b9 100644
--- a/hadoop-tools/hadoop-azure/src/site/markdown/abfs.md
+++ b/hadoop-tools/hadoop-azure/src/site/markdown/abfs.md
@@ -878,8 +878,8 @@ ABFS Driver can use the following networking libraries:
The networking library can be configured using the configuration
`fs.azure.networking.library`
while initializing the filesystem.
Following are the supported values:
-- `APACHE_HTTP_CLIENT` : Use Apache HttpClient [Default]
-- `JDK_HTTP_URL_CONNECTION` : Use JDK networking library
+- `JDK_HTTP_URL_CONNECTION` : Use JDK networking library [Default]
+- `APACHE_HTTP_CLIENT` : Use Apache HttpClient
#### <a href="ahc_networking_conf"></a>ApacheHttpClient networking layer
configuration Options:
---------------------------------------------------------------------
To unsubscribe, e-mail: common-commits-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-commits-h...@hadoop.apache.org
