[
https://issues.apache.org/jira/browse/HADOOP-18845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17755966#comment-17755966
]
ASF GitHub Bot commented on HADOOP-18845:
-----------------------------------------
steveloughran commented on code in PR #5948:
URL: https://github.com/apache/hadoop/pull/5948#discussion_r1298437465
##########
hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/Constants.java:
##########
@@ -156,10 +156,16 @@ private Constants() {
public static final String MAXIMUM_CONNECTIONS = "fs.s3a.connection.maximum";
public static final int DEFAULT_MAXIMUM_CONNECTIONS = 96;
- // Expiration time of s3 http connection from the connection pool.
- // See {@code com.amazonaws.ClientConfiguration#setConnectionTTL}
+ /**
+ * Configuration option to configure expiration time of
+ * s3 http connection from the connection pool in milliseconds: {@value}.
+ */
public static final String CONNECTION_TTL = "fs.s3a.connection.ttl";
- public static final long DEFAULT_CONNECTION_TTL =
ClientConfiguration.DEFAULT_CONNECTION_TTL;
+
+ /**
+ * Default value for {@value CONNECTION_TTL}: {@value}.
Review Comment:
should be @link or @code, surely
##########
hadoop-tools/hadoop-aws/src/site/markdown/tools/hadoop-aws/troubleshooting_s3a.md:
##########
@@ -1782,20 +1782,23 @@ will attempt to retry the operation; it may just be a
transient event. If there
are many such exceptions in logs, it may be a symptom of connectivity or
network
problems.
-Above error could be because of a stale http connections. By default,
connections
-in the http connection pool are reused indefinitely. To discard connections
after
-a specific period of time please configure fs.s3a.connection.ttl.
+The above error could be because of a stale http connections.Default value in
AWS
+SDK is set to -1 (infinite) which means connection will be reused indefinitely.
+We have introduced a new config fs.s3a.connection.ttl to configure this.
Review Comment:
surround the config option with backticks
##########
hadoop-tools/hadoop-aws/src/site/markdown/tools/hadoop-aws/troubleshooting_s3a.md:
##########
@@ -1782,20 +1782,23 @@ will attempt to retry the operation; it may just be a
transient event. If there
are many such exceptions in logs, it may be a symptom of connectivity or
network
problems.
-Above error could be because of a stale http connections. By default,
connections
-in the http connection pool are reused indefinitely. To discard connections
after
-a specific period of time please configure fs.s3a.connection.ttl.
+The above error could be because of a stale http connections.Default value in
AWS
Review Comment:
add space after .; change "Default" to "The default"
##########
hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/Constants.java:
##########
@@ -156,10 +156,16 @@ private Constants() {
public static final String MAXIMUM_CONNECTIONS = "fs.s3a.connection.maximum";
public static final int DEFAULT_MAXIMUM_CONNECTIONS = 96;
- // Expiration time of s3 http connection from the connection pool.
- // See {@code com.amazonaws.ClientConfiguration#setConnectionTTL}
+ /**
+ * Configuration option to configure expiration time of
+ * s3 http connection from the connection pool in milliseconds: {@value}.
+ */
public static final String CONNECTION_TTL = "fs.s3a.connection.ttl";
- public static final long DEFAULT_CONNECTION_TTL =
ClientConfiguration.DEFAULT_CONNECTION_TTL;
+
+ /**
+ * Default value for {@value CONNECTION_TTL}: {@value}.
+ */
+ public static final long DEFAULT_CONNECTION_TTL = 5 * 60 * 1000;
Review Comment:
use 60_000 for the minute value
##########
hadoop-tools/hadoop-aws/src/site/markdown/tools/hadoop-aws/troubleshooting_s3a.md:
##########
@@ -1782,20 +1782,23 @@ will attempt to retry the operation; it may just be a
transient event. If there
are many such exceptions in logs, it may be a symptom of connectivity or
network
problems.
-Above error could be because of a stale http connections. By default,
connections
-in the http connection pool are reused indefinitely. To discard connections
after
-a specific period of time please configure fs.s3a.connection.ttl.
+The above error could be because of a stale http connections.Default value in
AWS
+SDK is set to -1 (infinite) which means connection will be reused indefinitely.
Review Comment:
"the connection"
> Add ability to configure ConnectionTTL of http connections while creating S3
> Client.
> ------------------------------------------------------------------------------------
>
> Key: HADOOP-18845
> URL: https://issues.apache.org/jira/browse/HADOOP-18845
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Affects Versions: 3.3.6
> Reporter: Mukund Thakur
> Assignee: Mukund Thakur
> Priority: Major
> Labels: pull-request-available
> Fix For: 3.3.9
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
