steveloughran commented on code in PR #5948:
URL: https://github.com/apache/hadoop/pull/5948#discussion_r1298437465
##########
hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/Constants.java:
##########
@@ -156,10 +156,16 @@ private Constants() {
public static final String MAXIMUM_CONNECTIONS = "fs.s3a.connection.maximum";
public static final int DEFAULT_MAXIMUM_CONNECTIONS = 96;
- // Expiration time of s3 http connection from the connection pool.
- // See {@code com.amazonaws.ClientConfiguration#setConnectionTTL}
+ /**
+ * Configuration option to configure expiration time of
+ * s3 http connection from the connection pool in milliseconds: {@value}.
+ */
public static final String CONNECTION_TTL = "fs.s3a.connection.ttl";
- public static final long DEFAULT_CONNECTION_TTL =
ClientConfiguration.DEFAULT_CONNECTION_TTL;
+
+ /**
+ * Default value for {@value CONNECTION_TTL}: {@value}.
Review Comment:
should be @link or @code, surely
##########
hadoop-tools/hadoop-aws/src/site/markdown/tools/hadoop-aws/troubleshooting_s3a.md:
##########
@@ -1782,20 +1782,23 @@ will attempt to retry the operation; it may just be a
transient event. If there
are many such exceptions in logs, it may be a symptom of connectivity or
network
problems.
-Above error could be because of a stale http connections. By default,
connections
-in the http connection pool are reused indefinitely. To discard connections
after
-a specific period of time please configure fs.s3a.connection.ttl.
+The above error could be because of a stale http connections.Default value in
AWS
+SDK is set to -1 (infinite) which means connection will be reused indefinitely.
+We have introduced a new config fs.s3a.connection.ttl to configure this.
Review Comment:
surround the config option with backticks
##########
hadoop-tools/hadoop-aws/src/site/markdown/tools/hadoop-aws/troubleshooting_s3a.md:
##########
@@ -1782,20 +1782,23 @@ will attempt to retry the operation; it may just be a
transient event. If there
are many such exceptions in logs, it may be a symptom of connectivity or
network
problems.
-Above error could be because of a stale http connections. By default,
connections
-in the http connection pool are reused indefinitely. To discard connections
after
-a specific period of time please configure fs.s3a.connection.ttl.
+The above error could be because of a stale http connections.Default value in
AWS
Review Comment:
add space after .; change "Default" to "The default"
##########
hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/Constants.java:
##########
@@ -156,10 +156,16 @@ private Constants() {
public static final String MAXIMUM_CONNECTIONS = "fs.s3a.connection.maximum";
public static final int DEFAULT_MAXIMUM_CONNECTIONS = 96;
- // Expiration time of s3 http connection from the connection pool.
- // See {@code com.amazonaws.ClientConfiguration#setConnectionTTL}
+ /**
+ * Configuration option to configure expiration time of
+ * s3 http connection from the connection pool in milliseconds: {@value}.
+ */
public static final String CONNECTION_TTL = "fs.s3a.connection.ttl";
- public static final long DEFAULT_CONNECTION_TTL =
ClientConfiguration.DEFAULT_CONNECTION_TTL;
+
+ /**
+ * Default value for {@value CONNECTION_TTL}: {@value}.
+ */
+ public static final long DEFAULT_CONNECTION_TTL = 5 * 60 * 1000;
Review Comment:
use 60_000 for the minute value
##########
hadoop-tools/hadoop-aws/src/site/markdown/tools/hadoop-aws/troubleshooting_s3a.md:
##########
@@ -1782,20 +1782,23 @@ will attempt to retry the operation; it may just be a
transient event. If there
are many such exceptions in logs, it may be a symptom of connectivity or
network
problems.
-Above error could be because of a stale http connections. By default,
connections
-in the http connection pool are reused indefinitely. To discard connections
after
-a specific period of time please configure fs.s3a.connection.ttl.
+The above error could be because of a stale http connections.Default value in
AWS
+SDK is set to -1 (infinite) which means connection will be reused indefinitely.
Review Comment:
"the connection"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
