[
https://issues.apache.org/jira/browse/HADOOP-18915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17790701#comment-17790701
]
ASF GitHub Bot commented on HADOOP-18915:
-----------------------------------------
steveloughran commented on code in PR #6180:
URL: https://github.com/apache/hadoop/pull/6180#discussion_r1408134803
##########
hadoop-tools/hadoop-aws/src/site/markdown/tools/hadoop-aws/performance.md:
##########
@@ -205,31 +205,86 @@ The default pool sizes are intended to strike a balance
between performance
and memory/thread use.
You can have a larger pool of (reused) HTTP connections and threads
-for parallel IO (especially uploads) by setting the properties
+for parallel IO (especially uploads, prefetching and vector reads) by setting
the appropriate
+properties. Note: S3A Connectors have their own thread pools for job commit,
but
+everything uses the same HTTP connection pool.
+| Property | Default | Meaning
|
+|--------------------------------|---------|------------------------------------------|
+| `fs.s3a.threads.max` | `96` | Threads in the thread pool
|
+| `fs.s3a.threads.keepalivetime` | `60s` | Threads in the thread pool
|
Review Comment:
actually i should cut there
> Extend S3A http client connection timeouts
> ------------------------------------------
>
> Key: HADOOP-18915
> URL: https://issues.apache.org/jira/browse/HADOOP-18915
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Affects Versions: 3.4.0
> Reporter: Ahmar Suhail
> Assignee: Steve Loughran
> Priority: Major
> Labels: pull-request-available
>
> * Add ability to configure *all* timeouts, especially acquisition time
> * recognise ApiCallTimeout and map tp a retryable exception
> * use getDuration so suffixes can be used -so remove all ambiguity about time
> unit
> * use units in core-default.xml so warnings aren't printed
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
