[
https://issues.apache.org/jira/browse/IMPALA-11514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joe McDonnell updated IMPALA-11514:
-----------------------------------
Labels: broken-build (was: )
> Workaround s3 timeout waiting for connection from pool (HADOOP-18410)
> ---------------------------------------------------------------------
>
> Key: IMPALA-11514
> URL: https://issues.apache.org/jira/browse/IMPALA-11514
> Project: IMPALA
> Issue Type: Bug
> Components: Infrastructure
> Affects Versions: Impala 4.2.0
> Reporter: Joe McDonnell
> Assignee: Joe McDonnell
> Priority: Blocker
> Labels: broken-build
>
> When testing on s3, we see dataload fail when trying to load testcases:
> {noformat}
> 12:00:17 Creating tpcds testcase data (logging to
> /data/jenkins/workspace/impala-asf-master-core-s3/repos/Impala/logs/data_loading/create-tpcds-testcase-data.log)...
>
> 12:00:17 FAILED (Took: 0 min 13 sec)
> 12:00:30
> '/data/jenkins/workspace/impala-asf-master-core-s3/repos/Impala/testdata/bin/create-tpcds-testcase-files.sh'
> failed. Tail of log:
> 12:00:30 order by t_s_secyear.customer_id
> 12:00:30 ,t_s_secyear.customer_first_name
> 12:00:30 ,t_s_secyear.customer_last_name
> 12:00:30 ,t_s_secyear.customer_email_address
> 12:00:30 limit 100
> 12:00:30 Query submitted at: 2022-08-18 12:00:25 (Coordinator:
> http://hostname:25000)
> 12:00:30 ERROR: AnalysisException: getFileStatus on
> s3a://bucketname/test-warehouse/tpcds-testcase-data:
> com.amazonaws.SdkClientException: Unable to execute HTTP request: Timeout
> waiting for connection from pool
> 12:00:30 CAUSED BY: InterruptedIOException: getFileStatus on
> s3a://bucketname/test-warehouse/tpcds-testcase-data:
> com.amazonaws.SdkClientException: Unable to execute HTTP request: Timeout
> waiting for connection from pool
> 12:00:30 CAUSED BY: SdkClientException: Unable to execute HTTP request:
> Timeout waiting for connection from pool
> 12:00:30 CAUSED BY: ConnectionPoolTimeoutException: Timeout waiting for
> connection from pool{noformat}
> This has been tracked down to
> https://issues.apache.org/jira/browse/HADOOP-18410
> A temporary workaround is to specify fs.s3a.input.async.drain.threshold=512G
> in core-site.xml.
> We should work around this issue until the fix arrives.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
