Joe McDonnell created IMPALA-11514:
--------------------------------------
Summary: Workaround s3 functionality issue in HADOOP-18410
Key: IMPALA-11514
URL: https://issues.apache.org/jira/browse/IMPALA-11514
Project: IMPALA
Issue Type: Bug
Components: Infrastructure
Affects Versions: Impala 4.2.0
Reporter: Joe McDonnell
Assignee: Joe McDonnell
When testing on s3, we see dataload fail when trying to load testcases:
{noformat}
12:00:17 Creating tpcds testcase data (logging to
/data/jenkins/workspace/impala-asf-master-core-s3/repos/Impala/logs/data_loading/create-tpcds-testcase-data.log)...
12:00:17 FAILED (Took: 0 min 13 sec)
12:00:30
'/data/jenkins/workspace/impala-asf-master-core-s3/repos/Impala/testdata/bin/create-tpcds-testcase-files.sh'
failed. Tail of log:
12:00:30 order by t_s_secyear.customer_id
12:00:30 ,t_s_secyear.customer_first_name
12:00:30 ,t_s_secyear.customer_last_name
12:00:30 ,t_s_secyear.customer_email_address
12:00:30 limit 100
12:00:30 Query submitted at: 2022-08-18 12:00:25 (Coordinator:
http://hostname:25000)
12:00:30 ERROR: AnalysisException: getFileStatus on
s3a://bucketname/test-warehouse/tpcds-testcase-data:
com.amazonaws.SdkClientException: Unable to execute HTTP request: Timeout
waiting for connection from pool
12:00:30 CAUSED BY: InterruptedIOException: getFileStatus on
s3a://bucketname/test-warehouse/tpcds-testcase-data:
com.amazonaws.SdkClientException: Unable to execute HTTP request: Timeout
waiting for connection from pool
12:00:30 CAUSED BY: SdkClientException: Unable to execute HTTP request: Timeout
waiting for connection from pool
12:00:30 CAUSED BY: ConnectionPoolTimeoutException: Timeout waiting for
connection from pool{noformat}
This has been tracked down to https://issues.apache.org/jira/browse/HADOOP-18410
A temporary workaround is to specify fs.s3a.input.async.drain.threshold=512G in
core-site.xml.
We should work around this issue until the fix arrives.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
