Balaji Varadarajan created HUDI-1497:
----------------------------------------
Summary: Timeout Exception during getFileStatus()
Key: HUDI-1497
URL: https://issues.apache.org/jira/browse/HUDI-1497
Project: Apache Hudi
Issue Type: Sub-task
Components: Writer Core
Reporter: Balaji Varadarajan
Seeing this happening when running RFC-15 branch in long running mode. There
could be a resource leak as I am seeing this consistently after every 1 or 2
hour period runs. The below log shows it is during accessing bootstrap index
but I am seeing it in getFileStatus() for other files too.
Caused by: java.io.InterruptedIOException: getFileStatus on
s3://robinhood-encrypted-hudi-data-cove/dummy/balaji/sickle/public/client_ledger_clientledgerbalance/test_v4/.hoodie/.aux/.bootstrap/.partitions/00000000-0000-0000-0000-000000000000-0_1-0-1_00000000000001.hfile:
com.amazonaws.SdkClientException: Unable to execute HTTP request: Timeout
waiting for connection from poolCaused by: java.io.InterruptedIOException:
getFileStatus on
s3://robinhood-encrypted-hudi-data-cove/dummy/balaji/sickle/public/client_ledger_clientledgerbalance/test_v4/.hoodie/.aux/.bootstrap/.partitions/00000000-0000-0000-0000-000000000000-0_1-0-1_00000000000001.hfile:
com.amazonaws.SdkClientException: Unable to execute HTTP request: Timeout
waiting for connection from pool at
org.apache.hadoop.fs.s3a.S3AUtils.translateException(S3AUtils.java:141) at
org.apache.hadoop.fs.s3a.S3AUtils.translateException(S3AUtils.java:117) at
org.apache.hadoop.fs.s3a.S3AFileSystem.s3GetFileStatus(S3AFileSystem.java:1859)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.innerGetFileStatus(S3AFileSystem.java:1823)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.getFileStatus(S3AFileSystem.java:1763)
at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1627) at
org.apache.hadoop.fs.s3a.S3AFileSystem.exists(S3AFileSystem.java:2500) at
org.apache.hudi.common.fs.HoodieWrapperFileSystem.exists(HoodieWrapperFileSystem.java:549)
at
org.apache.hudi.common.bootstrap.index.HFileBootstrapIndex.<init>(HFileBootstrapIndex.java:102)
... 33 moreCaused by: com.amazonaws.SdkClientException: Unable to execute HTTP
request: Timeout waiting for connection from pool at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleRetryableException(AmazonHttpClient.java:1113)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1063)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:743)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:717)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:699)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:667)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:649)
at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:513) at
com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4229) at
com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4176) at
com.amazonaws.services.s3.AmazonS3Client.getObjectMetadata(AmazonS3Client.java:1253)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.getObjectMetadata(S3AFileSystem.java:1053)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.s3GetFileStatus(S3AFileSystem.java:1841)
... 39 more
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
