[
https://issues.apache.org/jira/browse/HUDI-1497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vinoth Chandar resolved HUDI-1497.
----------------------------------
Fix Version/s: 0.7.0
Resolution: Fixed
This was due to the leaking reader/writers. Fixed in 0.7.0
> Timeout Exception during getFileStatus()
> -----------------------------------------
>
> Key: HUDI-1497
> URL: https://issues.apache.org/jira/browse/HUDI-1497
> Project: Apache Hudi
> Issue Type: Sub-task
> Components: Writer Core
> Reporter: Balaji Varadarajan
> Assignee: Balaji Varadarajan
> Priority: Major
> Fix For: 0.7.0
>
>
> Seeing this happening when running RFC-15 branch in long running mode. There
> could be a resource leak as I am seeing this consistently after every 1 or 2
> hour period runs. The below log shows it is during accessing bootstrap index
> but I am seeing it in getFileStatus() for other files too.
>
>
> Caused by: java.io.InterruptedIOException: getFileStatus on
> s3://robinhood-encrypted-hudi-data-cove/dummy/balaji/sickle/public/client_ledger_clientledgerbalance/test_v4/.hoodie/.aux/.bootstrap/.partitions/00000000-0000-0000-0000-000000000000-0_1-0-1_00000000000001.hfile:
> com.amazonaws.SdkClientException: Unable to execute HTTP request: Timeout
> waiting for connection from poolCaused by: java.io.InterruptedIOException:
> getFileStatus on
> s3://robinhood-encrypted-hudi-data-cove/dummy/balaji/sickle/public/client_ledger_clientledgerbalance/test_v4/.hoodie/.aux/.bootstrap/.partitions/00000000-0000-0000-0000-000000000000-0_1-0-1_00000000000001.hfile:
> com.amazonaws.SdkClientException: Unable to execute HTTP request: Timeout
> waiting for connection from pool at
> org.apache.hadoop.fs.s3a.S3AUtils.translateException(S3AUtils.java:141) at
> org.apache.hadoop.fs.s3a.S3AUtils.translateException(S3AUtils.java:117) at
> org.apache.hadoop.fs.s3a.S3AFileSystem.s3GetFileStatus(S3AFileSystem.java:1859)
> at
> org.apache.hadoop.fs.s3a.S3AFileSystem.innerGetFileStatus(S3AFileSystem.java:1823)
> at
> org.apache.hadoop.fs.s3a.S3AFileSystem.getFileStatus(S3AFileSystem.java:1763)
> at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1627) at
> org.apache.hadoop.fs.s3a.S3AFileSystem.exists(S3AFileSystem.java:2500) at
> org.apache.hudi.common.fs.HoodieWrapperFileSystem.exists(HoodieWrapperFileSystem.java:549)
> at
> org.apache.hudi.common.bootstrap.index.HFileBootstrapIndex.<init>(HFileBootstrapIndex.java:102)
> ... 33 moreCaused by: com.amazonaws.SdkClientException: Unable to execute
> HTTP request: Timeout waiting for connection from pool at
> com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleRetryableException(AmazonHttpClient.java:1113)
> at
> com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1063)
> at
> com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:743)
> at
> com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:717)
> at
> com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:699)
> at
> com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:667)
> at
> com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:649)
> at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:513) at
> com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4229) at
> com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4176) at
> com.amazonaws.services.s3.AmazonS3Client.getObjectMetadata(AmazonS3Client.java:1253)
> at
> org.apache.hadoop.fs.s3a.S3AFileSystem.getObjectMetadata(S3AFileSystem.java:1053)
> at
> org.apache.hadoop.fs.s3a.S3AFileSystem.s3GetFileStatus(S3AFileSystem.java:1841)
> ... 39 more
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
