[
https://issues.apache.org/jira/browse/HADOOP-18410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17581997#comment-17581997
]
ASF GitHub Bot commented on HADOOP-18410:
-----------------------------------------
steveloughran commented on PR #4766:
URL: https://github.com/apache/hadoop/pull/4766#issuecomment-1221019682
HADOOP-18410. copy fields to variables before use in a lambda expression
the key cause is that even though the fields passed in to drain() were
converted to references through the methods, in the lambda expression
passed in to submit, they are direct references
```
operation = client.submit(
() -> drain(uri, streamStatistics,
false, reason, remaining,
object, wrappedStream)); /* here */
```
the fields are only read during the async execution, not during the submit
phase.
the next step the code does is reset those fields; the async work is
failing with an NPE which isn't being noted.
Adding the join() after submit works *because this was inserted before
the fields were set to null...so no NPE being silently raised.
> S3AInputStream.unbuffer() async drain not releasing http connections
> --------------------------------------------------------------------
>
> Key: HADOOP-18410
> URL: https://issues.apache.org/jira/browse/HADOOP-18410
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Affects Versions: 3.3.9
> Reporter: Steve Loughran
> Assignee: Steve Loughran
> Priority: Major
> Labels: pull-request-available
>
> Impala tcp-ds setup to s3 is hitting problems with timeout fetching http
> connections from the s3a fs pool. Disabling s3a async drain makes this
> problem *go away*. assumption, either those async ops are blocking, or they
> are not releasing references properly.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
