[
https://issues.apache.org/jira/browse/HADOOP-19451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17924903#comment-17924903
]
Steve Loughran commented on HADOOP-19451:
-----------------------------------------
...although we have stream leak detection, do you mean extra stuff for the
actual factory?
the key leak problems we have hit so far are
* libraries/apps forgetting to close() streams.
* code which builds a large list of streams and only calls close() on them at
the end of a larger piece of work. (iceberg did this at some point).
the stream factory has its lifecycle tied to that of the store, so provided the
fs closes that, it will release all its resources. and as the s3 clients are
released at the same time, their pools close
> S3A: AAL - Add support for stream leak detection
> ------------------------------------------------
>
> Key: HADOOP-19451
> URL: https://issues.apache.org/jira/browse/HADOOP-19451
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Reporter: Ahmar Suhail
> Assignee: Steve Loughran
> Priority: Major
> Fix For: 3.5.0
>
>
> AAL currently does not support leak detection.
> It may not require this, as individual streams do not hold only to any
> connections/resources, the factory does. We should verify if it's required,
> and if yes, add support.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
