[
https://issues.apache.org/jira/browse/HADOOP-16728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16983256#comment-16983256
]
Benoit Perroud commented on HADOOP-16728:
-----------------------------------------
While digging into this issue, I found 2 more problems:
* Closed connections are added back into the pool. They should be filtered out.
* The SFTPInputStream is closing the stream instead of returning it into the
pool.
I'll provide an update of the PR to fix these two issues, too.
> SFTPFileSystem connection pooling is not handling the pool properly
> -------------------------------------------------------------------
>
> Key: HADOOP-16728
> URL: https://issues.apache.org/jira/browse/HADOOP-16728
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
> Affects Versions: 3.2.1
> Reporter: Benoit Perroud
> Priority: Major
>
> This issue is trying to address few issues in the SFTP filesystem, and
> especially in SFTPConnectionPool:
> * all channels for a given connectionInfo are removed from the pool instead
> of only the selected channel.
> * Closed connections are added back into the pool. They should be filtered
> out.
> * The SFTPInputStream is closing the stream instead of returning it into the
> pool.
> * Some connection leak could happen in some cases
> As a result, if you have multiple threads sharing the pool, the SFTP
> filesystem is underperforming by untimely re-establishing SSH connections.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
