[
https://issues.apache.org/jira/browse/HADOOP-13973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15818246#comment-15818246
]
Steve Loughran commented on HADOOP-13973:
-----------------------------------------
The problem arises in S3A reopen(), which attempts to open/reopen HTTP
connection. We currently don't do any reties there.
Some hints
*
http://stackoverflow.com/questions/29488106/java-lang-illegalstateexception-connection-pool-shut-down-while-using-spring-re
*
http://stackoverflow.com/questions/25889925/apache-poolinghttpclientconnectionmanager-throwing-illegal-state-exception
the second hints that there's a fix in org.apache.httpcomponents:httpclient
4.4+; though you need to make an extra method call in setup: I don't know if
the AWS SDK does this
> S3 requests failing: java.lang.IllegalStateException: Connection is not open
> ----------------------------------------------------------------------------
>
> Key: HADOOP-13973
> URL: https://issues.apache.org/jira/browse/HADOOP-13973
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Affects Versions: 2.8.0
> Environment: EC2 cluster
> Reporter: Rajesh Balamohan
> Assignee: Steve Loughran
> Fix For: 2.8.0
>
>
> S3 requests failing with an error coming from Http client,
> "java.lang.IllegalStateException: Connection is not open"
> Some online discussion implies that this is related to shared connection pool
> shutdown & fixed in http client 4.4+. Hadoop & AWS SDK use v 4.5.2 so the fix
> is in, we just need to make sure the pool is being set up right.
> There's a problem here of course: it may require moving to a later version of
> the AWS SDK, with the consequences on jackson &c, as seen in HADOOP-13050.
> And that's if there is a patched version out there
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
