[
https://issues.apache.org/jira/browse/JCLOUDS-628?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14064033#comment-14064033
]
Shri Javadekar commented on JCLOUDS-628:
----------------------------------------
The JavaUrlHttpCommandExecutorService internally uses the
java.net.HTTPUrlConnection and javax.net.ssl.HttpsURLConnection objects.
JavaUrlHttpCommandExecutorService correctly sets the "http.maxConnections"
system variable when it initializes. But looks like this does not get honored?
Unfortunately, there is no way to find out whether a new HTTPUrlConnection
object is internally using the same port as any other HTTPUrlConnection object.
So the only way I could test is to create several connections to some https
endpoint and separately use "netstat -a" to find out the open connections.
I created a simple unit test [1] in
BaseHttpCommandExecutorServiceIntegrationTest that created thousands of such
http connections to a MockWebServer. It did show that the number of opened
connections far exceeds
PROPERTY_MAX_CONNECTIONS_PER_CONTEXT.
However, I still don't know the root cause of this problem.
One problem I found that looked suspicious was that in
BaseHttpCommandExecutoreService.invoke(), the nativeRequest gets set to null in
the try block. As a result when the finally block calls cleanup(),
nativeRequest is null and therefore does not get disconnected. I fixed that and
retried my test. But that does not seem to fix the problem.
Also, when the InputStream from the URLConnection gets closed, IOExceptions if
any are swallowed. I changed that too such that they don't get swallowed.
However, this didn't provide any new information.
I didn't find any documentation about this being a know problem with
javax.net.ssl.HttpsURLConnection.
[1] https://gist.github.com/shrinandj/be84c912eb3a0a6024b1
> PROPERTY_MAX_CONNECTIONS_PER_CONTEXT not honored for https connections
> ----------------------------------------------------------------------
>
> Key: JCLOUDS-628
> URL: https://issues.apache.org/jira/browse/JCLOUDS-628
> Project: jclouds
> Issue Type: Bug
> Components: jclouds-blobstore
> Affects Versions: 1.7.3
> Reporter: Shri Javadekar
> Assignee: Shri Javadekar
> Fix For: 1.8.0, 1.7.4
>
>
> The property PROPERTY_MAX_CONNECTIONS_PER_CONTEXT in jclouds governs how many
> connections are open to an endpoint per blobstore context. This along with
> PROPERTY_MAX_CONNECTION_REUSE makes sure that we don't run out of ports or
> fds on the underlying OS.
> I ran a simple test for verifying the PROPERTY_MAX_CONNECTIONS_PER_CONTEXT
> option. I created a container with ~10K blobs. I then used jclouds-cli to
> clear the container.
> I explicitly set jclouds.max-connections-per-context to 20. When the endpoint
> was http, things worked just fine.
> $ netstat -a | grep blah | grep ESTABL | wc -l
> 20
> $ netstat -a | grep blah | grep ESTABL | wc -l
> 21
> $ netstat -a | grep blah | grep ESTABL | wc -l
> 20
> However, when the endpoint was https, this config option did not seem to be
> honored. The endpoint was https://api.atmosonline.com for the numbers below,
> but I also tried with a local swift endpoint which was https.
> $ netstat -a | grep atmos | grep ESTABL | wc -l
> 516
> $ netstat -a | grep atmos | grep ESTABL | wc -l
> 602
> I've already run into a situation where we exhausted all the available ports
> and operations such as container-clear or container-delete failed.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
