John Burwell created CLOUDSTACK-9394:
----------------------------------------
Summary: HttpTemplateDownloader Causes Hanging Connections
Key: CLOUDSTACK-9394
URL: https://issues.apache.org/jira/browse/CLOUDSTACK-9394
Project: CloudStack
Issue Type: Bug
Security Level: Public (Anyone can view this level - this is the default.)
Components: Secondary Storage
Affects Versions: 4.5.2
Reporter: John Burwell
A GetMethod request is created and used to initially connect to the web server
where the template resides. It uses this connection to get the size and
content-type, but as of commit 6de4dcb it doesn't use this connection to
download. It creates a separate connection to download the template via a URL
object, while the initial connection is left hanging with a body to read. Then
at the very end, in a finally clause, it calls request.releaseConnection() for
the original connection. In some of our environments this final close hangs,
and seems to be correlated with the size of the template (how long it took to
download) and the firewalls in between the SSVM and webserver, implying that
the idle connection is timed out at some layer and can't be released. This may
further trickle down into issues with the deprecated Commons HttpClient, but we
believe the core issue is letting that initial connection go stale.
There is no justification in the commit notes for lines 236-238. We have tested
rolling back just those lines (along with adjusting the imports to match), and
it doesn't seem to affect speed or function and fixes the symptoms.
https://github.com/apache/cloudstack/commit/6de4dcb10a076671acc0f7486d493e914b59d1d4
You can verify the double connection via netstat, while a template is
downloading you should see two connections, one with a full receive queue
waiting for something to read the HTTP body, and one with an empty receive
queue (the active one).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
