[
https://issues.apache.org/jira/browse/AVRO-1069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13263787#comment-13263787
]
Thomas Andrews commented on AVRO-1069:
--------------------------------------
When I rolled my own transceiver which was a copy of HttpTransceiver with
close() calls to the InputStream and OutputStream added, I got much better
connection re-use in Java. Specifically, in load test with four streams sending
requests to the server, using Avro's HttpTransceiver, I used "netstat" to see
that there were anywhere from 3-4 sockets open to handle the requests, but when
I used my custom transceiver, it used only one socket for all four streams.
Since Java (by default) has a maximum of 5 cached open connections, this means
at slightly higher load, HttpTransceiver is going to be opening lots of
one-shot sockets.
> HttpTransceiver never closes its OutputStream
> ---------------------------------------------
>
> Key: AVRO-1069
> URL: https://issues.apache.org/jira/browse/AVRO-1069
> Project: Avro
> Issue Type: Bug
> Components: java
> Reporter: Thomas Andrews
>
> I'm not sure if this is the cause of my underlying problem, but the class
> org.apache.avro.ipc.HttpTransceiver opens an OutputStream and never
> explicitly closes it. That seems like very bad behavior, especially
> depending on whether the HttpURLConnection class holds onto the OutputStream
> or not - if it does, then potentially the Transceiver will never close the
> OutputStream.
> The specific oddity I am seeing is related to Flume. I have a Flume agent
> listening on port 60000 and a client using a FlumeEventAvroServer with the
> Avro HttpTransceiver class.
> Occasionally, when the Avro agent gets refreshed or otherwise reset, it fails
> with a BindException error.
> Even rarer, but still often enough to cause concern, the client software
> holds onto something which blocks the Flume agent from rebinding to the port
> even when the Flume agent is restarted. Essentially, something is now
> "occupying" the port. It is my guess that it is the transceiver which has
> failed to close the OutputStream, but I'm not 100% sure.
> I think you should also be closing the InputStream, as well.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
