[
https://issues.apache.org/jira/browse/AVRO-1069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thomas Andrews updated AVRO-1069:
---------------------------------
Description:
I'm not sure if this is the cause of my underlying problem, but the class
org.apache.avro.ipc.HttpTransceiver opens an OutputStream and never explicitly
closes it. That seems like very bad behavior, especially depending on whether
the HttpURLConnection class holds onto the OutputStream or not - if it does,
then potentially the Transceiver will never close the OutputStream.
I think you should also be closing the InputStream, as well.
In particular, Java has built-in the ability to keep HttpURLConnections open,
and re-use them. You might think that not closing these streams would help
Java in this effort, but actually, the streams are not the raw connections, but
wrappers. The javadoc says: "Calling the close() methods on the InputStream or
OutputStream of an HttpURLConnection after a request may free network resources
associated with this instance but has no effect on any shared persistent
connection."
In other words, when you fail to close these streams, Java doesn't know you are
done with the request, so it cannot re-use the connection. You only end up
able to re-use the connection when the HttpURLConnection gets garbage-collected.
was:
I'm not sure if this is the cause of my underlying problem, but the class
org.apache.avro.ipc.HttpTransceiver opens an OutputStream and never explicitly
closes it. That seems like very bad behavior, especially depending on whether
the HttpURLConnection class holds onto the OutputStream or not - if it does,
then potentially the Transceiver will never close the OutputStream.
The specific oddity I am seeing is related to Flume. I have a Flume agent
listening on port 60000 and a client using a FlumeEventAvroServer with the Avro
HttpTransceiver class.
Occasionally, when the Avro agent gets refreshed or otherwise reset, it fails
with a BindException error.
Even rarer, but still often enough to cause concern, the client software holds
onto something which blocks the Flume agent from rebinding to the port even
when the Flume agent is restarted. Essentially, something is now "occupying"
the port. It is my guess that it is the transceiver which has failed to close
the OutputStream, but I'm not 100% sure.
I think you should also be closing the InputStream, as well.
> HttpTransceiver never closes its OutputStream
> ---------------------------------------------
>
> Key: AVRO-1069
> URL: https://issues.apache.org/jira/browse/AVRO-1069
> Project: Avro
> Issue Type: Bug
> Components: java
> Reporter: Thomas Andrews
>
> I'm not sure if this is the cause of my underlying problem, but the class
> org.apache.avro.ipc.HttpTransceiver opens an OutputStream and never
> explicitly closes it. That seems like very bad behavior, especially
> depending on whether the HttpURLConnection class holds onto the OutputStream
> or not - if it does, then potentially the Transceiver will never close the
> OutputStream.
> I think you should also be closing the InputStream, as well.
> In particular, Java has built-in the ability to keep HttpURLConnections open,
> and re-use them. You might think that not closing these streams would help
> Java in this effort, but actually, the streams are not the raw connections,
> but wrappers. The javadoc says: "Calling the close() methods on the
> InputStream or OutputStream of an HttpURLConnection after a request may free
> network resources associated with this instance but has no effect on any
> shared persistent connection."
> In other words, when you fail to close these streams, Java doesn't know you
> are done with the request, so it cannot re-use the connection. You only end
> up able to re-use the connection when the HttpURLConnection gets
> garbage-collected.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
