[
https://issues.apache.org/jira/browse/NIFI-1035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14953900#comment-14953900
]
Alan Jackoway commented on NIFI-1035:
-------------------------------------
Pull request at https://github.com/apache/nifi/pull/101
I think that would work for the resources example, but we load our schemas from
HDFS. The result is that lsof has thousands of lines that look like this:
{noformat}
java 743 cops *419u IPv4 360974466 0t0 TCP
snow-004.edh.cloudera.com:59765->hodor-024.edh.cloudera.com:1004 (CLOSE_WAIT)
java 743 cops *420u IPv4 360974467 0t0 TCP
snow-004.edh.cloudera.com:57716->hodor-027.edh.cloudera.com:1004 (CLOSE_WAIT)
java 743 cops *421u IPv4 360974468 0t0 TCP
snow-004.edh.cloudera.com:56605->hodor-014.edh.cloudera.com:1004 (CLOSE_WAIT)
{noformat}
I'm pretty sure they're coming from the read schema, but since it's basically
just a connection to a Datanode, it isn't exactly a guarantee.
> AbstractKiteProcessor.getSchema leaks file handles
> --------------------------------------------------
>
> Key: NIFI-1035
> URL: https://issues.apache.org/jira/browse/NIFI-1035
> Project: Apache NiFi
> Issue Type: Bug
> Affects Versions: 0.3.0
> Reporter: Alan Jackoway
>
> We had NiFi nodes crash today with the dreaded too many open files error
> (similar to what I saw in NIFI-888).
> This time, the file handles were all connections to Datanodes in the
> CLOSE_WAIT status.
> The most recent thing I added to our pipeline were ConvertAvroSchema and
> ConvertJSONToAvro processors. I started looking there for a leak and I
> believe I found it in AbstractKiteProcessor.getSchema.
> Pull Request coming shortly.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
