MergeContent exists in the flow in several places. I haven't noticed any UnknownHostExceptions, though my grep of the logs is still running ...
-- Mike On Fri, Mar 4, 2016 at 5:13 PM, Joe Gresock <[email protected]> wrote: > Are there any UnknownHostExceptions in the logs prior to entering this > state? We had a similar problem where NiFi kept opening sockets to connect > to an RPG and eventually ran into "too many open files" after several > days. We had to add the fqdn of the RPG host to our /etc/hosts file before > NiFi would resolve it, and the problem went away. > > On Fri, Mar 4, 2016 at 5:04 PM, Joe Witt <[email protected]> wrote: > > > Mike, > > > > Does this flow have MergeContent processor on it? > > > > Thanks > > Joe > > > > On Fri, Mar 4, 2016 at 4:59 PM, Michael Moser <[email protected]> > wrote: > > > Thanks for the reply, Mark. > > > > > > NIFI-1577 isn't the cause because I don't think we were using any > > processor > > > that does ProcessSession.append(). > > > NIFI-1527 mentions a problem that occurs when NiFi starts, and our NiFi > > had > > > been running for several days. > > > > > > Setting aside the "Too many open files" cause for the moment. Here's > > what > > > we saw when the NiFi JVM encountered Too many open files: > > > > > > ERROR [Site-to-Site Worker Thread] > > o.a.nifi.remote.SocketRemoteSiteListener > > > Unable to communicate with remote instance due to > > > o.a.nifi.processor.exception.ProcessException: > > > o.a.nifi.processor.exception.FlowFileAccessException: Failed to import > > data > > > from org.apache.nifi.stream.io.MinimumLengthInputStream@1234 for > > > StandardFlowFileRecord[uuid=foo,claim=,offset=0,name=filename,size=0] > due > > > to org.apache.nifi.processor.exception.FlowFileAccessException: Unable > to > > > create ContentClaim due to java.io.FileNotFoundException: > > > content_repository/1/1-1 (Too many open files); closing connection > > > > > > This NiFi instance was using a remote process group Input Port to > accept > > > new files. It appears after the exception that a flowfile exists in > the > > > flowfile_repository but the ContentClaim doesn't get a chance to exist > in > > > the content_repository. > > > > > > -- Mike > > > > > > > > > > > > On Fri, Mar 4, 2016 at 3:03 PM, Mark Payne <[email protected]> > wrote: > > > > > >> Tony, > > >> > > >> The two tickets that come to mind are: > > >> https://issues.apache.org/jira/browse/NIFI-1577 < > > >> https://issues.apache.org/jira/browse/NIFI-1577> (Too many open > files) > > >> https://issues.apache.org/jira/browse/NIFI-1527 < > > >> https://issues.apache.org/jira/browse/NIFI-1527> (ContentNotFound) > > >> > > >> Do these sound like they may be what is causing your issues? > > >> > > >> Thanks > > >> -Mark > > >> > > >> > > >> > On Mar 4, 2016, at 2:57 PM, Tony Kurc <[email protected]> wrote: > > >> > > > >> > All, > > >> > I wanted to describe an issue on a nifi instance we've been using > > 0.4.1 > > >> on, > > >> > and why diagnosing it and reproducing it may be difficult. This is > on > > a > > >> > linux server, where we have a reasonably high load, and the error > > happens > > >> > infrequently, but when it does, it really gums up operations. > > >> > > > >> > At some point we get an IOException for too many open files. (with > an > > >> > awfully high limit of open files in ulimit, so not sure why that is > > >> > happening). > > >> > > > >> > Some time later, when trying to read a flowfile in a processor, we > > get a > > >> > ContentNotFoundException because presumably a flowfile is pointing > to > > >> > content that was never written. When this happens, we basically have > > to > > >> > remove the flowfile manually (and if no one is watching at the > moment > > or > > >> > the processor that reads isn't configured to handle this, or if > you're > > >> not > > >> > using 0.5.x where you can selectively remove flowfiles from a queue > > this > > >> > can cause operational challenges). > > >> > > > >> > Because this happens so infrequently, I'm not sure if others have > seen > > >> > this. I'm not sure if something in the framework may need to > > adjustment > > >> if > > >> > a content claim goes wrong, but I really didn't expect that a > flowfile > > >> with > > >> > no actual content should be able to be created, which seems to be > what > > >> > happened (rather than the content being deleted or corrupted). > > >> > > > >> > Anyone else experience this, or know maybe if something in 0.5.X may > > have > > >> > addressed this (looking through the release notes, nothing jumped > > out). > > >> > > > >> > Tony > > >> > > >> > > > > > > -- > I know what it is to be in need, and I know what it is to have plenty. I > have learned the secret of being content in any and every situation, > whether well fed or hungry, whether living in plenty or in want. I can do > all this through him who gives me strength. *-Philippians 4:12-13* > > This email has been sent from a virus-free computer protected by Avast. > www.avast.com > < > https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail > > > <#DDB4FAA8-2DD7-40BB-A1B8-4E2AA1F9FDF2> >
