Hi Sourav, Are you able to share any of the code from your onTrigger method so we can see how you are opening/closing the file reader?
-Bryan On Wed, Apr 27, 2016 at 7:43 AM, Sourav Gulati <[email protected]> wrote: > I have introduced yield of 5 secs and after that it started working as > expected. > > Regards, > Sourav Gulati > > -----Original Message----- > From: Sourav Gulati [mailto:[email protected]] > Sent: Wednesday, April 27, 2016 11:34 AM > To: [email protected] > Subject: RE: Reg: Issues in Custom Processor > > Joe, > > Now I have increased number of open files limit. > ulimit -n > 50000 > > How still I am seeing same exception > > Regards, > Sourav Gulati > > -----Original Message----- > From: Sourav Gulati [mailto:[email protected]] > Sent: Tuesday, April 26, 2016 11:48 PM > To: [email protected] > Subject: Re: Reg: Issues in Custom Processor > > Thanks Joe. I will try and update you > > Regards, > Sourav Gulati > Big Data > > > ________________________________________ > From: Joe Witt <[email protected]> > Sent: Tuesday, April 26, 2016 11:26 PM > To: [email protected] > Subject: Re: Reg: Issues in Custom Processor > > Sourav, > > On twitter you asked "why is it opening so many file handles" > > Your processor is not. It may well be just fine. However, at startup nifi > loads many files into the classloader and it is at the point now where it > references more than 1024 objects (files, sockets, etc..) so it is > important to set a higher default. > > I encourage you to go ahead and follow the best practices mentioned in the > documentation and I suspect you'll find your custom processor is doing what > you expected. > > Thanks > Joe > > On Tue, Apr 26, 2016 at 1:47 PM, Joe Witt <[email protected]> wrote: > > Sourav, > > > > Please make sure you've followed these recommendations > > https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html# > > configuration-best-practices > > > > Specifically make sure you update the max open file handles. > > > > Thanks > > Joe > > > > On Tue, Apr 26, 2016 at 9:34 AM, Sourav Gulati > > <[email protected]> wrote: > >> Hi Team, > >> > >> I am using "@TriggerSerially" annotation while creating processor and > inside "On Trigger" method I am creating a file reader and after doing the > processing I am closing the reader connection as well. As per my > assumption, if I am using @TriggerSerially, then "On trigger" method runs > in serial fashion. > >> > >> However, while executing processor I am getting "too many open files" > error . I check with lsof command as well, it has opened lot of reader > connections to file. > >> > >> What should I change ? Need urgent help > >> > >> > >> > >> Regards, > >> Sourav Gulati > >> > >> > >> ________________________________ > >> > >> > >> > >> > >> > >> > >> NOTE: This message may contain information that is confidential, > proprietary, privileged or otherwise protected by law. The message is > intended solely for the named addressee. If received in error, please > destroy and notify the sender. Any use of this email is prohibited when > received in error. Impetus does not represent, warrant and/or guarantee, > that the integrity of this communication has been maintained nor that the > communication is free of errors, virus, interception or interference. > > ________________________________ > > > > > > > NOTE: This message may contain information that is confidential, > proprietary, privileged or otherwise protected by law. The message is > intended solely for the named addressee. If received in error, please > destroy and notify the sender. Any use of this email is prohibited when > received in error. Impetus does not represent, warrant and/or guarantee, > that the integrity of this communication has been maintained nor that the > communication is free of errors, virus, interception or interference. > > ________________________________ > > > > > > > NOTE: This message may contain information that is confidential, > proprietary, privileged or otherwise protected by law. The message is > intended solely for the named addressee. If received in error, please > destroy and notify the sender. Any use of this email is prohibited when > received in error. Impetus does not represent, warrant and/or guarantee, > that the integrity of this communication has been maintained nor that the > communication is free of errors, virus, interception or interference. > > ________________________________ > > > > > > > NOTE: This message may contain information that is confidential, > proprietary, privileged or otherwise protected by law. The message is > intended solely for the named addressee. If received in error, please > destroy and notify the sender. Any use of this email is prohibited when > received in error. Impetus does not represent, warrant and/or guarantee, > that the integrity of this communication has been maintained nor that the > communication is free of errors, virus, interception or interference. >
