You can add an appender in the conf/logback.xml file which handles 
“org.apache.nifi” and anything of ERROR level and writes to a separate log 
file, which you can then monitor/parse/send wherever you like. This will filter 
out the normal operations logging from the failures you are interested in. 
Unfortunately, by default the full stacktraces and some other error information 
will be included here as well.


Andy LoPresto
alopre...@apache.org
alopresto.apa...@gmail.com
PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69

> On Aug 8, 2017, at 1:09 PM, James Farrington <nifini...@gmail.com> wrote:
> 
> Hi Andy,
> 
> I am trying to catch any error that happens from any processor in my flow.
> So adding a PutFile for each and every processor would not be ideal. And I
> don't need the data to be transformed into a usable type. I am passing this
> data to Logstash as a csv file (so raw data is just fine). Any thoughts?
> 
> Thank you,
> James
> 
> On Mon, Aug 7, 2017 at 11:35 PM, Andy LoPresto <alopre...@apache.org 
> <mailto:alopre...@apache.org>> wrote:
> 
>> Hi James,
>> 
>> The app log will definitely contain a lot of relevant information about
>> flowfile failure, but you can also make this easier for yourself by routing
>> the failure connection of the relevant processor to a PutFile/PutEmail
>> processor which outputs the flowfile UUID and content claim size to a
>> special error destination. This means you won’t have to parse the app log
>> and filter the non-error information. You may also want to look at the
>> SiteToSiteBulletinReportingTask and SiteToSiteProvenanceReportingTask to
>> transform the metadata (errors = bulletins, provenance for failure events)
>> into raw data that you can then operate on directly.
>> 
>> Obviously, you can also have that failure connection go to other
>> processors, back to the original to retry, etc.
>> 
>> Andy LoPresto
>> alopre...@apache.org
>> *alopresto.apa...@gmail.com <mailto:alopresto.apa...@gmail.com> 
>> <alopresto.apa...@gmail.com <mailto:alopresto.apa...@gmail.com>>*
>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>> 
>> On Aug 7, 2017, at 12:09 PM, James Farrington <nifini...@gmail.com> wrote:
>> 
>> Hi everyone,
>> 
>> Whenever there is an error in a flow and some flow file is not processed
>> properly, I need to retrieve the Flow File UUID and the size of that file.
>> I checked the nifi-app.log file and it seemed that most (if not all) of the
>> time this information was being sent there on a processor error. Does
>> anyone know if this will always be the case? Or if there is a better way to
>> capture all flow files that failed to process?
>> 
>> Any help would be great!
>> 
>> Thank you.

Attachment: signature.asc
Description: Message signed with OpenPGP using GPGMail

Reply via email to