I am trying to catch any error that happens from any processor in my flow.
So adding a PutFile for each and every processor would not be ideal. And I
don't need the data to be transformed into a usable type. I am passing this
data to Logstash as a csv file (so raw data is just fine). Any thoughts?
On Mon, Aug 7, 2017 at 11:35 PM, Andy LoPresto <alopre...@apache.org> wrote:
> Hi James,
> The app log will definitely contain a lot of relevant information about
> flowfile failure, but you can also make this easier for yourself by routing
> the failure connection of the relevant processor to a PutFile/PutEmail
> processor which outputs the flowfile UUID and content claim size to a
> special error destination. This means you won’t have to parse the app log
> and filter the non-error information. You may also want to look at the
> SiteToSiteBulletinReportingTask and SiteToSiteProvenanceReportingTask to
> transform the metadata (errors = bulletins, provenance for failure events)
> into raw data that you can then operate on directly.
> Obviously, you can also have that failure connection go to other
> processors, back to the original to retry, etc.
> Andy LoPresto
> *alopresto.apa...@gmail.com <alopresto.apa...@gmail.com>*
> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69
> On Aug 7, 2017, at 12:09 PM, James Farrington <nifini...@gmail.com> wrote:
> Hi everyone,
> Whenever there is an error in a flow and some flow file is not processed
> properly, I need to retrieve the Flow File UUID and the size of that file.
> I checked the nifi-app.log file and it seemed that most (if not all) of the
> time this information was being sent there on a processor error. Does
> anyone know if this will always be the case? Or if there is a better way to
> capture all flow files that failed to process?
> Any help would be great!
> Thank you.