Hi, I am maintaining several dataflows and I am facing this issue in practice: Lets say, I have several points of possible failure within the dataflow (nearly every processor have failure output), I route all of these into my general failure handler subgroup, which basically does some filtering and formatting before issuing a notification by email.
>From my email notifications, I get the FlowFile UUID and in case i am curious on what happened, I go into NiFi and search provenance events for this particular FlowFile. And here comes the point: Sometimes I find hard to find, which processor was the first one which sent the file into the 'Failure path'. Shouldn't processor which does the 'failure' routing send a ProvenanceEvent with type ProvenanceEventType.Route to the flowfile history for Dataflow manager to know when this unfortunate event happened? Is this the guideline which Processors do not obey? Or maybe, I do something wrong when search for events/history of the FlowFile. To get into the concrete example, let me point out that PostHTTP processor never issues any provenance event regarding the failure (nor it fills any execution details into attributes, like does the ExecuteStreamCommand do, for example, there you have execution.error which contains the stderr). So locating the error to be in PostHTTP is just heuristic from my side and I cannot find any HTTP -verbose output (like in curl -v for example), with headers, response from server or at least 'connection timeout' if that is the case... Thanks for suggestions and opinions. Michal Klempa
