[
https://issues.apache.org/jira/browse/NIFI-1754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15285269#comment-15285269
]
Joseph Witt commented on NIFI-1754:
-----------------------------------
{quote}
The original requirement was on an operations need, without being able to
identify flow files causing the Administrative Yield they couldn't track
whether the root causes were system issues, framework issues, processor issues,
or flow file problems.
{quote}
They are only issues of 'what happens inside processor executed code' and those
issues could be as you list (bad system state, bad framework thing, bad
processor code). But we cannot know the difference. The question is 'what
information can and should we make available through the REST API and UI when
this occurs'. I don't believe this has been fully thought through yet. The
relationship between a flowfile and a session that happened to get rolled back
is not clear. We can only really say "hey this flowfile was part of a session
recently that had to get rolled back and here the stacktrace that prompted the
rollback". I agree that information is useful to a developer and that is why I
am in favor of making that information available through the logs. I think we
can do more than current API/UI enabled information of "hey a rollback
happened" but I don't think that is changing attributes on a flow file. More
thought and user experience discussion needs to occur there.
{quote}
1. Move the rollback count value from the FF attribute to a FlowFileRecord
property.
{quote}
In my opinion this should be avoided at this point. We do not have a fully
thought through user experience for this scenario. I would like to see much
more discussion around that user experience and how that might work. For
example we should track process session statistics tied to a given
component/processor and in that we could include information about recent
rollback which could correlate to flowfiles. Just ideas...but point is we need
to think about the user experience and then think about code/api implications.
{quote}
2. Move the unacknowledged FF logging from AbstractProcessor into the
ProcessSession where the counting already is being done.
{quote}
Yes I agree that is a good next step. I'd keep all of the session logging
within the session class itself and not alter the existing API at all. This
allows us to make available more helpful debug data while we evaluate other
options for offering a better 'developer' centric user experience to expose
these things if that makes sense. We need to think about things like
authorization as well because this could include some rather sensitive
information.
{quote}
3. Eliminate the configuration entries by moving the logging to DEBUG level to
allow it to be controlled by logging configuration if necessary and always
counting rollbacks on the FlowFileRecord.
{quote}
I believe the item in #2 will require no changes to properties so yes I'd say
remove those. I agree just logback.xml changes would be needed for cases where
people want those details in the logs. For counting rollbacks my response from
#1 applies.
{quote}
4. Rename the FlowDebugger processor to DebugFlow and incorporate as a separate
ticket and PR.
{quote}
Yes please. I did not honestly dig into the details of DebugFlow yet but very
much think it is a cool concept and something I had been wanting to do/have for
a long time. Happy to review that one as well.
Thanks for the discussion. I absolutely want to respect what has clearly been
a lot of thought and effort.
> Rollback log messages should include the flowfile filename and UUID to assist
> in flow management.
> -------------------------------------------------------------------------------------------------
>
> Key: NIFI-1754
> URL: https://issues.apache.org/jira/browse/NIFI-1754
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Core Framework
> Affects Versions: 1.0.0, 0.7.0
> Reporter: Joe Skora
> Assignee: Joe Skora
>
> When a processor calls rollback on a flowfile, the filename and UUID of the
> flowfile should be included in log entry to make it easier for the offending
> files to be found and fixed or removed from the flow by the dataflow managers.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
