[
https://issues.apache.org/jira/browse/NIFI-1447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15121824#comment-15121824
]
Mark Payne commented on NIFI-1447:
----------------------------------
[~ozhurakousky] - the reason that we didn't always log a stack trace for ERROR
messages is because we have cases where a FlowFile gets routed to 'failure' for
instance and that is a self-loop and may be processed thousands of times, each
time erroring out. We didn't want to fill the logs with hundreds of MB's (or
more) of stack traces. So we made it only spit out a stack trace if debug is
enabled. Now, this decision was made before the notion of Data Provenance
really existed, so we had to rely on the logs to understand what was happening
in the flow and stack traces made the logs very noisy. Now that we have Data
Provenance to explain how each FlowFile was processed, the INFO level logs are
of far less value, so adding the stack traces may not be such a hinderance.
I can get behind the idea of logging stack traces for ERROR level messages,
though. It can be frustrating to have to change logger to DEBUG to get the
stack trace and if it's not a problem that is easily repeatable it may be far
more difficult.
We should consider the concept of WARN level messages though. I think these
should stay as-is, not logging stack traces unless debug is enabled. While this
is odd in a Java sense, I think it makes sense for NiFi. This is because there
are cases where a FlowFIle comes into a Processor and some Exception occurs.
The FlowFile can still be processed, though. For example, maybe some sort of
enrich is being done on XML and a date/time attribute has an invalid value. In
that case, the rest of the process can still continue. From the Processor's
perspective, a WARN message would be logged with a stack trace. In some use
cases, though, every FlowFile that comes in may cause that Exception to get
thrown. And for this use case it may be totally okay. We don't want to log a
stack trace for every single FlowFile. So we made it so that it only happens on
debug.
But for ERROR I think I'm okay with it.
> Improve exception logging for Processors
> ----------------------------------------
>
> Key: NIFI-1447
> URL: https://issues.apache.org/jira/browse/NIFI-1447
> Project: Apache NiFi
> Issue Type: Bug
> Reporter: Oleg Zhurakousky
> Assignee: Oleg Zhurakousky
> Fix For: 0.5.0
>
>
> Unfortunately while the title says _"improve"_, I am logging it as a BUG for
> the following reasons.
> A client reported an issue with one of the processors where the only visible
> message in both _bulletin and logs_ is the message produced by ProcessLogger
> which does not seem to log a full stack trace. A typical pattern in current
> processor implementation is to have a significant code in _onTrigger()_
> method to be wrapped in _try/catch_ where exception is logged and then
> re-thrown. Unfortunately it's not the case for this particular processor. As
> a result it is impossible to determine the cause of the root problem.
> While immediate solution is to modify the specific Processor by wrapping its
> significant code in _try/catch_ a better solution would be to improve
> ProcessLogger to log full stack trace whenever there is an error.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
