[
https://issues.apache.org/jira/browse/NIFI-10579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Handermann updated NIFI-10579:
------------------------------------
Labels: (was: easyfix newbie)
> Improve standard out handling
> -----------------------------
>
> Key: NIFI-10579
> URL: https://issues.apache.org/jira/browse/NIFI-10579
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Configuration
> Affects Versions: 1.16.3
> Environment: All
> Reporter: Dylan Klomparens
> Priority: Minor
>
> h1. Problem statement and context
> NiFi intentionally captures and redirects everything sent to standard out and
> standard error. It does so in the [RunNiFi class, seen
> here|https://github.com/apache/nifi/blob/7823156606ca541ef9cae7192b092efd2cfe4e9a/nifi-bootstrap/src/main/java/org/apache/nifi/bootstrap/RunNiFi.java#L1490].
> This approach has the benefit of assigning logback logger classification to
> output that is sent to standard out and standard error. Thus, messages that
> are printed to standard out are labeled as originating from the
> "{{org.apache.nifi.StdOut}}" logback logger, and output from the "{{NiFi
> logging handler}}" thread. This catches all stray messages that may casually
> be output to standard out or standard error for debugging purposes...
> This approach also has drawbacks. It does not permit the NiFi administrator
> to redirect standard error or standard out easily. For example, if NiFi is
> configured to write all logs to standard out, this "encases" *every* log
> message as if it were written to the "{{org.apache.nifi.StdOut}}" logger at
> the {{INFO}} log level. For example, suppose logback is configured to write
> all messages to the console (standard out) with the message format: [date]
> [log level] [thread] [logger] [message]. All subsequent output would be
> "double wrapped", first being output by the logger it was emitted from, then
> being output by the logger handling standard out, and would look like the
> sample below. (Red is the "wrapping" logger, blue is the original message).
> {color:#DE350B}[date] INFO [NiFi logging handler]
> org.apache.nifi.StdOut{color} {color:#0747A6}[date] [original log level]
> [original thread] [log message]{color}
> This approach has an additional drawback that all log messages emitted from
> {{org.apache.nifi.StdOut}} are output at the {{INFO}} log level. Furthermore,
> all log messages emitted from {{org.apache.nifi.StdErr}} are emitted at the
> {{ERROR}} log level. This secludes the log level from the source logger,
> making log filtering more difficult if the log messages passes through
> standard out or standard error. Every log message is forcibly set to one log
> level.
> h1. Why this should be improved
> It would be beneficial for system administrators to allow redirection of
> standard out and standard error. The ability to pipe log messages to the
> external logging solution of their choice would be an important improvement.
> For example, when NiFi is run in a Docker container, [Docker can send logs
> elsewhere using its configurable log
> driver|https://docs.docker.com/config/containers/logging/configure/]. This
> means Docker is responsible for buffering the logs and forwarding them on to
> an external log solution such as syslog, AWS CloudWatch, Fluent, Google Cloud
> Logging, Splunk, and others.
> It is beneficial to use an external log solution so that logs do not fill up
> the disk of the computer running NiFi.
> h1. Proposed improvement
> There are at least two approaches that would improve NiFi.
> Approach #1: remove the redirection of standard out and standard error in the
> RunNiFi class. This would be the most disruptive approach from a deployment
> perspective, and may affect some existing instances of NiFi. This is the
> easiest technical solution and the most simple.
> Approach #2: add a configuration parameter to bootstrap.conf to disable
> redirection of standard out and standard error. The default would be that
> redirection _does_ still occur. This would be the least disruptive approach,
> and would not affect existing instances of NiFi unless the system
> administrator specifically configures the setting.
> h1. Questions and Implementation
> * Would the NiFi developer community be amenable to this change?
> * If so, which approach is preferred?
> If there is a positive response to this improvement, I will follow the
> [Contributor
> Guide|https://cwiki.apache.org/confluence/display/NIFI/Contributor+Guide] and
> implement the desired improvement approach, then submit a pull request.
> Thank you for your time and attention.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
