Mark,
When did this happen? I'm looking at (an as yet unchanged from original
download) /co//nf///logback.xml/ in 0.6.1 and I'm seeing:
- no string of characters "org.apache.nifi.processors" and
- INFO all over in places like <logger name="org.apache.nifi"
level="INFO"/>
What am I missing? Was your comment valid from 0.7.0-onward?
Russ
On 06/10/2016 10:53 AM, Mark Payne wrote:
Hi Huagen,
This is typically the type of logging you will see in NiFi. Each
processor will generally log at an INFO
level what it is doing for each FlowFile. Unfortunately, though, this
can become extremely verbose,
and many people want that logging toned down, so in the master branch
of NiFi, the minimum log
level for processors is set to WARN instead of INFO. You can change
this by updating the conf/logback.xml
and setting the log level of "org.apache.nifi.processors" to INFO
instead of WARN.
The main reason that we have changed the default log level though is
that in NiFi, it is very rare to need
to go through all of the tedious labor of grepping through logs.
Instead, the recommended approach is to
use the Data Provenance features [1]. This will allow you to search
for data of interest to you and see exactly
how it was processed throughout the flow. Additionally, this provides
you access to the FlowFile attributes as
they were each step along the way, and the ability to click-to-content
to see how the data looked at that point
in the flow as well. It also allows you to visualize what happened to
the data, even if it is split into many smaller
pieces of data (potentially with different filenames) or merged
together with other data, so that you don't have
to worry about the filename.
I hope this helps!
-Mark
[1]
http://nifi.apache.org/docs/nifi-docs/html/user-guide.html#data-provenance
On Jun 10, 2016, at 11:58 AM, Huagen peng <[email protected]
<mailto:[email protected]>> wrote:
Hi,
I would like to learn about some better practices on logging. Here
is what I would imagine in an ideal log for a flow like fetching
files from SFTP, processing the files in certain way, and then saving
the file to the disk. In the log, I would see that the SFTP step is
triggered, with the filename in clear text. I would then see that
the file processing is started, and that the file is saved. If there
are errors, I would also see the errors in the log as well.
How would I achieve that or something close to that in NiFi?
Thanks,
Huagen
