Hi Huagen, This is typically the type of logging you will see in NiFi. Each processor will generally log at an INFO level what it is doing for each FlowFile. Unfortunately, though, this can become extremely verbose, and many people want that logging toned down, so in the master branch of NiFi, the minimum log level for processors is set to WARN instead of INFO. You can change this by updating the conf/logback.xml and setting the log level of "org.apache.nifi.processors" to INFO instead of WARN.
The main reason that we have changed the default log level though is that in NiFi, it is very rare to need to go through all of the tedious labor of grepping through logs. Instead, the recommended approach is to use the Data Provenance features [1]. This will allow you to search for data of interest to you and see exactly how it was processed throughout the flow. Additionally, this provides you access to the FlowFile attributes as they were each step along the way, and the ability to click-to-content to see how the data looked at that point in the flow as well. It also allows you to visualize what happened to the data, even if it is split into many smaller pieces of data (potentially with different filenames) or merged together with other data, so that you don't have to worry about the filename. I hope this helps! -Mark [1] http://nifi.apache.org/docs/nifi-docs/html/user-guide.html#data-provenance <http://nifi.apache.org/docs/nifi-docs/html/user-guide.html#data-provenance> > On Jun 10, 2016, at 11:58 AM, Huagen peng <[email protected]> wrote: > > Hi, > > I would like to learn about some better practices on logging. Here is what I > would imagine in an ideal log for a flow like fetching files from SFTP, > processing the files in certain way, and then saving the file to the disk. > In the log, I would see that the SFTP step is triggered, with the filename in > clear text. I would then see that the file processing is started, and that > the file is saved. If there are errors, I would also see the errors in the > log as well. > > How would I achieve that or something close to that in NiFi? > > Thanks, > > Huagen
