Hello.
What you can do it to prefix your log message, with something like
[MY_TABLE]
and then in the logback configuration use an EvaluatorFilter.
Something like this :
<appender name="CUSTOM_APPENDER"
class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>${org.apache.nifi.bootstrap.config.log.dir}/loreal-sync-dctm-process.log</file>
<rollingPolicy
class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<!--
For daily rollover, use 'user_%d.log'.
For hourly rollover, use 'user_%d{yyyy-MM-dd_HH}.log'.
To GZIP rolled files, replace '.log' with '.log.gz'.
To ZIP rolled files, replace '.log' with '.log.zip'.
-->
<fileNamePattern>${org.apache.nifi.bootstrap.config.log.dir}/mytable-logger_%d{yyyy-MM-dd_HH}.log</fileNamePattern>
<!-- keep 30 log files worth of history -->
<maxHistory>10</maxHistory>
<cleanHistoryOnStart>true</cleanHistoryOnStart>
</rollingPolicy>
<filter class="ch.qos.logback.core.filter.EvaluatorFilter">
<evaluator>
<matcher>
<Name>syncNuxeo</Name>
<regex>\[MY_TABLE\] </regex>
</matcher>
<expression>syncNuxeo.matches(formattedMessage)</expression>
</evaluator>
<OnMismatch>DENY</OnMismatch>
<OnMatch>NEUTRAL</OnMatch>
</filter>
<encoder
class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
<pattern>%date %level [%thread] %logger{40} %msg%n</pattern>
</encoder>
</appender>
Then use this new appender for LogMessage
<logger name="org.apache.nifi.processors.standard.LogMessage"
level="INFO">
<appender-ref ref=" CUSTOM_APPENDER "/>
</logger>
But this means you have to deploy specific libraries in the NiFi lib folder.
org.codehaus.janino.commons-compiler
org.codehaus.janino.janino
Those libraries are potentially already used in standard processors /
services.
Depending on your NiFi version, check which version you need to deploy.
Regards
Etienne Jouvin
Le jeu. 25 mars 2021 à 11:23, Vibhath Ileperuma <[email protected]>
a écrit :
> Hi All,
>
> I'm developing a NIFI flow which fetches data from a set of database
> tables. To log custom messages (no. of rows fetched, query execution time
> etc.), I'm using a 'LogMessage' processor. By changing the logback.xml
> file, I was able to write my custom messages in a separate log file. But
> it's better if I can maintain log files separately for different tables.
> Is there a way to do this in NIFI.
>
> Thanks & Regards
>
> *Vibhath Ileperuma*
>
>
>
>
