If you know all your tables, and there is too much you can duplicate this configuration for all messages.
Le jeu. 25 mars 2021 à 13:03, Vibhath Ileperuma <[email protected]> a écrit : > Hello Etienne, > > With the configuration you have mentioned, we can log messages related to > only one table using the LogMessage processor, can't we? > But what I really want is to log custom messages related to all the tables > using the LogMessage processor(s). But the messages should be written to > different log files based on the table name. > > Thanks & Regards > > *Vibhath Ileperuma* > > > > On Thu, Mar 25, 2021 at 4:18 PM Etienne Jouvin <[email protected]> > wrote: > >> Hello. >> >> What you can do it to prefix your log message, with something like >> [MY_TABLE] >> and then in the logback configuration use an EvaluatorFilter. >> >> Something like this : >> >> <appender name="CUSTOM_APPENDER" >> class="ch.qos.logback.core.rolling.RollingFileAppender"> >> >> <file>${org.apache.nifi.bootstrap.config.log.dir}/loreal-sync-dctm-process.log</file> >> <rollingPolicy >> class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy"> >> <!-- >> For daily rollover, use 'user_%d.log'. >> For hourly rollover, use 'user_%d{yyyy-MM-dd_HH}.log'. >> To GZIP rolled files, replace '.log' with '.log.gz'. >> To ZIP rolled files, replace '.log' with '.log.zip'. >> --> >> >> <fileNamePattern>${org.apache.nifi.bootstrap.config.log.dir}/mytable-logger_%d{yyyy-MM-dd_HH}.log</fileNamePattern> >> <!-- keep 30 log files worth of history --> >> <maxHistory>10</maxHistory> >> <cleanHistoryOnStart>true</cleanHistoryOnStart> >> </rollingPolicy> >> >> <filter class="ch.qos.logback.core.filter.EvaluatorFilter"> >> <evaluator> >> <matcher> >> <Name>syncNuxeo</Name> >> <regex>\[MY_TABLE\] </regex> >> </matcher> >> >> >> <expression>syncNuxeo.matches(formattedMessage)</expression> >> </evaluator> >> <OnMismatch>DENY</OnMismatch> >> <OnMatch>NEUTRAL</OnMatch> >> </filter> >> >> <encoder >> class="ch.qos.logback.classic.encoder.PatternLayoutEncoder"> >> <pattern>%date %level [%thread] %logger{40} %msg%n</pattern> >> </encoder> >> </appender> >> >> Then use this new appender for LogMessage >> <logger name="org.apache.nifi.processors.standard.LogMessage" >> level="INFO"> >> <appender-ref ref=" CUSTOM_APPENDER "/> >> </logger> >> >> But this means you have to deploy specific libraries in the NiFi lib >> folder. >> org.codehaus.janino.commons-compiler >> org.codehaus.janino.janino >> >> Those libraries are potentially already used in standard processors / >> services. >> Depending on your NiFi version, check which version you need to deploy. >> >> Regards >> >> Etienne Jouvin >> >> Le jeu. 25 mars 2021 à 11:23, Vibhath Ileperuma < >> [email protected]> a écrit : >> >>> Hi All, >>> >>> I'm developing a NIFI flow which fetches data from a set of database >>> tables. To log custom messages (no. of rows fetched, query execution time >>> etc.), I'm using a 'LogMessage' processor. By changing the logback.xml >>> file, I was able to write my custom messages in a separate log file. But >>> it's better if I can maintain log files separately for different tables. >>> Is there a way to do this in NIFI. >>> >>> Thanks & Regards >>> >>> *Vibhath Ileperuma* >>> >>> >>> >>>
