Looks like what I’m looking for can be done using collector not appender.
org.apache.karaf.decanter.collector.log.cfg: fields.remove.renderedMessage=true Cheers! On Mon, 24 Jun 2019 at 11:54 am, Noobtube Account <spidyt...@gmail.com> wrote: > Thanks JB - I think the EventFilter and the properties along with it are > used to include/exclude entire events, correct? > > In that case, I’m not able to use it to exclude fields of the resulting > json log but instead exclude entire logs based on the content of the log > message. > > I guess I’ll need to put this functionality in a custom marshaller? > > > Thanks > > On Fri, 21 Jun 2019 at 5:49 pm, Jean-Baptiste Onofré <j...@nanthrax.net> > wrote: > >> Hi, >> >> I guess you are talking about filtering of the events to send to the >> appender right ? >> >> I will update the Decanter documentation with some example and how it >> works. >> >> In the mean time, you can take a look on the test illustrating the >> events filtering: >> >> >> https://github.com/apache/karaf-decanter/blob/master/appender/utils/src/test/java/org/apache/karaf/decanter/appender/utils/EventFilterTest.java >> >> Regards >> JB >> >> On 21/06/2019 09:41, Noobtube Account wrote: >> > Hi JB, >> > >> > Could you show me where I could add the filter properties as you >> > mentioned or if it’s not implemented yet, some guidance on accessing the >> > appender or Marshaller properties file from inside the default decanter >> > json appender? >> > >> > >> > Thanks >> > >> > >> > >> > On Wed, 12 Jun 2019 at 11:53 am, Noobtube Account <spidyt...@gmail.com >> > <mailto:spidyt...@gmail.com>> wrote: >> > >> > Thanks JB! I've got the following config for appender to try out the >> > field exclude: >> > >> > # File >> > filename=${karaf.data}/log/karaf-json.log >> > >> > # Marshaller >> > marshaller.target=(dataFormat=json) >> > >> > # Append file >> > append=true >> > >> > # Fields Excluded by Name >> > event.property.name.exclude=MDC >> > >> > >> > This seems to stop the appender from producing any json log. >> Removing the event.property.name.exclude=MDC >> > >> > Also I couldn't find these properties documented anywhere nor could >> I find them in karaf decanter source code. Is the format I used correct? >> > >> > >> > >> > On Sun, 9 Jun 2019 at 02:58, Jean-Baptiste Onofré <j...@nanthrax.net >> > <mailto:j...@nanthrax.net>> wrote: >> > >> > Yes, you can create your own marshaller is you want to ;) >> > >> > By default, Decanter collectors send a Map to the Decanter >> > appenders. >> > Then the appenders use eventually marshaller to convert the Map >> to >> > something else. You can control the keys or values in this map >> > that you >> > want to include/exclude by directly configuring in the appender >> > configuration (with >> > >> >> event.property.name.exclude/event.property.name.include/event.property.value.exclude/event.property.value.include >> > properties). >> > By default, the JSON Marshaller convert the map as a json using >> > jackson. >> > If you want a full control, you can also create your own >> Marshaller >> > service. But I think that just the filtering on the appender >> > should be >> > OK for you. >> > Let me know if you need help about that. >> > >> > Regards >> > JB >> > >> > On 08/06/2019 18:52, Noobtube Account wrote: >> > > Great! Are we also able to control the format or turn fields >> off? >> > > >> > > >> > > On Sun, 9 Jun 2019 at 2:51 am, Jean-Baptiste Onofré >> > <j...@nanthrax.net <mailto:j...@nanthrax.net> >> > > <mailto:j...@nanthrax.net <mailto:j...@nanthrax.net>>> wrote: >> > > >> > > Yes, Decanter Log Collector is a global Pax Logging >> > "sniffer". So it >> > > takes messages from any bundle by default. >> > > >> > > However, you can configure this directly in >> > > etc/org.ops4j.pax.logging.cfg. >> > > >> > > Regards >> > > JB >> > > >> > > On 08/06/2019 18:07, Noobtube Account wrote: >> > > > Oh, scratch that. Uncommenting the filename in >> > > > etc/org.apache.karaf.decanter.appender.file.cfg seemed >> > to have made it >> > > > work. It probably wasn't due to that though and likely >> > something >> > > else I >> > > > did. Anyway, thanks for pointing out karaf decanter. >> > > > >> > > > Will the log collector collect log executions made >> > through any >> > > bundle? >> > > > >> > > > >> > > > On Sun, 9 Jun 2019 at 01:55, Noobtube Account >> > <spidyt...@gmail.com <mailto:spidyt...@gmail.com> >> > > <mailto:spidyt...@gmail.com <mailto:spidyt...@gmail.com>> >> > > > <mailto:spidyt...@gmail.com <mailto:spidyt...@gmail.com >> > >> > <mailto:spidyt...@gmail.com <mailto:spidyt...@gmail.com>>>> >> wrote: >> > > > >> > > > Thanks JB. I'm trying the decanter as you >> mentioned. >> > > > >> > > > I've done: >> > > > feature:repo-add decanter >> > > > feature:install decanter-collector-log >> > > > feature:install decanter-appender-file >> > > > >> > > > Then I edited >> > etc/org.apache.karaf.decanter.appender.file.cfg and >> > > > added: >> > > > # Marshaller >> > > > marshaller.target=(dataFormat=json) >> > > > >> > > > But I don't see any logs created. Do we need to >> > install the >> > > marshaller? >> > > > >> > > > On Sun, 9 Jun 2019 at 00:33, Jean-Baptiste Onofré >> > > <j...@nanthrax.net <mailto:j...@nanthrax.net> >> > <mailto:j...@nanthrax.net <mailto:j...@nanthrax.net>> >> > > > <mailto:j...@nanthrax.net <mailto:j...@nanthrax.net> >> > <mailto:j...@nanthrax.net <mailto:j...@nanthrax.net>>>> wrote: >> > > > >> > > > By the way, instead of doing that, you can use >> > regular logging >> > > > and use >> > > > Decanter with the log collector and the json >> > marshaller to >> > > > create json >> > > > output (to log, file, elasticsearch or >> whatever). >> > > > >> > > > Regards >> > > > JB >> > > > >> > > > On 08/06/2019 16:30, Jean-Baptiste Onofré wrote: >> > > > > Hi >> > > > > >> > > > > You don't have to add logback-core as >> > pax-logging-logback >> > > > bundle already >> > > > > embed it. >> > > > > >> > > > > Secondly it seems jackson is not imported >> > correctly. The >> > > root >> > > > cause is: >> > > > > >> > > > > Caused by: java.lang.ClassNotFoundException: >> > > > >> com.fasterxml.jackson.databind.ObjectMapper >> > not found by >> > > > >> org.ops4j.pax.logging.pax-logging-logback >> > > > > >> > > > > You can try a dynamic import on >> > pax-logging-logback. >> > > > > >> > > > > Else a simple workaround is to create a >> > fragment with >> > > > > pax-logging-logback as host and embedding >> > (private-package) >> > > > the classes >> > > > > you need. >> > > > > >> > > > > I did that for json layout with >> > pax-logging-log4j2. >> > > > > >> > > > > Regards >> > > > > JB >> > > > > >> > > > > On 08/06/2019 16:17, Noobtube Account wrote: >> > > > >> I'm trying to get JSON format logging enabled >> > in Karaf. >> > > > Trying to do >> > > > >> something similar to what's described >> > > > >> in >> https://ops4j1.jira.com/browse/PAXLOGGING-168 >> > > > >> >> > > > >> On a fresh copy of Karaf 4.2.5 I've added to >> > > > /etc/startup.properties: >> > > > >> >> > > > >> >> > mvn\:com.fasterxml.jackson.core/jackson-core/2.9.5 = 8 >> > > > >> >> > > mvn\:com.fasterxml.jackson.core/jackson-annotations/2.9.5 >> = 8 >> > > > >> >> > mvn\:com.fasterxml.jackson.core/jackson-databind/2.9.5 = 8 >> > > > >> mvn\:ch.qos.logback/logback-classic/1.2.2 = 8 >> > > > >> mvn\:ch.qos.logback/logback-core/1.2.2 = 8 >> > > > >> >> > mvn\:ch.qos.logback.contrib/logback-json-core/0.1.5 = 8 >> > > > >> >> > mvn\:ch.qos.logback.contrib/logback-json-classic/0.1.5 = 8 >> > > > >> >> > mvn\:org.ops4j.pax.logging/pax-logging-logback/1.10.1 = 8 >> > > > >> >> > > > >> The above jars have been added to the system >> > folder. They >> > > > look like: >> > > > >> logback: https://pastebin.com/raw/60Pnu8f1 >> > > > >> jackson: https://pastebin.com/raw/NkUqZrGS >> > > > >> >> > > > >> My etc/org.ops4j.pax.logging.cfg contains the >> > line: >> > > > >> >> > > > >> > > >> > >> org.ops4j.pax.logging.logback.config.file=${karaf.base}/etc/logback.xml >> > > > >> >> > > > >> The log back is the same one >> > > > >> >> > from https://ops4j1.jira.com/browse/PAXLOGGING-168 >> > > > >> >> > > > >> When I startup Karaf I get the following CNF >> > error >> > > related to >> > > > Jackson. >> > > > >> Shouldn't the startup.properties file make >> > those classes >> > > > available? >> > > > >> >> > > > >> WARN in >> > > > >> > ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@5c3aaee6 - >> > > > >> SizeAndTimeBasedFNATP is deprecated. Use >> > > > SizeAndTimeBasedRollingPolicy >> > > > >> instead >> > > > >> WARN in >> > > > >> > ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@5c3aaee6 >> > > - For >> > > > >> more information see >> > > > >> >> > > > >> > > >> > >> http://logback.qos.ch/manual/appenders.html#SizeAndTimeBasedRollingPolicy >> > > > >> 00:14:49.839 [CM Configuration Updater >> > (ManagedService >> > > Update: >> > > > >> pid=[org.ops4j.pax.logging])] ERROR >> > > > org.apache.felix.configadmin - >> > > > >> [org.osgi.service.log.LogService, >> > > > >> org.knopflerfish.service.log.LogService, >> > > > >> org.ops4j.pax.logging.PaxLoggingService, >> > > > >> org.osgi.service.cm >> > <http://org.osgi.service.cm> <http://org.osgi.service.cm> >> > > > <http://org.osgi.service.cm>.ManagedService, >> id=15, >> > > > >> >> > > >> > >> bundle=13/mvn:org.ops4j.pax.logging/pax-logging-logback/1.10.1]: >> > > > >> Unexpected problem updating configuration >> > > org.ops4j.pax.logging >> > > > >> java.lang.NoClassDefFoundError: >> > > > com/fasterxml/jackson/databind/ObjectMapper >> > > > >> at >> > > > >> >> > > > >> > > >> > >> >> ch.qos.logback.contrib.jackson.JacksonJsonFormatter.<init>(JacksonJsonFormatter.java:41) >> > > > >> ~[?:?] >> > > > >> at >> > > > >> > sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native >> > > > Method) >> > > > >> ~[?:?] >> > > > >> at >> > > > >> >> > > > >> > > >> > >> >> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) >> > > > >> ~[?:?] >> > > > >> at >> > > > >> >> > > > >> > > >> > >> >> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) >> > > > >> ~[?:?] >> > > > >> at >> > > > >> > > >> > >> java.lang.reflect.Constructor.newInstance(Constructor.java:423) >> > > > ~[?:?] >> > > > >> at >> > java.lang.Class.newInstance(Class.java:442) ~[?:?] >> > > > >> at >> > > > >> >> > > > >> > > >> > >> >> ch.qos.logback.core.joran.action.NestedComplexPropertyIA.begin(NestedComplexPropertyIA.java:121) >> > > > >> ~[?:?] >> > > > >> at >> > > > >> >> > > > >> > > >> > >> >> ch.qos.logback.core.joran.spi.Interpreter.callBeginAction(Interpreter.java:269) >> > > > >> ~[?:?] >> > > > >> at >> > > > >> >> > > > >> > > >> > >> >> ch.qos.logback.core.joran.spi.Interpreter.startElement(Interpreter.java:145) >> > > > >> ~[?:?] >> > > > >> at >> > > > >> >> > > > >> > > >> > >> >> ch.qos.logback.core.joran.spi.Interpreter.startElement(Interpreter.java:128) >> > > > >> ~[?:?] >> > > > >> at >> > > > >> > > >> > >> ch.qos.logback.core.joran.spi.EventPlayer.play(EventPlayer.java:50) >> > > > >> ~[?:?] >> > > > >> at >> > > > >> >> > > > >> > > >> > >> >> ch.qos.logback.core.joran.GenericConfigurator.doConfigure(GenericConfigurator.java:165) >> > > > >> ~[?:?] >> > > > >> at >> > > > >> >> > > > >> > > >> > >> >> ch.qos.logback.core.joran.GenericConfigurator.doConfigure(GenericConfigurator.java:152) >> > > > >> ~[?:?] >> > > > >> at >> > > > >> >> > > > >> > > >> > >> >> ch.qos.logback.core.joran.GenericConfigurator.doConfigure(GenericConfigurator.java:110) >> > > > >> ~[?:?] >> > > > >> at >> > > > >> >> > > > >> > > >> > >> >> ch.qos.logback.core.joran.GenericConfigurator.doConfigure(GenericConfigurator.java:81) >> > > > >> ~[?:?] >> > > > >> at >> > > > >> >> > > > >> > > >> > >> >> org.ops4j.pax.logging.logback.internal.PaxLoggingServiceImpl.configureLogback(PaxLoggingServiceImpl.java:325) >> > > > >> ~[?:?] >> > > > >> at >> > > > >> >> > > > >> > > >> > >> >> org.ops4j.pax.logging.logback.internal.PaxLoggingServiceImpl.updated(PaxLoggingServiceImpl.java:258) >> > > > >> ~[?:?] >> > > > >> at >> > > > >> >> > > > >> > > >> > >> >> org.ops4j.pax.logging.logback.internal.PaxLoggingServiceImpl$1ManagedPaxLoggingService.updated(PaxLoggingServiceImpl.java:471) >> > > > >> ~[?:?] >> > > > >> at >> > > > >> >> > > > >> > > >> > >> >> org.apache.felix.cm.impl.helper.ManagedServiceTracker.updated(ManagedServiceTracker.java:189) >> > > > >> ~[9:org.apache.felix.configadmin:1.9.14] >> > > > >> at >> > > > >> >> > > > >> > > >> > >> >> org.apache.felix.cm.impl.helper.ManagedServiceTracker.updateService(ManagedServiceTracker.java:152) >> > > > >> [9:org.apache.felix.configadmin:1.9.14] >> > > > >> at >> > > > >> >> > > > >> > > >> > >> >> org.apache.felix.cm.impl.helper.ManagedServiceTracker.provideConfiguration(ManagedServiceTracker.java:85) >> > > > >> [9:org.apache.felix.configadmin:1.9.14] >> > > > >> at >> > > > >> >> > > > >> > > >> > >> >> org.apache.felix.cm.impl.ConfigurationManager$ManagedServiceUpdate.provide(ConfigurationManager.java:1113) >> > > > >> [9:org.apache.felix.configadmin:1.9.14] >> > > > >> at >> > > > >> >> > > > >> > > >> > >> >> org.apache.felix.cm.impl.ConfigurationManager$ManagedServiceUpdate.run(ConfigurationManager.java:1069) >> > > > >> [9:org.apache.felix.configadmin:1.9.14] >> > > > >> at >> > > > >> > > >> > >> org.apache.felix.cm.impl.UpdateThread.run0(UpdateThread.java:138) >> > > > >> [9:org.apache.felix.configadmin:1.9.14] >> > > > >> at >> > > > >> > > >> > >> org.apache.felix.cm.impl.UpdateThread.run(UpdateThread.java:105) >> > > > >> [9:org.apache.felix.configadmin:1.9.14] >> > > > >> at java.lang.Thread.run(Thread.java:748) >> [?:?] >> > > > >> Caused by: java.lang.ClassNotFoundException: >> > > > >> com.fasterxml.jackson.databind.ObjectMapper >> > not found by >> > > > >> org.ops4j.pax.logging.pax-logging-logback >> [13] >> > > > >> at >> > > > >> >> > > > >> > > >> > >> >> org.apache.felix.framework.BundleWiringImpl.findClassOrResourceByDelegation(BundleWiringImpl.java:1639) >> > > > >> ~[?:?] >> > > > >> at >> > > > >> >> > > > >> > > >> > >> >> org.apache.felix.framework.BundleWiringImpl.access$200(BundleWiringImpl.java:80) >> > > > >> ~[?:?] >> > > > >> at >> > > > >> >> > > > >> > > >> > >> >> org.apache.felix.framework.BundleWiringImpl$BundleClassLoader.loadClass(BundleWiringImpl.java:2053) >> > > > >> ~[?:?] >> > > > >> at >> > > java.lang.ClassLoader.loadClass(ClassLoader.java:357) >> ~[?:?] >> > > > >> ... 26 more >> > > > >> >> > > > >> >> > > > > >> > > > >> > > > -- >> > > > Jean-Baptiste Onofré >> > > > jbono...@apache.org <mailto:jbono...@apache.org >> > >> > <mailto:jbono...@apache.org <mailto:jbono...@apache.org>> >> > > <mailto:jbono...@apache.org <mailto:jbono...@apache.org> >> > <mailto:jbono...@apache.org <mailto:jbono...@apache.org>>> >> > > > http://blog.nanthrax.net >> > > > Talend - http://www.talend.com >> > > > >> > > >> > > -- >> > > Jean-Baptiste Onofré >> > > jbono...@apache.org <mailto:jbono...@apache.org> >> > <mailto:jbono...@apache.org <mailto:jbono...@apache.org>> >> > > http://blog.nanthrax.net >> > > Talend - http://www.talend.com >> > > >> > >> > -- >> > Jean-Baptiste Onofré >> > jbono...@apache.org <mailto:jbono...@apache.org> >> > http://blog.nanthrax.net >> > Talend - http://www.talend.com >> > >> >> -- >> Jean-Baptiste Onofré >> jbono...@apache.org >> http://blog.nanthrax.net >> Talend - http://www.talend.com >> >
