Actually druid uses SLF4J, so you should be able to add in the logback libs to the classpath and things *should* just work.
On Fri, Feb 8, 2019 at 10:09 AM Don Bowman <d...@agilicus.com> wrote: > thanks! > > is there any movement to move to 'logback' from log4j? That would allow > using e.g. https://github.com/logstash/logstash-logback-encoder > > the problem w/ the JsonLayout approach is that it doesn't seem to get > exceptions properly. > > E.g. I get something like below. > > > {"timeMillis":1549648630326,"thread":"main","level":"INFO","loggerName":"org.apache.druid.java.util.common.lifecycle.Lifecycle$AnnotationBasedHandler","message":"Invoking > stop method[public void > > org.apache.druid.initialization.Log4jShutterDownerModule$Log4jShutterDowner.stop()] > on > > object[org.apache.druid.initialization.Log4jShutterDownerModule$Log4jShutterDowner@61d60e38 > > ].","endOfBatch":false,"loggerFqcn":"org.apache.logging.slf4j.Log4jLogger","contextMap":[]} > 2019-02-08 17:57:10,342 main ERROR Unable to register shutdown hook because > JVM is shutting down. java.lang.IllegalStateException: Not started > at > > org.apache.druid.common.config.Log4jShutdown.addShutdownCallback(Log4jShutdown.java:48) > at > > org.apache.logging.log4j.core.impl.Log4jContextFactory.addShutdownCallback(Log4jContextFactory.java:273) > at > > org.apache.logging.log4j.core.LoggerContext.setUpShutdownHook(LoggerContext.java:256) > at > org.apache.logging.log4j.core.LoggerContext.start(LoggerContext.java:216) > at > > org.apache.logging.log4j.core.impl.Log4jContextFactory.getContext(Log4jContextFactory.java:145) > at > > org.apache.logging.log4j.core.impl.Log4jContextFactory.getContext(Log4jContextFactory.java:41) > at org.apache.logging.log4j.LogManager.getContext(LogManager.java:182) > at > > org.apache.logging.log4j.spi.AbstractLoggerAdapter.getContext(AbstractLoggerAdapter.java:103) > at > > org.apache.logging.slf4j.Log4jLoggerFactory.getContext(Log4jLoggerFactory.java:43) > at > > org.apache.logging.log4j.spi.AbstractLoggerAdapter.getLogger(AbstractLoggerAdapter.java:42) > at > > org.apache.logging.slf4j.Log4jLoggerFactory.getLogger(Log4jLoggerFactory.java:29) > at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:253) > at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:265) > at > > org.apache.curator.utils.CloseableExecutorService.<init>(CloseableExecutorService.java:40) > at > > org.apache.druid.curator.cache.PathChildrenCacheFactory.make(PathChildrenCacheFactory.java:55) > at > > org.apache.druid.curator.inventory.CuratorInventoryManager.start(CuratorInventoryManager.java:109) > at > > org.apache.druid.client.AbstractCuratorServerInventoryView.start(AbstractCuratorServerInventoryView.java:168) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > > org.apache.druid.java.util.common.lifecycle.Lifecycle$AnnotationBasedHandler.start(Lifecycle.java:427) > at > > org.apache.druid.java.util.common.lifecycle.Lifecycle.start(Lifecycle.java:323) > at org.apache.druid.guice.LifecycleModule$2.start(LifecycleModule.java:138) > at org.apache.druid.cli.GuiceRunnable.initLifecycle(GuiceRunnable.java:107) > at org.apache.druid.cli.ServerRunnable.run(ServerRunnable.java:58) > at org.apache.druid.cli.Main.main(Main.java:118) > > > > On Fri, 8 Feb 2019 at 12:27, Charles Allen <charles.al...@snap.com > .invalid> > wrote: > > > Structured logging is not in a very good state in Druid right now (or the > > industry in general). Part of the issue is that the log4j standard json > > format is not very compatible with modern json logging systems. I even > > modified the sumologic appender to get better productionized items into > it > > at https://github.com/metamx/sumologic-log4j2-appender . I do not know > if > > there is a stackdriver friendly log4j formatter out there, but it would > not > > surprise me if there was but it pulled in guava version future+inifnity. > > > > My current log4j2 layout > > < > https://urldefense.proofpoint.com/v2/url?u=https-3A__logging.apache.org_log4j_2.x_manual_layouts.html-23JSONLayout&d=DwIBaQ&c=ncDTmphkJTvjIDPh0hpF_w&r=HrLGT1qWNhseJBMYABL0GFSZESht5gBoLejor3SqMSo&m=v3J-Uid1SfjBwbQj0bygNz7-FkK1Cw4ubdPwAFDUeLk&s=s49QFMDklbZ3qj13P53cf8fzxtS-phSdM2B3OeWfuNw&e=> > line > > looks like this: > > <JsonLayout compact="true" eventEol="true" complete="false" properties= > > "true" /> > > > > Which is kind of annoying, especially with stack traces which can easily > > overflow a "single line length" in logging infrastructures. > > > > If you ever do find it worth your while to go down the route of writing a > > custom format for your logging system. The easiest way I found was to > > piggyback on the jackson format and just annotate a special class with > > whatever fields in whichever format you want: > > > > > > > https://github.com/metamx/sumologic-log4j2-appender/blob/master/src/main/java/com/sumologic/log4j/core/SumoJsonLayout.java#L117-L237 > > > > > > > > On Fri, Feb 8, 2019 at 5:22 AM Don Bowman <d...@agilicus.com> wrote: > > > > > I am finding that the logging produced is very complex to manage. I > would > > > like to add logstash json_event ( > > > https://github.com/logstash/log4j-jsonevent-layout). > > > Any comments before I start? > > > > > > In my environment I'm using fluent-bit, others are using logstash. Its > > > inconvenient to have multi-line log messages, or have to fish through > > > exceptions. > > > > > > Is there any other way to achieve? > > > > > > Some have suggested e.g. > > > > > > log4j.appender.stdout=org.apache.log4j.ConsoleAppender > > > log4j.appender.stdout.encoding=UTF-8 > > > log4j.appender.stdout.layout=org.apache.log4j.PatternLayout > > > > > > > > > log4j.appender.stdout.layout.ConversionPattern={"level":"%p","timestamp":"%d{ISO8601}","thread":"%t","file":"%F", > > > "line":"%L","message":"%m"}%n > > > > > > but this won't work if there's a quote or newline. > > > > > >
