Structured logging is not in a very good state in Druid right now (or the industry in general). Part of the issue is that the log4j standard json format is not very compatible with modern json logging systems. I even modified the sumologic appender to get better productionized items into it at https://github.com/metamx/sumologic-log4j2-appender . I do not know if there is a stackdriver friendly log4j formatter out there, but it would not surprise me if there was but it pulled in guava version future+inifnity.
My current log4j2 layout <https://logging.apache.org/log4j/2.x/manual/layouts.html#JSONLayout> line looks like this: <JsonLayout compact="true" eventEol="true" complete="false" properties= "true" /> Which is kind of annoying, especially with stack traces which can easily overflow a "single line length" in logging infrastructures. If you ever do find it worth your while to go down the route of writing a custom format for your logging system. The easiest way I found was to piggyback on the jackson format and just annotate a special class with whatever fields in whichever format you want: https://github.com/metamx/sumologic-log4j2-appender/blob/master/src/main/java/com/sumologic/log4j/core/SumoJsonLayout.java#L117-L237 On Fri, Feb 8, 2019 at 5:22 AM Don Bowman <d...@agilicus.com> wrote: > I am finding that the logging produced is very complex to manage. I would > like to add logstash json_event ( > https://github.com/logstash/log4j-jsonevent-layout). > Any comments before I start? > > In my environment I'm using fluent-bit, others are using logstash. Its > inconvenient to have multi-line log messages, or have to fish through > exceptions. > > Is there any other way to achieve? > > Some have suggested e.g. > > log4j.appender.stdout=org.apache.log4j.ConsoleAppender > log4j.appender.stdout.encoding=UTF-8 > log4j.appender.stdout.layout=org.apache.log4j.PatternLayout > > log4j.appender.stdout.layout.ConversionPattern={"level":"%p","timestamp":"%d{ISO8601}","thread":"%t","file":"%F", > "line":"%L","message":"%m"}%n > > but this won't work if there's a quote or newline. >
