Hey Ralph,
Here is my status update:
-~- Benchmarks -~-
I have removed the benchmarks results. It takes ~8h for a complete run
and I don't want to repeat that cycle after every change. I might
revisit this idea once JTL becomes more stable in terms of its
features.
-~- Flattening of MDC fields -~-
I have changed the MDC directive as follows:
mdc
mdc:flatten[=<prefix>][,stringify]
mdc:pattern=<pattern>[,flatten=<prefix>][,stringify]
mdc:key=<key>[,stringify]
This also allowed JsonTemplateLayout to produce the *exact* output as
GelfLayout and EcsLayout, where MDC keys are flattened and values are
stringified.
-~- Null termination, newlines, Logstash, and etc. -~-
I have tried to reproduce the experiment you have shared, though could
not really succeed due to some Docker command failure in restartApp.sh.
Further, I find the associated Spring setup quite cumbersome to wrap my
mind around it. That said, I have done something else: I have improved
LogstashIT such that
1. Logstash is configured with both "tcp" and "gelf" input plugins,
2. LogstashIT employs JsonTemplateLayout against both inputs, repeats
the same using GelfLayout and EcsLayout, and verifies the populated
content in Elasticsearch.
One can easily validate this on the branch as follows:
$ ./mvnw clean package -DskipTests
$ ./mvnw \
verify -o -P docker \
-pl
log4j-layout-jackson-json,log4j-plugins,log4j-core,log4j-api,log4j-layout-json-template
\
-Dtest="Dummy.java" -DtrimStackTrace=false -DfailIfNoTests=false
One can repeat the very same by running LogstashIT in IDE after
starting the containers:
$ ./mvnw \
docker:start -o -P docker \
-pl
log4j-layout-jackson-json,log4j-plugins,log4j-core,log4j-api,log4j-layout-json-template
I don't know what is wrong with the Spring Cloud Config setup you have
shared, but I have no reasons to believe that JsonTemplateLayout is the
suspect.
Regarding your remark about everything getting escaped... This might
happen when Logstash fails to read an input. In such a case, it puts
the entire payload into the "message" field (hence, the escaping) and
stores it like that.
>From now on, I don't know how to proceed with this problem, in
particular, given I believe the problem is in somewhere else but
JsonTemplateLayout.
-~- Merging branch to master -~-
Unless there are objections, I want to merge the branch to master.
There on I will share json-template-layout.md with the community to
collect some feedback on the API. In particular, I have existing users
of LogstashLayout in mind.
Kind regards.
On Mon, May 18, 2020 at 1:18 AM Ralph Goers <ralph.go...@dslextreme.com> wrote:
>
>
>
> > On May 17, 2020, at 2:37 PM, Volkan Yazıcı <volkan.yaz...@gmail.com> wrote:
> >
> > Thanks so much for the thorough review Ralph, really appreciated! I
> > will address issues you have raised.
> >
> > [As a side note, I have pushed changes containing performance
> > improvements and benchmark results. The module is still dependency
> > free and performance-wise pretty good.]
> >
> >> all the default templates separate the message from the exception
> >
> > Yes, this I have also realized while studying the source code of
> > GelfLayout. Some random thoughts:
> >
> > 1. How about introducing a firstNonNull-like operator:
> > ${json:op:firstNonNull:${json:message},${json:exception:message}}.
> > Parsing this would not be trivial, given the list of variables
> > can include comma in various forms. Though the rest should
> > be tractable. (Feels like JsonTemplateLayout directives will at
> > some point catch up the with ones in PatternLayout.)
>
> The problem here is that I want newlines in the message. I’m not sure what
> firstNonNull means in the context of a message. The message itself won’t have
> nulls. The Gelf format dictates the null is at the end of the JSON string.
>
> >
> > 2. How about introducing a fallback parameter to the "message"
> > directive: ${json:message:fallback=exception:message}.
>
> Again, I don’t know what that means or how it solves my problem of wanting
> newlines in the message.
>
> >
> > 3. May I challenge the request: Do one really need it? You store
> > individual values, i.e, message and exception, in individual fields.
> > When one is missing and the other is present, isn't it just a
> > presentation layer issue to display it properly?
>
> Picture how a log event looks in a file with a “standard” PatternLayout. In
> Kibana by default you get 3 columns, the timestamp, a field I don’t care
> about and the message. I want the message to appear just as it would in a log
> file without the timestamp. Yes, I can click on each message and see its
> attributes, but that takes longer. And I want exceptions to jump out at me
> when I am visually scrolling through messages. In fact, I typically don’t
> want the stack trace in a variable of its own. Kibana doesn’t take the
> pieces of the message and put them together like the pattern layout does.
> Instead it lets you create columns for individual attributes. The problem
> with that is the columns waste a lot of space, especially if they frequently
> are empty as a stack trace would be.
>
> So yes, I really need it.
>
> >
> >> In the logged event all newlines seem to be stripped from the exception.
> >
> > I really don't get this Ralph. Have you seen the tests in LogstashIT?
> > I have just added a new one testing only newlines. Would you mind
> > fiddling around with LogstashIT to reproduce the issue, please? Or
> > sharing a recipe for me to reproduce it?
>
> The way I do it is to install Logstash, ElasticSearch and Kibana locally on
> my Mac as well as Docker Desktop for Mac. I then run SpringCloud config
> server from the Log4j Spring Cloud Config project and tailor the log4j2.xml
> that is in the config-repo directory of that project. Note that all Log4j
> Lookup variables have to have the $ escaped or Spring Cloud Config will try
> to resolve them. It also requires having the Socket appender connect to a
> host of “host.docker.internal” to be able to connect to Logstash outside of
> the docker container. I configure Logstash using the configuration documented
> at http://logging.apache.org/log4j/2.x/manual/cloud.html
> <http://logging.apache.org/log4j/2.x/manual/cloud.html>, at least when using
> Gelf. I configure Kibana with an index of app-*.
>
> I then modifythe Spring Cloud Config Sample app as needed to work with this
> setup and start it using docker/restartApp.sh.
>
> After starting I do a few things:
> 1. Enter “docker logs app-container”. This should show it is running and the
> last message will be the Spring logo.
> 2. Look in Kibana and click on the logs. You should see logs from the server.
> 3. In a browser enter http://localhost:8080/sample/exception
> <http://localhost:8080/sample/exception> - unless you have changed the port.
> For some reason I had something running on 8080 and had to change the sample
> app to use 8090. This will generate a stack trace on the web page and in the
> log.
> 4. Look in Kibana. You should see the exception and the stack trace properly
> formatted in the message column. It should also display the thread id,
> log-level and other items configured in the pattern.
>
> >
> >> Logstash does not seem to be getting a null character
> >> (I tried with both “\0” and “\u0000”)
> >
> > This is weird. Below is how GelfLayout does it:
> >
> > if (includeNullDelimiter) {
> > builder.append('\0');
> > }
> >
> > And here is how JsonTemplateLayout does it:
> >
> > stringBuilder.append(eventDelimiter);
> >
> > Further, there is even a test for this:
> > JsonTemplateLayout#test_null_eventDelimiter():
> >
> > final String serializedLogEvent = layout.toSerializable(logEvent);
> > assertThat(serializedLogEvent).isEqualTo(eventTemplate + '\0');
> >
> > Any ideas on what might I be missing?
>
> No - I just know that Logstash wasn’t detecting the null. It looked like the
> whole string was being escaped as all the double quotes were also escaped as
> you can see from the example. I should note that I directly copied that from
> the Logstash log that was written to the terminal window with debug logging
> enabled, so it is possible it was caused by that, but this is also what I see
> in Kibana.
>
> >
> >> This isn’t valid in the Gelf spec as additional attributes in Gelf
> >> must match the regex - ^[\w\.\-]*$. This means you need to
> >> parse the MDC and add each key as an additional to be able
> >> to create valid Gelf. Personally, I’d like that option anyway.
> >
> > For this purpose, I need to introduce an operator similar to
> > unquote-splicing in Lisp to merge the contents of the child with the
> > parent. That said, I have my doubts about the practical benefit of
> > such a feature. To the best of my knowledge, almost every (sanely
> > configured?) log pipeline ends up with a structured storage system
> > containing whitelisted fields. Consider ELK stack. Would you allow
> > developers to introduce fields at their will? If so, it is a matter of
> > time somebody will take down the entire Elasticsearch cluster by
> > flooding it with ContextData.put(userInput, "doh"). Hence, you employ
> > the identical whitelisting strategy at the layout as well:
> >
> > {
> > "version": "1.1",
> > "host": "${hostName}",
> > ...,
> > "_mdc.loginId": "${json:mdc:loginId}",
> > "_mdc.requestId": "${json:mdc:requestId}"
> > }
> >
> > What do you think? Shall we really introduce unquote-splicing?
>
> I would have preferred a syntax like
>
> {
> “version” “1.1”,
> …
> ${json.map:mdc:prefix=“_”}
> }
>
> This would cause the whole map to be emitted. Although it would be nice to
> have a version of that which also could specify the keys there probably isn’t
> a point to that since you can directly specify them as you have shown.
>
>
> As I said previously, now that I understand what is going on I am comfortable
> with you merging the PR. It is going to the master branch so it won’t be what
> users see on the web site right now. By the time it makes it there we can get
> this all straightened out. If nothing else we could have the current
> GelfLayout and the JsonTemplateLayout both on the page and explain the
> differences between them, although in the end I agree it would be better to
> have the JsonTemplateLayout replace both the JsonLayout and GelfLayout.
>
> Ralph
>
> >
> > Kind regards.
> >
> > On Sat, May 16, 2020 at 1:28 AM Ralph Goers <ralph.go...@dslextreme.com>
> > wrote:
> >>
> >> I finally got time to spend looking at the Layout. I now understand why it
> >> works for you and sort of understand why it doesn’t work for me.
> >>
> >> First, all the default templates separate the message from the exception.
> >> In the logged event all newlines seem to be stripped from the exception.
> >> This makes viewing them in Kibana pretty awful, but the do get logged
> >> successfully.
> >>
> >> To include newlines in a log event you either have to play games in
> >> Logstash (or Fluentd) and try to put the event back together again after
> >> it has been separated into multiple lines or you have to use a different
> >> delimiter. Gelf specifies that a null character be used.
> >>
> >> I then tried to use GelfLayout.json with
> >>
> >> <Socket name="Logstash"
> >> host="\${sys:logstash.host:-host.docker.internal}"
> >> port="12222"
> >> protocol="tcp"
> >> bufferedIo="true">
> >> <JsonTemplateLayout eventTemplateUri="classpath:GelfLayout.json"
> >> eventDelimiter="\0"
> >> locationInfoEnabled="true">
> >> <EventTemplateAdditionalFields>
> >> <KeyValuePair key="hostName" value="${hostName}"/>
> >> <KeyValuePair key="_containerId" value="\${docker:containerId:-}"/>
> >> <KeyValuePair key="_application"
> >> value="$\${lower:\${spring:spring.application.name:-spring}}"/>
> >> </EventTemplateAdditionalFields>
> >> </JsonTemplateLayout>
> >> </Socket>
> >> but this did not work at all. Logstash does not seem to be getting a null
> >> character (I tried with both “\0” and “\u0000”) and all the events buffer
> >> in Logstash until I shutdown the application. Worse the formatting looks
> >> like escaped JSON rather than raw json.
> >>
> >> Also, the GelfTemplate contains
> >> {
> >> "version": "1.1",
> >> "host": "${hostName}",
> >> "short_message": "${json:message}",
> >> "full_message": "${json:exception:stackTrace:text}",
> >> "timestamp": "${json:timestamp:epoch:secs}",
> >> "level": "${json:level:severity:code}",
> >> "_logger": "${json:logger:name}",
> >> "_thread": "${json:thread:name}",
> >> "_mdc": "${json:mdc}"
> >> }
> >> This causes the output to contain
> >>
> >> \\\"_mdc\\\":{\\\"loginId\\\":\\\"rgoers\\\",\\\"requestId\\\":\\\"266196c1-9702-11ea-9e4a-0242ac120006\\\"}
> >>
> >> This isn’t valid in the Gelf spec as additional attributes in Gelf must
> >> match the regex - ^[\w\.\-]*$. This means you need to parse the MDC and
> >> add each key as an additional to be able to create valid Gelf. Personally,
> >> I’d like that option anyway.
> >>
> >> I would suggest you attempt to work to make a template that matches the
> >> result of the GelfLayout I had previously documented.
> >>
> >> But now that I understand what is going on I am OK with this contribution
> >> being merged. I would like to get these issues addressed and update the
> >> Logging in the Cloud documentation so that it can document both ways of
> >> sending to Logstash but we can do that after the code is merged.
> >>
> >> Ralph
> >>
> >>
> >>
> >>
> >>
> >>> On Apr 21, 2020, at 2:23 PM, Ralph Goers <ralph.go...@dslextreme.com>
> >>> wrote:
> >>>
> >>> Ok. Now that a release has been cut spending time with your contribution
> >>> is at the top of my list.
> >>>
> >>> Ralph
> >>>
> >>>> On Apr 21, 2020, at 1:24 PM, Volkan Yazıcı <volkan.yaz...@gmail.com>
> >>>> wrote:
> >>>>
> >>>> Ralph, I've just pushed a commit that adds PatternLayout support, FYI.
> >>>>
> >>>> On Mon, Apr 20, 2020 at 2:08 AM Ralph Goers <ralph.go...@dslextreme.com>
> >>>> wrote:
> >>>>>
> >>>>> I completely understand how messed up people’s lives are right now. I’m
> >>>>> lucky, all my kids are long gone out of the house. With COVID-19 I am
> >>>>> able to work from home and while that saves me over 2 hours a day in
> >>>>> commute time and more time because there isn’t much else to do, I end
> >>>>> up spending a lot of the extra time on my $dayjob (and feel lucky I
> >>>>> still have it).
> >>>>>
> >>>>> 1. You should care about the checkstyle errors as much as everyone else
> >>>>> ;-). They don’t seem to be a high priority for people to fix.
> >>>>>
> >>>>> 2. Maybe I didn’t make my need for the Pattern Layout clear. Adding the
> >>>>> pattern layout for the JTL is ONLY for the message attribute (just as I
> >>>>> did for the GelfLayout). This allows what is displayed in Kibana to
> >>>>> look like what you would see in the log file while still allowing
> >>>>> filtering on all the attributes. For me, this is a big deal - I won’t
> >>>>> use the Layout without it. That doesn’t mean it has to be included in
> >>>>> the initial commit. I don’t believe JsonLayout supports it either but
> >>>>> I am hoping to be able to do that today, although I have several other
> >>>>> things I need to do. What you would end up with is a new attribute
> >>>>> called messagePattern that takes a pattern that is exactly what you
> >>>>> would give to PatternLayout.
> >>>>>
> >>>>> 3. You can handle the performance issue however you want but if it was
> >>>>> me I would just revert back to using Jackson without a second thought.
> >>>>> I just don’t see the point of trying to roll your own.
> >>>>>
> >>>>> 4. It will take me a while to see what your integration test does and
> >>>>> if it resolves my concerns regarding updating the logging in the cloud
> >>>>> page.
> >>>>>
> >>>>> Ralph
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>> On Apr 19, 2020, at 2:30 PM, Volkan Yazıcı <volkan.yaz...@gmail.com>
> >>>>>> wrote:
> >>>>>>
> >>>>>> Hello Ralph,
> >>>>>>
> >>>>>> [Due to #COVID-19, day cares are closed. I can hardly spare time to
> >>>>>> work, including Log4j. Hence, apologies for my slowed down progress &
> >>>>>> responses.]
> >>>>>>
> >>>>>> I could not put my finger on the performance regression, yet. I might
> >>>>>> opt to switch back to Jackson, but I am not gonna do that without
> >>>>>> spotting that darn pain in the back. (You might have noticed, there is
> >>>>>> actually a class for generating a report out of the available
> >>>>>> JSON-emitting layouts. I will skip incorporating its output into the
> >>>>>> manually until I address this performance regression.)
> >>>>>>
> >>>>>> I've just pushed a commit, i.e., LogstashIT, which tests
> >>>>>> JsonTemplateLayout against ELK stack. Would you mind skimming through
> >>>>>> that (small) change, please?
> >>>>>>
> >>>>>> I am not inclined to incorporate PatternLayout (PL) into
> >>>>>> JsonTemplateLayout (JTL). JTL is supposed to (ideally) deliver all the
> >>>>>> functionality provided by PL but in a structured format, i.e., JSON.
> >>>>>> If there arises a need to fallback to PL, I'd rather add that missing
> >>>>>> functionality to JTL.
> >>>>>>
> >>>>>> You've mentioned about the checkstyle errors. How much shall I be
> >>>>>> concerned about them? (It even complains about missing JavaDocs on
> >>>>>> class fields.) Are there any particular ones you want me to address?
> >>>>>>
> >>>>>> Besides these... I think I am done.
> >>>>>>
> >>>>>> Kind regards.
> >>>>>>
> >>>>>> On Thu, Apr 16, 2020 at 10:49 PM Ralph Goers
> >>>>>> <ralph.go...@dslextreme.com> wrote:
> >>>>>>>
> >>>>>>> Volkan,
> >>>>>>>
> >>>>>>> I’m just wondering if you have had a chance to get to this. As I
> >>>>>>> said, as far as I am concerned this is the only thing getting in the
> >>>>>>> way of merging the PR. If you would prefer you can just remove the
> >>>>>>> update to the “Logging in the Cloud” page and push the rest and then
> >>>>>>> update that page after you have the testing done.
> >>>>>>>
> >>>>>>> Ralph
> >>>>>>>
> >>>>>>>> On Apr 12, 2020, at 12:01 AM, Ralph Goers
> >>>>>>>> <ralph.go...@dslextreme.com> wrote:
> >>>>>>>>
> >>>>>>>> I think I said this before, but I see no reason you can’t have a
> >>>>>>>> dependency on Jackson. We currently do for almost all of our JSON
> >>>>>>>> processing. I wouldn’t spend any more time trying to improve the
> >>>>>>>> performance of your custom support, especially since - as I recall -
> >>>>>>>> you had to drop support of a feature because it was hard to
> >>>>>>>> implement.
> >>>>>>>>
> >>>>>>>> As for replacing JsonLayout and GelfLayout, I would envision that
> >>>>>>>> those would continue to exist for backwards compatibility but would
> >>>>>>>> simply pass their configuration too JsonTemplateLayout.
> >>>>>>>>
> >>>>>>>> As for moving JSON out of core, we may end up doing that. I am
> >>>>>>>> pretty sure the xml support will require a dependency on the
> >>>>>>>> java.xml module so either that will move out of core or we will have
> >>>>>>>> a transitive dependency on it. Whichever way we go with that I would
> >>>>>>>> expect would be the same with Json and Yaml.
> >>>>>>>>
> >>>>>>>> Adding support for formatting the message with the PatternLayout was
> >>>>>>>> pretty trivial. I would prefer to continue to have that in the
> >>>>>>>> example so I would hope it could be added if the example changes.
> >>>>>>>>
> >>>>>>>> As I said, the issue is Logstash or Fluentd and how they process
> >>>>>>>> messages. Adding the null terminator in the configuration would make
> >>>>>>>> it work on the log4j side as long as you left the logstash
> >>>>>>>> configuration alone. My devops team has tried everything they can
> >>>>>>>> think of to get exceptions to work and this was the only thing that
> >>>>>>>> did without having to invent convoluted parsing rules. But yet,
> >>>>>>>> before we publish that I would need to know it had been tested all
> >>>>>>>> the way to Kibana. It was pretty easy for me to set it all up in
> >>>>>>>> docker containers. I think the log4j-spring-cloud-sample project has
> >>>>>>>> reference to the docker setup I used.
> >>>>>>>>
> >>>>>>>> Ralph
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>> On Apr 11, 2020, at 11:23 PM, Volkan Yazıcı
> >>>>>>>>> <volkan.yaz...@gmail.com> wrote:
> >>>>>>>>>
> >>>>>>>>> Hey Ralph,
> >>>>>>>>>
> >>>>>>>>> Thanks so much for taking time to review such a lengthy PR, much
> >>>>>>>>> appreciated. Let me try to address your concerns and share some of
> >>>>>>>>> mine:
> >>>>>>>>>
> >>>>>>>>> Does JTL (i.e., JsonTemplateLayout) support Logstash? I've tried it
> >>>>>>>>> locally against an actually running Logstash instance and it has
> >>>>>>>>> worked fine. Though I did not test newlines. I will do that. In any
> >>>>>>>>> case, AFAIC, JTL should also support null-termination via
> >>>>>>>>> eventDelimiter="\u0000" configuration. Nevertheless, I will try both
> >>>>>>>>> and report the outcome. (I wish we would support Docker in ITs to
> >>>>>>>>> allow tests against such external services, rather than us doing
> >>>>>>>>> that
> >>>>>>>>> manually.)
> >>>>>>>>>
> >>>>>>>>> Does "message" directive in JTL support patterns ala PatternLayout?
> >>>>>>>>> No, but it is on my TODO list.
> >>>>>>>>>
> >>>>>>>>> Can one extend the existing JTL template resolvers, i.e.,
> >>>>>>>>> EventResolverFactories? [I've extracted this remark from your most
> >>>>>>>>> recent GitHub comment.] No, but sounds like a great idea! This is
> >>>>>>>>> not
> >>>>>>>>> a necessity for the first release (as you also have stated), but I
> >>>>>>>>> will see what I can do about it.
> >>>>>>>>>
> >>>>>>>>> Right now I am busy with a performance regression. After replacing
> >>>>>>>>> the
> >>>>>>>>> Jackson JsonGenerator with the in-house JsonWriter, I've observed a
> >>>>>>>>> ~2x performance degradation. I've been busy with it for almost a
> >>>>>>>>> month, but it is really difficult to spare time for it due to
> >>>>>>>>> full-time parenting enforced by closed day cares. I do not want to
> >>>>>>>>> have a release before fixing this issue.
> >>>>>>>>>
> >>>>>>>>> I can imagine at some point 2.x will ship 3 contending JSON layouts:
> >>>>>>>>> GelfLayout, JsonLayout, and JsonTemplateLayout. That said, for 3.x I
> >>>>>>>>> suggest to deprecate (maybe even remove?) the former two.
> >>>>>>>>> Personally,
> >>>>>>>>> as a developer, what I find most confusing is a library providing
> >>>>>>>>> multiple solutions for the same problem without much tangible
> >>>>>>>>> benefit
> >>>>>>>>> between each other. I would rather strain from that dilemma in a
> >>>>>>>>> library that I contribute to.
> >>>>>>>>>
> >>>>>>>>> Speaking of 3.x enhancements I have in mind, I am in favor of
> >>>>>>>>> removing
> >>>>>>>>> all JSON logic (i.e., escaping/quoting routines) out of core module.
> >>>>>>>>> As a motivation, see my earlier post regarding discrepancies between
> >>>>>>>>> multiple JSON quoting methods. Further, many (incl. you) also have
> >>>>>>>>> stated their concerns about the size of "core". I don't think a
> >>>>>>>>> logger
> >>>>>>>>> should ship JSON encoders, decoders, and such. Though I did not
> >>>>>>>>> investigate this issue in detail, that is, what are the existing
> >>>>>>>>> dependencies, etc. Nevertheless, I wanted to raise this point.
> >>>>>>>>>
> >>>>>>>>> Kind regards.
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> On Sat, Apr 11, 2020 at 11:22 PM Ralph Goers
> >>>>>>>>> <ralph.go...@dslextreme.com> wrote:
> >>>>>>>>>>
> >>>>>>>>>> Volkan,
> >>>>>>>>>>
> >>>>>>>>>> I have looked at the PR and the only remaining thing I have an
> >>>>>>>>>> issue with are the changes you made to the logging in the cloud
> >>>>>>>>>> documentation. In my testing, only the Gelf Layout worked properly
> >>>>>>>>>> with an ELK stack. This is because Logstash (as well as Fluentd)
> >>>>>>>>>> are parsing the input trying to figure out where each log event
> >>>>>>>>>> begins and ends. They typically do that using some sort of pattern
> >>>>>>>>>> but by default look for newlines. This will cause log events that
> >>>>>>>>>> contain exceptions, which inherently include newlines) or other
> >>>>>>>>>> events that include newlines in the message to be broken into
> >>>>>>>>>> multiple lines in Kibana and make it impossible to filter on them
> >>>>>>>>>> properly.
> >>>>>>>>>>
> >>>>>>>>>> Have you tested the changes you have made to the documentation
> >>>>>>>>>> with a full ELK stack and verified that all log events, including
> >>>>>>>>>> events with exceptions, are displayed correctly?
> >>>>>>>>>>
> >>>>>>>>>> Also, you will notice that the Gelf Layout includes a
> >>>>>>>>>> MessagePattern attribute that allows the message element to be
> >>>>>>>>>> formatted using the PatternLayout. Does the Json Template Layout
> >>>>>>>>>> support that? I don’t see examples of that in any of the template
> >>>>>>>>>> files if it does. If it does not, that is another reason I would
> >>>>>>>>>> prefer leaving the page as is for now.
> >>>>>>>>>>
> >>>>>>>>>> Ralph
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>
> >>>>>
> >>>>
> >>>
> >>>
> >>>
> >>
> >
>
