Somebody mind giving me write access to the GitHub repo, please? I am
not able to merge the PR.
On Fri, May 22, 2020 at 3:53 PM Apache <ralph.go...@dslextreme.com> wrote:
>
> Feel free to merge it. I will test it there when I can.
>
> Ralph
>
>
> > On May 22, 2020, at 4:50 AM, Volkan Yazıcı <volkan.yaz...@gmail.com> wrote:
> >
> > Hey Ralph,
> >
> > Here is my status update:
> >
> > -~- Benchmarks -~-
> >
> > I have removed the benchmarks results. It takes ~8h for a complete run
> > and I don't want to repeat that cycle after every change. I might
> > revisit this idea once JTL becomes more stable in terms of its
> > features.
> >
> > -~- Flattening of MDC fields -~-
> >
> > I have changed the MDC directive as follows:
> >
> > mdc
> > mdc:flatten[=<prefix>][,stringify]
> > mdc:pattern=<pattern>[,flatten=<prefix>][,stringify]
> > mdc:key=<key>[,stringify]
> >
> > This also allowed JsonTemplateLayout to produce the *exact* output as
> > GelfLayout and EcsLayout, where MDC keys are flattened and values are
> > stringified.
> >
> > -~- Null termination, newlines, Logstash, and etc. -~-
> >
> > I have tried to reproduce the experiment you have shared, though could
> > not really succeed due to some Docker command failure in restartApp.sh.
> > Further, I find the associated Spring setup quite cumbersome to wrap my
> > mind around it. That said, I have done something else: I have improved
> > LogstashIT such that
> >
> > 1. Logstash is configured with both "tcp" and "gelf" input plugins,
> >
> > 2. LogstashIT employs JsonTemplateLayout against both inputs, repeats
> > the same using GelfLayout and EcsLayout, and verifies the populated
> > content in Elasticsearch.
> >
> > One can easily validate this on the branch as follows:
> >
> > $ ./mvnw clean package -DskipTests
> > $ ./mvnw \
> > verify -o -P docker \
> > -pl
> > log4j-layout-jackson-json,log4j-plugins,log4j-core,log4j-api,log4j-layout-json-template
> > \
> > -Dtest="Dummy.java" -DtrimStackTrace=false -DfailIfNoTests=false
> >
> > One can repeat the very same by running LogstashIT in IDE after
> > starting the containers:
> >
> > $ ./mvnw \
> > docker:start -o -P docker \
> > -pl
> > log4j-layout-jackson-json,log4j-plugins,log4j-core,log4j-api,log4j-layout-json-template
> >
> > I don't know what is wrong with the Spring Cloud Config setup you have
> > shared, but I have no reasons to believe that JsonTemplateLayout is the
> > suspect.
> >
> > Regarding your remark about everything getting escaped... This might
> > happen when Logstash fails to read an input. In such a case, it puts
> > the entire payload into the "message" field (hence, the escaping) and
> > stores it like that.
> >
> > From now on, I don't know how to proceed with this problem, in
> > particular, given I believe the problem is in somewhere else but
> > JsonTemplateLayout.
> >
> > -~- Merging branch to master -~-
> >
> > Unless there are objections, I want to merge the branch to master.
> > There on I will share json-template-layout.md with the community to
> > collect some feedback on the API. In particular, I have existing users
> > of LogstashLayout in mind.
> >
> > Kind regards.
> >
> >> On Mon, May 18, 2020 at 1:18 AM Ralph Goers <ralph.go...@dslextreme.com>
> >> wrote:
> >>
> >>
> >>
> >>>> On May 17, 2020, at 2:37 PM, Volkan Yazıcı <volkan.yaz...@gmail.com>
> >>>> wrote:
> >>>
> >>> Thanks so much for the thorough review Ralph, really appreciated! I
> >>> will address issues you have raised.
> >>>
> >>> [As a side note, I have pushed changes containing performance
> >>> improvements and benchmark results. The module is still dependency
> >>> free and performance-wise pretty good.]
> >>>
> >>>> all the default templates separate the message from the exception
> >>>
> >>> Yes, this I have also realized while studying the source code of
> >>> GelfLayout. Some random thoughts:
> >>>
> >>> 1. How about introducing a firstNonNull-like operator:
> >>> ${json:op:firstNonNull:${json:message},${json:exception:message}}.
> >>> Parsing this would not be trivial, given the list of variables
> >>> can include comma in various forms. Though the rest should
> >>> be tractable. (Feels like JsonTemplateLayout directives will at
> >>> some point catch up the with ones in PatternLayout.)
> >>
> >> The problem here is that I want newlines in the message. I’m not sure what
> >> firstNonNull means in the context of a message. The message itself won’t
> >> have nulls. The Gelf format dictates the null is at the end of the JSON
> >> string.
> >>
> >>>
> >>> 2. How about introducing a fallback parameter to the "message"
> >>> directive: ${json:message:fallback=exception:message}.
> >>
> >> Again, I don’t know what that means or how it solves my problem of wanting
> >> newlines in the message.
> >>
> >>>
> >>> 3. May I challenge the request: Do one really need it? You store
> >>> individual values, i.e, message and exception, in individual fields.
> >>> When one is missing and the other is present, isn't it just a
> >>> presentation layer issue to display it properly?
> >>
> >> Picture how a log event looks in a file with a “standard” PatternLayout.
> >> In Kibana by default you get 3 columns, the timestamp, a field I don’t
> >> care about and the message. I want the message to appear just as it would
> >> in a log file without the timestamp. Yes, I can click on each message and
> >> see its attributes, but that takes longer. And I want exceptions to jump
> >> out at me when I am visually scrolling through messages. In fact, I
> >> typically don’t want the stack trace in a variable of its own. Kibana
> >> doesn’t take the pieces of the message and put them together like the
> >> pattern layout does. Instead it lets you create columns for individual
> >> attributes. The problem with that is the columns waste a lot of space,
> >> especially if they frequently are empty as a stack trace would be.
> >>
> >> So yes, I really need it.
> >>
> >>>
> >>>> In the logged event all newlines seem to be stripped from the exception.
> >>>
> >>> I really don't get this Ralph. Have you seen the tests in LogstashIT?
> >>> I have just added a new one testing only newlines. Would you mind
> >>> fiddling around with LogstashIT to reproduce the issue, please? Or
> >>> sharing a recipe for me to reproduce it?
> >>
> >> The way I do it is to install Logstash, ElasticSearch and Kibana locally
> >> on my Mac as well as Docker Desktop for Mac. I then run SpringCloud config
> >> server from the Log4j Spring Cloud Config project and tailor the
> >> log4j2.xml that is in the config-repo directory of that project. Note that
> >> all Log4j Lookup variables have to have the $ escaped or Spring Cloud
> >> Config will try to resolve them. It also requires having the Socket
> >> appender connect to a host of “host.docker.internal” to be able to connect
> >> to Logstash outside of the docker container. I configure Logstash using
> >> the configuration documented at
> >> http://logging.apache.org/log4j/2.x/manual/cloud.html
> >> <http://logging.apache.org/log4j/2.x/manual/cloud.html>, at least when
> >> using Gelf. I configure Kibana with an index of app-*.
> >>
> >> I then modifythe Spring Cloud Config Sample app as needed to work with
> >> this setup and start it using docker/restartApp.sh.
> >>
> >> After starting I do a few things:
> >> 1. Enter “docker logs app-container”. This should show it is running and
> >> the last message will be the Spring logo.
> >> 2. Look in Kibana and click on the logs. You should see logs from the
> >> server.
> >> 3. In a browser enter http://localhost:8080/sample/exception
> >> <http://localhost:8080/sample/exception> - unless you have changed the
> >> port. For some reason I had something running on 8080 and had to change
> >> the sample app to use 8090. This will generate a stack trace on the web
> >> page and in the log.
> >> 4. Look in Kibana. You should see the exception and the stack trace
> >> properly formatted in the message column. It should also display the
> >> thread id, log-level and other items configured in the pattern.
> >>
> >>>
> >>>> Logstash does not seem to be getting a null character
> >>>> (I tried with both “\0” and “\u0000”)
> >>>
> >>> This is weird. Below is how GelfLayout does it:
> >>>
> >>> if (includeNullDelimiter) {
> >>> builder.append('\0');
> >>> }
> >>>
> >>> And here is how JsonTemplateLayout does it:
> >>>
> >>> stringBuilder.append(eventDelimiter);
> >>>
> >>> Further, there is even a test for this:
> >>> JsonTemplateLayout#test_null_eventDelimiter():
> >>>
> >>> final String serializedLogEvent = layout.toSerializable(logEvent);
> >>> assertThat(serializedLogEvent).isEqualTo(eventTemplate + '\0');
> >>>
> >>> Any ideas on what might I be missing?
> >>
> >> No - I just know that Logstash wasn’t detecting the null. It looked like
> >> the whole string was being escaped as all the double quotes were also
> >> escaped as you can see from the example. I should note that I directly
> >> copied that from the Logstash log that was written to the terminal window
> >> with debug logging enabled, so it is possible it was caused by that, but
> >> this is also what I see in Kibana.
> >>
> >>>
> >>>> This isn’t valid in the Gelf spec as additional attributes in Gelf
> >>>> must match the regex - ^[\w\.\-]*$. This means you need to
> >>>> parse the MDC and add each key as an additional to be able
> >>>> to create valid Gelf. Personally, I’d like that option anyway.
> >>>
> >>> For this purpose, I need to introduce an operator similar to
> >>> unquote-splicing in Lisp to merge the contents of the child with the
> >>> parent. That said, I have my doubts about the practical benefit of
> >>> such a feature. To the best of my knowledge, almost every (sanely
> >>> configured?) log pipeline ends up with a structured storage system
> >>> containing whitelisted fields. Consider ELK stack. Would you allow
> >>> developers to introduce fields at their will? If so, it is a matter of
> >>> time somebody will take down the entire Elasticsearch cluster by
> >>> flooding it with ContextData.put(userInput, "doh"). Hence, you employ
> >>> the identical whitelisting strategy at the layout as well:
> >>>
> >>> {
> >>> "version": "1.1",
> >>> "host": "${hostName}",
> >>> ...,
> >>> "_mdc.loginId": "${json:mdc:loginId}",
> >>> "_mdc.requestId": "${json:mdc:requestId}"
> >>> }
> >>>
> >>> What do you think? Shall we really introduce unquote-splicing?
> >>
> >> I would have preferred a syntax like
> >>
> >> {
> >> “version” “1.1”,
> >> …
> >> ${json.map:mdc:prefix=“_”}
> >> }
> >>
> >> This would cause the whole map to be emitted. Although it would be nice to
> >> have a version of that which also could specify the keys there probably
> >> isn’t a point to that since you can directly specify them as you have
> >> shown.
> >>
> >>
> >> As I said previously, now that I understand what is going on I am
> >> comfortable with you merging the PR. It is going to the master branch so
> >> it won’t be what users see on the web site right now. By the time it makes
> >> it there we can get this all straightened out. If nothing else we could
> >> have the current GelfLayout and the JsonTemplateLayout both on the page
> >> and explain the differences between them, although in the end I agree it
> >> would be better to have the JsonTemplateLayout replace both the JsonLayout
> >> and GelfLayout.
> >>
> >> Ralph
> >>
> >>>
> >>> Kind regards.
> >>>
> >>> On Sat, May 16, 2020 at 1:28 AM Ralph Goers <ralph.go...@dslextreme.com>
> >>> wrote:
> >>>>
> >>>> I finally got time to spend looking at the Layout. I now understand why
> >>>> it works for you and sort of understand why it doesn’t work for me.
> >>>>
> >>>> First, all the default templates separate the message from the
> >>>> exception. In the logged event all newlines seem to be stripped from the
> >>>> exception. This makes viewing them in Kibana pretty awful, but the do
> >>>> get logged successfully.
> >>>>
> >>>> To include newlines in a log event you either have to play games in
> >>>> Logstash (or Fluentd) and try to put the event back together again after
> >>>> it has been separated into multiple lines or you have to use a different
> >>>> delimiter. Gelf specifies that a null character be used.
> >>>>
> >>>> I then tried to use GelfLayout.json with
> >>>>
> >>>> <Socket name="Logstash"
> >>>> host="\${sys:logstash.host:-host.docker.internal}"
> >>>> port="12222"
> >>>> protocol="tcp"
> >>>> bufferedIo="true">
> >>>> <JsonTemplateLayout eventTemplateUri="classpath:GelfLayout.json"
> >>>> eventDelimiter="\0"
> >>>> locationInfoEnabled="true">
> >>>> <EventTemplateAdditionalFields>
> >>>> <KeyValuePair key="hostName" value="${hostName}"/>
> >>>> <KeyValuePair key="_containerId" value="\${docker:containerId:-}"/>
> >>>> <KeyValuePair key="_application"
> >>>> value="$\${lower:\${spring:spring.application.name:-spring}}"/>
> >>>> </EventTemplateAdditionalFields>
> >>>> </JsonTemplateLayout>
> >>>> </Socket>
> >>>> but this did not work at all. Logstash does not seem to be getting a
> >>>> null character (I tried with both “\0” and “\u0000”) and all the events
> >>>> buffer in Logstash until I shutdown the application. Worse the
> >>>> formatting looks like escaped JSON rather than raw json.
> >>>>
> >>>> Also, the GelfTemplate contains
> >>>> {
> >>>> "version": "1.1",
> >>>> "host": "${hostName}",
> >>>> "short_message": "${json:message}",
> >>>> "full_message": "${json:exception:stackTrace:text}",
> >>>> "timestamp": "${json:timestamp:epoch:secs}",
> >>>> "level": "${json:level:severity:code}",
> >>>> "_logger": "${json:logger:name}",
> >>>> "_thread": "${json:thread:name}",
> >>>> "_mdc": "${json:mdc}"
> >>>> }
> >>>> This causes the output to contain
> >>>>
> >>>> \\\"_mdc\\\":{\\\"loginId\\\":\\\"rgoers\\\",\\\"requestId\\\":\\\"266196c1-9702-11ea-9e4a-0242ac120006\\\"}
> >>>>
> >>>> This isn’t valid in the Gelf spec as additional attributes in Gelf must
> >>>> match the regex - ^[\w\.\-]*$. This means you need to parse the MDC and
> >>>> add each key as an additional to be able to create valid Gelf.
> >>>> Personally, I’d like that option anyway.
> >>>>
> >>>> I would suggest you attempt to work to make a template that matches the
> >>>> result of the GelfLayout I had previously documented.
> >>>>
> >>>> But now that I understand what is going on I am OK with this
> >>>> contribution being merged. I would like to get these issues addressed
> >>>> and update the Logging in the Cloud documentation so that it can
> >>>> document both ways of sending to Logstash but we can do that after the
> >>>> code is merged.
> >>>>
> >>>> Ralph
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>> On Apr 21, 2020, at 2:23 PM, Ralph Goers <ralph.go...@dslextreme.com>
> >>>>> wrote:
> >>>>>
> >>>>> Ok. Now that a release has been cut spending time with your
> >>>>> contribution is at the top of my list.
> >>>>>
> >>>>> Ralph
> >>>>>
> >>>>>> On Apr 21, 2020, at 1:24 PM, Volkan Yazıcı <volkan.yaz...@gmail.com>
> >>>>>> wrote:
> >>>>>>
> >>>>>> Ralph, I've just pushed a commit that adds PatternLayout support, FYI.
> >>>>>>
> >>>>>> On Mon, Apr 20, 2020 at 2:08 AM Ralph Goers
> >>>>>> <ralph.go...@dslextreme.com> wrote:
> >>>>>>>
> >>>>>>> I completely understand how messed up people’s lives are right now.
> >>>>>>> I’m lucky, all my kids are long gone out of the house. With COVID-19
> >>>>>>> I am able to work from home and while that saves me over 2 hours a
> >>>>>>> day in commute time and more time because there isn’t much else to
> >>>>>>> do, I end up spending a lot of the extra time on my $dayjob (and feel
> >>>>>>> lucky I still have it).
> >>>>>>>
> >>>>>>> 1. You should care about the checkstyle errors as much as everyone
> >>>>>>> else ;-). They don’t seem to be a high priority for people to fix.
> >>>>>>>
> >>>>>>> 2. Maybe I didn’t make my need for the Pattern Layout clear. Adding
> >>>>>>> the pattern layout for the JTL is ONLY for the message attribute
> >>>>>>> (just as I did for the GelfLayout). This allows what is displayed in
> >>>>>>> Kibana to look like what you would see in the log file while still
> >>>>>>> allowing filtering on all the attributes. For me, this is a big deal
> >>>>>>> - I won’t use the Layout without it. That doesn’t mean it has to be
> >>>>>>> included in the initial commit. I don’t believe JsonLayout supports
> >>>>>>> it either but I am hoping to be able to do that today, although I
> >>>>>>> have several other things I need to do. What you would end up with is
> >>>>>>> a new attribute called messagePattern that takes a pattern that is
> >>>>>>> exactly what you would give to PatternLayout.
> >>>>>>>
> >>>>>>> 3. You can handle the performance issue however you want but if it
> >>>>>>> was me I would just revert back to using Jackson without a second
> >>>>>>> thought. I just don’t see the point of trying to roll your own.
> >>>>>>>
> >>>>>>> 4. It will take me a while to see what your integration test does and
> >>>>>>> if it resolves my concerns regarding updating the logging in the
> >>>>>>> cloud page.
> >>>>>>>
> >>>>>>> Ralph
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>> On Apr 19, 2020, at 2:30 PM, Volkan Yazıcı <volkan.yaz...@gmail.com>
> >>>>>>>> wrote:
> >>>>>>>>
> >>>>>>>> Hello Ralph,
> >>>>>>>>
> >>>>>>>> [Due to #COVID-19, day cares are closed. I can hardly spare time to
> >>>>>>>> work, including Log4j. Hence, apologies for my slowed down progress &
> >>>>>>>> responses.]
> >>>>>>>>
> >>>>>>>> I could not put my finger on the performance regression, yet. I might
> >>>>>>>> opt to switch back to Jackson, but I am not gonna do that without
> >>>>>>>> spotting that darn pain in the back. (You might have noticed, there
> >>>>>>>> is
> >>>>>>>> actually a class for generating a report out of the available
> >>>>>>>> JSON-emitting layouts. I will skip incorporating its output into the
> >>>>>>>> manually until I address this performance regression.)
> >>>>>>>>
> >>>>>>>> I've just pushed a commit, i.e., LogstashIT, which tests
> >>>>>>>> JsonTemplateLayout against ELK stack. Would you mind skimming through
> >>>>>>>> that (small) change, please?
> >>>>>>>>
> >>>>>>>> I am not inclined to incorporate PatternLayout (PL) into
> >>>>>>>> JsonTemplateLayout (JTL). JTL is supposed to (ideally) deliver all
> >>>>>>>> the
> >>>>>>>> functionality provided by PL but in a structured format, i.e., JSON.
> >>>>>>>> If there arises a need to fallback to PL, I'd rather add that missing
> >>>>>>>> functionality to JTL.
> >>>>>>>>
> >>>>>>>> You've mentioned about the checkstyle errors. How much shall I be
> >>>>>>>> concerned about them? (It even complains about missing JavaDocs on
> >>>>>>>> class fields.) Are there any particular ones you want me to address?
> >>>>>>>>
> >>>>>>>> Besides these... I think I am done.
> >>>>>>>>
> >>>>>>>> Kind regards.
> >>>>>>>>
> >>>>>>>> On Thu, Apr 16, 2020 at 10:49 PM Ralph Goers
> >>>>>>>> <ralph.go...@dslextreme.com> wrote:
> >>>>>>>>>
> >>>>>>>>> Volkan,
> >>>>>>>>>
> >>>>>>>>> I’m just wondering if you have had a chance to get to this. As I
> >>>>>>>>> said, as far as I am concerned this is the only thing getting in
> >>>>>>>>> the way of merging the PR. If you would prefer you can just remove
> >>>>>>>>> the update to the “Logging in the Cloud” page and push the rest and
> >>>>>>>>> then update that page after you have the testing done.
> >>>>>>>>>
> >>>>>>>>> Ralph
> >>>>>>>>>
> >>>>>>>>>> On Apr 12, 2020, at 12:01 AM, Ralph Goers
> >>>>>>>>>> <ralph.go...@dslextreme.com> wrote:
> >>>>>>>>>>
> >>>>>>>>>> I think I said this before, but I see no reason you can’t have a
> >>>>>>>>>> dependency on Jackson. We currently do for almost all of our JSON
> >>>>>>>>>> processing. I wouldn’t spend any more time trying to improve the
> >>>>>>>>>> performance of your custom support, especially since - as I recall
> >>>>>>>>>> - you had to drop support of a feature because it was hard to
> >>>>>>>>>> implement.
> >>>>>>>>>>
> >>>>>>>>>> As for replacing JsonLayout and GelfLayout, I would envision that
> >>>>>>>>>> those would continue to exist for backwards compatibility but
> >>>>>>>>>> would simply pass their configuration too JsonTemplateLayout.
> >>>>>>>>>>
> >>>>>>>>>> As for moving JSON out of core, we may end up doing that. I am
> >>>>>>>>>> pretty sure the xml support will require a dependency on the
> >>>>>>>>>> java.xml module so either that will move out of core or we will
> >>>>>>>>>> have a transitive dependency on it. Whichever way we go with that
> >>>>>>>>>> I would expect would be the same with Json and Yaml.
> >>>>>>>>>>
> >>>>>>>>>> Adding support for formatting the message with the PatternLayout
> >>>>>>>>>> was pretty trivial. I would prefer to continue to have that in the
> >>>>>>>>>> example so I would hope it could be added if the example changes.
> >>>>>>>>>>
> >>>>>>>>>> As I said, the issue is Logstash or Fluentd and how they process
> >>>>>>>>>> messages. Adding the null terminator in the configuration would
> >>>>>>>>>> make it work on the log4j side as long as you left the logstash
> >>>>>>>>>> configuration alone. My devops team has tried everything they can
> >>>>>>>>>> think of to get exceptions to work and this was the only thing
> >>>>>>>>>> that did without having to invent convoluted parsing rules. But
> >>>>>>>>>> yet, before we publish that I would need to know it had been
> >>>>>>>>>> tested all the way to Kibana. It was pretty easy for me to set it
> >>>>>>>>>> all up in docker containers. I think the log4j-spring-cloud-sample
> >>>>>>>>>> project has reference to the docker setup I used.
> >>>>>>>>>>
> >>>>>>>>>> Ralph
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>> On Apr 11, 2020, at 11:23 PM, Volkan Yazıcı
> >>>>>>>>>>> <volkan.yaz...@gmail.com> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>> Hey Ralph,
> >>>>>>>>>>>
> >>>>>>>>>>> Thanks so much for taking time to review such a lengthy PR, much
> >>>>>>>>>>> appreciated. Let me try to address your concerns and share some of
> >>>>>>>>>>> mine:
> >>>>>>>>>>>
> >>>>>>>>>>> Does JTL (i.e., JsonTemplateLayout) support Logstash? I've tried
> >>>>>>>>>>> it
> >>>>>>>>>>> locally against an actually running Logstash instance and it has
> >>>>>>>>>>> worked fine. Though I did not test newlines. I will do that. In
> >>>>>>>>>>> any
> >>>>>>>>>>> case, AFAIC, JTL should also support null-termination via
> >>>>>>>>>>> eventDelimiter="\u0000" configuration. Nevertheless, I will try
> >>>>>>>>>>> both
> >>>>>>>>>>> and report the outcome. (I wish we would support Docker in ITs to
> >>>>>>>>>>> allow tests against such external services, rather than us doing
> >>>>>>>>>>> that
> >>>>>>>>>>> manually.)
> >>>>>>>>>>>
> >>>>>>>>>>> Does "message" directive in JTL support patterns ala
> >>>>>>>>>>> PatternLayout?
> >>>>>>>>>>> No, but it is on my TODO list.
> >>>>>>>>>>>
> >>>>>>>>>>> Can one extend the existing JTL template resolvers, i.e.,
> >>>>>>>>>>> EventResolverFactories? [I've extracted this remark from your most
> >>>>>>>>>>> recent GitHub comment.] No, but sounds like a great idea! This is
> >>>>>>>>>>> not
> >>>>>>>>>>> a necessity for the first release (as you also have stated), but I
> >>>>>>>>>>> will see what I can do about it.
> >>>>>>>>>>>
> >>>>>>>>>>> Right now I am busy with a performance regression. After
> >>>>>>>>>>> replacing the
> >>>>>>>>>>> Jackson JsonGenerator with the in-house JsonWriter, I've observed
> >>>>>>>>>>> a
> >>>>>>>>>>> ~2x performance degradation. I've been busy with it for almost a
> >>>>>>>>>>> month, but it is really difficult to spare time for it due to
> >>>>>>>>>>> full-time parenting enforced by closed day cares. I do not want to
> >>>>>>>>>>> have a release before fixing this issue.
> >>>>>>>>>>>
> >>>>>>>>>>> I can imagine at some point 2.x will ship 3 contending JSON
> >>>>>>>>>>> layouts:
> >>>>>>>>>>> GelfLayout, JsonLayout, and JsonTemplateLayout. That said, for
> >>>>>>>>>>> 3.x I
> >>>>>>>>>>> suggest to deprecate (maybe even remove?) the former two.
> >>>>>>>>>>> Personally,
> >>>>>>>>>>> as a developer, what I find most confusing is a library providing
> >>>>>>>>>>> multiple solutions for the same problem without much tangible
> >>>>>>>>>>> benefit
> >>>>>>>>>>> between each other. I would rather strain from that dilemma in a
> >>>>>>>>>>> library that I contribute to.
> >>>>>>>>>>>
> >>>>>>>>>>> Speaking of 3.x enhancements I have in mind, I am in favor of
> >>>>>>>>>>> removing
> >>>>>>>>>>> all JSON logic (i.e., escaping/quoting routines) out of core
> >>>>>>>>>>> module.
> >>>>>>>>>>> As a motivation, see my earlier post regarding discrepancies
> >>>>>>>>>>> between
> >>>>>>>>>>> multiple JSON quoting methods. Further, many (incl. you) also have
> >>>>>>>>>>> stated their concerns about the size of "core". I don't think a
> >>>>>>>>>>> logger
> >>>>>>>>>>> should ship JSON encoders, decoders, and such. Though I did not
> >>>>>>>>>>> investigate this issue in detail, that is, what are the existing
> >>>>>>>>>>> dependencies, etc. Nevertheless, I wanted to raise this point.
> >>>>>>>>>>>
> >>>>>>>>>>> Kind regards.
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> On Sat, Apr 11, 2020 at 11:22 PM Ralph Goers
> >>>>>>>>>>> <ralph.go...@dslextreme.com> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>> Volkan,
> >>>>>>>>>>>>
> >>>>>>>>>>>> I have looked at the PR and the only remaining thing I have an
> >>>>>>>>>>>> issue with are the changes you made to the logging in the cloud
> >>>>>>>>>>>> documentation. In my testing, only the Gelf Layout worked
> >>>>>>>>>>>> properly with an ELK stack. This is because Logstash (as well as
> >>>>>>>>>>>> Fluentd) are parsing the input trying to figure out where each
> >>>>>>>>>>>> log event begins and ends. They typically do that using some
> >>>>>>>>>>>> sort of pattern but by default look for newlines. This will
> >>>>>>>>>>>> cause log events that contain exceptions, which inherently
> >>>>>>>>>>>> include newlines) or other events that include newlines in the
> >>>>>>>>>>>> message to be broken into multiple lines in Kibana and make it
> >>>>>>>>>>>> impossible to filter on them properly.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Have you tested the changes you have made to the documentation
> >>>>>>>>>>>> with a full ELK stack and verified that all log events,
> >>>>>>>>>>>> including events with exceptions, are displayed correctly?
> >>>>>>>>>>>>
> >>>>>>>>>>>> Also, you will notice that the Gelf Layout includes a
> >>>>>>>>>>>> MessagePattern attribute that allows the message element to be
> >>>>>>>>>>>> formatted using the PatternLayout. Does the Json Template Layout
> >>>>>>>>>>>> support that? I don’t see examples of that in any of the
> >>>>>>>>>>>> template files if it does. If it does not, that is another
> >>>>>>>>>>>> reason I would prefer leaving the page as is for now.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Ralph
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>
> >>>
> >>
> >
>
>
