This is an automated email from the ASF dual-hosted git repository. vy pushed a commit to branch release-2.x in repository https://gitbox.apache.org/repos/asf/logging-log4j2.git
commit 20d5463ab4114b12c4c95a8704fd5cbabfc17949 Author: Volkan Yazıcı <[email protected]> AuthorDate: Tue Aug 25 09:22:00 2020 +0200 #335 Migrate JsonTemplateLayout manual from Xdoc to AsciiDoc. --- ...te-layout.adoc => json-template-layout.adoc.vm} | 344 ++--- src/site/xdoc/manual/json-template-layout.xml.vm | 1526 -------------------- 2 files changed, 177 insertions(+), 1693 deletions(-) diff --git a/src/site/asciidoc/manual/json-template-layout.adoc b/src/site/asciidoc/manual/json-template-layout.adoc.vm similarity index 58% rename from src/site/asciidoc/manual/json-template-layout.adoc rename to src/site/asciidoc/manual/json-template-layout.adoc.vm index 1be3f5f..c2da984 100644 --- a/src/site/asciidoc/manual/json-template-layout.adoc +++ b/src/site/asciidoc/manual/json-template-layout.adoc.vm @@ -17,33 +17,33 @@ = JSON Template Layout Volkan Yazıcı <[email protected]> -<code>JsonTemplateLayout</code> is a customizable, efficient, and garbage-free JSON -emitting layout. It encodes <code>LogEvent</code>s according to the structure described +`JsonTemplateLayout` is a customizable, efficient, and garbage-free JSON +emitting layout. It encodes ``LogEvent``s according to the structure described by the JSON template provided. In a nutshell, it shines with its -* Customizable JSON structure (see <code>eventTemplate[Uri]</code> and - <code>stackTraceElementTemplate[Uri]</code> parameters) +* Customizable JSON structure (see `eventTemplate[Uri]` and + `stackTraceElementTemplate[Uri]` parameters) -* Customizable timestamp formatting (see <code>timestamp</code> parameter) +* Customizable timestamp formatting (see `timestamp` parameter) [#usage] == Usage -Adding <code>log4j-layout-json-template</code> artifact to your list of dependencies is -enough to enable access to <code>JsonTemplateLayout</code> in your Log4j configuration: +Adding `log4j-layout-json-template` artifact to your list of dependencies is +enough to enable access to `JsonTemplateLayout` in your Log4j configuration: [source,xml] ---- <dependency> <groupId>org.apache.logging.log4j</groupId> <artifactId>log4j-layout-json-template</artifactId> - <version>${log4j.version}</version> + <version>${Log4jReleaseVersion}</version> </dependency> ---- For instance, given the following JSON template modelling the https://github.com/logstash/log4j-jsonevent-layout[the official Logstash -<code>JSONEventLayoutV1</code>] (accessible via <code>classpath:LogstashJsonEventLayoutV1.json</code>) +`JSONEventLayoutV1`] (accessible via `classpath:LogstashJsonEventLayoutV1.json`) [source,json] ---- @@ -107,14 +107,14 @@ https://github.com/logstash/log4j-jsonevent-layout[the official Logstash } ---- -in combination with the below <code>log4j2.xml</code> configuration: +in combination with the below `log4j2.xml` configuration: [source,xml] ---- <JsonTemplateLayout eventTemplateUri="classpath:LogstashJsonEventLayoutV1.json"/> ---- -or with the below <code>log4j2.properties</code> configuration: +or with the below `log4j2.properties` configuration: [source,ini] ---- @@ -122,7 +122,7 @@ appender.console.json.type = JsonTemplateLayout appender.console.json.eventTemplateUri = classpath:LogstashJsonEventLayoutV1.json ---- -<code>JsonTemplateLayout</code> emits JSON strings as follows: +`JsonTemplateLayout` emits JSON strings as follows: [source,json] ---- @@ -149,9 +149,9 @@ appender.console.json.eventTemplateUri = classpath:LogstashJsonEventLayoutV1.jso [#layout-config] == Layout Configuration -<code>JsonTemplateLayout</code> is configured with the following parameters: +`JsonTemplateLayout` is configured with the following parameters: -.<code>JsonTemplateLayout</code> parameters +.`JsonTemplateLayout` parameters [cols="1m,1m,4"] |=== | Parameter Name @@ -160,29 +160,29 @@ appender.console.json.eventTemplateUri = classpath:LogstashJsonEventLayoutV1.jso | charset | Charset -| <code>Charset</code> used for <code>String</code> encoding +| `Charset` used for `String` encoding | locationInfoEnabled | boolean -| toggles access to the <code>LogEvent</code> source; file name, line number, etc. - (defaults to <code>false</code> set by <code>log4j.layout.jsonTemplate.locationInfoEnabled</code> +| toggles access to the `LogEvent` source; file name, line number, etc. + (defaults to `false` set by `log4j.layout.jsonTemplate.locationInfoEnabled` property) | stackTraceEnabled | boolean -| toggles access to the stack traces (defaults to <code>true</code> set by - <code>log4j.layout.jsonTemplate.stackTraceEnabled</code> property) +| toggles access to the stack traces (defaults to `true` set by + `log4j.layout.jsonTemplate.stackTraceEnabled` property) | eventTemplate | String -| inline JSON template for rendering <code>LogEvent</code>s (has priority over - <code>eventTemplateUri</code>, defaults to <code>null</code> set by - <code>log4j.layout.jsonTemplate.eventTemplate</code> property) +| inline JSON template for rendering ``LogEvent``s (has priority over + `eventTemplateUri`, defaults to `null` set by + `log4j.layout.jsonTemplate.eventTemplate` property) | eventTemplateUri | String -| URI pointing to the JSON template for rendering <code>LogEvent</code>s (defaults to - <code>classpath:EcsLayout.json</code> set by <code>log4j.layout.jsonTemplate.eventTemplateUri</code> +| URI pointing to the JSON template for rendering ``LogEvent``s (defaults to + `classpath:EcsLayout.json` set by `log4j.layout.jsonTemplate.eventTemplateUri` property) | eventTemplateAdditionalFields @@ -191,43 +191,43 @@ appender.console.json.eventTemplateUri = classpath:LogstashJsonEventLayoutV1.jso | stackTraceElementTemplate | String -| inline JSON template for rendering <code>StackTraceElement</code>s (has priority over - <code>stackTraceElementTemplateUri</code>, defaults to <code>null</code> set by - <code>log4j.layout.jsonTemplate.stackTraceElementTemplate</code> property) +| inline JSON template for rendering ``StackTraceElement``s (has priority over + `stackTraceElementTemplateUri`, defaults to `null` set by + `log4j.layout.jsonTemplate.stackTraceElementTemplate` property) | stackTraceElementTemplateUri | String -| JSON template for rendering <code>StackTraceElement</code>s (defaults to - <code>classpath:StackTraceElementLayout.json</code> set by - <code>log4j.layout.jsonTemplate.stackTraceElementTemplateUri</code> property) +| JSON template for rendering ``StackTraceElement``s (defaults to + `classpath:StackTraceElementLayout.json` set by + `log4j.layout.jsonTemplate.stackTraceElementTemplateUri` property) | eventDelimiter | String -| delimiter used for separating emitted <code>LogEvent</code>s (defaults to - <code>System.lineSeparator()</code> set by <code>log4j.layout.jsonTemplate.eventDelimiter</code> +| delimiter used for separating emitted ``LogEvent``s (defaults to + `System.lineSeparator()` set by `log4j.layout.jsonTemplate.eventDelimiter` property) | nullEventDelimiterEnabled | boolean -| append <code>\0</code> (<code>null</code>) character to the end of every emitted <code>eventDelimiter</code> - (defaults to <code>false</code> set by - <code>log4j.layout.jsonTemplate.nullEventDelimiterEnabled</code> property) +| append `\0` (`null`) character to the end of every emitted `eventDelimiter` + (defaults to `false` set by + `log4j.layout.jsonTemplate.nullEventDelimiterEnabled` property) | maxStringLength | int | truncate string values longer than the specified limit (defaults to 16384 set - by <code>log4j.layout.jsonTemplate.maxStringLength</code> property) + by `log4j.layout.jsonTemplate.maxStringLength` property) | truncatedStringSuffix | String -| suffix to append to strings truncated due to exceeding <code>maxStringLength</code> - (defaults to <code>…</code> set by <code>log4j.layout.jsonTemplate.truncatedStringSuffix</code> +| suffix to append to strings truncated due to exceeding `maxStringLength` + (defaults to `…` set by `log4j.layout.jsonTemplate.truncatedStringSuffix` property) | recyclerFactory | RecyclerFactory -| recycling strategy that can either be <code>dummy</code>, <code>threadLocal</code>, or <code>queue</code> - (set by <code>log4j.layout.jsonTemplate.recyclerFactory</code> property) +| recycling strategy that can either be `dummy`, `threadLocal`, or `queue` + (set by `log4j.layout.jsonTemplate.recyclerFactory` property) |=== [#additional-event-template-fields] @@ -235,7 +235,7 @@ appender.console.json.eventTemplateUri = classpath:LogstashJsonEventLayoutV1.jso Additional event template field is a convenient short-cut to add custom fields to a template or override the fields of a template. Following configuration -overrides the <code>host</code> field of the <code>GelfLayout.json</code> template and adds two new +overrides the `host` field of the `GelfLayout.json` template and adds two new custom fields: [source,xml] @@ -270,30 +270,30 @@ One can also pass JSON literals into additional fields: [#recycling-strategy] === Recycling strategy -<code>RecyclerFactory</code> plays a crucial role for determining the memory footprint of +`RecyclerFactory` plays a crucial role for determining the memory footprint of the layout. Template resolvers employ it to create recyclers for objects that -they can reuse. The function of each <code>RecyclerFactory</code> and when one should +they can reuse. The function of each `RecyclerFactory` and when one should prefer one over another is explained below: -* <code>dummy</code> performs no recycling, hence each recycling attempt will result in a +* `dummy` performs no recycling, hence each recycling attempt will result in a new instance. This will obviously create a load on the garbage-collector. It is a good choice for applications with low and medium log rate. -* <code>threadLocal</code> performs the best, since every instance is stored in -<code>ThreadLocal</code>s and accessed without any synchronization cost. Though this +* `threadLocal` performs the best, since every instance is stored in +``ThreadLocal``s and accessed without any synchronization cost. Though this might not be a desirable option for applications running with hundreds of threads or more, e.g., a web servlet. -* <code>queue</code> is the best of both worlds. It allows recycling of objects up to a -certain number (<code>capacity</code>). When this limit is exceeded due to excessive -concurrent load (e.g., <code>capacity</code> is 50 but there are 51 threads concurrently -trying to log), it starts allocating. <code>queue</code> is a good strategy where -<code>threadLocal</code> is not desirable. +* `queue` is the best of both worlds. It allows recycling of objects up to a +certain number (`capacity`). When this limit is exceeded due to excessive +concurrent load (e.g., `capacity` is 50 but there are 51 threads concurrently +trying to log), it starts allocating. `queue` is a good strategy where +`threadLocal` is not desirable. + -<code>queue</code> also accepts optional <code>supplier</code> (of type <code>java.util.Queue</code>, defaults to - <code>org.jctools.queues.MpmcArrayQueue.new</code> if JCTools is in the classpath; -otherwise <code>java.util.concurrent.ArrayBlockingQueue.new</code>) and <code>capacity</code> (of -type <code>int</code>, defaults to <code>max(8,2*cpuCount+1)</code>) parameters: +`queue` also accepts optional `supplier` (of type `java.util.Queue`, defaults to + `org.jctools.queues.MpmcArrayQueue.new` if JCTools is in the classpath; +otherwise `java.util.concurrent.ArrayBlockingQueue.new`) and `capacity` (of +type `int`, defaults to `max(8,2*cpuCount+1)`) parameters: + [source] ---- @@ -302,26 +302,26 @@ queue:capacity=10 queue:supplier=java.util.concurrent.ArrayBlockingQueue.new,capacity=50 ---- -The default <code>RecyclerFactory</code> is <code>threadLocal</code>, if -<code>log4j2.enable.threadlocals=true</code>; otherwise, <code>queue</code>. +The default `RecyclerFactory` is `threadLocal`, if +`log4j2.enable.threadlocals=true`; otherwise, `queue`. [#template-config] == Template Configuration -Templates are configured by means of the following <code>JsonTemplateLayout</code> +Templates are configured by means of the following `JsonTemplateLayout` parameters: -- <code>eventTemplate[Uri]</code> (for serializing <code>LogEvent</code>s) -- <code>stackTraceElementTemplate[Uri]</code> (for serializing <code>StackStraceElement</code>s) -- <code>eventTemplateAdditionalFields</code> (for extending the used event template) +- `eventTemplate[Uri]` (for serializing ``LogEvent``s) +- `stackTraceElementTemplate[Uri]` (for serializing ``StackStraceElement``s) +- `eventTemplateAdditionalFields` (for extending the used event template) [#event-templates] === Event Templates -<code>eventTemplate[Uri]</code> describes the JSON structure <code>JsonTemplateLayout</code> uses to -serialize <code>LogEvent</code>s. The default configuration (accessible by -<code>log4j.layout.jsonTemplate.eventTemplate[Uri]</code> property) is set to -<code>classpath:EcsLayout.json</code> provided by the <code>log4j-layout-json-template</code> +`eventTemplate[Uri]` describes the JSON structure `JsonTemplateLayout` uses to +serialize ``LogEvent``s. The default configuration (accessible by +`log4j.layout.jsonTemplate.eventTemplate[Uri]` property) is set to +`classpath:EcsLayout.json` provided by the `log4j-layout-json-template` artifact: [source,json] @@ -375,34 +375,34 @@ artifact: ---- -<code>log4j-layout-json-template</code> artifact contains the following predefined event +`log4j-layout-json-template` artifact contains the following predefined event templates: -- https://github.com/apache/logging-log4j2/tree/master/log4j-layout-json-template/src/main/resources/EcsLayout.json[<code>EcsLayout.json</code>] +- https://github.com/apache/logging-log4j2/tree/master/log4j-layout-json-template/src/main/resources/EcsLayout.json[`EcsLayout.json`] described by https://www.elastic.co/guide/en/ecs/current/ecs-reference.html[the Elastic Common Schema (ECS) specification] -- https://github.com/apache/logging-log4j2/tree/master/log4j-layout-json-template/src/main/resources/LogstashJsonEventLayoutV1.json[<code>LogstashJsonEventLayoutV1.json</code>] +- https://github.com/apache/logging-log4j2/tree/master/log4j-layout-json-template/src/main/resources/LogstashJsonEventLayoutV1.json[`LogstashJsonEventLayoutV1.json`] described in https://github.com/logstash/log4j-jsonevent-layout[Logstash - <code>json_event</code> pattern for log4j] + `json_event` pattern for log4j] -- https://github.com/apache/logging-log4j2/tree/master/log4j-layout-json-template/src/main/resources/GelfLayout.json[<code>GelfLayout.json</code>] +- https://github.com/apache/logging-log4j2/tree/master/log4j-layout-json-template/src/main/resources/GelfLayout.json[`GelfLayout.json`] described by https://docs.graylog.org/en/3.1/pages/gelf.html#gelf-payload-specification[the Graylog Extended Log Format (GELF) payload specification] with additional - <code>_thread</code> and <code>_logger</code> fields. (Here it is advised to override the obligatory - <code>host</code> field with a user provided constant via <code>eventTemplateAdditionalFields</code> - to avoid <code>hostName</code> property lookup at runtime, which incurs an extra cost.) - -- https://github.com/apache/logging-log4j2/tree/master/log4j-layout-json-template/src/main/resources/JsonLayout.json[<code>JsonLayout.json</code>] - providing the exact JSON structure generated by link:layouts.html#JSONLayout[<code>JsonLayout</code>] - with the exception of <code>thrown</code> field. (<code>JsonLayout</code> serializes the <code>Throwable</code> - as is via Jackson <code>ObjectMapper</code>, whereas <code>JsonLayout.json</code> template of - <code>JsonTemplateLayout</code> employs the <code>StackTraceElementLayout.json</code> template + `_thread` and `_logger` fields. (Here it is advised to override the obligatory + `host` field with a user provided constant via `eventTemplateAdditionalFields` + to avoid `hostName` property lookup at runtime, which incurs an extra cost.) + +- https://github.com/apache/logging-log4j2/tree/master/log4j-layout-json-template/src/main/resources/JsonLayout.json[`JsonLayout.json`] + providing the exact JSON structure generated by link:layouts.html#JSONLayout[`JsonLayout`] + with the exception of `thrown` field. (`JsonLayout` serializes the `Throwable` + as is via Jackson `ObjectMapper`, whereas `JsonLayout.json` template of + `JsonTemplateLayout` employs the `StackTraceElementLayout.json` template for stack traces to generate a document-store-friendly flat structure.) Below is the list of supported event template resolvers: [#event-template-resolvers] -.<code>LogEvent</code> template resolvers +.`LogEvent` template resolvers [cols="1m,3,2,2,4"] |=== | Resolver Name @@ -413,7 +413,7 @@ Below is the list of supported event template resolvers: | endOfBatch | -| <code>logEvent.isEndOfBatch()</code> +| `logEvent.isEndOfBatch()` | none a| [source,json] @@ -435,14 +435,14 @@ field = "field" -> ( stringified = "stringified" -> boolean ---- a| -Resolves fields of the <code>Throwable</code> returned by <code>logEvent.getThrown()</code>. +Resolves fields of the `Throwable` returned by `logEvent.getThrown()`. Note that this resolver is toggled by -<code>log4j.layout.jsonTemplate.stackTraceEnabled</code> property. -| Since <code>Throwable#getStackTrace()</code> clones the original <code>StackTraceElement[]</code>, +`log4j.layout.jsonTemplate.stackTraceEnabled` property. +| Since `Throwable#getStackTrace()` clones the original `StackTraceElement[]`, access to (and hence rendering of) stack traces are not garbage-free. a| -Resolve <code>logEvent.getThrown().getClass().getCanonicalName()</code>: +Resolve `logEvent.getThrown().getClass().getCanonicalName()`: [source,json] ---- @@ -452,7 +452,7 @@ Resolve <code>logEvent.getThrown().getClass().getCanonicalName()</code>: } ---- -Resolve the stack trace into a list of <code>StackTraceElement</code> objects: +Resolve the stack trace into a list of `StackTraceElement` objects: [source,json] ---- @@ -474,15 +474,15 @@ Resolve the stack trace into a string field: ---- | exceptionRootCause -| identical to <code>exception</code> resolver +| identical to `exception` resolver a| -Resolves the fields of the innermost <code>Throwable</code> returned by -<code>logEvent.getThrown()</code>. +Resolves the fields of the innermost `Throwable` returned by +`logEvent.getThrown()`. Note that this resolver is toggled by -<code>log4j.layout.jsonTemplate.stackTraceEnabled</code> property. -| identical to <code>exception</code> resolver -| identical to <code>exception</code> resolver +`log4j.layout.jsonTemplate.stackTraceEnabled` property. +| identical to `exception` resolver +| identical to `exception` resolver | level a| @@ -493,7 +493,7 @@ field = "field" -> ( "name" \| "severity" ) severity = severity-field severity-field = "field" -> ( "keyword" \| "code" ) ---- -| resolves the fields of the <code>logEvent.getLevel()</code> +| resolves the fields of the `logEvent.getLevel()` | none a| Resolve the level name: @@ -540,7 +540,7 @@ a| ---- config = "field" -> ( "name" \| "fqcn" ) ---- -| resolves <code>logEvent.getLoggerFqcn()</code> and <code>logEvent.getLoggerName()</code> +| resolves `logEvent.getLoggerFqcn()` and `logEvent.getLoggerName()` | none a| Resolve the logger name: @@ -563,7 +563,7 @@ Resolve the logger's fully qualified class name: } ---- -| main:<key> +| main a| [source] ---- @@ -572,10 +572,10 @@ index = "index" -> number key = "key" -> string ---- | performs link:lookups.html#AppMainArgsLookup[Main Argument Lookup] for the - given <code>index</code> or <code>key</code> + given `index` or `key` | none a| -Resolve the 1st <code>main()</code> method argument: +Resolve the 1st `main()` method argument: [source,json] ---- @@ -585,7 +585,7 @@ Resolve the 1st <code>main()</code> method argument: } ---- -Resolve the argument coming right after <code>--userId</code>: +Resolve the argument coming right after `--userId`: [source,json] ---- @@ -603,11 +603,11 @@ config = key , [ stringified ] key = "key" -> string stringified = "stringified" -> boolean ---- -| resolves the given <code>key</code> of <code>MapMessage</code>s -| <code>stringified</code> flag translates to <code>String.valueOf(value)</code>, hence mind - not-<code>String</code>-typed values. +| resolves the given `key` of ``MapMessage``s +| `stringified` flag translates to `String.valueOf(value)`, hence mind + not-`String`-typed values. a| -Resolve the <code>userRole</code> field of the message: +Resolve the `userRole` field of the message: [source,json] ---- @@ -623,7 +623,7 @@ a| ---- config = "field" -> "name" ---- -| <code>logEvent.getMarker().getName()</code> +| `logEvent.getMarker().getName()` | none a| Resolve the marker name: @@ -654,25 +654,25 @@ flattenPrefix = "prefix" -> string ---- a| Mapped Diagnostic Context (MDC), aka. Thread Context Data, resolver. -<code>singleAccess</code> resolves the MDC value as is, whilst <code>multiAccess</code> resolves a -multitude of MDC values. If <code>flatten</code> is provided, <code>multiAccess</code> merges the +`singleAccess` resolves the MDC value as is, whilst `multiAccess` resolves a +multitude of MDC values. If `flatten` is provided, `multiAccess` merges the values with the parent, otherwise creates a new JSON object containing the values. -Enabling <code>stringified</code> flag converts each value to its string representation. +Enabling `stringified` flag converts each value to its string representation. -Regex provided in the <code>pattern</code> is used to match against the keys. +Regex provided in the `pattern` is used to match against the keys. a| -<code>log4j2.garbagefreeThreadContextMap</code> flag needs to be turned on to iterate +`log4j2.garbagefreeThreadContextMap` flag needs to be turned on to iterate the map without allocations. -<code>stringified</code> allocates a new <code>String</code> for values that are not of type <code>String</code>. +`stringified` allocates a new `String` for values that are not of type `String`. -Writing certain non-primitive values (e.g., <code>BigDecimal</code>, <code>Set</code>, etc.) to JSON -generates garbage, though most (e.g., <code>int</code>, <code>long</code>, <code>String</code>, <code>List</code>, -<code>boolean[]</code>, etc.) don't. +Writing certain non-primitive values (e.g., `BigDecimal`, `Set`, etc.) to JSON +generates garbage, though most (e.g., `int`, `long`, `String`, `List`, +`boolean[]`, etc.) don't. a| -Resolve the <code>userRole</code> MDC value: +Resolve the `userRole` MDC value: [source,json] ---- @@ -682,7 +682,7 @@ Resolve the <code>userRole</code> MDC value: } ---- -Resolve the string representation of the <code>userRank</code> MDC value: +Resolve the string representation of the `userRank` MDC value: [source,json] ---- @@ -712,7 +712,7 @@ Resolve all MDC entries into an object such that values are converted to string: } ---- -Merge all MDC entries whose keys are matching with the <code>user(Role\|Rank)</code> regex +Merge all MDC entries whose keys are matching with the `user(Role\|Rank)` regex into the parent: [source,json] @@ -725,7 +725,7 @@ into the parent: ---- After converting the corresponding entries to string, merge all MDC entries to -parent such that keys are prefixed with <code>_</code>: +parent such that keys are prefixed with `_`: [source,json] ---- @@ -748,9 +748,9 @@ includeStackTrace = "includeStacktrae" -> boolean stringified = "stringified" -> boolean fallbackKey = "fallbackKey" -> string ---- -a| <code>logEvent.getMessage()</code> +a| `logEvent.getMessage()` | For simple string messages, the resolution is performed without allocations. - For <code>ObjectMessage</code>s and <code>MultiformatMessage</code>s, it depends. + For ``ObjectMessage``s and ``MultiformatMessage``s, it depends. a| Resolve the message into a string: @@ -762,6 +762,17 @@ Resolve the message into a string: } ---- +Resolve the message into a string using a pattern: + +[source,json] +---- +{ + "$resolver": "message", + "pattern": ""[%t] %-5p %X{requestId, sessionId, loginId, userId, ipAddress, corpAcctNumber} %C{1.}.%M:%L - %m"", + "stringified": true +} +---- + Resolve the message such that if it is an `ObjectMessage` or a `MultiformatMessage` with JSON support, its type (string, list, object, etc.) will be retained: @@ -773,12 +784,12 @@ will be retained: } ---- -Given the above configuration, a <code>SimpleMessage</code> will generate a <code>"sample log -message"</code>, whereas a <code>MapMessage</code> will generate a <code>{"action": "login", -"sessionId": "87asd97a"}</code>. Certain indexed log storage systems (e.g., +Given the above configuration, a `SimpleMessage` will generate a `"sample log +message"`, whereas a `MapMessage` will generate a `{"action": "login", +"sessionId": "87asd97a"}`. Certain indexed log storage systems (e.g., https://www.elastic.co/elasticsearch/[Elasticsearch]) will not allow both values -to coexist due to type mismatch: one is a <code>string</code> while the other is an <code>object</code>. -Here one can use a <code>fallbackKey</code> to work around the problem: +to coexist due to type mismatch: one is a `string` while the other is an `object`. +Here one can use a `fallbackKey` to work around the problem: [source,json] ---- @@ -788,10 +799,10 @@ Here one can use a <code>fallbackKey</code> to work around the problem: } ---- -Using this configuration, a <code>SimpleMessage</code> will generate a -<code>{"formattedMessage": "sample log message"}</code> and a <code>MapMessage</code> will generate a -<code>{"action": "login", "sessionId": "87asd97a"}</code>. Note that both emitted JSONs are -of type <code>object</code> and have no type-conflicting fields. +Using this configuration, a `SimpleMessage` will generate a +`{"formattedMessage": "sample log message"}` and a `MapMessage` will generate a +`{"action": "login", "sessionId": "87asd97a"}`. Note that both emitted JSONs are +of type `object` and have no type-conflicting fields. | ndc a| @@ -801,7 +812,7 @@ config = [ pattern ] pattern = "pattern" -> string ---- | Resolves the Nested Diagnostic Context (NDC), aka. Thread Context Stack, - <code>String[]</code> returned by <code>logEvent.getContextStack()</code> + `String[]` returned by `logEvent.getContextStack()` | none a| Resolve all NDC values into a list: @@ -813,7 +824,7 @@ Resolve all NDC values into a list: } ---- -Resolve all NDC values matching with the <code>pattern</code> regex: +Resolve all NDC values matching with the `pattern` regex: [source,json] ---- @@ -832,13 +843,13 @@ pattern = "pattern" -> string stackTraceEnabled = "stackTraceEnabled" -> boolean ---- a| -Resolver delegating to link:layouts.html#PatternLayout[<code>PatternLayout</code>]. +Resolver delegating to link:layouts.html#PatternLayout[`PatternLayout`]. -The default value of <code>stackTraceEnabled</code> is inherited from the parent -<code>JsonTemplateLayout</code>. +The default value of `stackTraceEnabled` is inherited from the parent +`JsonTemplateLayout`. | none a| -Resolve the string produced by <code>%p %c{1.} [%t] %X{userId} %X %m%ex</code> pattern: +Resolve the string produced by `%p %c{1.} [%t] %X{userId} %X %m%ex` pattern: [source,json] ---- @@ -859,11 +870,11 @@ config = "field" -> ( "lineNumber" ) ---- a| -Resolves the fields of the <code>StackTraceElement</code> returned by -<code>logEvent.getSource()</code>. +Resolves the fields of the `StackTraceElement` returned by +`logEvent.getSource()`. Note that this resolver is toggled by -<code>log4j.layout.jsonTemplate.locationInfoEnabled</code> property. +`log4j.layout.jsonTemplate.locationInfoEnabled` property. | none a| Resolve the line number: @@ -882,8 +893,8 @@ a| ---- config = "field" -> ( "name" \| "id" \| "priority" ) ---- -| resolves <code>logEvent.getThreadId()</code>, <code>logEvent.getThreadName()</code>, - <code>logEvent.getThreadPriority()</code> +| resolves `logEvent.getThreadId()`, `logEvent.getThreadName()`, + `logEvent.getThreadPriority()` | none a| Resolve the thread name: @@ -924,10 +935,10 @@ unit = "unit" -> ( ) rounded = "rounded" -> boolean ---- -| resolves <code>logEvent.getInstant()</code> in various forms +| resolves `logEvent.getInstant()` in various forms | none a| -.<code>timestamp</code> template resolver examples +.`timestamp` template resolver examples [cols="5,2m"] !=== ! Configuration @@ -1047,11 +1058,11 @@ a! [#stack-trace-element-templates] === Stack Trace Element Templates -<code>stackTraceElement[Uri]</code> describes the JSON structure <code>JsonTemplateLayout</code> uses -to format <code>StackTraceElement</code>s. The default configuration (accessible by -<code>log4j.layout.jsonTemplate.stackTraceElementTemplate[Uri]</code> property) is set to -<code>classpath:StackTraceElementLayout.json</code> provided by the -<code>log4j-layout-json-template</code> artifact: +`stackTraceElement[Uri]` describes the JSON structure `JsonTemplateLayout` uses +to format ``StackTraceElement``s. The default configuration (accessible by +`log4j.layout.jsonTemplate.stackTraceElementTemplate[Uri]` property) is set to +`classpath:StackTraceElementLayout.json` provided by the +`log4j-layout-json-template` artifact: [source,json] ---- @@ -1086,22 +1097,22 @@ config = "field" -> ( "lineNumber" ) ---- -All above accesses to <code>StackTraceElement</code> is garbage-free. +All above accesses to `StackTraceElement` is garbage-free. [#features] == Features -Below is a feature comparison matrix between <code>JsonTemplateLayout</code> and +Below is a feature comparison matrix between `JsonTemplateLayout` and alternatives. .Feature comparison matrix [cols="3,1,1,1,1"] |=== | Feature -| <code>JsonTemplateLayout</code> -| link:layouts.html#JSONLayout[<code>JsonLayout</code>] -| link:layouts.html#GELFLayout[<code>GelfLayout</code>] -| https://github.com/elastic/java-ecs-logging/tree/master/log4j2-ecs-layout[<code>EcsLayout</code>] +| `JsonTemplateLayout` +| link:layouts.html#JSONLayout[`JsonLayout`] +| link:layouts.html#GELFLayout[`GelfLayout`] +| https://github.com/elastic/java-ecs-logging/tree/master/log4j2-ecs-layout[`EcsLayout`] | Java version | 8 @@ -1115,7 +1126,7 @@ alternatives. | None | None -| Full schema customization? +| Schema customization? | ✓ | ✕ | ✕ @@ -1133,13 +1144,13 @@ alternatives. | ✓ | ✓ -| Custom typed <code>Message</code> serialization? +| Custom typed `Message` serialization? | ✓ | ✕ | ✕ -| ?footnote:[Only for <code>ObjectMessage</code>s and if Jackson is in the classpath.] +| ?footnote:[Only for ``ObjectMessage``s and if Jackson is in the classpath.] -| Custom typed <code>MDC</code> value serialization? +| Custom typed `MDC` value serialization? | ✓ | ✕ | ✕ @@ -1157,7 +1168,7 @@ alternatives. | ✕ | ✕ -| Additional fields? +| Additional field support? | ✓ | ✓ | ✓ @@ -1170,29 +1181,28 @@ alternatives. [#faq-lookups] === Are lookups supported in templates? -Yes, link:lookups.html[lookups] (e.g., <code>${java:version}</code>, <code>${env:USER}</code>, -<code>${date:MM-dd-yyyy}</code>) are supported in string literals of templates. Though note -that they are not garbage-free. +Yes, link:lookups.html[lookups] (e.g., `${dollar}{java:version}`, +`${dollar}{env:USER}`, `${dollar}{date:MM-dd-yyyy}`) are supported in string +literals of templates. Though note that they are not garbage-free. [#faq-garbage-free] -=== Is <code>JsonTemplateLayout</code> garbage-free? +=== Is `JsonTemplateLayout` garbage-free? Yes, if the garbage-free layout behaviour toggling properties -<code>log4j2.enableDirectEncoders</code> and <code>log4j2.garbagefreeThreadContextMap</code> are +`log4j2.enableDirectEncoders` and `log4j2.garbagefreeThreadContextMap` are enabled. Take into account the following caveats: * The configured link:#recycling-strategy[recycling strategy] might not be garbage-free. -* Since <code>Throwable#getStackTrace()</code> clones the original <code>StackTraceElement[]</code>, +* Since `Throwable#getStackTrace()` clones the original `StackTraceElement[]`, access to (and hence rendering of) stack traces are not garbage-free. -* Serialization of <code>MapMessage</code>s and <code>ObjectMessage</code>s are mostly - garbage-free except for certain types (e.g., <code>BigDecimal</code>, <code>BigInteger</code>, - <code>Collection</code>s with the exception of <code>List</code>). +* Serialization of ``MapMessage``s and ``ObjectMessage``s are mostly + garbage-free except for certain types (e.g., `BigDecimal`, `BigInteger`, + ``Collection``s with the exception of `List`). -* link:lookups.html[Lookups] (that is, <code>${...}</code> variables, excluding - <code>${json:...}</code> ones) are not garbage-free. +* link:lookups.html[Lookups] (that is, `${...}` variables) are not garbage-free. Don't forget to checkout link:#event-template-resolvers[the notes on garbage footprint of resolvers] you employ in templates. diff --git a/src/site/xdoc/manual/json-template-layout.xml.vm b/src/site/xdoc/manual/json-template-layout.xml.vm deleted file mode 100644 index be05a1c..0000000 --- a/src/site/xdoc/manual/json-template-layout.xml.vm +++ /dev/null @@ -1,1526 +0,0 @@ -<?xml version="1.0"?> -<!-- - Licensed to the Apache Software Foundation (ASF) under one or more - contributor license agreements. See the NOTICE file distributed with - this work for additional information regarding copyright ownership. - The ASF licenses this file to You under the Apache License, Version 2.0 - (the "License"); you may not use this file except in compliance with - the License. You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. ---> - -<document xmlns="http://maven.apache.org/XDOC/2.0"; - xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"; - xsi:schemaLocation="http://maven.apache.org/XDOC/2.0 http://maven.apache.org/xsd/xdoc-2.0.xsd";> - - <properties> - <title>JSON Template Layout</title> - <author email="[email protected]">Volkan Yazıcı</author> - </properties> - - <body> - - <section name="JSON Template Layout"> - - <p> - <code>JsonTemplateLayout</code> is a customizable, efficient, and - garbage-free JSON emitting layout. It encodes <code>LogEvent</code>s - according to the structure described by the JSON template provided. In a - nutshell, it shines with its - </p> - - <ul> - <li> - Customizable JSON structure (see <code>eventTemplate[Uri]</code> and - <code>stackTraceElementTemplate[Uri]</code> parameters) - </li> - <li> - Customizable timestamp formatting (see <code>timestamp</code> parameter) - </li> - </ul> - - <subsection name="Usage"> - - <a name="usage"/> - - <p> - Adding <code>log4j-layout-json-template</code> artifact to your list - of dependencies is enough to enable access to - <code>JsonTemplateLayout</code> in your Log4j configuration: - </p> - - <pre class="prettyprint linenums"><![CDATA[ -<dependency> - <groupId>org.apache.logging.log4j</groupId> - <artifactId>log4j-layout-json-template</artifactId> - <version>${Log4jReleaseVersion}</version> -</dependency>]]></pre> - - <p> - For instance, given the following JSON template modelling the - <a href="https://github.com/logstash/log4j-jsonevent-layout";>the - official Logstash <code>JSONEventLayoutV1</code></a> (accessible via - <code>classpath:LogstashJsonEventLayoutV1.json</code>) - </p> - - <pre class="prettyprint linenums"><![CDATA[ -{ - "mdc": { - "$resolver": "mdc" - }, - "exception": { - "exception_class": { - "$resolver": "exception", - "field": "className" - }, - "exception_message": { - "$resolver": "exception", - "field": "message", - "stringified": true - }, - "stacktrace": { - "$resolver": "exception", - "field": "stackTrace", - "stringified": true - } - }, - "line_number": { - "$resolver": "source", - "field": "lineNumber" - }, - "class": { - "$resolver": "source", - "field": "className" - }, - "@version": 1, - "source_host": "${hostName}", - "message": { - "$resolver": "message", - "stringified": true - }, - "thread_name": { - "$resolver": "thread", - "field": "name" - }, - "@timestamp": { - "$resolver": "timestamp" - }, - "level": { - "$resolver": "level", - "field": "name" - }, - "file": { - "$resolver": "source", - "field": "fileName" - }, - "method": { - "$resolver": "source", - "field": "methodName" - }, - "logger_name": { - "$resolver": "logger", - "field": "name" - } -}]]></pre> - - <p> - in combination with the below <code>log4j2.xml</code> configuration: - </p> - - <pre class="prettyprint linenums"><![CDATA[ -<JsonTemplateLayout eventTemplateUri="classpath:LogstashJsonEventLayoutV1.json"/>]]></pre> - - <p> - or with the below <code>log4j2.properties</code> configuration: - </p> - - <pre class="prettyprint linenums"><![CDATA[ -appender.console.json.type = JsonTemplateLayout -appender.console.json.eventTemplateUri = classpath:LogstashJsonEventLayoutV1.json]]></pre> - - <p> - <code>JsonTemplateLayout</code> emits JSON strings as follows: - </p> - - <pre class="prettyprint linenums"><![CDATA[ -{ - "exception": { - "exception_class": "java.lang.RuntimeException", - "exception_message": "test", - "stacktrace": "java.lang.RuntimeException: test\n\tat org.apache.logging.log4j.JsonTemplateLayoutDemo.main(JsonTemplateLayoutDemo.java:11)\n" - }, - "line_number": 12, - "class": "org.apache.logging.log4j.JsonTemplateLayoutDemo", - "@version": 1, - "source_host": "varlik", - "message": "Hello, error!", - "thread_name": "main", - "@timestamp": "2017-05-25T19:56:23.370+02:00", - "level": "ERROR", - "file": "JsonTemplateLayoutDemo.java", - "method": "main", - "logger_name": "org.apache.logging.log4j.JsonTemplateLayoutDemo" -}]]></pre> - - </subsection> - - <subsection name="Layout Configuration"> - - <a name="layout-config"/> - - <p> - <code>JsonTemplateLayout</code> is configured with the following - parameters: - </p> - - <table> - - <caption align="top"><code>JsonTemplateLayout</code> parameters</caption> - - <thead> - <tr> - <th>Parameter Name</th> - <th>Type</th> - <th>Description</th> - </tr> - </thead> - - <tbody> - - <tr> - <td>charset</td> - <td>Charset</td> - <td><code>Charset</code> used for <code>String</code> encoding</td> - </tr> - - <tr> - <td>locationInfoEnabled</td> - <td>boolean</td> - <td> - toggles access to the <code>LogEvent</code> source; file name, - line number, etc. (defaults to <code>false</code> set by - <code>log4j.layout.jsonTemplate.locationInfoEnabled</code> - property) - </td> - </tr> - - <tr> - <td>stackTraceEnabled</td> - <td>boolean</td> - <td> - toggles access to the stack traces (defaults to - <code>true</code> set by - <code>log4j.layout.jsonTemplate.stackTraceEnabled</code> - property) - </td> - </tr> - - <tr> - <td>eventTemplate</td> - <td>String</td> - <td> - inline JSON template for rendering <code>LogEvent</code>s (has - priority over <code>eventTemplateUri</code>, defaults to - <code>null</code> set by - <code>log4j.layout.jsonTemplate.eventTemplate</code> property) - </td> - </tr> - - <tr> - <td>eventTemplateUri</td> - <td>String</td> - <td> - URI pointing to the JSON template for rendering - <code>LogEvent</code>s (defaults to - <code>classpath:EcsLayout.json</code> set by - <code>log4j.layout.jsonTemplate.eventTemplateUri</code> - property) - </td> - </tr> - - <tr> - <td>eventTemplateAdditionalFields</td> - <td>EventTemplateAdditionalField[]</td> - <td> - additional key-value pairs appended to the root of the event - template - </td> - </tr> - - <tr> - <td>stackTraceElementTemplate</td> - <td>String</td> - <td> - inline JSON template for rendering <code>StackTraceElement</code>s - (has priority over <code>stackTraceElementTemplateUri</code>, - defaults to <code>null</code> set by - <code>log4j.layout.jsonTemplate.stackTraceElementTemplate</code> - property) - </td> - </tr> - - <tr> - <td>stackTraceElementTemplateUri</td> - <td>String</td> - <td> - JSON template for rendering <code>StackTraceElement</code>s - (defaults to <code>classpath:StackTraceElementLayout.json</code> - set by - <code>log4j.layout.jsonTemplate.stackTraceElementTemplateUri</code> - property) - </td> - </tr> - - <tr> - <td>eventDelimiter</td> - <td>String</td> - <td> - delimiter used for separating emitted <code>LogEvent</code>s - (defaults to <code>System.lineSeparator()</code> set by - <code>log4j.layout.jsonTemplate.eventDelimiter</code> - property) - </td> - </tr> - - <tr> - <td>nullEventDelimiterEnabled</td> - <td>boolean</td> - <td> - append <code>\0</code> (<code>null</code>) character to the end - of every emitted <code>eventDelimiter</code> (defaults to - <code>false</code> set by - <code>log4j.layout.jsonTemplate.nullEventDelimiterEnabled</code> - property) - </td> - </tr> - - <tr> - <td>maxStringLength</td> - <td>int</td> - <td> - truncate string values longer than the specified limit (defaults - to 16384 set by <code>log4j.layout.jsonTemplate.maxStringLength</code> - property) - </td> - </tr> - - <tr> - <td>truncatedStringSuffix</td> - <td>String</td> - <td> - suffix to append to strings truncated due to exceeding - <code>maxStringLength</code> (defaults to <code>…</code> set by - <code>log4j.layout.jsonTemplate.truncatedStringSuffix</code> - property) - </td> - </tr> - - <tr> - <td>recyclerFactory</td> - <td>RecyclerFactory</td> - <td> - recycling strategy that can either be <code>dummy</code>, - <code>threadLocal</code>, or <code>queue</code> (set by - <code>log4j.layout.jsonTemplate.recyclerFactory</code> property) - </td> - </tr> - - </tbody> - - </table> - - <h4>Additonal event template fields</h4> - - <a name="additional-event-template-fields"/> - - <p> - Additional event template field is a convenient short-cut to add - custom fields to a template or override the fields of a template. - Following configuration overrides the <code>host</code> field of the - <code>GelfLayout.json</code> template and adds two new custom fields: - </p> - - <pre class="prettyprint linenums"><![CDATA[ -<JsonTemplateLayout eventTemplateUri="classpath:GelfLayout.json"> - <EventTemplateAdditionalFields> - <EventTemplateAdditionalField key="host" value="www.apache.org"/> - <EventTemplateAdditionalField key="_serviceName" value="auth-service"/> - <EventTemplateAdditionalField key="_containerId" value="6ede3f0ca7d9"/> - </EventTemplateAdditionalFields> -</JsonTemplateLayout>]]></pre> - - <p> - One can also pass JSON literals into additional fields: - </p> - - <pre class="prettyprint linenums"><![CDATA[<EventTemplateAdditionalField - key="marker" - type="JSON" - value='{"$resolver": "marker", "field": "name"}'/> -<EventTemplateAdditionalField - key="aNumber" - type="JSON" - value="1"/> -<EventTemplateAdditionalField - key="aList" - type="JSON" - value='[1,2,"string"]'/>]]></pre> - - <h4>Recycling strategy</h4> - - <a name="recycling-strategy"/> - - <p> - <code>RecyclerFactory</code> plays a crucial role for determining the - memory footprint of the layout. Template resolvers employ it to create - recyclers for objects that they can reuse. The function of each - <code>RecyclerFactory</code> and when one should prefer one over - another is explained below: - </p> - - <ul> - - <li> - <code>dummy</code> performs no recycling, hence each recycling - attempt will result in a new instance. This will obviously create a - load on the garbage-collector. It is a good choice for applications - with low and medium log rate. - </li> - - <li> - <code>threadLocal</code> performs the best, since every instance is - stored in <code>ThreadLocal</code>s and accessed without any - synchronization cost. Though this might not be a desirable option - for applications running with hundreds of threads or more, e.g., a - web servlet. - </li> - - <li> - - <p> - <code>queue</code> is the best of both worlds. It allows recycling - of objects up to a certain number (<code>capacity</code>). When - this limit is exceeded due to excessive concurrent load (e.g., - <code>capacity</code> is 50 but there are 51 threads concurrently - trying to log), it starts allocating. <code>queue</code> is a good - strategy where <code>threadLocal</code> is not desirable. - </p> - - <p> - <code>queue</code> also accepts optional <code>supplier</code> (of - type <code>java.util.Queue</code>, defaults to - <code>org.jctools.queues.MpmcArrayQueue.new</code> if JCTools is - in the classpath; otherwise - <code>java.util.concurrent.ArrayBlockingQueue.new</code>) and - <code>capacity</code> (of type <code>int</code>, defaults to - <code>max(8,2*cpuCount+1)</code>) parameters: - </p> - - <pre class="prettyprint linenums"><![CDATA[ -queue:supplier=org.jctools.queues.MpmcArrayQueue.new -queue:capacity=10 -queue:supplier=java.util.concurrent.ArrayBlockingQueue.new,capacity=50]]></pre> - - <p> - The default <code>RecyclerFactory</code> is - <code>threadLocal</code>, if - <code>log4j2.enable.threadlocals=true</code>; otherwise, - <code>queue</code>. - </p> - - </li> - - </ul> - - </subsection> - - <subsection name="Template Configuration"> - - <a name="template-config"/> - - <p> - Templates are configured by means of the following - <code>JsonTemplateLayout</code> parameters: - </p> - - <ul> - <li> - <code>eventTemplate[Uri]</code> (for serializing - <code>LogEvent</code>s) - </li> - <li> - <code>stackTraceElementTemplate[Uri]</code> (for serializing - <code>StackStraceElement</code>s) - </li> - <li> - <code>eventTemplateAdditionalFields</code> (for extending the used - event template) - </li> - </ul> - - <h4>Event Templates</h4> - - <a name="event-templates"/> - - <p> - <code>eventTemplate[Uri]</code> describes the JSON structure - <code>JsonTemplateLayout</code> uses to serialize - <code>LogEvent</code>s. The default configuration (accessible by - <code>log4j.layout.jsonTemplate.eventTemplate[Uri]</code> property) is - set to <code>classpath:EcsLayout.json</code> provided by the - <code>log4j-layout-json-template</code> artifact: - </p> - - <pre class="prettyprint linenums"><![CDATA[ -{ - "@timestamp": { - "$resolver": "timestamp", - "pattern": { - "format": "yyyy-MM-dd'T'HH:mm:ss.SSS'Z'", - "timeZone": "UTC" - } - }, - "log.level": { - "$resolver": "level", - "field": "name" - }, - "message": { - "$resolver": "message", - "stringified": true - }, - "process.thread.name": { - "$resolver": "thread", - "field": "name" - }, - "log.logger": { - "$resolver": "logger", - "field": "name" - }, - "labels": { - "$resolver": "mdc", - "flatten": true, - "stringified": true - }, - "tags": { - "$resolver": "ndc" - }, - "error.type": { - "$resolver": "exception", - "field": "className" - }, - "error.message": { - "$resolver": "exception", - "field": "message" - }, - "error.stack_trace": { - "$resolver": "exception", - "field": "stackTrace", - "stringified": true - } -}]]></pre> - - <p> - <code>log4j-layout-json-template</code> artifact contains the - following predefined event templates: - </p> - - <ul> - <li> - <a href="https://github.com/apache/logging-log4j2/tree/master/log4j-layout-json-template/src/main/resources/EcsLayout.json";><code>EcsLayout.json</code></a> - described by <a href="https://www.elastic.co/guide/en/ecs/current/ecs-reference.html";>the - Elastic Common Schema (ECS) specification</a> - </li> - <li> - <a href="https://github.com/apache/logging-log4j2/tree/master/log4j-layout-json-template/src/main/resources/LogstashJsonEventLayoutV1.json";><code>LogstashJsonEventLayoutV1.json</code></a> - described in <a href="https://github.com/logstash/log4j-jsonevent-layout";>Logstash - <code>json_event</code> pattern for log4j</a> - </li> - <li> - <a href="https://github.com/apache/logging-log4j2/tree/master/log4j-layout-json-template/src/main/resources/GelfLayout.json";><code>GelfLayout.json</code></a> - described by <a href="https://docs.graylog.org/en/3.1/pages/gelf.html#gelf-payload-specification";>the - Graylog Extended Log Format (GELF) payload specification</a> with - additional <code>_thread</code> and <code>_logger</code> fields. - (Here it is advised to override the obligatory <code>host</code> - field with a user provided constant via <code>eventTemplateAdditionalFields</code> - to avoid <code>hostName</code> property lookup at runtime, which - incurs an extra cost.) - </li> - <li> - <a href="https://github.com/apache/logging-log4j2/tree/master/log4j-layout-json-template/src/main/resources/JsonLayout.json";><code>JsonLayout.json</code></a> - providing the exact JSON structure generated by - <a href="layouts.html#JSONLayout"><code>JsonLayout</code></a> with - the exception of <code>thrown</code> field. (<code>JsonLayout</code> - serializes the <code>Throwable</code> as is via Jackson - <code>ObjectMapper</code>, whereas <code>JsonLayout.json</code> - template of <code>JsonTemplateLayout</code> employs the - <code>StackTraceElementLayout.json</code> template for stack traces - to generate a document-store-friendly flat structure.) - </li> - </ul> - - <p> - Below is the list of supported event template resolvers: - </p> - - <a name="event-template-resolvers"/> - - <table> - - <caption align="top"><code>LogEvent</code> template resolvers</caption> - - <thead> - <tr> - <th width="10%">Resolver Name</th> - <th width="30%">Syntax</th> - <th width="15%">Description</th> - <th width="15%">Garbage Footprint</th> - <th width="30%">Examples</th> - </tr> - </thead> - - <tbody> - - <tr> - <td>endOfBatch</td> - <td/> - <td><code>`logEvent.isEndOfBatch()`</code></td> - <td>none</td> - <td> - <pre class="prettyprint linenums"><![CDATA[ -{ - "$resolver": "endOfBatch" -}]]></pre> - </td> - </tr> - - <tr> - <td>exception</td> - <td> - <pre class="prettyprint linenums"><![CDATA[ -config = field , [ stringified ] -field = "field" -> ( - "className" | - "message" | - "stackTrace" ) -stringified = "stringified" -> boolean]]></pre> - </td> - <td> - <p> - Resolves fields of the <code>Throwable</code> returned by - <code>logEvent.getThrown()</code>. - </p> - <p> - Note that this resolver is toggled by - <code>log4j.layout.jsonTemplate.stackTraceEnabled</code> - property. - </p> - </td> - <td> - Since <code>Throwable#getStackTrace()</code> clones the original - <code>StackTraceElement[]</code>, access to (and hence rendering - of) stack traces are not garbage-free. - </td> - <td> - <p> - Resolve <code>logEvent.getThrown().getClass().getCanonicalName()</code>: - </p> - <pre class="prettyprint linenums"><![CDATA[ -{ - "$resolver": "exception", - "field": "className" -}]]></pre> - <p> - Resolve the stack trace into a list of - <code>StackTraceElement</code> objects: - </p> - <pre class="prettyprint linenums"><![CDATA[ -{ - "$resolver": "exception", - "field": "stackTrace" -}]]></pre> - <p>Resolve the stack trace into a string field:</p> - <pre class="prettyprint linenums"><![CDATA[ -{ - "$resolver": "exception", - "field": "stackTrace", - "stringified": true -}]]></pre> - </td> - </tr> - - <tr> - <td>exceptionRootCause</td> - <td>identical to <code>exception</code> resolver</td> - <td> - <p> - Resolves the fields of the innermost <code>Throwable</code> - returned by <code>logEvent.getThrown()</code>. - </p> - <p> - Note that this resolver is toggled by - <code>log4j.layout.jsonTemplate.stackTraceEnabled</code> property. - </p> - </td> - <td>identical to <code>exception</code> resolver</td> - <td>identical to <code>exception</code> resolver</td> - </tr> - - <tr> - <td>endOfBatch</td> - <td> - <pre class="prettyprint linenums"><![CDATA[ -config = field , [ severity ] -field = "field" -> ( "name" | "severity" ) -severity = severity-field -severity-field = "field" -> ( "keyword" | "code" )]]></pre> - </td> - <td>resolves the fields of the <code>logEvent.getLevel()</code></td> - <td>none</td> - <td> - <p>Resolve the level name:</p> - <pre class="prettyprint linenums"><![CDATA[ -{ - "$resolver": "level", - "field": "name" -}]]></pre> - <p> - Resolve the - <a href="https://en.wikipedia.org/wiki/Syslog#Severity_levels";>Syslog - severity</a> keyword: - </p> - <pre class="prettyprint linenums"><![CDATA[ -{ - "$resolver": "level", - "field": "severity", - "severity": { - "field": "keyword" - } -}]]></pre> - <p> - Resolve the - <a href="https://en.wikipedia.org/wiki/Syslog#Severity_levels";>Syslog - severity</a> code: - </p> - <pre class="prettyprint linenums"><![CDATA[ -{ - "$resolver": "level", - "field": "severity", - "severity": { - "field": "code" - } -}]]></pre> - </td> - </tr> - - <tr> - <td>logger</td> - <td> - <pre class="prettyprint linenums"><![CDATA[ -config = "field" -> ( "name" | "fqcn" )]]></pre> - </td> - <td> - resolves <code>logEvent.getLoggerFqcn()</code> and - <code>logEvent.getLoggerName()</code> - </td> - <td>none</td> - <td> - <p>Resolve the logger name:</p> - <pre class="prettyprint linenums"><![CDATA[ -{ - "$resolver": "logger", - "field": "name" -}]]></pre> - <p>Resolve the logger's fully qualified class name:</p> - <pre class="prettyprint linenums"><![CDATA[ -{ - "$resolver": "logger", - "field": "fqcn" -}]]></pre> - </td> - </tr> - - <tr> - <td>main</td> - <td> - <pre class="prettyprint linenums"><![CDATA[ -config = ( index | key ) -index = "index" -> number -key = "key" -> string]]></pre> - </td> - <td> - performs <a href="lookups.html#AppMainArgsLookup">Main Argument - Lookup</a> for the given <code>index</code> or <code>key</code> - </td> - <td>none</td> - <td> - <p> - Resolve the 1<sup>st</sup> <code>main()</code> method - argument: - </p> - <pre class="prettyprint linenums"><![CDATA[ -{ - "$resolver": "main", - "index": 0 -}]]></pre> - <p> - Resolve the argument coming right after <code>--userId</code>: - </p> - <pre class="prettyprint linenums"><![CDATA[ -{ - "$resolver": "main", - "key": "--userId" -}]]></pre> - </td> - </tr> - - <tr> - <td>map</td> - <td> - <pre class="prettyprint linenums"><![CDATA[ -config = key , [ stringified ] -key = "key" -> string -stringified = "stringified" -> boolean]]></pre> - </td> - <td> - resolves the given <code>key</code> of <code>MapMessage</code>s - </td> - <td> - <code>stringified</code> flag translates to - <code>String.valueOf(value)</code>, hence mind - not-<code>String</code>-typed values. - </td> - <td> - <p>Resolve the <code>userRole</code> field of the message:</p> - <pre class="prettyprint linenums"><![CDATA[ -{ - "$resolver": "map", - "key": "userRole" -}]]></pre> - </td> - </tr> - - <tr> - <td>marker</td> - <td> - <pre class="prettyprint linenums"><![CDATA[ -config = "field" -> "name"]]></pre> - </td> - <td><code>logEvent.getMarker().getName()</code></td> - <td>none</td> - <td> - <p>Resolve the marker name:</p> - <pre class="prettyprint linenums"><![CDATA[ -{ - "$resolver": "marker", - "field": "name" -}]]></pre> - </td> - </tr> - - <tr> - <td>mdc</td> - <td> - <pre class="prettyprint linenums"><![CDATA[ -config = singleAccess | multiAccess - -singleAccess = key , [ stringified ] -key = "key" -> string -stringified = "stringified" -> boolean - -multi-access = [ pattern ] , [ flatten ] , [ stringified ] -pattern = "pattern" -> string -flatten = "flatten" -> ( boolean | flattenConfig ) -flattenConfig = [ flattenPrefix ] -flattenPrefix = "prefix" -> string]]></pre> - </td> - <td> - <p> - Mapped Diagnostic Context (MDC), aka. Thread Context Data, - resolver. - </p> - <p> - <code>singleAccess</code> resolves the MDC value as is, whilst - <code>multiAccess</code> resolves a multitude of MDC values. - If <code>flatten</code> is provided, <code>multiAccess</code> - merges the values with the parent, otherwise creates a new - JSON object containing the values. - </p> - <p> - Enabling <code>stringified</code> flag converts each value to - its string representation. - </p> - <p> - Regex provided in the <code>pattern</code> is used to match - against the keys. - </p> - </td> - <td> - <p> - <code>log4j2.garbagefreeThreadContextMap</code> flag needs to - be turned on to iterate the map without allocations. - </p> - <p> - <code>stringified</code> allocates a new <code>String</code> - for values that are not of type <code>String</code>. - </p> - <p> - Writing certain non-primitive values (e.g., - <code>BigDecimal</code>, <code>Set</code>, etc.) to JSON - generates garbage, though most (e.g., <code>int</code>, - <code>long</code>, <code>String</code>, <code>List</code>, - <code>boolean[]</code>, etc.) don't. - </p> - </td> - <td> - <p>Resolve the <code>userRole</code> MDC value:</p> - <pre class="prettyprint linenums"><![CDATA[ -{ - "$resolver": "mdc", - "key": "userRole" -}]]></pre> - <p> - Resolve the string representation of the <code>userRank</code> - MDC value: - </p> - <pre class="prettyprint linenums"><![CDATA[ -{ - "$resolver": "mdc", - "key": "userRank", - "stringified": true -}]]></pre> - <p>Resolve all MDC entries into an object:</p> - <pre class="prettyprint linenums"><![CDATA[ -{ - "$resolver": "mdc" -}]]></pre> - <p> - Resolve all MDC entries into an object such that values are - converted to string: - </p> - <pre class="prettyprint linenums"><![CDATA[ -{ - "$resolver": "mdc", - "stringified": true -}]]></pre> - <p> - Merge all MDC entries whose keys are matching with the - <code>user(Role|Rank)</code> regex into the parent: - </p> - <pre class="prettyprint linenums"><![CDATA[ -{ - "$resolver": "mdc", - "flatten": true, - "pattern": "user(Role|Rank)" -}]]></pre> - <p> - After converting the corresponding entries to string, merge - all MDC entries to parent such that keys are prefixed with - <code>_</code>: - </p> - <pre class="prettyprint linenums"><![CDATA[ -{ - "$resolver": "mdc", - "stringified": true, - "flatten": { - "prefix": "_" - } -}]]></pre> - </td> - </tr> - - <tr> - <td>message</td> - <td> - <pre class="prettyprint linenums"><![CDATA[ -config = [ stringified ] , [ fallbackKey ] -pattern = "pattern" -> string -includeStackTrace = "includeStacktrae" -> boolean -stringified = "stringified" -> boolean -fallbackKey = "fallbackKey" -> string]]></pre> - </td> - <td><code>logEvent.getMessage()</code></td> - <td> - For simple string messages, the resolution is performed without - allocations. For <code>ObjectMessage</code>s and - <code>MultiformatMessage</code>s, it depends. - </td> - <td> - <p>Resolve the message into a string:</p> - <pre class="prettyprint linenums"><![CDATA[ -{ - "$resolver": "message", - "stringified": true -}]]></pre> - <p>Resolve the message into a string using a pattern:</p> - <pre class="prettyprint linenums"><![CDATA[ -{ - "$resolver": "message", - "stringified": true -}]]></pre> - <p> - Resolve the message such that if it is an - <code>ObjectMessage</code> or a - <code>MultiformatMessage</code> with JSON support, its type - (string, list, object, etc.) will be retained: - </p> - <pre class="prettyprint linenums"><![CDATA[ -{ - "$resolver": "message" -}]]></pre> - <p> - Given the above configuration, a <code>SimpleMessage</code> - will generate a <code>"sample log message"</code>, whereas a - <code>MapMessage</code> will generate a <code>{"action": "login", - "sessionId": "87asd97a"}</code>. Certain indexed log storage - systems (e.g., - <a href="https://www.elastic.co/elasticsearch/";>Elasticsearch</a>) - will not allow both values to coexist due to type mismatch: - one is a <code>string</code> while the other is an - <code>object</code>. Here one can use a - <code>fallbackKey</code> to work around the problem: - </p> - <pre class="prettyprint linenums"><![CDATA[ -{ - "$resolver": "message", - "fallbackKey": "formattedMessage" -}]]></pre> - <p> - Using this configuration, a <code>SimpleMessage</code> will - generate a <code>{"formattedMessage": "sample log message"}</code> - and a <code>MapMessage</code> will generate a - <code>{"action": "login", "sessionId": "87asd97a"}</code>. - Note that both emitted JSONs are of type <code>object</code> - and have no type-conflicting fields. - </p> - </td> - </tr> - - <tr> - <td>ndc</td> - <td> - <pre class="prettyprint linenums"><![CDATA[ -config = [ pattern ] -pattern = "pattern" -> string]]></pre> - </td> - <td> - Resolves the Nested Diagnostic Context (NDC), aka. Thread - Context Stack, <code>String[]</code> returned by - <code>logEvent.getContextStack()</code> - </td> - <td>none</td> - <td> - <p>Resolve all NDC values into a list:</p> - <pre class="prettyprint linenums"><![CDATA[ -{ - "$resolver": "ndc" -}]]></pre> - <p> - Resolve all NDC values matching with the <code>pattern</code> - regex: - </p> - <pre class="prettyprint linenums"><![CDATA[ -{ - "$resolver": "ndc", - "pattern": "user(Role|Rank):\w+" -}]]></pre> - </td> - </tr> - - <tr> - <td>pattern</td> - <td> - <pre class="prettyprint linenums"><![CDATA[ -config = pattern , [ stackTraceEnabled ] -pattern = "pattern" -> string -stackTraceEnabled = "stackTraceEnabled" -> boolean]]></pre> - </td> - <td> - <p> - Resolver delegating to - <a href="layouts.html#PatternLayout"><code>PatternLayout</code></a>. - </p> - <p> - The default value of <code>stackTraceEnabled</code> is - inherited from the parent <code>JsonTemplateLayout</code>. - </p> - </td> - <td>none</td> - <td> - <p> - Resolve the string produced by <code>%p %c{1.} [%t] %X{userId} - %X %m%ex</code> pattern: - </p> - <pre class="prettyprint linenums"><![CDATA[ -{ - "$resolver": "pattern", - "pattern": "%p %c{1.} [%t] %X{userId} %X %m%ex" -}]]></pre> - </td> - </tr> - - <tr> - <td>source</td> - <td> - <pre class="prettyprint linenums"><![CDATA[ -config = "field" -> ( - "className" | - "fileName" | - "methodName" | - "lineNumber" )]]></pre> - </td> - <td> - <p> - Resolves the fields of the <code>StackTraceElement</code> - returned by <code>logEvent.getSource()</code>. - </p> - <p> - Note that this resolver is toggled by - <code>log4j.layout.jsonTemplate.locationInfoEnabled</code> - property. - </p> - </td> - <td>none</td> - <td> - <p>Resolve the line number:</p> - <pre class="prettyprint linenums"><![CDATA[ -{ - "$resolver": "source", - "field": "lineNumber" -}]]></pre> - </td> - </tr> - - <tr> - <td>thread</td> - <td> - <pre class="prettyprint linenums"><![CDATA[ -config = "field" -> ( "name" | "id" | "priority" )]]></pre> - </td> - <td> - resolves <code>logEvent.getThreadId()</code>, - <code>logEvent.getThreadName()</code>, - <code>logEvent.getThreadPriority()</code> - </td> - <td>none</td> - <td> - <p>Resolve the thread name:</p> - <pre class="prettyprint linenums"><![CDATA[ -{ - "$resolver": "thread", - "field": "name" -}]]></pre> - </td> - </tr> - - <tr> - <td>timestamp</td> - <td> - <pre class="prettyprint linenums"><![CDATA[ -config = [ patternConfig | epochConfig ] - -patternConfig = "pattern" -> ( - [ format ] , - [ timeZone ] , - [ locale ] ) -format = "format" -> string -timeZone = "timeZone" -> string -locale = "locale" -> ( - language | - ( language , "_" , country ) | - ( language , "_" , country , "_" , variant ) - ) - -epochConfig = "epoch" -> ( unit , [ rounded ] ) -unit = "unit" -> ( - "nanos" | - "millis" | - "secs" | - "millis.nanos" | - "secs.nanos" | - ) -rounded = "rounded" -> boolean]]></pre> - </td> - <td> - resolves <code>logEvent.getInstant()</code> in various forms - </td> - <td>none</td> - <td> - <table> - - <caption align="top"> - <code>timestamp</code> template resolver examples - </caption> - - <thead> - <tr> - <th>Configuration</th> - <th>Output</th> - </tr> - </thead> - - <tbody> - - <tr> - <td> - <pre class="prettyprint linenums"><![CDATA[ -{ - "$resolver": "timestamp" -}]]></pre> - </td> - <td>2020-02-07T13:38:47.098+02:00</td> - </tr> - - <tr> - <td> - <pre class="prettyprint linenums"><![CDATA[ -{ - "$resolver": "timestamp", - "pattern": { - "format": "yyyy-MM-dd'T'HH:mm:ss.SSS'Z'", - "timeZone": "UTC", - "locale": "en_US" - } -}]]></pre> - </td> - <td>2020-02-07T13:38:47.098Z</td> - </tr> - - <tr> - <td> - <pre class="prettyprint linenums"><![CDATA[ -{ - "$resolver": "timestamp", - "epoch": { - "unit": "secs" - } -}]]></pre> - </td> - <td>1581082727.982123456</td> - </tr> - - <tr> - <td> - <pre class="prettyprint linenums"><![CDATA[ -{ - "$resolver": "timestamp", - "epoch": { - "unit": "secs", - "rounded": true - } -}]]></pre> - </td> - <td>1581082727</td> - </tr> - - <tr> - <td> - <pre class="prettyprint linenums"><![CDATA[ -{ - "$resolver": "timestamp", - "epoch": { - "unit": "secs.nanos" - } -}]]></pre> - </td> - <td>982123456</td> - </tr> - - <tr> - <td> - <pre class="prettyprint linenums"><![CDATA[ -{ - "$resolver": "timestamp", - "epoch": { - "unit": "millis" - } -}]]></pre> - </td> - <td>1581082727982.123456</td> - </tr> - - <tr> - <td> - <pre class="prettyprint linenums"><![CDATA[ -{ - "$resolver": "timestamp", - "epoch": { - "unit": "millis", - "rounded": true - } -}]]></pre> - </td> - <td>1581082727982</td> - </tr> - - <tr> - <td> - <pre class="prettyprint linenums"><![CDATA[ -{ - "$resolver": "timestamp", - "epoch": { - "unit": "millis.nanos" - } -}]]></pre> - </td> - <td>123456</td> - </tr> - - <tr> - <td> - <pre class="prettyprint linenums"><![CDATA[ -{ - "$resolver": "timestamp", - "epoch": { - "unit": "nanos" - } -}]]></pre> - </td> - <td>1581082727982123456</td> - </tr> - - </tbody> - - </table> - </td> - </tr> - - </tbody> - - </table> - - <h4>Stack Trace Element Templates</h4> - - <a name="stack-trace-element-templates"/> - - <p> - <code>stackTraceElement[Uri]</code> describes the JSON structure - <code>JsonTemplateLayout</code> uses to format - <code>StackTraceElement</code>s. The default configuration (accessible - by <code>log4j.layout.jsonTemplate.stackTraceElementTemplate[Uri]</code> - property) is set to <code>classpath:StackTraceElementLayout.json</code> - provided by the <code>log4j-layout-json-template</code> artifact: - </p> - - <pre class="prettyprint linenums"><![CDATA[ -{ - "class": { - "$resolver": "stackTraceElement", - "field": "className" - }, - "method": { - "$resolver": "stackTraceElement", - "field": "methodName" - }, - "file": { - "$resolver": "stackTraceElement", - "field": "fileName" - }, - "line": { - "$resolver": "stackTraceElement", - "field": "lineNumber" - } -}]]></pre> - - <p>The allowed template configuration syntax is as follows:</p> - - <pre class="prettyprint linenums"><![CDATA[ -config = "field" -> ( - "className" | - "fileName" | - "methodName" | - "lineNumber" )]]></pre> - - </subsection> - - <p> - All above accesses to <code>StackTraceElement</code> is garbage-free. - </p> - - </section> - - <section name="Features"> - - <a name="features"/> - - <p> - Below is a feature comparison matrix between - <code>JsonTemplateLayout</code> and alternatives. - </p> - - <table> - - <caption align="top">Feature comparison matrix</caption> - - <thead> - <tr> - <th>Feature</th> - <th><code>JsonTemplateLayout</code></th> - <th><a href="layouts.html#JSONLayout"><code>JsonLayout</code></a></th> - <th><a href="layouts.html#GELFLayout"><code>GelfLayout</code></a></th> - <th><a href="https://github.com/elastic/java-ecs-logging/tree/master/log4j2-ecs-layout";><code>EcsLayout</code></a></th> - </tr> - </thead> - - <tbody> - - <tr> - <td>Java version</td> - <td>8</td> - <td>8</td> - <td>8</td> - <td>6</td> - </tr> - - <tr> - <td>Dependencies</td> - <td>None</td> - <td>Jackson</td> - <td>None</td> - <td>None</td> - </tr> - - <tr> - <td>Schema customization?</td> - <td>✓</td> - <td>✕</td> - <td>✕</td> - <td>✕</td> - </tr> - - <tr> - <td>Timestamp customization?</td> - <td>✓</td> - <td>✕</td> - <td>✕</td> - <td>✕</td> - </tr> - - <tr> - <td>(Almost) garbage-free?</td> - <td>✓</td> - <td>✕</td> - <td>✓</td> - <td>✓</td> - </tr> - - <tr> - <td>Custom typed <code>Message</code> serialization?</td> - <td>✓</td> - <td>✕</td> - <td>✕</td> - <td>?<sup>1</sup></td> - </tr> - - <tr> - <td>Custom typed <code>MDC</code> value serialization?</td> - <td>✓</td> - <td>✕</td> - <td>✕</td> - <td>✕</td> - </tr> - - <tr> - <td>Rendering stack traces as array?</td> - <td>✓</td> - <td>✓</td> - <td>✕</td> - <td>✓</td> - </tr> - - <tr> - <td>JSON pretty print?</td> - <td>✕</td> - <td>✓</td> - <td>✕</td> - <td>✕</td> - </tr> - - <tr> - <td>Additional field support?</td> - <td>✓</td> - <td>✓</td> - <td>✓</td> - <td>✓</td> - </tr> - - </tbody> - - </table> - - <p> - <sup>1</sup> Only for <code>ObjectMessage</code>s and if Jackson is in - the classpath. - </p> - - </section> - - <section name="F.A.Q"> - - <a name="faq"/> - - <h4>Are lookups supported in templates?</h4> - - <a name="faq-lookups"/> - - <p> - Yes, <a href="lookups.html">lookups</a> (e.g., - <code>${java:version}</code>, <code>${env:USER}</code>, - <code>${date:MM-dd-yyyy}</code>) are supported in string - literals of templates. Though note that they are not garbage-free. - </p> - - <h4>Is <code>JsonTemplateLayout</code> garbage-free?</h4> - - <a name="faq-garbage-free"/> - - <p> - Yes, if the garbage-free layout behaviour toggling properties - <code>log4j2.enableDirectEncoders</code> and - <code>log4j2.garbagefreeThreadContextMap</code> are enabled. Take into - account the following caveats: - </p> - - <ul> - - <li> - The configured <a href="#recycling-strategy">recycling strategy</a> - might not be garbage-free. - </li> - - <li> - Since <code>Throwable#getStackTrace()</code> clones the original - <code>StackTraceElement[]</code>, access to (and hence rendering of) - stack traces are not garbage-free. - </li> - - <li> - Serialization of <code>MapMessage</code>s and - <code>ObjectMessage</code>s are mostly garbage-free except for certain - types (e.g., <code>BigDecimal</code>, <code>BigInteger</code>, - <code>Collection</code>s with the exception of <code>List</code>). - </li> - - <li> - <a href="lookups.html">Lookups</a> (that is, <code>${...}</code> - variables) are not garbage-free. - </li> - - </ul> - - <p> - Don't forget to checkout <a href="#event-template-resolvers">the notes - on garbage footprint of resolvers</a> you employ in templates. - </p> - - </section> - - </body> - -</document>
