ppkarwasz commented on code in PR #2717:
URL: https://github.com/apache/logging-log4j2/pull/2717#discussion_r1668209963
##########
src/site/antora/modules/ROOT/pages/manual/filters.adoc:
##########
@@ -17,1038 +17,1407 @@
[id=filters]
= Filters
+:stem:
-Log4j supports filtering of log events at each level of the logging pipeline
using two features:
+Filters are Log4j plugins that evaluate the parameters of a logging call or a
log event and return one of three results:
-* the `level` attributes that can be set on loggers and appender references,
-* filter components that can be attached to loggers, appenders, appender
references or the global configuration object.
+ACCEPT:: The log event is accepted by the filter and goes to the next stage of
the logging pipeline.
-Filters evaluate the parameters of a logging call (context-wide filter) or a
log event and return one of three results:
+DENY:: The log event is unconditionally dropped.
-ACCEPT:: The log event is accepted by the filter and goes to the next stage of
the logging pipeline,
+NEUTRAL:: Log4j behaves as if the filter was not present.
-DENY:: The log event is unconditionally dropped,
+Filters can be used at each level of the
+xref:manual/architecture.adoc#architecture-diagram[logging pipeline]:
-NEUTRAL:: Log4j behaves as if the filter was not present.
+* the global configuration element can contain a
xref:manual/configuration.adoc#global-filters[global filter].
+* each xref:manual/configuration.adoc#configuring-loggers[logger]
configuration element can contain a
xref:manual/configuration.adoc#logger-elements-filters[logger filter].
+* each xref:manual/configuration.adoc#configuring-appenderrefs[appender
reference] configuration element can contain an
xref:manual/configuration.adoc#appenderrefs-elements-filters[appender reference
filter].
+* each xref:manual/appenders.adoc[appender] configuration element can contain
an xref:manual/appenders.adoc[appender filter].
+
+Additionally, the following configuration attributes take part in the
filtering process:
+
+* the xref:manual/configuration.adoc#logger-attributes-level[`level`
attribute] of logger configuration elements.
+* the xref:manual/configuration.adoc#appenderref-attributes-level[`level`
attribute] of appender reference configuration elements.
+
+[#filtering-process]
+== Filtering process
+
+Due to the interaction of many elements,
+the filtering process in Log4j is quite complex and can be divided in four
stages:
+
+. <<logger-stage,`Logger` stage>>
+. <<logger-config-stage,`LoggerConfig` stage>>
+. <<appender-control-stage,`AppenderControl` stage>>
+. <<appender-stage,`Appender` stage>>
+
+[IMPORTANT]
+====
+For performance reasons, log events should be filtered at the earliest
possible stage.
+This reduces the cost of disabled log events:
+e.g., log event creation, population of context data, formatting, transfer
through an asynchronous barrier.
+====
+
+[#logger-stage]
+=== 1. `Logger` stage
+
+[plantuml]
+....
+@startuml
+start
+group Logger
+
+:A Logger method;
+
+switch (Apply global filter)
+case (DENY)
+ #pink:Discard;
+ detach
+case (ACCEPT)
+case (NEUTRAL)
+ if (Is less severe than logger level?) then (yes)
+ #pink:Discard;
+ detach
+ else (no)
+ ' The label improves spacing
+ label a1
+ endif
+endswitch
+end group
+:Create LogEvent;
+stop
+....
+
+The parameters of the logging call are passed to the global filter.
+If the global filter returns:
+
+DENY:: The log message is immediately discarded.
+NEUTRAL:: If the level of the log message is less severe than the configured
logger threshold, the message is discarded.
+Otherwise, a
+link:../javadoc/log4j-core/org/apache/logging/log4j/core/LogEvent.html[`LogEvent`]
is created and processing continues.
+ACCEPT:: A `LogEvent` is created and processing continues in the next stage.
+
+[TIP]
+====
+Filtering logging calls at this stage provides the best performance:
+
+* this stage precedes the creation of log events, therefore operations like the
+xref:manual/thread-context.adoc[injection of context data],
+xref:manual/layouts.adoc#LocationInformation[computation of location
information]
+will not be performed for disabled log statements.
+* this stage precedes the asynchronous calls performed by either
+xref:manual/async.adoc[asynchronous loggers]
+or
+xref:manual/appenders.adoc#AsyncAppender[asynchronous appenders].
+====
+
+[#logger-config-stage]
+=== 2. `LoggerConfig` stage
+
+[plantuml]
+....
+@startuml
+start
+group LoggerConfig
+repeat
+
+:LoggerConfig.log();
+
+if (Apply logger filter) then (DENY)
+ #pink:Discard;
+ detach
+else (not DENY)
+ ' The label improves spacing
+ label a1
+endif
+repeat while (Go to parent logger?) is (yes)
+-> no;
+end group
+stop
+@enduml
+....
+
+In this stage, log events are evaluated by all the
+xref:manual/configuration.adoc#logger-elements-filters[logger filters]
+that stand on the path from the logger to an appender.
+Due to the
+xref:manual/configuration.adoc#logger-attributes-additivity[additivity of
logger configurations],
+this means that a log event must also pass the filters of all the parent
loggers,
+until it reaches the logger that references the chosen appender.
+
+[#appender-control-stage]
+=== 3. `AppenderControl` stage
+
+[plantuml]
+....
+@startuml
+!pragma useVerticalIf on
+start
+group AppenderControl
+
+:AppenderControl.callAppender();
+
+if (Is less severe then appender reference level?) then (yes)
+ #pink:Discard;
+ detach
+else (no)
+ ' The label improves spacing
+ label a2
+endif
+if (Apply appender reference filter) then (DENY)
+ #pink:Discard;
+ detach
+else (not DENY)
+ ' The label improves spacing
+ label a1
+endif
+end group
+stop
+@enduml
+....
+
+To pass this stage, log events must satisfy both conditions:
+
+* the log event must be at least as severe as the
+xref:manual/configuration.adoc#appenderref-attributes-level[`level` attribute]
+of the appender reference.
+* the xref:manual/configuration.adoc#appenderrefs-elements-filters[appender
reference filter] must return `ACCEPT` or `NEUTRAL`,
+
+[#appender-stage]
+=== 4. `Appender` stage
+
+[plantuml]
+....
+@startuml
+start
+group Appender
+
+:Appender.append();
+
+if (Apply appender filter) then (DENY)
+ #pink:Discard;
+ detach
+else (not DENY)
+ ' The label improves spacing
+ label a1
+endif
+end group
+#palegreen:Log message;
+@enduml
+....
+
+When log events reach an appender,
+the filter attached to an appender is evaluated and if the result is `DENY`,
+the log event is discarded.
+
+[NOTE]
+====
+Some appenders like the
+xref:manual/appenders.adoc#AsyncAppender[asynchronous appender]
+use appender references to transfer log events to other appenders.
+In such a case, the filtering process goes back to the
<<appender-control-stage,`AppenderControl` stage>>.
+====
+
+[TIP]
+====
+Users migrating from Log4j 1 often replace the `threshold` property of a Log4j
1 appender with a <<ThresholdFilter>> on the equivalent Log4j 2 appender.
+
+Using the `level` property of appender references will give a better
performance.
+====
+
+[WARNING]
+====
+Configuring filters is a measure of last resort,
+since it adds a large overhead to disabled log events.
+You should rather configure the filtering in one of the previous stages.
+====
-To decide whether a log event from a certain logger is delivered to a specific
appender, the following procedure is followed:
+[#example-configuration-file]
+=== Example configuration file
+
+The following example shows the order in which filters are evaluated:
[tabs]
====
XML::
+
-.Snippet from an example
{antora-examples-url}/manual/filters/filters.xml[`log4j2.xml`]
[source,xml]
----
-include::example$manual/filters/filters.xml[lines=23..41,indent=0]
+include::example$manual/filters/filters.xml[lines=1;18..-1]
----
JSON::
+
-.Snippet from an example
{antora-examples-url}/manual/filters/filters.json[`log4j2.json`]
[source,json]
----
-include::example$manual/filters/filters.json[lines=3..35,indent=0]
+include::example$manual/filters/filters.json[]
----
YAML::
+
-.Snippet from an example
{antora-examples-url}/manual/filters/filters.yaml[`log4j2.yaml`]
[source,yaml]
----
-include::example$manual/filters/filters.yaml[lines=18..-1]
+include::example$manual/filters/filters.yaml[lines=17..-1]
----
Properties::
+
-.Snippet from an example
{antora-examples-url}/manual/filters/filters.properties[`log4j2.properties`]
[source,properties]
----
include::example$manual/filters/filters.properties[lines=17..-1]
----
====
-<1> First the context-wide filter is consulted.
-If it returns `ACCEPT` the log message goes directly to point 3.
-<2> Then Log4j checks the message against the configured logger level.
-<3> The filter configured on a logger is applied next.
-If the logger is additive, the filter on the parent logger is applied
recursively until we end up on the logger that references the given appender.
-<4> Next comes the turn of the filter configured on an appender reference,
-<5> followed by a level check against the configured level of the reference.
-<6> The process ends with the filter attached to an appender.
+<1> Global filter
+<2> Logger `level` attribute (it is skipped if the event matches the global
filter)
+<3> Filter of the "org.example" logger
+<4> Filter of the root logger (it is the parent of the "org.example" logger)
+<5> Appender reference `level` attribute
+<6> Filter of the appender reference
+<7> Filter of the appender
-[WARNING]
-====
-For performance reasons, log events should be filtered as soon as possible in
the logging pipeline.
-This reduces the costs (formatting, transfer through an asynchronous barrier)
of disabled log events.
-====
+[#common-configuration]
+== Common configuration
-[TIP]
-====
-Users migrating from Log4j 1 often replace the `threshold` property of a Log4j
1 appender with a <<ThresholdFilter>> on the equivalent Log4j 2 appender.
+[#common-configuration-attributes]
+=== Common configuration attributes
-Using the `level` property of appender references will give a better
performance.
-====
+The default behavior of filters is in line with the `filter()` methods of
functional interfaces, such as
+https://docs.oracle.com/javase/{java-target-version}/docs/api/java/util/Optional.html#filter-java.util.function.Predicate-[`Optional.filter()`]
+or
+https://docs.oracle.com/javase/{java-target-version}/docs/api/java/util/stream/Stream.html#filter-java.util.function.Predicate-[`Stream.filter()`]:
+filters pass matching events to the next filter and drop those that do not
match.
+
+To allow for a larger spectrum of behaviors,
+all standard filters, except `CompositeFilter` and `DenyAllFilter`, accept the
following configuration attributes:
+
+.Common filter configuration attributes
+[cols="1m,1,1,4"]
+|===
+|Attribute |Type | Default value |Description
+
+| [[onMatch]]onMatch
+|
link:../javadoc/log4j-core/org/apache/logging/log4j/core/Filter.Result.html[`Result`]
+|
link:../javadoc/log4j-core/org/apache/logging/log4j/core/Filter.Result.html#NEUTRAL[`NEUTRAL`]
+| Result returned if the condition matches.
+
+| [[onMismatch]]onMismatch
+|
link:../javadoc/log4j-core/org/apache/logging/log4j/core/Filter.Result.html[`Result`]
+|
link:../javadoc/log4j-core/org/apache/logging/log4j/core/Filter.Result.html#DENY[`DENY`]
+| Result returned if the condition does not match.
+
+|===
+
+[#CompositeFilter]
+=== Composing filters
+
+Filters usually test for a single condition.
+To express a more complex filtering logic, Log4j provides a
+xref:plugin-reference.adoc#org-apache-logging-log4j_log4j-core_org-apache-logging-log4j-core-filter-CompositeFilter[`Filters`]
+plugin.
+This plugin can contain a sequence of filters and has no other configuration
option.
+
+The `Filters` plugin sequentially evaluates each sub-filter and:
+
+* if the sub-filter returns `ACCEPT` (resp. `DENY`), the `Filters` plugin
returns `ACCEPT` (resp. `DENY`).
+* if the sub-filter return `NEUTRAL`, the `Filters` plugin evaluates the next
sub-filter in the chain.
+* if the last sub-filter returns `NEUTRAL`, the `Filters` plugin returns
`NEUTRAL`.
+
+The `Filters` plugin together with the ternary logic of filters, can be used
to express all the boolean operators.
+In the following examples `A` and `B` are two filters.
+
+`NOT A`::
+You can invert the functionality of filter `A` by swapping the `onMatch` and
`onMismatch`:
++
+[source,xml]
+----
+<A onMatch="DENY" onMismatch="NEUTRAL"/>
+----
+
+`A AND B`::
+To select the events that match both `A` and `B` you can use:
++
+[source,xml]
+----
+<Filters>
+ <A/>
+ <B/>
+</Filters>
+----
+
+`A OR B`::
+To select the events that match `A` or `B` we can replace `NEUTRAL` with
`ACCEPT` in the `onMatch` attribute:
++
+[source,xml]
+----
+<Filters>
+ <A onMatch="ACCEPT"/>
+ <B onMatch="ACCEPT"/>
+</Filters>
+----
+
+xref:plugin-reference.adoc#org-apache-logging-log4j_log4j-core_org-apache-logging-log4j-core-filter-CompositeFilter[📖
Plugin reference for `Filters`]
[#collection]
== Collection
-Log4j bundles several predefined filters to assist in several common
deployment use cases.
-Following sections explain all these in detail.
+Log4j Core provides the following filters out-of-the-box.
+
+[#timestamp-filters]
+=== Timestamp filters
+
+Timestamp filters use the timestamp of log events to decide whether to log
them or not.
[#BurstFilter]
-=== BurstFilter
+==== `BurstFilter`
-The BurstFilter provides a mechanism to control the rate at which LogEvents
are processed by silently discarding events after the maximum limit has been
reached.
+The `BurstFilter` uses the _sliding window log_ algorithm to limit the rate of
log events.
Review Comment:
I add a description in 6448bdbb604651d99c71df1cda9b17f887f1bd29.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
