Re: Context data propagation and logger-bound context data

[Raman Gupta]

On Thu, Mar 21, 2024 at 9:58 PM Ralph Goers 
wrote:

>
> > On Mar 21, 2024, at 2:48 PM, Raman Gupta  wrote:
> >
> > On Thu, Mar 21, 2024 at 5:30 AM Piotr P. Karwasz <
> piotr.karw...@gmail.com>
> > wrote:
> >
> >> Hi Ralph,
> >>
> >> On Thu, 21 Mar 2024 at 07:03, Ralph Goers 
> >> wrote:
>  1. Raman and Mikko would like to bind context data to an object
>  implementing the `Logger` interface or more generally to a service
>  object (e.g. resource manager, DAO and variants),
> >>>
> >>> Yes, I’ve seen that. Personally, I am not much of a fan of this use
> case
> >> as it is pretty easy to add the data you want to a single class. That
> said,
> >> we already offer a solution. Allowing a MessageFactory to be provided
> on a
> >> Logger was done for exactly this reason.
> >>>
> >>> For example, a User could configure a custom MessageFactory that
> >> provides an extension of MapMessage that causes the message to include
> data
> >> from the class or resource. Going to the extreme of trying to shoehorn
> in
> >> Context data as well simply isn’t necessary.
> >>
> >> Great point! This effectively solves Raman's and Mikko's problem. We
> >> just need to introduce a new sub-interface of `Message` and require
> >> all implementations to support it.
> >>
> >>
> > I'm not sure it does. I differentiate between context and message. When I
> > want the data from the resource in the context, it's because the data is
> > contextual, not part of the message.
> >
> > This approach seems to be not much better than making sure all log
> messages
> > in a class have a consistent string format so it can be grepped.
>
> No, it is nothing like that. LogEvents effectively support 2 kinds of
> Maps; those associated with the ThreadContext (or injected into the
> LogEvents contextMap, or key/value pairs associated with MapMessages. In
> the PatternLayout these are distinguished by %X and %K. If you know the
> keys you want you can use a pattern like:
>
> %d %p %C{1.} [%t] {requestId=%X{requestId}
> customerNumber=%X{customerNumber} userName=%K{userName} %{userType} - %m%n
>
> Thus making the ThreadContext key/values and Message key/values virtually
> indistinguishable.


When one controls the layout with something like PatternLayout, yes.
However in structured logging use cases (e.g. to Google Cloud Logging) the
context and message maps are shown under separate "context" and "message"
keys. As they should be because they are different things.

We have context and we have a message -- I don't think it should be
considered best practice to conflate these.


> The only issue at the moment is the result of %m on a MapMessage isn’t
> really what you would like. I am looking into addressing that as I have run
> across this before in things I have tried to do.
>
> Ralph

Re: Context data propagation and logger-bound context data

[Ralph Goers]

> On Mar 21, 2024, at 3:19 PM, Piotr P. Karwasz  wrote:
> 
> 
> PS: In the `main` branch I plan to replace the default
> ReusableMessageFactory with ReusableLogEventFactory:
> 
> * this change should have a minimal impact on memory (events are not
> much larger than messages),
> * it can skip a data transfer between ReusableMessage and MutableLogEvent,
> * it allows the message factory to collect data that is usually
> reserved for events.

I have to question this. I don’t understand the concept of replacing a 
MessageFactory with a LogEventFactory. They are completely different things. 
What does this even mean?

Ralph

Re: Context data propagation and logger-bound context data

[Ralph Goers]

> On Mar 21, 2024, at 2:48 PM, Raman Gupta  wrote:
> 
> On Thu, Mar 21, 2024 at 5:30 AM Piotr P. Karwasz 
> wrote:
> 
>> Hi Ralph,
>> 
>> On Thu, 21 Mar 2024 at 07:03, Ralph Goers 
>> wrote:
 1. Raman and Mikko would like to bind context data to an object
 implementing the `Logger` interface or more generally to a service
 object (e.g. resource manager, DAO and variants),
>>> 
>>> Yes, I’ve seen that. Personally, I am not much of a fan of this use case
>> as it is pretty easy to add the data you want to a single class. That said,
>> we already offer a solution. Allowing a MessageFactory to be provided on a
>> Logger was done for exactly this reason.
>>> 
>>> For example, a User could configure a custom MessageFactory that
>> provides an extension of MapMessage that causes the message to include data
>> from the class or resource. Going to the extreme of trying to shoehorn in
>> Context data as well simply isn’t necessary.
>> 
>> Great point! This effectively solves Raman's and Mikko's problem. We
>> just need to introduce a new sub-interface of `Message` and require
>> all implementations to support it.
>> 
>> 
> I'm not sure it does. I differentiate between context and message. When I
> want the data from the resource in the context, it's because the data is
> contextual, not part of the message.
> 
> This approach seems to be not much better than making sure all log messages
> in a class have a consistent string format so it can be grepped.

No, it is nothing like that. LogEvents effectively support 2 kinds of Maps; 
those associated with the ThreadContext (or injected into the LogEvents 
contextMap, or key/value pairs associated with MapMessages. In the 
PatternLayout these are distinguished by %X and %K. If you know the keys you 
want you can use a pattern like:

%d %p %C{1.} [%t] {requestId=%X{requestId} customerNumber=%X{customerNumber} 
userName=%K{userName} %{userType} - %m%n

Thus making the ThreadContext key/values and Message key/values virtually 
indistinguishable. 

The only issue at the moment is the result of %m on a MapMessage isn’t really 
what you would like. I am looking into addressing that as I have run across 
this before in things I have tried to do.

Ralph

Re: Context data propagation and logger-bound context data

[Raman Gupta]

On Thu, Mar 21, 2024 at 6:19 PM Piotr P. Karwasz 
wrote:

> Hi Raman,
>
> On Thu, 21 Mar 2024 at 22:51, Raman Gupta  wrote:
> > > > For example, a User could configure a custom MessageFactory that
> > > provides an extension of MapMessage that causes the message to include
> data
> > > from the class or resource. Going to the extreme of trying to shoehorn
> in
> > > Context data as well simply isn’t necessary.
> > >
> > > Great point! This effectively solves Raman's and Mikko's problem. We
> > > just need to introduce a new sub-interface of `Message` and require
> > > all implementations to support it.
> > >
> > >
> > I'm not sure it does. I differentiate between context and message. When I
> > want the data from the resource in the context, it's because the data is
> > contextual, not part of the message.
> >
> > This approach seems to be not much better than making sure all log
> messages
> > in a class have a consistent string format so it can be grepped.
>
> I was thinking about a code like this:
>
> private static final Logger mainLogger = LogManager.getLogger();
>
> private final Logger logger;
>
> public ResourceManager() {
> Map contextData = Map.of(...);
> logger = mainLogger.withMessageFactory(new
> ContextualMessageFactory(contextData));
> }
>
> The new `Logger#withMessageFactory` method will give you a logger with
> the same name as `mainLogger`, but with a different message factory.
> The message factory will create LogEvents initialized with the given
> context, which will be further complemented by the context data
> injected by ContextDataInjector. Is this the effect you are looking
> for?
>

Yes, your description sounds like what I'm looking for, but I am confused
with the name `withMessageFactory`. As I understand it, the
`MessageFactory` interface allows creating Messages, not LogEvents with
context.

Thanks,
Raman


> Piotr
>
> PS: In the `main` branch I plan to replace the default
> ReusableMessageFactory with ReusableLogEventFactory:
>
> * this change should have a minimal impact on memory (events are not
> much larger than messages),
> * it can skip a data transfer between ReusableMessage and MutableLogEvent,
> * it allows the message factory to collect data that is usually
> reserved for events.
>

Re: Context data propagation and logger-bound context data

[Piotr P. Karwasz]

Hi Raman,

On Thu, 21 Mar 2024 at 22:51, Raman Gupta  wrote:
> > > For example, a User could configure a custom MessageFactory that
> > provides an extension of MapMessage that causes the message to include data
> > from the class or resource. Going to the extreme of trying to shoehorn in
> > Context data as well simply isn’t necessary.
> >
> > Great point! This effectively solves Raman's and Mikko's problem. We
> > just need to introduce a new sub-interface of `Message` and require
> > all implementations to support it.
> >
> >
> I'm not sure it does. I differentiate between context and message. When I
> want the data from the resource in the context, it's because the data is
> contextual, not part of the message.
>
> This approach seems to be not much better than making sure all log messages
> in a class have a consistent string format so it can be grepped.

I was thinking about a code like this:

private static final Logger mainLogger = LogManager.getLogger();

private final Logger logger;

public ResourceManager() {
Map contextData = Map.of(...);
logger = mainLogger.withMessageFactory(new
ContextualMessageFactory(contextData));
}

The new `Logger#withMessageFactory` method will give you a logger with
the same name as `mainLogger`, but with a different message factory.
The message factory will create LogEvents initialized with the given
context, which will be further complemented by the context data
injected by ContextDataInjector. Is this the effect you are looking
for?

Piotr

PS: In the `main` branch I plan to replace the default
ReusableMessageFactory with ReusableLogEventFactory:

* this change should have a minimal impact on memory (events are not
much larger than messages),
* it can skip a data transfer between ReusableMessage and MutableLogEvent,
* it allows the message factory to collect data that is usually
reserved for events.

Re: Context data propagation and logger-bound context data

[Raman Gupta]

On Thu, Mar 21, 2024 at 5:30 AM Piotr P. Karwasz 
wrote:

> Hi Ralph,
>
> On Thu, 21 Mar 2024 at 07:03, Ralph Goers 
> wrote:
> > > 1. Raman and Mikko would like to bind context data to an object
> > > implementing the `Logger` interface or more generally to a service
> > > object (e.g. resource manager, DAO and variants),
> >
> > Yes, I’ve seen that. Personally, I am not much of a fan of this use case
> as it is pretty easy to add the data you want to a single class. That said,
> we already offer a solution. Allowing a MessageFactory to be provided on a
> Logger was done for exactly this reason.
> >
> > For example, a User could configure a custom MessageFactory that
> provides an extension of MapMessage that causes the message to include data
> from the class or resource. Going to the extreme of trying to shoehorn in
> Context data as well simply isn’t necessary.
>
> Great point! This effectively solves Raman's and Mikko's problem. We
> just need to introduce a new sub-interface of `Message` and require
> all implementations to support it.
>
>
I'm not sure it does. I differentiate between context and message. When I
want the data from the resource in the context, it's because the data is
contextual, not part of the message.

This approach seems to be not much better than making sure all log messages
in a class have a consistent string format so it can be grepped.



> To be sure we don't cherry-pick each element of a log event with a
> different interface (like we did with `TimeStampMessage`), let's use a
> single interface to supply all the additional data the user might want
> to have. Let's throw in it Ceki's `KeyValuePair`s[1]
>
> Piotr
>
> [1] https://www.slf4j.org/api/org/slf4j/event/LoggingEvent.html
>

Re: [VOTE] Release Apache Log4j Tools 0.8.0 (RC3)

[Piotr P. Karwasz]

+1, release the artifacts.

I repeated the same checks as in RC2:

* distribution hashes: OK
* distribution signatures: OK
* Maven artifacts are reproducible: OK
* Maven artifacts are the same as binary distribution: OK
* new artifacts are documented on the website.

Piotr

On Thu, 21 Mar 2024 at 16:46, Volkan Yazıcı  wrote:
>
> This is a vote to release the Apache Log4j Tools 0.8.0 (RC3).
>
> Website: https://logging.staged.apache.org/log4j/tools
> GitHub: https://github.com/apache/logging-log4j-tools
> Commit: 7d157b61e198e3baca816129d8d406b300436e2f
> Distribution: https://dist.apache.org/repos/dist/dev/logging/log4j-tools
> Nexus:
> https://repository.apache.org/content/repositories/orgapachelogging-1266
> Signing key: 0x077e8893a6dcc33dd4a4d5b256e73ba9a0b592d0
>
> Please download, test, and cast your votes on this mailing list.
>
> [ ] +1, release the artifacts
> [ ] -1, don't release, because...
>
> This vote is open until 2024-03-22 12:30 (CET) and will pass unless
> getting a net negative vote count. All votes are welcome and we
> encourage everyone to test the release, but only the Logging
> Services PMC votes are officially counted. At least 3 +1 votes and
> more positive than negative votes are required.
>
> === Timing & Differences
>
> I will keep this vote open until 2024-03-22 12:30 (CET), that is, the
> official end date of the RC2, since RC3 contains a minor difference:
>
> https://github.com/apache/logging-log4j-tools/compare/2bb07037bbbfe14fe1c224d46a3e4135b48ffde6...7d157b61e198e3baca816129d8d406b300436e2f
>
> === Review kit
>
> The minimum set of steps needed to review the uploaded distribution
> files in the Subversion repository can be summarized as follows:
>
> # Check out the distribution
> svn co https://dist.apache.org/repos/... && cd $_
>
> # Verify checksums
> shasum --check *.sha512
>
> # Verify signatures
> wget -O - https://downloads.apache.org/logging/KEYS | gpg --import
> for sigFile in *.asc; do gpg --verify $sigFile; done
>
> # Verify reproduciblity
> umask 0022
> unzip *-src.zip -d src
> cd src
> export NEXUS_REPO=https://repository.apache.org/content/...
> sh mvnw -Prelease verify artifact:compare -Dreference.repo=$NEXUS_REPO
> # If preferred, augment `mvnw` with `-DskipTests` to speed things up
>
> === Release notes
>
> This release delivers the first version of Log4j Docgen (Documentation
> Generator).
> It is a set of tools to auto-generate the Log4j plugin documentation (to be
> integrated into the website) and the Log4j configuration XSD file (for
> assisting the configuration of the Log4j runtime, i.e., `log4j2.xml`) from
> the Log4j source code. See the project website for details.
>
>  Added
>
> * Add the `log4j-docgen` et al. containing a universal XML model to
> document Log4j plugins
>
>  Changed
>
> * Move Log4j Changelog XML namespace and schema location to `
> https://logging.apache.org/xml/ns` and `
> https://logging.apache.org/xml/ns/log4j-changelog-0.xsd`, respectively
>
>  Removed
>
> * Remove `author` from Log4j Changelog. It was yet another bit to maintain
> and created role-related (who did what) problems. Many modern software
> projects use a VCS (e.g., Git) and support services (e.g., GitHub) which
> make it trivial to trace back the origin of a change using commit and issue
> IDs.
>
>  Updated
>
> * Update `org.apache.logging:logging-parent` to version `10.6.0`
> * Update `jakarta.inject:jakarta.inject-api` to version `2.0.1` (#94)
> * Update `org.apache.logging.log4j:log4j-core` to version `2.23.1` (#108)
> * Update `org.apache.logging.log4j:log4j-plugins` to version `3.0.0-beta2`
> (#107)
> * Update `org.apache.maven.plugin-tools:maven-plugin-annotations` to
> version `3.11.0` (#98)
> * Update `org.assertj:assertj-core` to version `3.25.3` (#104)
> * Update `org.codehaus.modello:modello-maven-plugin` to version `2.3.0`
> (#105)
> * Update `org.junit:junit-bom` to version `5.10.2` (#103)

Re: Context data propagation and logger-bound context data

[Matt Sicker]

https://www.javadoc.io/doc/org.apache.logging.log4j/log4j-api/3.0.0-alpha1/org.apache.logging.log4j/org/apache/logging/log4j/LogManager.html

Pass your custom MessageFactory here. It’s an optional argument when creating 
the Logger.

Also, I’m not sure where to even find the current javadocs for the API. 
javadoc.io  only seems to have this alpha release.


> On Mar 21, 2024, at 04:34, Volkan Yazıcı  wrote:
> 
> Ralph, could you show how those two users can use a `MessageFactory` to
> create `Logger`s with predefined additional context data?
> 
> On Thu, Mar 21, 2024 at 7:25 AM Ralph Goers  wrote:
> 
>> Unfortunately this is another message I somehow didn't get in my inbox.
>> Replying to it via lists.a.o is not a great experience but is the best I
>> can do.
>> 
>> On 2024/03/20 13:51:56 Volkan Yazıcı wrote:
>>> I agree with the way Piotr dissects the problem. I think `ScopedContext`,
>>> even though it has its own merits, doesn't address the problem reported
>> by
>>> users. They simply want a new logger associated with some additional
>>> context data.
>> 
>> Two users do.  I have personally been asked for something like
>> ScopedContext several times.
>> As I replied to Piotr, we already solved the problem of adding data to
>> Loggers. That is what MessageFactories are intended for.
>> 
>>> 
>>> *[See my comments below.]*
>>> 
>>> On Mon, Mar 18, 2024 at 10:40 AM Piotr P. Karwasz <
>> piotr.karw...@gmail.com>
>>> wrote:
>>> 
 * we can create a `Logger` wrapper "bound" to context data as Mikko
 does. This wrapper will take care of setting the `ThreadContext`
 before the logger call and restore it after it.
>>> 
>>> Creating a wrapper `Logger` can work without needing to deal with
>>> `ThreadContext`. I can think of two different ways to carry this out:
>>> 
>>>   1. Currently, `AbstractLogger` only creates `Message`s. We can rework
>> it
>>>   to create `LogEvent`s too. Once `AbstractLogger` gets its hand on a
>>>   `LogEvent`, it can enrich its context data as it wishes.
>>>   2. We can extend `ContextDataInjector` with a new `void
>>>   injectContextData(Logger logger, StringMap target)` method, provide a
>>>   `ContextDataInjector` implementation that branches on `logger
>> instanceof
>>>   ContextDataProvider`, and call `ContextDataInjector` with the
>> associated
>>>   `Logger` in `LogEventFactory`.
>> 
>> We can do lots of things, most of which I wouldn't recommend. As to yours:
>> 1. Logger/AbstractLogger got very complex with Async, Garbage Free,
>> Reliablity Strategies, etc. Trying to move creating the LogEvent sooner is
>> likely to be a major PITA and could seriously impact performance. While we
>> could add a context map to AbstractLogger we would have to pass that on the
>> logging calls to LoggerConfig and deal with all that that means - remember,
>> a LoggerConfig can be handling multiple Loggers.
>> 2. I don't recommend extending ContextDataInjector. That proved difficult
>> to work with which is why we now recommend using ContextDataProviders. You
>> really can only have one ContextDataInjector. Also, please note that
>> ContextDataInjector is called while constructing the LogEvent. The LogEvent
>> isn't passed the Logger, only the LoggerName. Looking up the Logger to do
>> this is yet another way to slow down logging.
>> 
>>> 
>>> On Tue, Mar 19, 2024 at 7:45 AM Ralph Goers 
>>> wrote:
 In the meantime, I provided
>>> https://github.com/apache/logging-log4j2/pull/2385 which I very loosely
>>> modeled after ScopedValues.
>>> 
>>> The fact that `ScopedContext` tries to imitate `ScopedValue` using
>>> `ThreadLocal`s is extremely confusing (from a user's pov) and risky
>>> liability (from a maintainer's pov). I guess you wanted to implement *a*
>>> `ScopedValue` without using *the* `ScopedValue` to be compatible with
>> Java
>>> 8. If so, that really sounds like the `o.a.l.log4j.util.Supplier`
>> downward
>>> spiral. We can rather have an *internal* `Log4jScopedValue` interface and
>>> provide Java 8 (using `InheritableThreadLocal`) and Java 21+ (using
>>> `ScopedValue`) compatible solutions in an MRJ (Multi-Release JAR).
>> 
>> I am NOT trying to make ScopedContext compatible with ScopedValue. I am
>> trying to make it conceptually close enough to ScopedValue that users will
>> understand what it is doing.
>> We can argue about naming if you want. Gary has already expressed his
>> opinion.
>>> 
>>> We can integrate `ScopedContext` to the `LogEventFactory` by providing a
>>> specialized `ContextDataInjector` plugin – assuming `LogEventFactory`
>>> employs all available `ContextDataInjector` plugins.
>> 
>> ScopedContext is integrated with a ContextDataProvider, which is the
>> supported way to do this. Again, you cannot have more than one
>> ContextDataInjector so providing "specialized versions" is a pipe dream.
>> You will simply have to enhance the one we already have.
>> ContextDataInjector is NOT a plugin.
>> 
>>> 
>>> I find the current ceremony 

Re: [VOTE] Release Apache Log4j Tools 0.8.0 (RC3)

[Volkan Yazıcı]

What do you mean? That is exactly the link I shared in the very first
section of the VOTE email you responded to. See the `=== Timing &
Differences` header. Note that you don't need to trust the GitHub view, you
can diff RC2 and RC3 using their commit IDs available in the associated
VOTE emails.

On Thu, Mar 21, 2024 at 5:00 PM Gary Gregory  wrote:

> What's the difference with RC2?
>
> TY,
> Gary
>
> On Thu, Mar 21, 2024, 11:46 AM Volkan Yazıcı  wrote:
>
> > This is a vote to release the Apache Log4j Tools 0.8.0 (RC3).
> >
> > Website: https://logging.staged.apache.org/log4j/tools
> > GitHub: https://github.com/apache/logging-log4j-tools
> > Commit: 7d157b61e198e3baca816129d8d406b300436e2f
> > Distribution: https://dist.apache.org/repos/dist/dev/logging/log4j-tools
> > Nexus:
> > https://repository.apache.org/content/repositories/orgapachelogging-1266
> > Signing key: 0x077e8893a6dcc33dd4a4d5b256e73ba9a0b592d0
> >
> > Please download, test, and cast your votes on this mailing list.
> >
> > [ ] +1, release the artifacts
> > [ ] -1, don't release, because...
> >
> > This vote is open until 2024-03-22 12:30 (CET) and will pass unless
> > getting a net negative vote count. All votes are welcome and we
> > encourage everyone to test the release, but only the Logging
> > Services PMC votes are officially counted. At least 3 +1 votes and
> > more positive than negative votes are required.
> >
> > === Timing & Differences
> >
> > I will keep this vote open until 2024-03-22 12:30 (CET), that is, the
> > official end date of the RC2, since RC3 contains a minor difference:
> >
> >
> >
> https://github.com/apache/logging-log4j-tools/compare/2bb07037bbbfe14fe1c224d46a3e4135b48ffde6...7d157b61e198e3baca816129d8d406b300436e2f
> >
> > === Review kit
> >
> > The minimum set of steps needed to review the uploaded distribution
> > files in the Subversion repository can be summarized as follows:
> >
> > # Check out the distribution
> > svn co https://dist.apache.org/repos/... && cd $_
> >
> > # Verify checksums
> > shasum --check *.sha512
> >
> > # Verify signatures
> > wget -O - https://downloads.apache.org/logging/KEYS | gpg --import
> > for sigFile in *.asc; do gpg --verify $sigFile; done
> >
> > # Verify reproduciblity
> > umask 0022
> > unzip *-src.zip -d src
> > cd src
> > export NEXUS_REPO=https://repository.apache.org/content/...
> > sh mvnw -Prelease verify artifact:compare
> -Dreference.repo=$NEXUS_REPO
> > # If preferred, augment `mvnw` with `-DskipTests` to speed things up
> >
> > === Release notes
> >
> > This release delivers the first version of Log4j Docgen (Documentation
> > Generator).
> > It is a set of tools to auto-generate the Log4j plugin documentation (to
> be
> > integrated into the website) and the Log4j configuration XSD file (for
> > assisting the configuration of the Log4j runtime, i.e., `log4j2.xml`)
> from
> > the Log4j source code. See the project website for details.
> >
> >  Added
> >
> > * Add the `log4j-docgen` et al. containing a universal XML model to
> > document Log4j plugins
> >
> >  Changed
> >
> > * Move Log4j Changelog XML namespace and schema location to `
> > https://logging.apache.org/xml/ns`  <
> https://logging.apache.org/xml/ns>
> > and `
> > https://logging.apache.org/xml/ns/log4j-changelog-0.xsd`
> 
> > , respectively
> >
> >  Removed
> >
> > * Remove `author` from Log4j Changelog. It was yet another bit to
> maintain
> > and created role-related (who did what) problems. Many modern software
> > projects use a VCS (e.g., Git) and support services (e.g., GitHub) which
> > make it trivial to trace back the origin of a change using commit and
> issue
> > IDs.
> >
> >  Updated
> >
> > * Update `org.apache.logging:logging-parent` to version `10.6.0`
> > * Update `jakarta.inject:jakarta.inject-api` to version `2.0.1` (#94)
> > * Update `org.apache.logging.log4j:log4j-core` to version `2.23.1` (#108)
> > * Update `org.apache.logging.log4j:log4j-plugins` to version
> `3.0.0-beta2`
> > (#107)
> > * Update `org.apache.maven.plugin-tools:maven-plugin-annotations` to
> > version `3.11.0` (#98)
> > * Update `org.assertj:assertj-core` to version `3.25.3` (#104)
> > * Update `org.codehaus.modello:modello-maven-plugin` to version `2.3.0`
> > (#105)
> > * Update `org.junit:junit-bom` to version `5.10.2` (#103)
> >
>

Re: [VOTE] Release Apache Log4j Tools 0.8.0 (RC3)

[Gary Gregory]

What's the difference with RC2?

TY,
Gary

On Thu, Mar 21, 2024, 11:46 AM Volkan Yazıcı  wrote:

> This is a vote to release the Apache Log4j Tools 0.8.0 (RC3).
>
> Website: https://logging.staged.apache.org/log4j/tools
> GitHub: https://github.com/apache/logging-log4j-tools
> Commit: 7d157b61e198e3baca816129d8d406b300436e2f
> Distribution: https://dist.apache.org/repos/dist/dev/logging/log4j-tools
> Nexus:
> https://repository.apache.org/content/repositories/orgapachelogging-1266
> Signing key: 0x077e8893a6dcc33dd4a4d5b256e73ba9a0b592d0
>
> Please download, test, and cast your votes on this mailing list.
>
> [ ] +1, release the artifacts
> [ ] -1, don't release, because...
>
> This vote is open until 2024-03-22 12:30 (CET) and will pass unless
> getting a net negative vote count. All votes are welcome and we
> encourage everyone to test the release, but only the Logging
> Services PMC votes are officially counted. At least 3 +1 votes and
> more positive than negative votes are required.
>
> === Timing & Differences
>
> I will keep this vote open until 2024-03-22 12:30 (CET), that is, the
> official end date of the RC2, since RC3 contains a minor difference:
>
>
> https://github.com/apache/logging-log4j-tools/compare/2bb07037bbbfe14fe1c224d46a3e4135b48ffde6...7d157b61e198e3baca816129d8d406b300436e2f
>
> === Review kit
>
> The minimum set of steps needed to review the uploaded distribution
> files in the Subversion repository can be summarized as follows:
>
> # Check out the distribution
> svn co https://dist.apache.org/repos/... && cd $_
>
> # Verify checksums
> shasum --check *.sha512
>
> # Verify signatures
> wget -O - https://downloads.apache.org/logging/KEYS | gpg --import
> for sigFile in *.asc; do gpg --verify $sigFile; done
>
> # Verify reproduciblity
> umask 0022
> unzip *-src.zip -d src
> cd src
> export NEXUS_REPO=https://repository.apache.org/content/...
> sh mvnw -Prelease verify artifact:compare -Dreference.repo=$NEXUS_REPO
> # If preferred, augment `mvnw` with `-DskipTests` to speed things up
>
> === Release notes
>
> This release delivers the first version of Log4j Docgen (Documentation
> Generator).
> It is a set of tools to auto-generate the Log4j plugin documentation (to be
> integrated into the website) and the Log4j configuration XSD file (for
> assisting the configuration of the Log4j runtime, i.e., `log4j2.xml`) from
> the Log4j source code. See the project website for details.
>
>  Added
>
> * Add the `log4j-docgen` et al. containing a universal XML model to
> document Log4j plugins
>
>  Changed
>
> * Move Log4j Changelog XML namespace and schema location to `
> https://logging.apache.org/xml/ns` 
> and `
> https://logging.apache.org/xml/ns/log4j-changelog-0.xsd`
> , respectively
>
>  Removed
>
> * Remove `author` from Log4j Changelog. It was yet another bit to maintain
> and created role-related (who did what) problems. Many modern software
> projects use a VCS (e.g., Git) and support services (e.g., GitHub) which
> make it trivial to trace back the origin of a change using commit and issue
> IDs.
>
>  Updated
>
> * Update `org.apache.logging:logging-parent` to version `10.6.0`
> * Update `jakarta.inject:jakarta.inject-api` to version `2.0.1` (#94)
> * Update `org.apache.logging.log4j:log4j-core` to version `2.23.1` (#108)
> * Update `org.apache.logging.log4j:log4j-plugins` to version `3.0.0-beta2`
> (#107)
> * Update `org.apache.maven.plugin-tools:maven-plugin-annotations` to
> version `3.11.0` (#98)
> * Update `org.assertj:assertj-core` to version `3.25.3` (#104)
> * Update `org.codehaus.modello:modello-maven-plugin` to version `2.3.0`
> (#105)
> * Update `org.junit:junit-bom` to version `5.10.2` (#103)
>

[VOTE] Release Apache Log4j Tools 0.8.0 (RC3)

[Volkan Yazıcı]

This is a vote to release the Apache Log4j Tools 0.8.0 (RC3).

Website: https://logging.staged.apache.org/log4j/tools
GitHub: https://github.com/apache/logging-log4j-tools
Commit: 7d157b61e198e3baca816129d8d406b300436e2f
Distribution: https://dist.apache.org/repos/dist/dev/logging/log4j-tools
Nexus:
https://repository.apache.org/content/repositories/orgapachelogging-1266
Signing key: 0x077e8893a6dcc33dd4a4d5b256e73ba9a0b592d0

Please download, test, and cast your votes on this mailing list.

[ ] +1, release the artifacts
[ ] -1, don't release, because...

This vote is open until 2024-03-22 12:30 (CET) and will pass unless
getting a net negative vote count. All votes are welcome and we
encourage everyone to test the release, but only the Logging
Services PMC votes are officially counted. At least 3 +1 votes and
more positive than negative votes are required.

=== Timing & Differences

I will keep this vote open until 2024-03-22 12:30 (CET), that is, the
official end date of the RC2, since RC3 contains a minor difference:

https://github.com/apache/logging-log4j-tools/compare/2bb07037bbbfe14fe1c224d46a3e4135b48ffde6...7d157b61e198e3baca816129d8d406b300436e2f

=== Review kit

The minimum set of steps needed to review the uploaded distribution
files in the Subversion repository can be summarized as follows:

# Check out the distribution
svn co https://dist.apache.org/repos/... && cd $_

# Verify checksums
shasum --check *.sha512

# Verify signatures
wget -O - https://downloads.apache.org/logging/KEYS | gpg --import
for sigFile in *.asc; do gpg --verify $sigFile; done

# Verify reproduciblity
umask 0022
unzip *-src.zip -d src
cd src
export NEXUS_REPO=https://repository.apache.org/content/...
sh mvnw -Prelease verify artifact:compare -Dreference.repo=$NEXUS_REPO
# If preferred, augment `mvnw` with `-DskipTests` to speed things up

=== Release notes

This release delivers the first version of Log4j Docgen (Documentation
Generator).
It is a set of tools to auto-generate the Log4j plugin documentation (to be
integrated into the website) and the Log4j configuration XSD file (for
assisting the configuration of the Log4j runtime, i.e., `log4j2.xml`) from
the Log4j source code. See the project website for details.

 Added

* Add the `log4j-docgen` et al. containing a universal XML model to
document Log4j plugins

 Changed

* Move Log4j Changelog XML namespace and schema location to `
https://logging.apache.org/xml/ns` and `
https://logging.apache.org/xml/ns/log4j-changelog-0.xsd`, respectively

 Removed

* Remove `author` from Log4j Changelog. It was yet another bit to maintain
and created role-related (who did what) problems. Many modern software
projects use a VCS (e.g., Git) and support services (e.g., GitHub) which
make it trivial to trace back the origin of a change using commit and issue
IDs.

 Updated

* Update `org.apache.logging:logging-parent` to version `10.6.0`
* Update `jakarta.inject:jakarta.inject-api` to version `2.0.1` (#94)
* Update `org.apache.logging.log4j:log4j-core` to version `2.23.1` (#108)
* Update `org.apache.logging.log4j:log4j-plugins` to version `3.0.0-beta2`
(#107)
* Update `org.apache.maven.plugin-tools:maven-plugin-annotations` to
version `3.11.0` (#98)
* Update `org.assertj:assertj-core` to version `3.25.3` (#104)
* Update `org.codehaus.modello:modello-maven-plugin` to version `2.3.0`
(#105)
* Update `org.junit:junit-bom` to version `5.10.2` (#103)

Re: [VOTE] Release Apache Log4j Tools 0.8.0 (RC2)

[Volkan Yazıcı]

I need to cancel this RC2 VOTE.
Encountered another shortcoming.
I will issue an RC3 promptly.

On Tue, Mar 19, 2024 at 12:34 PM Volkan Yazıcı  wrote:

> This is a vote to release the Apache Log4j Tools 0.8.0 (RC2).
>
> Website: https://logging.staged.apache.org/log4j/tools
> GitHub: https://github.com/apache/logging-log4j-tools
> Commit: 2bb07037bbbfe14fe1c224d46a3e4135b48ffde6
> Distribution: https://dist.apache.org/repos/dist/dev/logging/log4j-tools
> Nexus:
> https://repository.apache.org/content/repositories/orgapachelogging-1265
> Signing key: 0x077e8893a6dcc33dd4a4d5b256e73ba9a0b592d0
>
> Please download, test, and cast your votes on this mailing list.
>
> [ ] +1, release the artifacts
> [ ] -1, don't release, because...
>
> This vote is open for 72 hours and will pass unless getting a
> net negative vote count. All votes are welcome and we encourage
> everyone to test the release, but only the Logging Services PMC
> votes are officially counted. At least 3 +1 votes and more
> positive than negative votes are required.
>
> === Review kit
>
> The minimum set of steps needed to review the uploaded distribution
> files in the Subversion repository can be summarized as follows:
>
> # Check out the distribution
> svn co https://dist.apache.org/repos/... && cd $_
>
> # Verify checksums
> shasum --check *.sha512
>
> # Verify signatures
> wget -O - https://downloads.apache.org/logging/KEYS | gpg --import
> for sigFile in *.asc; do gpg --verify $sigFile; done
>
> # Verify reproduciblity
> umask 0022
> unzip *-src.zip -d src
> cd src
> export NEXUS_REPO=https://repository.apache.org/content/...
> sh mvnw -Prelease verify artifact:compare -Dreference.repo=$NEXUS_REPO
> # If preferred, augment `mvnw` with `-DskipTests` to speed things up
>
> === Release notes
>
> This release delivers the first version of Log4j Docgen (Documentation
> Generator). It is a set of tools to auto-generate the Log4j plugin
> documentation (to be integrated into the website) and the Log4j
> configuration XSD file (for assisting the configuration of the Log4j
> runtime, i.e., `log4j2.xml`) from the Log4j source code. See the
> project website for details.
>
>  Added
>
> * Add the `log4j-docgen` et al. containing a universal XML model to
> document Log4j plugins
>
>  Changed
>
> * Move Log4j Changelog XML namespace and schema location to
> `https://logging.apache.org/xml/ns` 
> and
> `https://logging.apache.org/xml/ns/log4j-changelog-0.xsd`
> ,
> respectively
>
>  Removed
>
> * Remove `author` from Log4j Changelog. It was yet another bit to
> maintain and created role-related (who did what) problems. Many modern
> software projects use a VCS (e.g., Git) and support services (e.g.,
> GitHub) which make it trivial to trace back the origin of a change
> using commit and issue IDs.
>
>  Updated
>
> * Update `org.apache.logging:logging-parent` to version `10.6.0`
> * Update `jakarta.inject:jakarta.inject-api` to version `2.0.1` (#94)
> * Update `org.apache.logging.log4j:log4j-core` to version `2.23.1` (#108)
> * Update `org.apache.logging.log4j:log4j-plugins` to version
> `3.0.0-beta2` (#107)
> * Update `org.apache.maven.plugin-tools:maven-plugin-annotations` to
> version `3.11.0` (#98)
> * Update `org.assertj:assertj-core` to version `3.25.3` (#104)
> * Update `org.codehaus.modello:modello-maven-plugin` to version `2.3.0`
> (#105)
> * Update `org.junit:junit-bom` to version `5.10.2` (#103)
>

Re: Context data propagation and logger-bound context data

[Volkan Yazıcı]

Ralph, could you show how those two users can use a `MessageFactory` to
create `Logger`s with predefined additional context data?

On Thu, Mar 21, 2024 at 7:25 AM Ralph Goers  wrote:

> Unfortunately this is another message I somehow didn't get in my inbox.
> Replying to it via lists.a.o is not a great experience but is the best I
> can do.
>
> On 2024/03/20 13:51:56 Volkan Yazıcı wrote:
> > I agree with the way Piotr dissects the problem. I think `ScopedContext`,
> > even though it has its own merits, doesn't address the problem reported
> by
> > users. They simply want a new logger associated with some additional
> > context data.
>
> Two users do.  I have personally been asked for something like
> ScopedContext several times.
> As I replied to Piotr, we already solved the problem of adding data to
> Loggers. That is what MessageFactories are intended for.
>
> >
> > *[See my comments below.]*
> >
> > On Mon, Mar 18, 2024 at 10:40 AM Piotr P. Karwasz <
> piotr.karw...@gmail.com>
> > wrote:
> >
> > > * we can create a `Logger` wrapper "bound" to context data as Mikko
> > > does. This wrapper will take care of setting the `ThreadContext`
> > > before the logger call and restore it after it.
> >
> > Creating a wrapper `Logger` can work without needing to deal with
> > `ThreadContext`. I can think of two different ways to carry this out:
> >
> >1. Currently, `AbstractLogger` only creates `Message`s. We can rework
> it
> >to create `LogEvent`s too. Once `AbstractLogger` gets its hand on a
> >`LogEvent`, it can enrich its context data as it wishes.
> >2. We can extend `ContextDataInjector` with a new `void
> >injectContextData(Logger logger, StringMap target)` method, provide a
> >`ContextDataInjector` implementation that branches on `logger
> instanceof
> >ContextDataProvider`, and call `ContextDataInjector` with the
> associated
> >`Logger` in `LogEventFactory`.
>
> We can do lots of things, most of which I wouldn't recommend. As to yours:
> 1. Logger/AbstractLogger got very complex with Async, Garbage Free,
> Reliablity Strategies, etc. Trying to move creating the LogEvent sooner is
> likely to be a major PITA and could seriously impact performance. While we
> could add a context map to AbstractLogger we would have to pass that on the
> logging calls to LoggerConfig and deal with all that that means - remember,
> a LoggerConfig can be handling multiple Loggers.
> 2. I don't recommend extending ContextDataInjector. That proved difficult
> to work with which is why we now recommend using ContextDataProviders. You
> really can only have one ContextDataInjector. Also, please note that
> ContextDataInjector is called while constructing the LogEvent. The LogEvent
> isn't passed the Logger, only the LoggerName. Looking up the Logger to do
> this is yet another way to slow down logging.
>
> >
> > On Tue, Mar 19, 2024 at 7:45 AM Ralph Goers 
> > wrote:
> > > In the meantime, I provided
> > https://github.com/apache/logging-log4j2/pull/2385 which I very loosely
> > modeled after ScopedValues.
> >
> > The fact that `ScopedContext` tries to imitate `ScopedValue` using
> > `ThreadLocal`s is extremely confusing (from a user's pov) and risky
> > liability (from a maintainer's pov). I guess you wanted to implement *a*
> > `ScopedValue` without using *the* `ScopedValue` to be compatible with
> Java
> > 8. If so, that really sounds like the `o.a.l.log4j.util.Supplier`
> downward
> > spiral. We can rather have an *internal* `Log4jScopedValue` interface and
> > provide Java 8 (using `InheritableThreadLocal`) and Java 21+ (using
> > `ScopedValue`) compatible solutions in an MRJ (Multi-Release JAR).
>
> I am NOT trying to make ScopedContext compatible with ScopedValue. I am
> trying to make it conceptually close enough to ScopedValue that users will
> understand what it is doing.
> We can argue about naming if you want. Gary has already expressed his
> opinion.
> >
> > We can integrate `ScopedContext` to the `LogEventFactory` by providing a
> > specialized `ContextDataInjector` plugin – assuming `LogEventFactory`
> > employs all available `ContextDataInjector` plugins.
>
> ScopedContext is integrated with a ContextDataProvider, which is the
> supported way to do this. Again, you cannot have more than one
> ContextDataInjector so providing "specialized versions" is a pipe dream.
> You will simply have to enhance the one we already have.
> ContextDataInjector is NOT a plugin.
>
> >
> > I find the current ceremony also too long:
> > `ScopedContext.getCurrent().where("key1", "value1").run(...)`. I would
> > rather aim for `ScopedContext.run(key, value, runnable)` and similar
> > `ScopedContext.op(..., runnable)` interaction.
>
> Those are going to be provided as well.
>
> Ralph
>

Re: Context data propagation and logger-bound context data

[Piotr P. Karwasz]

Hi Ralph,

On Thu, 21 Mar 2024 at 07:03, Ralph Goers  wrote:
> > 1. Raman and Mikko would like to bind context data to an object
> > implementing the `Logger` interface or more generally to a service
> > object (e.g. resource manager, DAO and variants),
>
> Yes, I’ve seen that. Personally, I am not much of a fan of this use case as 
> it is pretty easy to add the data you want to a single class. That said, we 
> already offer a solution. Allowing a MessageFactory to be provided on a 
> Logger was done for exactly this reason.
>
> For example, a User could configure a custom MessageFactory that provides an 
> extension of MapMessage that causes the message to include data from the 
> class or resource. Going to the extreme of trying to shoehorn in Context data 
> as well simply isn’t necessary.

Great point! This effectively solves Raman's and Mikko's problem. We
just need to introduce a new sub-interface of `Message` and require
all implementations to support it.

To be sure we don't cherry-pick each element of a log event with a
different interface (like we did with `TimeStampMessage`), let's use a
single interface to supply all the additional data the user might want
to have. Let's throw in it Ceki's `KeyValuePair`s[1]

Piotr

[1] https://www.slf4j.org/api/org/slf4j/event/LoggingEvent.html

Re: Context data propagation and logger-bound context data

[Piotr P. Karwasz]

Hi Ralph,

On Wed, 20 Mar 2024 at 17:11, Ralph Goers  wrote:
>
> 1. I am experimenting with adding Thread support to ScopedContext as we 
> speak. It should be straightforward.
>
> I am not sure we ever want to deal with ScopedValue. ScopedContext provides 
> the same functionality plus integration with logging.

>From what I read about ScopedValue I am also less and less convinced
we want to use it. It can be used this way:

ScopedValue.where(KEY1, "value1").where(KEY2, "value2").run(() ->
doSomething());

Normally the values are not propagated to other "threads" or anything
of this sort. Sure, `doSomething` can run as virtual thread on
multiple Java `Thread`s (and each of them can run on multiple CPU
cores), but as soon as `doSomething` calls `new Thread` or calls
`ExecutorService.submit`, the context is lost.

There is a **special** executor (`StructuredTaskScope`) that will
automatically copy the values bound to a `ScopedValue` before jumping
threads, but there are specialized executors that already do that.

Why don't we implement the same functionality with a **minimal** API extension?

CloseableThreadContext.put("key1", "value1").put("key2",
"value2").run(() -> doSomething());

Piotr