Re: Lock pattern

[Matt Sicker]

FWIW, I'm still interested in this AutoCloseableLock idea.

On 23 June 2016 at 22:25, Gary Gregory  wrote:

> Before you go on a dig, is the discussion thread you are looking for
> arguing against creating such a gadget as an AutoCloseableLock? As in, it's
> an aberration?
>
> Gary
>
> On Thu, Jun 23, 2016 at 7:49 PM, Paul Benedict 
> wrote:
>
>> If my memory serves me right, there is a whole thread on OpenJDK message
>> boards on this design. I will find it and send it, time permitting.
>>
>> On Jun 23, 2016 9:07 PM, "Matt Sicker"  wrote:
>>
>>> Any locks that don't use wait/notify could be converted to Object
>>> monitors instead. Using the Lock interface works best combined with
>>> Conditions as it's more powerful than the basic stuff in Object.
>>>
>>> Also, if you need something serializable as a lock, you can apparently
>>> use Object[0] because an empty array of anything is serializable.
>>>
>>> On 23 June 2016 at 20:48, Gary Gregory  wrote:
>>>
 On Thu, Jun 23, 2016 at 5:52 PM, Matt Sicker  wrote:

> I kind of hate to point this out, but what's wrong with this when
> you're not using multiple condition variables?
>
> final Object lock = new Object();
> synchronized (lock) {
>   // ...
> }
>

 Considering that we use a lot of ReentrantLocks, does this apply? We
 cannot replace ReentrantLocks with Objects. Or is that what you are
 proposing? I'm guessing you are making a more general point that
 a AutoCloseableLock is not a generic replacement for synchronized blocks.

 Gary

>
> It auto-unlocks at the end, too.
>
> On 23 June 2016 at 19:35, Gary Gregory  wrote:
>
>> On Thu, Jun 23, 2016 at 4:58 PM, Gary Gregory > > wrote:
>>
>>> On Thu, Jun 23, 2016 at 4:56 PM, Gary Gregory <
>>> garydgreg...@gmail.com> wrote:
>>>
 Like this:

 public static boolean hasManager(final String name) {
 try (Object o = AutoCloseableLock.lock(LOCK)) {
 return MAP.containsKey(name);
 }
 }

 With a new class:

 public class AutoCloseableLock implements AutoCloseable {

 public static AutoCloseableLock lock(final Lock lock) {
 Objects.requireNonNull(lock, "lock");
 lock.lock();
 return new AutoCloseableLock(lock);
 }

 private final Lock lock;

 public AutoCloseableLock(final Lock lock) {
 this.lock = lock;
 }

 @Override
 public void close() {
 this.lock.unlock();
 }

 public void lock() {
 this.lock.lock();
 }

>>>
>>> You do not even need a lock() method.
>>>
>>
>> But let's say you do for GC-free reasons:
>>
>> public AutoCloseableLock lock() {
>> this.lock.lock();
>> return this;
>> }
>>
>> to do:
>>
>> private AutoCloseableLock autoCloseableLock = new
>> AutoCloseableLock(new ReentrantLock());
>>
>> public static boolean hasManager(final String name) {
>> try (Object o = autoCloseableLock.lock()) {
>> return MAP.containsKey(name);
>> }
>> }
>>
>>  Gary
>>
>> Gary
>>>
 }

 The pro is less code and less chance for errors, the con is that
 you create a new AutoCloseableLock for lock/unlock sites.

 Gary

 On Thu, Jun 23, 2016 at 4:54 PM, Matt Sicker 
 wrote:

> Sounds like an opportunity for a small Commons or independent
> project. CloseableLock!
>
> On 23 June 2016 at 18:51, Greg Thomas 
> wrote:
>
>> I''m sure I read somewhere that it was a deliberate choice not to
>> make it, to stop people using the very common pattern of creating the
>> object in the try() - which isn't much use for a lock.
>>
>> Greg
>> --
>> Sent from my iPhone
>>
>> On 24 Jun 2016, at 00:45, Remko Popma 
>> wrote:
>>
>> Good idea!
>> Maybe propose this for Java 9?
>> Looks very reasonable to me.
>>
>> Sent from my iPhone
>>
>> On 2016/06/24, at 8:32, Gary Gregory 
>> wrote:
>>
>> I wonder if anyone knows why Lock is not AutoCloseable.
>>
>> This:
>>
>> public static boolean hasManager(final String name) {
>> 

[jira] [Commented] (LOG4J2-1010) Injectable context properties

[Ralph Goers (JIRA)]

[ 
https://issues.apache.org/jira/browse/LOG4J2-1010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15347633#comment-15347633
 ] 

Ralph Goers commented on LOG4J2-1010:
-

OK - I've read this whole thing:
1. I never really like the context stack and only brought it over because Log4j 
1 has it. Frankly, I'd be afraid applications would get the stack out of sync. 
So I don't think what you are proposing needs to support a stack.
2. I didn't like adding a parameter to logMessage. Glad you have discarded the 
idea.
3. I have no problem with the interface proposed by Remko. I believe it belongs 
in Core.

> Injectable context properties
> -
>
> Key: LOG4J2-1010
> URL: https://issues.apache.org/jira/browse/LOG4J2-1010
> Project: Log4j 2
>  Issue Type: Improvement
>  Components: API
>Affects Versions: 2.2
>Reporter: Mikael Ståldal
>Assignee: Remko Popma
> Fix For: 2.7
>
> Attachments: properties.patch
>
>
> It would be useful to have a way to inject context properties into a 
> {{LogEvent}}, as an alternative to {{ThreadContext}}.
> In an asynchronous environment, using ThreadContext as currently implemented 
> is not so useful since JVM threads might not be coupled to the logical flow 
> of the application.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: log4j-dev-unsubscr...@logging.apache.org
For additional commands, e-mail: log4j-dev-h...@logging.apache.org

Re: Lock pattern

[Gary Gregory]

Before you go on a dig, is the discussion thread you are looking for
arguing against creating such a gadget as an AutoCloseableLock? As in, it's
an aberration?

Gary

On Thu, Jun 23, 2016 at 7:49 PM, Paul Benedict  wrote:

> If my memory serves me right, there is a whole thread on OpenJDK message
> boards on this design. I will find it and send it, time permitting.
>
> On Jun 23, 2016 9:07 PM, "Matt Sicker"  wrote:
>
>> Any locks that don't use wait/notify could be converted to Object
>> monitors instead. Using the Lock interface works best combined with
>> Conditions as it's more powerful than the basic stuff in Object.
>>
>> Also, if you need something serializable as a lock, you can apparently
>> use Object[0] because an empty array of anything is serializable.
>>
>> On 23 June 2016 at 20:48, Gary Gregory  wrote:
>>
>>> On Thu, Jun 23, 2016 at 5:52 PM, Matt Sicker  wrote:
>>>
 I kind of hate to point this out, but what's wrong with this when
 you're not using multiple condition variables?

 final Object lock = new Object();
 synchronized (lock) {
   // ...
 }

>>>
>>> Considering that we use a lot of ReentrantLocks, does this apply? We
>>> cannot replace ReentrantLocks with Objects. Or is that what you are
>>> proposing? I'm guessing you are making a more general point that
>>> a AutoCloseableLock is not a generic replacement for synchronized blocks.
>>>
>>> Gary
>>>

 It auto-unlocks at the end, too.

 On 23 June 2016 at 19:35, Gary Gregory  wrote:

> On Thu, Jun 23, 2016 at 4:58 PM, Gary Gregory 
> wrote:
>
>> On Thu, Jun 23, 2016 at 4:56 PM, Gary Gregory > > wrote:
>>
>>> Like this:
>>>
>>> public static boolean hasManager(final String name) {
>>> try (Object o = AutoCloseableLock.lock(LOCK)) {
>>> return MAP.containsKey(name);
>>> }
>>> }
>>>
>>> With a new class:
>>>
>>> public class AutoCloseableLock implements AutoCloseable {
>>>
>>> public static AutoCloseableLock lock(final Lock lock) {
>>> Objects.requireNonNull(lock, "lock");
>>> lock.lock();
>>> return new AutoCloseableLock(lock);
>>> }
>>>
>>> private final Lock lock;
>>>
>>> public AutoCloseableLock(final Lock lock) {
>>> this.lock = lock;
>>> }
>>>
>>> @Override
>>> public void close() {
>>> this.lock.unlock();
>>> }
>>>
>>> public void lock() {
>>> this.lock.lock();
>>> }
>>>
>>
>> You do not even need a lock() method.
>>
>
> But let's say you do for GC-free reasons:
>
> public AutoCloseableLock lock() {
> this.lock.lock();
> return this;
> }
>
> to do:
>
> private AutoCloseableLock autoCloseableLock = new
> AutoCloseableLock(new ReentrantLock());
>
> public static boolean hasManager(final String name) {
> try (Object o = autoCloseableLock.lock()) {
> return MAP.containsKey(name);
> }
> }
>
>  Gary
>
> Gary
>>
>>> }
>>>
>>> The pro is less code and less chance for errors, the con is that you
>>> create a new AutoCloseableLock for lock/unlock sites.
>>>
>>> Gary
>>>
>>> On Thu, Jun 23, 2016 at 4:54 PM, Matt Sicker 
>>> wrote:
>>>
 Sounds like an opportunity for a small Commons or independent
 project. CloseableLock!

 On 23 June 2016 at 18:51, Greg Thomas 
 wrote:

> I''m sure I read somewhere that it was a deliberate choice not to
> make it, to stop people using the very common pattern of creating the
> object in the try() - which isn't much use for a lock.
>
> Greg
> --
> Sent from my iPhone
>
> On 24 Jun 2016, at 00:45, Remko Popma 
> wrote:
>
> Good idea!
> Maybe propose this for Java 9?
> Looks very reasonable to me.
>
> Sent from my iPhone
>
> On 2016/06/24, at 8:32, Gary Gregory 
> wrote:
>
> I wonder if anyone knows why Lock is not AutoCloseable.
>
> This:
>
> public static boolean hasManager(final String name) {
> LOCK.lock();
> try {
> return MAP.containsKey(name);
> } finally {
> LOCK.unlock();
> }
> }
>
>
> Seems lame in comparison to:
>
> public 

Re: Lock pattern

[Paul Benedict]

If my memory serves me right, there is a whole thread on OpenJDK message
boards on this design. I will find it and send it, time permitting.

On Jun 23, 2016 9:07 PM, "Matt Sicker"  wrote:

> Any locks that don't use wait/notify could be converted to Object monitors
> instead. Using the Lock interface works best combined with Conditions as
> it's more powerful than the basic stuff in Object.
>
> Also, if you need something serializable as a lock, you can apparently use
> Object[0] because an empty array of anything is serializable.
>
> On 23 June 2016 at 20:48, Gary Gregory  wrote:
>
>> On Thu, Jun 23, 2016 at 5:52 PM, Matt Sicker  wrote:
>>
>>> I kind of hate to point this out, but what's wrong with this when you're
>>> not using multiple condition variables?
>>>
>>> final Object lock = new Object();
>>> synchronized (lock) {
>>>   // ...
>>> }
>>>
>>
>> Considering that we use a lot of ReentrantLocks, does this apply? We
>> cannot replace ReentrantLocks with Objects. Or is that what you are
>> proposing? I'm guessing you are making a more general point that
>> a AutoCloseableLock is not a generic replacement for synchronized blocks.
>>
>> Gary
>>
>>>
>>> It auto-unlocks at the end, too.
>>>
>>> On 23 June 2016 at 19:35, Gary Gregory  wrote:
>>>
 On Thu, Jun 23, 2016 at 4:58 PM, Gary Gregory 
 wrote:

> On Thu, Jun 23, 2016 at 4:56 PM, Gary Gregory 
> wrote:
>
>> Like this:
>>
>> public static boolean hasManager(final String name) {
>> try (Object o = AutoCloseableLock.lock(LOCK)) {
>> return MAP.containsKey(name);
>> }
>> }
>>
>> With a new class:
>>
>> public class AutoCloseableLock implements AutoCloseable {
>>
>> public static AutoCloseableLock lock(final Lock lock) {
>> Objects.requireNonNull(lock, "lock");
>> lock.lock();
>> return new AutoCloseableLock(lock);
>> }
>>
>> private final Lock lock;
>>
>> public AutoCloseableLock(final Lock lock) {
>> this.lock = lock;
>> }
>>
>> @Override
>> public void close() {
>> this.lock.unlock();
>> }
>>
>> public void lock() {
>> this.lock.lock();
>> }
>>
>
> You do not even need a lock() method.
>

 But let's say you do for GC-free reasons:

 public AutoCloseableLock lock() {
 this.lock.lock();
 return this;
 }

 to do:

 private AutoCloseableLock autoCloseableLock = new
 AutoCloseableLock(new ReentrantLock());

 public static boolean hasManager(final String name) {
 try (Object o = autoCloseableLock.lock()) {
 return MAP.containsKey(name);
 }
 }

  Gary

 Gary
>
>> }
>>
>> The pro is less code and less chance for errors, the con is that you
>> create a new AutoCloseableLock for lock/unlock sites.
>>
>> Gary
>>
>> On Thu, Jun 23, 2016 at 4:54 PM, Matt Sicker 
>> wrote:
>>
>>> Sounds like an opportunity for a small Commons or independent
>>> project. CloseableLock!
>>>
>>> On 23 June 2016 at 18:51, Greg Thomas 
>>> wrote:
>>>
 I''m sure I read somewhere that it was a deliberate choice not to
 make it, to stop people using the very common pattern of creating the
 object in the try() - which isn't much use for a lock.

 Greg
 --
 Sent from my iPhone

 On 24 Jun 2016, at 00:45, Remko Popma 
 wrote:

 Good idea!
 Maybe propose this for Java 9?
 Looks very reasonable to me.

 Sent from my iPhone

 On 2016/06/24, at 8:32, Gary Gregory 
 wrote:

 I wonder if anyone knows why Lock is not AutoCloseable.

 This:

 public static boolean hasManager(final String name) {
 LOCK.lock();
 try {
 return MAP.containsKey(name);
 } finally {
 LOCK.unlock();
 }
 }


 Seems lame in comparison to:

 public static boolean hasManager(final String name) {
 try (LOCK.lock()) {
 return MAP.containsKey(name);
 }
 }

 Which, due to syntax really would be:

 public static boolean hasManager(final String name) {
 try (Object o = LOCK.lock()) {
 return MAP.containsKey(name);
  

Re: Lock pattern

[Matt Sicker]

Any locks that don't use wait/notify could be converted to Object monitors
instead. Using the Lock interface works best combined with Conditions as
it's more powerful than the basic stuff in Object.

Also, if you need something serializable as a lock, you can apparently use
Object[0] because an empty array of anything is serializable.

On 23 June 2016 at 20:48, Gary Gregory  wrote:

> On Thu, Jun 23, 2016 at 5:52 PM, Matt Sicker  wrote:
>
>> I kind of hate to point this out, but what's wrong with this when you're
>> not using multiple condition variables?
>>
>> final Object lock = new Object();
>> synchronized (lock) {
>>   // ...
>> }
>>
>
> Considering that we use a lot of ReentrantLocks, does this apply? We
> cannot replace ReentrantLocks with Objects. Or is that what you are
> proposing? I'm guessing you are making a more general point that
> a AutoCloseableLock is not a generic replacement for synchronized blocks.
>
> Gary
>
>>
>> It auto-unlocks at the end, too.
>>
>> On 23 June 2016 at 19:35, Gary Gregory  wrote:
>>
>>> On Thu, Jun 23, 2016 at 4:58 PM, Gary Gregory 
>>> wrote:
>>>
 On Thu, Jun 23, 2016 at 4:56 PM, Gary Gregory 
 wrote:

> Like this:
>
> public static boolean hasManager(final String name) {
> try (Object o = AutoCloseableLock.lock(LOCK)) {
> return MAP.containsKey(name);
> }
> }
>
> With a new class:
>
> public class AutoCloseableLock implements AutoCloseable {
>
> public static AutoCloseableLock lock(final Lock lock) {
> Objects.requireNonNull(lock, "lock");
> lock.lock();
> return new AutoCloseableLock(lock);
> }
>
> private final Lock lock;
>
> public AutoCloseableLock(final Lock lock) {
> this.lock = lock;
> }
>
> @Override
> public void close() {
> this.lock.unlock();
> }
>
> public void lock() {
> this.lock.lock();
> }
>

 You do not even need a lock() method.

>>>
>>> But let's say you do for GC-free reasons:
>>>
>>> public AutoCloseableLock lock() {
>>> this.lock.lock();
>>> return this;
>>> }
>>>
>>> to do:
>>>
>>> private AutoCloseableLock autoCloseableLock = new
>>> AutoCloseableLock(new ReentrantLock());
>>>
>>> public static boolean hasManager(final String name) {
>>> try (Object o = autoCloseableLock.lock()) {
>>> return MAP.containsKey(name);
>>> }
>>> }
>>>
>>>  Gary
>>>
>>> Gary

> }
>
> The pro is less code and less chance for errors, the con is that you
> create a new AutoCloseableLock for lock/unlock sites.
>
> Gary
>
> On Thu, Jun 23, 2016 at 4:54 PM, Matt Sicker  wrote:
>
>> Sounds like an opportunity for a small Commons or independent
>> project. CloseableLock!
>>
>> On 23 June 2016 at 18:51, Greg Thomas 
>> wrote:
>>
>>> I''m sure I read somewhere that it was a deliberate choice not to
>>> make it, to stop people using the very common pattern of creating the
>>> object in the try() - which isn't much use for a lock.
>>>
>>> Greg
>>> --
>>> Sent from my iPhone
>>>
>>> On 24 Jun 2016, at 00:45, Remko Popma  wrote:
>>>
>>> Good idea!
>>> Maybe propose this for Java 9?
>>> Looks very reasonable to me.
>>>
>>> Sent from my iPhone
>>>
>>> On 2016/06/24, at 8:32, Gary Gregory  wrote:
>>>
>>> I wonder if anyone knows why Lock is not AutoCloseable.
>>>
>>> This:
>>>
>>> public static boolean hasManager(final String name) {
>>> LOCK.lock();
>>> try {
>>> return MAP.containsKey(name);
>>> } finally {
>>> LOCK.unlock();
>>> }
>>> }
>>>
>>>
>>> Seems lame in comparison to:
>>>
>>> public static boolean hasManager(final String name) {
>>> try (LOCK.lock()) {
>>> return MAP.containsKey(name);
>>> }
>>> }
>>>
>>> Which, due to syntax really would be:
>>>
>>> public static boolean hasManager(final String name) {
>>> try (Object o = LOCK.lock()) {
>>> return MAP.containsKey(name);
>>> }
>>> }
>>>
>>> Just wonderin'...
>>>
>>> Gary
>>>
>>> --
>>> E-Mail: garydgreg...@gmail.com | ggreg...@apache.org
>>> Java Persistence with Hibernate, Second Edition
>>> 
>>> JUnit in Action, Second Edition 
>>> Spring Batch in Action 

[jira] [Updated] (LOG4J2-1010) Injectable context properties

[Remko Popma (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LOG4J2-1010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Remko Popma updated LOG4J2-1010:

 Assignee: Remko Popma
Fix Version/s: 2.7

> Injectable context properties
> -
>
> Key: LOG4J2-1010
> URL: https://issues.apache.org/jira/browse/LOG4J2-1010
> Project: Log4j 2
>  Issue Type: Improvement
>  Components: API
>Affects Versions: 2.2
>Reporter: Mikael Ståldal
>Assignee: Remko Popma
> Fix For: 2.7
>
> Attachments: properties.patch
>
>
> It would be useful to have a way to inject context properties into a 
> {{LogEvent}}, as an alternative to {{ThreadContext}}.
> In an asynchronous environment, using ThreadContext as currently implemented 
> is not so useful since JVM threads might not be coupled to the logical flow 
> of the application.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: log4j-dev-unsubscr...@logging.apache.org
For additional commands, e-mail: log4j-dev-h...@logging.apache.org

[jira] [Updated] (LOG4J2-1401) Support changing the log level for all messages related to some domain object

[Remko Popma (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LOG4J2-1401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Remko Popma updated LOG4J2-1401:

 Assignee: Remko Popma
Fix Version/s: 2.7

The use case in the description emphasizes using primitives rather than 
Objects. This is perhaps unusual, and it makes sense to create a custom 
solution rather than pushing this into the logging library.

Now that the ideas on LOG4J2-1010 (Injectable context properties), LOG4J2-1349 
(Garbage-free ThreadContext map) and LOG4J2-1447
(Garbage-free data structure for LogEvent's context map data) are starting to 
crystallize, I can see how these can be used as building blocks to implement 
this ticket.

First, the application can manage its context in a domain-specific class. 
Something that tracks the current user ID, order ID, instrument ID. This can be 
garbage-free and use primitives. (So there is no need to modify the 
ThreadContext interface to accommodate primitive values.)

The nice thing about the {{ContextInjector}} idea is that *Log4j does not need 
to know* about the domain-specific context. Only the ContextInjector needs to 
know about it. The application can have a custom injector that gets IDs as 
primitive values from the application context object and puts them in the 
LogEvent's CustomData.

Finally, for this to be garbage-free, the application would populate LogEvents 
with a custom implementation of the CustomData interface that can store 
primitive long values without creating temporary objects.

> Support changing the log level for all messages related to some domain object
> -
>
> Key: LOG4J2-1401
> URL: https://issues.apache.org/jira/browse/LOG4J2-1401
> Project: Log4j 2
>  Issue Type: New Feature
>  Components: Core, Filters
>Affects Versions: 2.6
>Reporter: Remko Popma
>Assignee: Remko Popma
> Fix For: 2.7
>
>
> Use case:
> We have a low-garbage trading system. When a production incident occurs we 
> want to temporarily increase visibility into what the system is doing by 
> increasing the log level. However, we want to do this selectively: we want 
> the ability to turn on detailed logging only for certain orders, for certain 
> instruments, for a certain user, or for some other attributes.
> The system processes many messages, say in the order of hundreds of thousands 
> of messages per second. When a message is processed it passes through many 
> components of the system, and when each of these components do logging, the 
> message context information like order ID, instrument etc. should be 
> available so the log event can be filtered in the normal case and enabled in 
> case of a production incident.
> We envision this "verbose logging" filtering to be switched on and off 
> manually by an operations team. In addition to manual operation, it would be 
> nice if the original verbosity can be restored automatically when some 
> condition no longer holds. For example, log at some verbose level 
> * for some period of time (5 minutes)
> * for some fixed number of events (10,000 log messages)
> * any other user-specified condition 
> There are two problems to solve:
> # efficient and garbage-free log event "tagging"
> # dynamic and garbage-free filtering
> *Problem 1: Log Event "Tagging"*
> Considerations:
> * Because of the volume of events we want the mechanism for tagging log 
> events to be garbage-free. 
> * Our order IDs and instrument IDs etc are modeled as primitives ({{long}} 
> and {{int}} values).
> Current options in Log4j for "tagging" a log event:
> * *ThreadContext map*. (This _almost_ does what we need.) Data in this map is 
> copied into each Log Event, and downstream components can query the data with 
> the {{LogEvent.getContextMap() : Map}} method. The drawback 
> of the ThreadContext API is that values must be Strings. We would like to be 
> able to store {{long}} primitives in the ThreadContext map. We can also take 
> this opportunity to provide a garbage-free version of the ThreadContext 
> (LOG4J2-1349) and the LogEvent properties map.
> * *Direct user access to the LogEvent properties map*. LOG4J2-1010 suggests 
> changing the Logger API to accomplish this, but there may be alternatives.
> * Update: the idea raised in LOG4J2-1010 to make initialization of the 
> LogEvent's {{contextMap}} data pluggable is relevant for this ticket also: It 
> means that context data can come from anywhere, not necessarily only from 
> Log4j API classes like ThreadContext. Applications can have their own custom 
> ThreadContext-like class with data that is copied into every log event, 
> without requiring changes to the Log4j API.
> * *Markers*: (Not a good option, mentioned here for completeness.) 
> String-based, name attribute only (no 

Re: Lock pattern

[Gary Gregory]

On Thu, Jun 23, 2016 at 5:52 PM, Matt Sicker  wrote:

> I kind of hate to point this out, but what's wrong with this when you're
> not using multiple condition variables?
>
> final Object lock = new Object();
> synchronized (lock) {
>   // ...
> }
>

Considering that we use a lot of ReentrantLocks, does this apply? We cannot
replace ReentrantLocks with Objects. Or is that what you are proposing? I'm
guessing you are making a more general point that a AutoCloseableLock is
not a generic replacement for synchronized blocks.

Gary

>
> It auto-unlocks at the end, too.
>
> On 23 June 2016 at 19:35, Gary Gregory  wrote:
>
>> On Thu, Jun 23, 2016 at 4:58 PM, Gary Gregory 
>> wrote:
>>
>>> On Thu, Jun 23, 2016 at 4:56 PM, Gary Gregory 
>>> wrote:
>>>
 Like this:

 public static boolean hasManager(final String name) {
 try (Object o = AutoCloseableLock.lock(LOCK)) {
 return MAP.containsKey(name);
 }
 }

 With a new class:

 public class AutoCloseableLock implements AutoCloseable {

 public static AutoCloseableLock lock(final Lock lock) {
 Objects.requireNonNull(lock, "lock");
 lock.lock();
 return new AutoCloseableLock(lock);
 }

 private final Lock lock;

 public AutoCloseableLock(final Lock lock) {
 this.lock = lock;
 }

 @Override
 public void close() {
 this.lock.unlock();
 }

 public void lock() {
 this.lock.lock();
 }

>>>
>>> You do not even need a lock() method.
>>>
>>
>> But let's say you do for GC-free reasons:
>>
>> public AutoCloseableLock lock() {
>> this.lock.lock();
>> return this;
>> }
>>
>> to do:
>>
>> private AutoCloseableLock autoCloseableLock = new
>> AutoCloseableLock(new ReentrantLock());
>>
>> public static boolean hasManager(final String name) {
>> try (Object o = autoCloseableLock.lock()) {
>> return MAP.containsKey(name);
>> }
>> }
>>
>>  Gary
>>
>> Gary
>>>
 }

 The pro is less code and less chance for errors, the con is that you
 create a new AutoCloseableLock for lock/unlock sites.

 Gary

 On Thu, Jun 23, 2016 at 4:54 PM, Matt Sicker  wrote:

> Sounds like an opportunity for a small Commons or independent project.
> CloseableLock!
>
> On 23 June 2016 at 18:51, Greg Thomas  wrote:
>
>> I''m sure I read somewhere that it was a deliberate choice not to
>> make it, to stop people using the very common pattern of creating the
>> object in the try() - which isn't much use for a lock.
>>
>> Greg
>> --
>> Sent from my iPhone
>>
>> On 24 Jun 2016, at 00:45, Remko Popma  wrote:
>>
>> Good idea!
>> Maybe propose this for Java 9?
>> Looks very reasonable to me.
>>
>> Sent from my iPhone
>>
>> On 2016/06/24, at 8:32, Gary Gregory  wrote:
>>
>> I wonder if anyone knows why Lock is not AutoCloseable.
>>
>> This:
>>
>> public static boolean hasManager(final String name) {
>> LOCK.lock();
>> try {
>> return MAP.containsKey(name);
>> } finally {
>> LOCK.unlock();
>> }
>> }
>>
>>
>> Seems lame in comparison to:
>>
>> public static boolean hasManager(final String name) {
>> try (LOCK.lock()) {
>> return MAP.containsKey(name);
>> }
>> }
>>
>> Which, due to syntax really would be:
>>
>> public static boolean hasManager(final String name) {
>> try (Object o = LOCK.lock()) {
>> return MAP.containsKey(name);
>> }
>> }
>>
>> Just wonderin'...
>>
>> Gary
>>
>> --
>> E-Mail: garydgreg...@gmail.com | ggreg...@apache.org
>> Java Persistence with Hibernate, Second Edition
>> 
>> JUnit in Action, Second Edition 
>> Spring Batch in Action 
>> Blog: http://garygregory.wordpress.com
>> Home: http://garygregory.com/
>> Tweet! http://twitter.com/GaryGregory
>>
>>
>
>
> --
> Matt Sicker 
>



 --
 E-Mail: garydgreg...@gmail.com | ggreg...@apache.org
 Java Persistence with Hibernate, Second Edition
 
 JUnit in Action, Second Edition 
 Spring Batch in Action 
 Blog: http://garygregory.wordpress.com
 Home: 

[jira] [Assigned] (LOG4J2-1349) Garbage-free ThreadContext map

[Remko Popma (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LOG4J2-1349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Remko Popma reassigned LOG4J2-1349:
---

Assignee: Remko Popma

> Garbage-free ThreadContext map
> --
>
> Key: LOG4J2-1349
> URL: https://issues.apache.org/jira/browse/LOG4J2-1349
> Project: Log4j 2
>  Issue Type: Improvement
>  Components: API
>Affects Versions: 2.5
>Reporter: Remko Popma
>Assignee: Remko Popma
> Fix For: 2.7
>
>
> The current ThreadContext map and stack implementations allocate temporary 
> objects. This ticket is to investigate and track the work for alternative 
> implementations that are garbage-free.
> Both DefaultThreadContextMap and DefaultThreadContextStack are copy-on-write 
> data structures: each modification replaces the ThreadLocal object with a 
> modified copy. The advantage of this approach is that there is no need to 
> make a copy for each LogEvent.
> Also, DefaultThreadContextMap uses a JDK map, the JDK collections tend to 
> allocate a lot of temporary objects.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: log4j-dev-unsubscr...@logging.apache.org
For additional commands, e-mail: log4j-dev-h...@logging.apache.org

Re: RollingAppenderSizeTest failures explained

[Gary Gregory]

Nope, I was wrong. The close shield is for the console appender...

Gary

On Thu, Jun 23, 2016 at 6:11 PM, Gary Gregory 
wrote:

> The failures in RollingAppenderSizeTest are due to the fact that the
> OutputStreamManager holds on to a CloseShieldOutputStream.
>
> Gary
>
> --
> E-Mail: garydgreg...@gmail.com | ggreg...@apache.org
> Java Persistence with Hibernate, Second Edition
> 
> JUnit in Action, Second Edition 
> Spring Batch in Action 
> Blog: http://garygregory.wordpress.com
> Home: http://garygregory.com/
> Tweet! http://twitter.com/GaryGregory
>



-- 
E-Mail: garydgreg...@gmail.com | ggreg...@apache.org
Java Persistence with Hibernate, Second Edition

JUnit in Action, Second Edition 
Spring Batch in Action 
Blog: http://garygregory.wordpress.com
Home: http://garygregory.com/
Tweet! http://twitter.com/GaryGregory

RollingAppenderSizeTest failures explained

[Gary Gregory]

The failures in RollingAppenderSizeTest are due to the fact that the
OutputStreamManager holds on to a CloseShieldOutputStream.

Gary

-- 
E-Mail: garydgreg...@gmail.com | ggreg...@apache.org
Java Persistence with Hibernate, Second Edition

JUnit in Action, Second Edition 
Spring Batch in Action 
Blog: http://garygregory.wordpress.com
Home: http://garygregory.com/
Tweet! http://twitter.com/GaryGregory

Jenkins build is back to stable : Log4j 2.x #2060

[Apache Jenkins Server]

See 


-
To unsubscribe, e-mail: log4j-dev-unsubscr...@logging.apache.org
For additional commands, e-mail: log4j-dev-h...@logging.apache.org

Re: Lock pattern

[Matt Sicker]

I kind of hate to point this out, but what's wrong with this when you're
not using multiple condition variables?

final Object lock = new Object();
synchronized (lock) {
  // ...
}

It auto-unlocks at the end, too.

On 23 June 2016 at 19:35, Gary Gregory  wrote:

> On Thu, Jun 23, 2016 at 4:58 PM, Gary Gregory 
> wrote:
>
>> On Thu, Jun 23, 2016 at 4:56 PM, Gary Gregory 
>> wrote:
>>
>>> Like this:
>>>
>>> public static boolean hasManager(final String name) {
>>> try (Object o = AutoCloseableLock.lock(LOCK)) {
>>> return MAP.containsKey(name);
>>> }
>>> }
>>>
>>> With a new class:
>>>
>>> public class AutoCloseableLock implements AutoCloseable {
>>>
>>> public static AutoCloseableLock lock(final Lock lock) {
>>> Objects.requireNonNull(lock, "lock");
>>> lock.lock();
>>> return new AutoCloseableLock(lock);
>>> }
>>>
>>> private final Lock lock;
>>>
>>> public AutoCloseableLock(final Lock lock) {
>>> this.lock = lock;
>>> }
>>>
>>> @Override
>>> public void close() {
>>> this.lock.unlock();
>>> }
>>>
>>> public void lock() {
>>> this.lock.lock();
>>> }
>>>
>>
>> You do not even need a lock() method.
>>
>
> But let's say you do for GC-free reasons:
>
> public AutoCloseableLock lock() {
> this.lock.lock();
> return this;
> }
>
> to do:
>
> private AutoCloseableLock autoCloseableLock = new
> AutoCloseableLock(new ReentrantLock());
>
> public static boolean hasManager(final String name) {
> try (Object o = autoCloseableLock.lock()) {
> return MAP.containsKey(name);
> }
> }
>
>  Gary
>
> Gary
>>
>>> }
>>>
>>> The pro is less code and less chance for errors, the con is that you
>>> create a new AutoCloseableLock for lock/unlock sites.
>>>
>>> Gary
>>>
>>> On Thu, Jun 23, 2016 at 4:54 PM, Matt Sicker  wrote:
>>>
 Sounds like an opportunity for a small Commons or independent project.
 CloseableLock!

 On 23 June 2016 at 18:51, Greg Thomas  wrote:

> I''m sure I read somewhere that it was a deliberate choice not to make
> it, to stop people using the very common pattern of creating the object in
> the try() - which isn't much use for a lock.
>
> Greg
> --
> Sent from my iPhone
>
> On 24 Jun 2016, at 00:45, Remko Popma  wrote:
>
> Good idea!
> Maybe propose this for Java 9?
> Looks very reasonable to me.
>
> Sent from my iPhone
>
> On 2016/06/24, at 8:32, Gary Gregory  wrote:
>
> I wonder if anyone knows why Lock is not AutoCloseable.
>
> This:
>
> public static boolean hasManager(final String name) {
> LOCK.lock();
> try {
> return MAP.containsKey(name);
> } finally {
> LOCK.unlock();
> }
> }
>
>
> Seems lame in comparison to:
>
> public static boolean hasManager(final String name) {
> try (LOCK.lock()) {
> return MAP.containsKey(name);
> }
> }
>
> Which, due to syntax really would be:
>
> public static boolean hasManager(final String name) {
> try (Object o = LOCK.lock()) {
> return MAP.containsKey(name);
> }
> }
>
> Just wonderin'...
>
> Gary
>
> --
> E-Mail: garydgreg...@gmail.com | ggreg...@apache.org
> Java Persistence with Hibernate, Second Edition
> 
> JUnit in Action, Second Edition 
> Spring Batch in Action 
> Blog: http://garygregory.wordpress.com
> Home: http://garygregory.com/
> Tweet! http://twitter.com/GaryGregory
>
>


 --
 Matt Sicker 

>>>
>>>
>>>
>>> --
>>> E-Mail: garydgreg...@gmail.com | ggreg...@apache.org
>>> Java Persistence with Hibernate, Second Edition
>>> 
>>> JUnit in Action, Second Edition 
>>> Spring Batch in Action 
>>> Blog: http://garygregory.wordpress.com
>>> Home: http://garygregory.com/
>>> Tweet! http://twitter.com/GaryGregory
>>>
>>
>>
>>
>> --
>> E-Mail: garydgreg...@gmail.com | ggreg...@apache.org
>> Java Persistence with Hibernate, Second Edition
>> 
>> JUnit in Action, Second Edition 
>> Spring Batch in Action 
>> Blog: http://garygregory.wordpress.com
>> Home: http://garygregory.com/
>> Tweet! http://twitter.com/GaryGregory
>>
>
>
>
> --
> E-Mail: garydgreg...@gmail.com | ggreg...@apache.org
> Java 

Re: Lock pattern

[Gary Gregory]

On Thu, Jun 23, 2016 at 4:58 PM, Gary Gregory 
wrote:

> On Thu, Jun 23, 2016 at 4:56 PM, Gary Gregory 
> wrote:
>
>> Like this:
>>
>> public static boolean hasManager(final String name) {
>> try (Object o = AutoCloseableLock.lock(LOCK)) {
>> return MAP.containsKey(name);
>> }
>> }
>>
>> With a new class:
>>
>> public class AutoCloseableLock implements AutoCloseable {
>>
>> public static AutoCloseableLock lock(final Lock lock) {
>> Objects.requireNonNull(lock, "lock");
>> lock.lock();
>> return new AutoCloseableLock(lock);
>> }
>>
>> private final Lock lock;
>>
>> public AutoCloseableLock(final Lock lock) {
>> this.lock = lock;
>> }
>>
>> @Override
>> public void close() {
>> this.lock.unlock();
>> }
>>
>> public void lock() {
>> this.lock.lock();
>> }
>>
>
> You do not even need a lock() method.
>

But let's say you do for GC-free reasons:

public AutoCloseableLock lock() {
this.lock.lock();
return this;
}

to do:

private AutoCloseableLock autoCloseableLock = new AutoCloseableLock(new
ReentrantLock());

public static boolean hasManager(final String name) {
try (Object o = autoCloseableLock.lock()) {
return MAP.containsKey(name);
}
}

 Gary

Gary
>
>> }
>>
>> The pro is less code and less chance for errors, the con is that you
>> create a new AutoCloseableLock for lock/unlock sites.
>>
>> Gary
>>
>> On Thu, Jun 23, 2016 at 4:54 PM, Matt Sicker  wrote:
>>
>>> Sounds like an opportunity for a small Commons or independent project.
>>> CloseableLock!
>>>
>>> On 23 June 2016 at 18:51, Greg Thomas  wrote:
>>>
 I''m sure I read somewhere that it was a deliberate choice not to make
 it, to stop people using the very common pattern of creating the object in
 the try() - which isn't much use for a lock.

 Greg
 --
 Sent from my iPhone

 On 24 Jun 2016, at 00:45, Remko Popma  wrote:

 Good idea!
 Maybe propose this for Java 9?
 Looks very reasonable to me.

 Sent from my iPhone

 On 2016/06/24, at 8:32, Gary Gregory  wrote:

 I wonder if anyone knows why Lock is not AutoCloseable.

 This:

 public static boolean hasManager(final String name) {
 LOCK.lock();
 try {
 return MAP.containsKey(name);
 } finally {
 LOCK.unlock();
 }
 }


 Seems lame in comparison to:

 public static boolean hasManager(final String name) {
 try (LOCK.lock()) {
 return MAP.containsKey(name);
 }
 }

 Which, due to syntax really would be:

 public static boolean hasManager(final String name) {
 try (Object o = LOCK.lock()) {
 return MAP.containsKey(name);
 }
 }

 Just wonderin'...

 Gary

 --
 E-Mail: garydgreg...@gmail.com | ggreg...@apache.org
 Java Persistence with Hibernate, Second Edition
 
 JUnit in Action, Second Edition 
 Spring Batch in Action 
 Blog: http://garygregory.wordpress.com
 Home: http://garygregory.com/
 Tweet! http://twitter.com/GaryGregory


>>>
>>>
>>> --
>>> Matt Sicker 
>>>
>>
>>
>>
>> --
>> E-Mail: garydgreg...@gmail.com | ggreg...@apache.org
>> Java Persistence with Hibernate, Second Edition
>> 
>> JUnit in Action, Second Edition 
>> Spring Batch in Action 
>> Blog: http://garygregory.wordpress.com
>> Home: http://garygregory.com/
>> Tweet! http://twitter.com/GaryGregory
>>
>
>
>
> --
> E-Mail: garydgreg...@gmail.com | ggreg...@apache.org
> Java Persistence with Hibernate, Second Edition
> 
> JUnit in Action, Second Edition 
> Spring Batch in Action 
> Blog: http://garygregory.wordpress.com
> Home: http://garygregory.com/
> Tweet! http://twitter.com/GaryGregory
>



-- 
E-Mail: garydgreg...@gmail.com | ggreg...@apache.org
Java Persistence with Hibernate, Second Edition

JUnit in Action, Second Edition 
Spring Batch in Action 
Blog: http://garygregory.wordpress.com
Home: http://garygregory.com/
Tweet! http://twitter.com/GaryGregory

[jira] [Comment Edited] (LOG4J2-1349) Garbage-free ThreadContext map

[Remko Popma (JIRA)]

[ 
https://issues.apache.org/jira/browse/LOG4J2-1349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15346312#comment-15346312
 ] 

Remko Popma edited comment on LOG4J2-1349 at 6/24/16 12:30 AM:
---

The current {{ThreadContext}} map implementation is a copy-on-write data 
structure. This is efficient because the number of _reads_ vastly outnumbers 
the number of _writes_. Every time a message is logged, the current 
ThreadContext map is passed to the LogEvent as is. This is safe because this 
map is immutable: it is replaced, not modified, when the user calls 
ThreadContext.put(key, somevalue).

The drawback is that a copy-on-write data structure is not garbage-free. 

A garbage-free ThreadContext map would have to be a mutable data structure. To 
ensure that LogEvents can safely be passed off to other threads, the LogEvent 
cannot have a direct reference to the ThreadContext map. Instead, the data from 
the ThreadContext map would need to be copied into the LogEvent's context map.

LOG4J2-1447 proposes a garbage-free data structure for LogEvent to carry 
context map data.


was (Author: rem...@yahoo.com):
The current {{ThreadContext}} map implementation is a copy-on-write data 
structure. This is efficient because the number of _reads_ vastly outnumbers 
the number of _writes_. Every time a message is logged, the current 
ThreadContext map is passed to the LogEvent as is. This is safe because this 
map is immutable: it is replaced, not modified, when the user calls 
ThreadContext.put(key, somevalue).

The drawback is that a copy-on-write data structure is not garbage-free. 

A garbage-free ThreadContext map would have to be a mutable data structure. To 
ensure that LogEvents can safely be passed off to other threads, the LogEvent 
cannot have a direct reference to the ThreadContext map. Instead, the data from 
the ThreadContext map would need to be copied into the LogEvent's context map.

For this to be garbage-free, LogEvent needs a different data structure to carry 
context map data. Currently LogEvent implementations use a 
{{java.util.Map}} for this. The JDK Map is not an easy data 
structure to make garbage-free. It would also be nice to have the ability in 
LogEvent to carry data of any type.

One idea is to introduce a small interface that is general enough to be 
map-like but can be implemented in a garbage-free manner. 

LogEvent implementations would have an instance of this interface instead of 
the current {{java.util.Map contextMap}} attribute.

Something like this:
{code}
interface CustomData {
/** Called to implement {@link LogEvent#getContextMap()}. */
Map asMap();

/** Put key-value pair into the table.
Remove key if value is null. */
void put(K key, V value);

/** Returns the value for the specified key. */
V getValue(K key);

/** Number of key-value pairs. */
int size();

/** Removes all key-value pairs. */
void clear();

// Instead of Iterators, use an index-based approach.

/** Returns the index of the specified key. */
int indexOfKey(K key);

/** Returns the i-th key. */
K getKeyAt(int i);

/** Returns the value for the i-th key. */
V getValueAt(int i);
}
{code}

The LogEvent interface would have an additional method {{getCustomData() : 
CustomData}} that gives downstream components direct access to the new 
data structure. 

Existing downstream components would still be able to call 
{{logEvent.getContextMap()}} to get a {{Map}} view of the 
context map data and this would work as expected (although it may not be 
garbage-free).

> Garbage-free ThreadContext map
> --
>
> Key: LOG4J2-1349
> URL: https://issues.apache.org/jira/browse/LOG4J2-1349
> Project: Log4j 2
>  Issue Type: Improvement
>  Components: API
>Affects Versions: 2.5
>Reporter: Remko Popma
> Fix For: 2.7
>
>
> The current ThreadContext map and stack implementations allocate temporary 
> objects. This ticket is to investigate and track the work for alternative 
> implementations that are garbage-free.
> Both DefaultThreadContextMap and DefaultThreadContextStack are copy-on-write 
> data structures: each modification replaces the ThreadLocal object with a 
> modified copy. The advantage of this approach is that there is no need to 
> make a copy for each LogEvent.
> Also, DefaultThreadContextMap uses a JDK map, the JDK collections tend to 
> allocate a lot of temporary objects.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: log4j-dev-unsubscr...@logging.apache.org
For additional commands, e-mail: log4j-dev-h...@logging.apache.org

[jira] [Created] (LOG4J2-1447) Garbage-free data structure for LogEvent's context map data

[Remko Popma (JIRA)]

Remko Popma created LOG4J2-1447:
---

 Summary: Garbage-free data structure for LogEvent's context map 
data
 Key: LOG4J2-1447
 URL: https://issues.apache.org/jira/browse/LOG4J2-1447
 Project: Log4j 2
  Issue Type: Improvement
  Components: Core
Affects Versions: 2.6.1
Reporter: Remko Popma
Assignee: Remko Popma
 Fix For: 2.7


With each logging call, context map data is copied from the {{ThreadContext}} 
map into the {{LogEvent}}.

The LogEvent interface exposes this data via the {{getContextData() : 
Map}} method, and this is implementated by storing the data in 
a Map.  The JDK Map is not an easy data structure to make 
garbage-free. It would also be nice to have the ability in LogEvent to carry 
data of any type.

One idea is to introduce a small interface that is general enough to be 
map-like but can be implemented in a garbage-free manner. 

LogEvent implementations would have an instance of this interface instead of 
the current {{java.util.Map contextMap}} attribute.

The interface could look something like this:
{code}
interface CustomData {
/** Called to implement {@link LogEvent#getContextMap()}. */
Map asMap();

/** Put key-value pair into the table.
Remove key if value is null. */
void put(K key, V value);

/** Returns the value for the specified key. */
V getValue(K key);

/** Number of key-value pairs. */
int size();

/** Removes all key-value pairs. */
void clear();

// Instead of Iterators, use an index-based approach.

/** Returns the index of the specified key. */
int indexOfKey(K key);

/** Returns the i-th key. */
K getKeyAt(int i);

/** Returns the value for the i-th key. */
V getValueAt(int i);
}
{code}

The LogEvent interface would have an additional method {{getCustomData() : 
CustomData}} that gives downstream components direct access to the new 
data structure. 

Existing downstream components would still be able to call 
{{logEvent.getContextMap()}} to get a {{Map}} view of the 
context map data and this would work as expected (although it may not be 
garbage-free).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: log4j-dev-unsubscr...@logging.apache.org
For additional commands, e-mail: log4j-dev-h...@logging.apache.org

[jira] [Closed] (LOG4J2-1445) OnStartupTriggeringPolicyTest fails on Windows saying the file is used by another process

[Ludovic HOCHET (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LOG4J2-1445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ludovic HOCHET closed LOG4J2-1445.
--

The fix works for me.

> OnStartupTriggeringPolicyTest fails on Windows saying the file is used by 
> another process
> -
>
> Key: LOG4J2-1445
> URL: https://issues.apache.org/jira/browse/LOG4J2-1445
> Project: Log4j 2
>  Issue Type: Bug
>  Components: Core
>Affects Versions: 2.6.2
> Environment: Windows 10, JDK 8
>Reporter: Ludovic HOCHET
>Priority: Trivial
> Fix For: 2.6.2
>
>
> I saw some mention on the mailing list that some tests were failing on 
> Windows and thought I'd give a look.
> Effectively OnStartupTriggeringPolicyTest is failing on Windows with:
> java.nio.file.FileSystemException: 
> target\testfile: Le processus ne peut pas accéder au fichier car ce fichier 
> est utilisé par un autre processus.
> at 
> org.apache.logging.log4j.core.appender.rolling.OnStartupTriggeringPolicyTest.cleanup(OnStartupTriggeringPolicyTest.java:77)
> This can be solved by ending the test with:
> manager.release();
> And is due to RollingFileManagerFactory.createManager(..) using new 
> FileOutputStream(..) which eventually calls Windows API CreateFileW(..) 
> without the FILE_SHARE_DELETE flag which prevent other calls to delete the 
> file.
> (Files.newOutputStream(..) eventually does the opposite)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: log4j-dev-unsubscr...@logging.apache.org
For additional commands, e-mail: log4j-dev-h...@logging.apache.org

[jira] [Updated] (LOG4J2-1349) Garbage-free ThreadContext map

[Remko Popma (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LOG4J2-1349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Remko Popma updated LOG4J2-1349:

Summary: Garbage-free ThreadContext map  (was: Garbage-free ThreadContext 
map and stack)

> Garbage-free ThreadContext map
> --
>
> Key: LOG4J2-1349
> URL: https://issues.apache.org/jira/browse/LOG4J2-1349
> Project: Log4j 2
>  Issue Type: Improvement
>  Components: API
>Affects Versions: 2.5
>Reporter: Remko Popma
> Fix For: 2.7
>
>
> The current ThreadContext map and stack implementations allocate temporary 
> objects. This ticket is to investigate and track the work for alternative 
> implementations that are garbage-free.
> Both DefaultThreadContextMap and DefaultThreadContextStack are copy-on-write 
> data structures: each modification replaces the ThreadLocal object with a 
> modified copy. The advantage of this approach is that there is no need to 
> make a copy for each LogEvent.
> Also, DefaultThreadContextMap uses a JDK map, the JDK collections tend to 
> allocate a lot of temporary objects.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: log4j-dev-unsubscr...@logging.apache.org
For additional commands, e-mail: log4j-dev-h...@logging.apache.org

[jira] [Comment Edited] (LOG4J2-1349) Garbage-free ThreadContext map and stack

[Remko Popma (JIRA)]

[ 
https://issues.apache.org/jira/browse/LOG4J2-1349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15346312#comment-15346312
 ] 

Remko Popma edited comment on LOG4J2-1349 at 6/24/16 12:09 AM:
---

The current {{ThreadContext}} map implementation is a copy-on-write data 
structure. This is efficient because the number of _reads_ vastly outnumbers 
the number of _writes_. Every time a message is logged, the current 
ThreadContext map is passed to the LogEvent as is. This is safe because this 
map is immutable: it is replaced, not modified, when the user calls 
ThreadContext.put(key, somevalue).

The drawback is that a copy-on-write data structure is not garbage-free. 

A garbage-free ThreadContext map would have to be a mutable data structure. To 
ensure that LogEvents can safely be passed off to other threads, the LogEvent 
cannot have a direct reference to the ThreadContext map. Instead, the data from 
the ThreadContext map would need to be copied into the LogEvent's context map.

For this to be garbage-free, LogEvent needs a different data structure to carry 
context map data. Currently LogEvent implementations use a 
{{java.util.Map}} for this. The JDK Map is not an easy data 
structure to make garbage-free. It would also be nice to have the ability in 
LogEvent to carry data of any type.

One idea is to introduce a small interface that is general enough to be 
map-like but can be implemented in a garbage-free manner. 

LogEvent implementations would have an instance of this interface instead of 
the current {{java.util.Map contextMap}} attribute.

Something like this:
{code}
interface CustomData {
/** Called to implement {@link LogEvent#getContextMap()}. */
Map asMap();

/** Put key-value pair into the table.
Remove key if value is null. */
void put(K key, V value);

/** Returns the value for the specified key. */
V getValue(K key);

/** Number of key-value pairs. */
int size();

/** Removes all key-value pairs. */
void clear();

// Instead of Iterators, use an index-based approach.

/** Returns the index of the specified key. */
int indexOfKey(K key);

/** Returns the i-th key. */
K getKeyAt(int i);

/** Returns the value for the i-th key. */
V getValueAt(int i);
}
{code}

The LogEvent interface would have an additional method {{getCustomData() : 
CustomData}} that gives downstream components direct access to the new 
data structure. 

Existing downstream components would still be able to call 
{{logEvent.getContextMap()}} to get a {{Map}} view of the 
context map data and this would work as expected (although it may not be 
garbage-free).


was (Author: rem...@yahoo.com):
The current {{ThreadContext}} map implementation is a copy-on-write data 
structure. This is efficient because the number of _reads_ vastly outnumbers 
the number of _writes_. Every time a message is logged, the current 
ThreadContext map is passed to the LogEvent as is. This is safe because this 
map is immutable: it is replaced, not modified, when the user calls 
ThreadContext.put(key, somevalue).

The drawback is that a copy-on-write data structure is not garbage-free. 

A garbage-free ThreadContext map would have to be a mutable data structure. To 
ensure that LogEvents can safely be passed off to other threads, the LogEvent 
cannot have a direct reference to the ThreadContext map. Instead, the data from 
the ThreadContext map would need to be copied into the LogEvent's context map.

For this to be garbage-free, LogEvent needs a different data structure to carry 
context map data. Currently LogEvent implementations use a 
{{java.util.Map}} for this. The JDK Map is not an easy data 
structure to make garbage-free. It would also be nice to have the ability in 
LogEvent to carry data of any type.

One idea is to introduce a small interface that is general enough to be 
map-like but can be implemented in a garbage-free manner. 

LogEvent implementations would have an instance of this interface instead of 
the current {{java.util.Map contextMap}} attribute.

Something like this:
{code}
interface CustomData {
/** Called to implement {@link LogEvent#getContextMap()}. */
Map asMap();

/** Put key-value pair into the table.
Remove key if value is null. */
void put(K key, V value);

/** Returns the value for the specified key. */
V getValue(K key);

/** Number of key-value pairs. */
int size();

/** Removes all key-value pairs. */
void clear();

// Instead of Iterators, use an index-based approach.

/** Returns the index of the specified key. */
int indexOfKey(K key);

/** Returns the i-th key. */
K getKey(int i);

/** Returns the value for the i-th key. */
V getValueAt(int i);
}

Jenkins build became unstable: Log4j 2.x #2059

[Apache Jenkins Server]

See 


-
To unsubscribe, e-mail: log4j-dev-unsubscr...@logging.apache.org
For additional commands, e-mail: log4j-dev-h...@logging.apache.org

Re: Lock pattern

[Gary Gregory]

On Thu, Jun 23, 2016 at 4:51 PM, Greg Thomas 
wrote:

> I''m sure I read somewhere that it was a deliberate choice not to make it,
> to stop people using the very common pattern of creating the object in the
> try() - which isn't much use for a lock.
>

It feels like there is a discrepancy between the narrow AutoCloseable use
case of "close me", and the more general try-with-resources which feels
more about do/undo, start/stop, being/end-type of pattern.

Gary

>
>
> Greg
> --
> Sent from my iPhone
>
> On 24 Jun 2016, at 00:45, Remko Popma  wrote:
>
> Good idea!
> Maybe propose this for Java 9?
> Looks very reasonable to me.
>
> Sent from my iPhone
>
> On 2016/06/24, at 8:32, Gary Gregory  wrote:
>
> I wonder if anyone knows why Lock is not AutoCloseable.
>
> This:
>
> public static boolean hasManager(final String name) {
> LOCK.lock();
> try {
> return MAP.containsKey(name);
> } finally {
> LOCK.unlock();
> }
> }
>
>
> Seems lame in comparison to:
>
> public static boolean hasManager(final String name) {
> try (LOCK.lock()) {
> return MAP.containsKey(name);
> }
> }
>
> Which, due to syntax really would be:
>
> public static boolean hasManager(final String name) {
> try (Object o = LOCK.lock()) {
> return MAP.containsKey(name);
> }
> }
>
> Just wonderin'...
>
> Gary
>
> --
> E-Mail: garydgreg...@gmail.com | ggreg...@apache.org
> Java Persistence with Hibernate, Second Edition
> 
> JUnit in Action, Second Edition 
> Spring Batch in Action 
> Blog: http://garygregory.wordpress.com
> Home: http://garygregory.com/
> Tweet! http://twitter.com/GaryGregory
>
>


-- 
E-Mail: garydgreg...@gmail.com | ggreg...@apache.org
Java Persistence with Hibernate, Second Edition

JUnit in Action, Second Edition 
Spring Batch in Action 
Blog: http://garygregory.wordpress.com
Home: http://garygregory.com/
Tweet! http://twitter.com/GaryGregory

Re: Lock pattern

[Gary Gregory]

On Thu, Jun 23, 2016 at 4:56 PM, Gary Gregory 
wrote:

> Like this:
>
> public static boolean hasManager(final String name) {
> try (Object o = AutoCloseableLock.lock(LOCK)) {
> return MAP.containsKey(name);
> }
> }
>
> With a new class:
>
> public class AutoCloseableLock implements AutoCloseable {
>
> public static AutoCloseableLock lock(final Lock lock) {
> Objects.requireNonNull(lock, "lock");
> lock.lock();
> return new AutoCloseableLock(lock);
> }
>
> private final Lock lock;
>
> public AutoCloseableLock(final Lock lock) {
> this.lock = lock;
> }
>
> @Override
> public void close() {
> this.lock.unlock();
> }
>
> public void lock() {
> this.lock.lock();
> }
>

You do not even need a lock() method.

Gary

> }
>
> The pro is less code and less chance for errors, the con is that you
> create a new AutoCloseableLock for lock/unlock sites.
>
> Gary
>
> On Thu, Jun 23, 2016 at 4:54 PM, Matt Sicker  wrote:
>
>> Sounds like an opportunity for a small Commons or independent project.
>> CloseableLock!
>>
>> On 23 June 2016 at 18:51, Greg Thomas  wrote:
>>
>>> I''m sure I read somewhere that it was a deliberate choice not to make
>>> it, to stop people using the very common pattern of creating the object in
>>> the try() - which isn't much use for a lock.
>>>
>>> Greg
>>> --
>>> Sent from my iPhone
>>>
>>> On 24 Jun 2016, at 00:45, Remko Popma  wrote:
>>>
>>> Good idea!
>>> Maybe propose this for Java 9?
>>> Looks very reasonable to me.
>>>
>>> Sent from my iPhone
>>>
>>> On 2016/06/24, at 8:32, Gary Gregory  wrote:
>>>
>>> I wonder if anyone knows why Lock is not AutoCloseable.
>>>
>>> This:
>>>
>>> public static boolean hasManager(final String name) {
>>> LOCK.lock();
>>> try {
>>> return MAP.containsKey(name);
>>> } finally {
>>> LOCK.unlock();
>>> }
>>> }
>>>
>>>
>>> Seems lame in comparison to:
>>>
>>> public static boolean hasManager(final String name) {
>>> try (LOCK.lock()) {
>>> return MAP.containsKey(name);
>>> }
>>> }
>>>
>>> Which, due to syntax really would be:
>>>
>>> public static boolean hasManager(final String name) {
>>> try (Object o = LOCK.lock()) {
>>> return MAP.containsKey(name);
>>> }
>>> }
>>>
>>> Just wonderin'...
>>>
>>> Gary
>>>
>>> --
>>> E-Mail: garydgreg...@gmail.com | ggreg...@apache.org
>>> Java Persistence with Hibernate, Second Edition
>>> 
>>> JUnit in Action, Second Edition 
>>> Spring Batch in Action 
>>> Blog: http://garygregory.wordpress.com
>>> Home: http://garygregory.com/
>>> Tweet! http://twitter.com/GaryGregory
>>>
>>>
>>
>>
>> --
>> Matt Sicker 
>>
>
>
>
> --
> E-Mail: garydgreg...@gmail.com | ggreg...@apache.org
> Java Persistence with Hibernate, Second Edition
> 
> JUnit in Action, Second Edition 
> Spring Batch in Action 
> Blog: http://garygregory.wordpress.com
> Home: http://garygregory.com/
> Tweet! http://twitter.com/GaryGregory
>



-- 
E-Mail: garydgreg...@gmail.com | ggreg...@apache.org
Java Persistence with Hibernate, Second Edition

JUnit in Action, Second Edition 
Spring Batch in Action 
Blog: http://garygregory.wordpress.com
Home: http://garygregory.com/
Tweet! http://twitter.com/GaryGregory

Re: Lock pattern

[Gary Gregory]

Like this:

public static boolean hasManager(final String name) {
try (Object o = AutoCloseableLock.lock(LOCK)) {
return MAP.containsKey(name);
}
}

With a new class:

public class AutoCloseableLock implements AutoCloseable {

public static AutoCloseableLock lock(final Lock lock) {
Objects.requireNonNull(lock, "lock");
lock.lock();
return new AutoCloseableLock(lock);
}

private final Lock lock;

public AutoCloseableLock(final Lock lock) {
this.lock = lock;
}

@Override
public void close() {
this.lock.unlock();
}

public void lock() {
this.lock.lock();
}
}

The pro is less code and less chance for errors, the con is that you create
a new AutoCloseableLock for lock/unlock sites.

Gary

On Thu, Jun 23, 2016 at 4:54 PM, Matt Sicker  wrote:

> Sounds like an opportunity for a small Commons or independent project.
> CloseableLock!
>
> On 23 June 2016 at 18:51, Greg Thomas  wrote:
>
>> I''m sure I read somewhere that it was a deliberate choice not to make
>> it, to stop people using the very common pattern of creating the object in
>> the try() - which isn't much use for a lock.
>>
>> Greg
>> --
>> Sent from my iPhone
>>
>> On 24 Jun 2016, at 00:45, Remko Popma  wrote:
>>
>> Good idea!
>> Maybe propose this for Java 9?
>> Looks very reasonable to me.
>>
>> Sent from my iPhone
>>
>> On 2016/06/24, at 8:32, Gary Gregory  wrote:
>>
>> I wonder if anyone knows why Lock is not AutoCloseable.
>>
>> This:
>>
>> public static boolean hasManager(final String name) {
>> LOCK.lock();
>> try {
>> return MAP.containsKey(name);
>> } finally {
>> LOCK.unlock();
>> }
>> }
>>
>>
>> Seems lame in comparison to:
>>
>> public static boolean hasManager(final String name) {
>> try (LOCK.lock()) {
>> return MAP.containsKey(name);
>> }
>> }
>>
>> Which, due to syntax really would be:
>>
>> public static boolean hasManager(final String name) {
>> try (Object o = LOCK.lock()) {
>> return MAP.containsKey(name);
>> }
>> }
>>
>> Just wonderin'...
>>
>> Gary
>>
>> --
>> E-Mail: garydgreg...@gmail.com | ggreg...@apache.org
>> Java Persistence with Hibernate, Second Edition
>> 
>> JUnit in Action, Second Edition 
>> Spring Batch in Action 
>> Blog: http://garygregory.wordpress.com
>> Home: http://garygregory.com/
>> Tweet! http://twitter.com/GaryGregory
>>
>>
>
>
> --
> Matt Sicker 
>



-- 
E-Mail: garydgreg...@gmail.com | ggreg...@apache.org
Java Persistence with Hibernate, Second Edition

JUnit in Action, Second Edition 
Spring Batch in Action 
Blog: http://garygregory.wordpress.com
Home: http://garygregory.com/
Tweet! http://twitter.com/GaryGregory

Re: Lock pattern

[Matt Sicker]

Sounds like an opportunity for a small Commons or independent project.
CloseableLock!

On 23 June 2016 at 18:51, Greg Thomas  wrote:

> I''m sure I read somewhere that it was a deliberate choice not to make it,
> to stop people using the very common pattern of creating the object in the
> try() - which isn't much use for a lock.
>
> Greg
> --
> Sent from my iPhone
>
> On 24 Jun 2016, at 00:45, Remko Popma  wrote:
>
> Good idea!
> Maybe propose this for Java 9?
> Looks very reasonable to me.
>
> Sent from my iPhone
>
> On 2016/06/24, at 8:32, Gary Gregory  wrote:
>
> I wonder if anyone knows why Lock is not AutoCloseable.
>
> This:
>
> public static boolean hasManager(final String name) {
> LOCK.lock();
> try {
> return MAP.containsKey(name);
> } finally {
> LOCK.unlock();
> }
> }
>
>
> Seems lame in comparison to:
>
> public static boolean hasManager(final String name) {
> try (LOCK.lock()) {
> return MAP.containsKey(name);
> }
> }
>
> Which, due to syntax really would be:
>
> public static boolean hasManager(final String name) {
> try (Object o = LOCK.lock()) {
> return MAP.containsKey(name);
> }
> }
>
> Just wonderin'...
>
> Gary
>
> --
> E-Mail: garydgreg...@gmail.com | ggreg...@apache.org
> Java Persistence with Hibernate, Second Edition
> 
> JUnit in Action, Second Edition 
> Spring Batch in Action 
> Blog: http://garygregory.wordpress.com
> Home: http://garygregory.com/
> Tweet! http://twitter.com/GaryGregory
>
>


-- 
Matt Sicker 

Re: Lock pattern

[Greg Thomas]

I''m sure I read somewhere that it was a deliberate choice not to make it, to 
stop people using the very common pattern of creating the object in the try() - 
which isn't much use for a lock. 

Greg
-- 
Sent from my iPhone

> On 24 Jun 2016, at 00:45, Remko Popma  wrote:
> 
> Good idea! 
> Maybe propose this for Java 9?
> Looks very reasonable to me. 
> 
> Sent from my iPhone
> 
>> On 2016/06/24, at 8:32, Gary Gregory  wrote:
>> 
>> I wonder if anyone knows why Lock is not AutoCloseable.
>> 
>> This:
>> 
>> public static boolean hasManager(final String name) {
>> LOCK.lock();
>> try {
>> return MAP.containsKey(name);
>> } finally {
>> LOCK.unlock();
>> }
>> }
>> 
>> 
>> Seems lame in comparison to:
>> 
>> public static boolean hasManager(final String name) {
>> try (LOCK.lock()) {
>> return MAP.containsKey(name);
>> }
>> }
>> 
>> Which, due to syntax really would be:
>> 
>> public static boolean hasManager(final String name) {
>> try (Object o = LOCK.lock()) {
>> return MAP.containsKey(name);
>> }
>> }
>> 
>> Just wonderin'...
>> 
>> Gary
>> 
>> -- 
>> E-Mail: garydgreg...@gmail.com | ggreg...@apache.org 
>> Java Persistence with Hibernate, Second Edition
>> JUnit in Action, Second Edition
>> Spring Batch in Action
>> Blog: http://garygregory.wordpress.com 
>> Home: http://garygregory.com/
>> Tweet! http://twitter.com/GaryGregory

Re: Lock pattern

[Remko Popma]

Good idea! 
Maybe propose this for Java 9?
Looks very reasonable to me. 

Sent from my iPhone

> On 2016/06/24, at 8:32, Gary Gregory  wrote:
> 
> I wonder if anyone knows why Lock is not AutoCloseable.
> 
> This:
> 
> public static boolean hasManager(final String name) {
> LOCK.lock();
> try {
> return MAP.containsKey(name);
> } finally {
> LOCK.unlock();
> }
> }
> 
> 
> Seems lame in comparison to:
> 
> public static boolean hasManager(final String name) {
> try (LOCK.lock()) {
> return MAP.containsKey(name);
> }
> }
> 
> Which, due to syntax really would be:
> 
> public static boolean hasManager(final String name) {
> try (Object o = LOCK.lock()) {
> return MAP.containsKey(name);
> }
> }
> 
> Just wonderin'...
> 
> Gary
> 
> -- 
> E-Mail: garydgreg...@gmail.com | ggreg...@apache.org 
> Java Persistence with Hibernate, Second Edition
> JUnit in Action, Second Edition
> Spring Batch in Action
> Blog: http://garygregory.wordpress.com 
> Home: http://garygregory.com/
> Tweet! http://twitter.com/GaryGregory

[jira] [Resolved] (LOG4J2-1445) OnStartupTriggeringPolicyTest fails on Windows saying the file is used by another process

[Gary Gregory (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LOG4J2-1445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gary Gregory resolved LOG4J2-1445.
--
   Resolution: Fixed
Fix Version/s: 2.6.2

Thank you for your suggestion. A fix is in Git master. Can you please verify 
and close this ticket if the fix works on your set up?

I works for me with:

Apache Maven 3.3.9 (bb52d8502b132ec0a5a3f4c09453c07478323dc5; 
2015-11-10T08:41:47-08:00)
Maven home: E:\Java\apache-maven-3.3.9
Java version: 1.7.0_79, vendor: Oracle Corporation
Java home: C:\Program Files\Java\jdk1.7.0_79\jre
Default locale: en_US, platform encoding: Cp1252
OS name: "windows 7", version: "6.1", arch: "amd64", family: "windows"

Thank you,
Gary

> OnStartupTriggeringPolicyTest fails on Windows saying the file is used by 
> another process
> -
>
> Key: LOG4J2-1445
> URL: https://issues.apache.org/jira/browse/LOG4J2-1445
> Project: Log4j 2
>  Issue Type: Bug
>  Components: Core
>Affects Versions: 2.6.2
> Environment: Windows 10, JDK 8
>Reporter: Ludovic HOCHET
>Priority: Trivial
> Fix For: 2.6.2
>
>
> I saw some mention on the mailing list that some tests were failing on 
> Windows and thought I'd give a look.
> Effectively OnStartupTriggeringPolicyTest is failing on Windows with:
> java.nio.file.FileSystemException: 
> target\testfile: Le processus ne peut pas accéder au fichier car ce fichier 
> est utilisé par un autre processus.
> at 
> org.apache.logging.log4j.core.appender.rolling.OnStartupTriggeringPolicyTest.cleanup(OnStartupTriggeringPolicyTest.java:77)
> This can be solved by ending the test with:
> manager.release();
> And is due to RollingFileManagerFactory.createManager(..) using new 
> FileOutputStream(..) which eventually calls Windows API CreateFileW(..) 
> without the FILE_SHARE_DELETE flag which prevent other calls to delete the 
> file.
> (Files.newOutputStream(..) eventually does the opposite)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: log4j-dev-unsubscr...@logging.apache.org
For additional commands, e-mail: log4j-dev-h...@logging.apache.org

Lock pattern

[Gary Gregory]

I wonder if anyone knows why Lock is not AutoCloseable.

This:

public static boolean hasManager(final String name) {
LOCK.lock();
try {
return MAP.containsKey(name);
} finally {
LOCK.unlock();
}
}


Seems lame in comparison to:

public static boolean hasManager(final String name) {
try (LOCK.lock()) {
return MAP.containsKey(name);
}
}

Which, due to syntax really would be:

public static boolean hasManager(final String name) {
try (Object o = LOCK.lock()) {
return MAP.containsKey(name);
}
}

Just wonderin'...

Gary

-- 
E-Mail: garydgreg...@gmail.com | ggreg...@apache.org
Java Persistence with Hibernate, Second Edition

JUnit in Action, Second Edition 
Spring Batch in Action 
Blog: http://garygregory.wordpress.com
Home: http://garygregory.com/
Tweet! http://twitter.com/GaryGregory

AbstractManager.release() -> AutoCloseable

[Gary Gregory]

I seems that AbstractManager should implement AutoCloseable where close()
does the same thing as release() and release() can be deprecated.

org.apache.logging.log4j.core.appender.AbstractManager.release()

This will let us rewrite things like the recently fixed
"OnStartupTriggeringPolicyTest
fails on Windows saying the file is used by another process"
https://issues.apache.org/jira/browse/LOG4J2-1445:

Is:

final RollingFileManager manager =
RollingFileManager.getFileManager(TARGET_FILE, TARGET_PATTERN, true, false,
policy, strategy, null, layout, 8192, true);
try {
manager.initialize();
assertTrue(Files.exists(target));
assertTrue(Files.size(target) == 0);
assertTrue(Files.exists(rolled));
assertTrue(Files.size(rolled) == size);
} finally {
manager.release();
}

Could be:

try (final RollingFileManager manager =
RollingFileManager.getFileManager(TARGET_FILE, TARGET_PATTERN, true, false,
policy, strategy, null, layout, 8192, true)) {
manager.initialize();
assertTrue(Files.exists(target));
assertTrue(Files.size(target) == 0);
assertTrue(Files.exists(rolled));
assertTrue(Files.size(rolled) == size);
}

Thoughts?

Gary


-- 
E-Mail: garydgreg...@gmail.com | ggreg...@apache.org
Java Persistence with Hibernate, Second Edition

JUnit in Action, Second Edition 
Spring Batch in Action 
Blog: http://garygregory.wordpress.com
Home: http://garygregory.com/
Tweet! http://twitter.com/GaryGregory

[jira] [Created] (LOG4J2-1446) RollingAppenderSizeTest fails on Windows saying the file is used by another process

[Ludovic HOCHET (JIRA)]

Ludovic HOCHET created LOG4J2-1446:
--

 Summary: RollingAppenderSizeTest fails on Windows saying the file 
is used by another process
 Key: LOG4J2-1446
 URL: https://issues.apache.org/jira/browse/LOG4J2-1446
 Project: Log4j 2
  Issue Type: Bug
  Components: Core
Affects Versions: 2.6.2
 Environment: Windows 10, JDK 8
Reporter: Ludovic HOCHET
Priority: Trivial


RollingAppenderSizeTest s fails on Windows saying the file are being used by 
another process.
This is like LOG4J2-1445 it seems.
The fix this time is to call LogManager.shutdown() before deleteDir().





--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: log4j-dev-unsubscr...@logging.apache.org
For additional commands, e-mail: log4j-dev-h...@logging.apache.org

[jira] [Created] (LOG4J2-1445) OnStartupTriggeringPolicyTest fails on Windows saying the file is used by another process

[Ludovic HOCHET (JIRA)]

Ludovic HOCHET created LOG4J2-1445:
--

 Summary: OnStartupTriggeringPolicyTest fails on Windows saying the 
file is used by another process
 Key: LOG4J2-1445
 URL: https://issues.apache.org/jira/browse/LOG4J2-1445
 Project: Log4j 2
  Issue Type: Bug
  Components: Core
Affects Versions: 2.6.2
 Environment: Windows 10, JDK 8
Reporter: Ludovic HOCHET
Priority: Trivial


I saw some mention on the mailing list that some tests were failing on Windows 
and thought I'd give a look.
Effectively OnStartupTriggeringPolicyTest is failing on Windows with:
java.nio.file.FileSystemException: 
target\testfile: Le processus ne peut pas accéder au fichier car ce fichier est 
utilisé par un autre processus.
at 
org.apache.logging.log4j.core.appender.rolling.OnStartupTriggeringPolicyTest.cleanup(OnStartupTriggeringPolicyTest.java:77)

This can be solved by ending the test with:
manager.release();

And is due to RollingFileManagerFactory.createManager(..) using new 
FileOutputStream(..) which eventually calls Windows API CreateFileW(..) without 
the FILE_SHARE_DELETE flag which prevent other calls to delete the file.

(Files.newOutputStream(..) eventually does the opposite)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: log4j-dev-unsubscr...@logging.apache.org
For additional commands, e-mail: log4j-dev-h...@logging.apache.org

Jenkins build is back to stable : Log4j 2.x #2058

[Apache Jenkins Server]

See 


-
To unsubscribe, e-mail: log4j-dev-unsubscr...@logging.apache.org
For additional commands, e-mail: log4j-dev-h...@logging.apache.org

Jenkins build became unstable: Log4j 2.x #2057

[Apache Jenkins Server]

See 


-
To unsubscribe, e-mail: log4j-dev-unsubscr...@logging.apache.org
For additional commands, e-mail: log4j-dev-h...@logging.apache.org

[jira] [Commented] (LOG4J2-1010) Injectable context properties

[Ralph Goers (JIRA)]

[ 
https://issues.apache.org/jira/browse/LOG4J2-1010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15346789#comment-15346789
 ] 

Ralph Goers commented on LOG4J2-1010:
-

I just saw my name mentioned here. I definitely will want to weigh in on this 
issue but I haven't had time to read it. You will have to give me a day or two.

> Injectable context properties
> -
>
> Key: LOG4J2-1010
> URL: https://issues.apache.org/jira/browse/LOG4J2-1010
> Project: Log4j 2
>  Issue Type: Improvement
>  Components: API
>Affects Versions: 2.2
>Reporter: Mikael Ståldal
> Attachments: properties.patch
>
>
> It would be useful to have a way to inject context properties into a 
> {{LogEvent}}, as an alternative to {{ThreadContext}}.
> In an asynchronous environment, using ThreadContext as currently implemented 
> is not so useful since JVM threads might not be coupled to the logical flow 
> of the application.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: log4j-dev-unsubscr...@logging.apache.org
For additional commands, e-mail: log4j-dev-h...@logging.apache.org

[jira] [Commented] (LOG4J2-1010) Injectable context properties

[JIRA]

[ 
https://issues.apache.org/jira/browse/LOG4J2-1010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15346781#comment-15346781
 ] 

Mikael Ståldal commented on LOG4J2-1010:


Don't get me wrong. I like this injector approach in general, it is better than 
my idea of adding a parameter to {{ExtendedLogger.logMessage}} (so let's forget 
about it).

We just have to sort out the details about how the interface should look, and 
where it should be (log4j-api or log4j-core). Maybe [~ralphgoers], 
[~garydgregory] or [~jvz] have some input on this?

BTW, I want it to be possible to inject String key-value pairs that can be 
consumed by existing Layouts and Appenders not aware of this new feature, to 
them it should look exactly like the current ThreadContext map.

BTW2, should we support injection of context stack as well, or can we consider 
that deprecated (I have never used that)?

> Injectable context properties
> -
>
> Key: LOG4J2-1010
> URL: https://issues.apache.org/jira/browse/LOG4J2-1010
> Project: Log4j 2
>  Issue Type: Improvement
>  Components: API
>Affects Versions: 2.2
>Reporter: Mikael Ståldal
> Attachments: properties.patch
>
>
> It would be useful to have a way to inject context properties into a 
> {{LogEvent}}, as an alternative to {{ThreadContext}}.
> In an asynchronous environment, using ThreadContext as currently implemented 
> is not so useful since JVM threads might not be coupled to the logical flow 
> of the application.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: log4j-dev-unsubscr...@logging.apache.org
For additional commands, e-mail: log4j-dev-h...@logging.apache.org

[jira] [Commented] (LOG4J2-1010) Injectable context properties

[Remko Popma (JIRA)]

[ 
https://issues.apache.org/jira/browse/LOG4J2-1010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15346749#comment-15346749
 ] 

Remko Popma commented on LOG4J2-1010:
-

The beauty of this approach is that it would provide a generic solution for 
this ticket as well as help solve LOG4J2-1401 and LOG4J2-1349. 

There have also been user requests in the past (cannot find the Jira now) to 
allow putting custom data in the LogEvent to be consumed by custom Layouts or 
Appenders. These are rare cases but until now the only answer we had was "put a 
String key-value pair in the ThreadContext map". With this approach users can 
provide their own injector to put data of arbitrary type in the log event.

> Injectable context properties
> -
>
> Key: LOG4J2-1010
> URL: https://issues.apache.org/jira/browse/LOG4J2-1010
> Project: Log4j 2
>  Issue Type: Improvement
>  Components: API
>Affects Versions: 2.2
>Reporter: Mikael Ståldal
> Attachments: properties.patch
>
>
> It would be useful to have a way to inject context properties into a 
> {{LogEvent}}, as an alternative to {{ThreadContext}}.
> In an asynchronous environment, using ThreadContext as currently implemented 
> is not so useful since JVM threads might not be coupled to the logical flow 
> of the application.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: log4j-dev-unsubscr...@logging.apache.org
For additional commands, e-mail: log4j-dev-h...@logging.apache.org

[jira] [Commented] (LOG4J2-1010) Injectable context properties

[JIRA]

[ 
https://issues.apache.org/jira/browse/LOG4J2-1010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15346716#comment-15346716
 ] 

Mikael Ståldal commented on LOG4J2-1010:


BTW, it is Finagle, not Finacle.

> Injectable context properties
> -
>
> Key: LOG4J2-1010
> URL: https://issues.apache.org/jira/browse/LOG4J2-1010
> Project: Log4j 2
>  Issue Type: Improvement
>  Components: API
>Affects Versions: 2.2
>Reporter: Mikael Ståldal
> Attachments: properties.patch
>
>
> It would be useful to have a way to inject context properties into a 
> {{LogEvent}}, as an alternative to {{ThreadContext}}.
> In an asynchronous environment, using ThreadContext as currently implemented 
> is not so useful since JVM threads might not be coupled to the logical flow 
> of the application.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: log4j-dev-unsubscr...@logging.apache.org
For additional commands, e-mail: log4j-dev-h...@logging.apache.org

[jira] [Commented] (LOG4J2-1010) Injectable context properties

[JIRA]

[ 
https://issues.apache.org/jira/browse/LOG4J2-1010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15346714#comment-15346714
 ] 

Mikael Ståldal commented on LOG4J2-1010:


Yes, this would work.

However, currently the Scala module only depends on log4j-api, not on 
log4j-core.

> Injectable context properties
> -
>
> Key: LOG4J2-1010
> URL: https://issues.apache.org/jira/browse/LOG4J2-1010
> Project: Log4j 2
>  Issue Type: Improvement
>  Components: API
>Affects Versions: 2.2
>Reporter: Mikael Ståldal
> Attachments: properties.patch
>
>
> It would be useful to have a way to inject context properties into a 
> {{LogEvent}}, as an alternative to {{ThreadContext}}.
> In an asynchronous environment, using ThreadContext as currently implemented 
> is not so useful since JVM threads might not be coupled to the logical flow 
> of the application.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: log4j-dev-unsubscr...@logging.apache.org
For additional commands, e-mail: log4j-dev-h...@logging.apache.org

[jira] [Comment Edited] (LOG4J2-1010) Injectable context properties

[Remko Popma (JIRA)]

[ 
https://issues.apache.org/jira/browse/LOG4J2-1010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15346691#comment-15346691
 ] 

Remko Popma edited comment on LOG4J2-1010 at 6/23/16 4:21 PM:
--

I think there is still some confusion on how to use this.

The {{ContextInjector}} interface is not intended to be used by the 
application. It is only used internally by Log4j.

I propose that the Scala module provides a utility class similar to the 
log4j-api ThreadContext class, that allows users to put key-value pairs in the 
Finacle Context (or in Locals). _This_ is the API that applications would use. 
I imagine this utility would be used similarly to the log4j-api ThreadContext.

The {{ScalaContextInjector}} implementation would copy the key-value pairs from 
the Locals into the LogEvent's CustomData every time something is logged.


was (Author: rem...@yahoo.com):
I think there is still some confusion on how to use this.

The {{ContextInjector}} interface is not intended to be used by the 
application. It is only used internally by Log4j.

I propose that the Scala module provides a utility class similar to the 
log4j-api ThreadContext class, that allows users to put key-value pairs in the 
Finacle Context (or in Locals). _This_ is the API that applications would use. 
I imagine this utility would be used similarly to the log4j-api ThreadContext.

The {{ScalaContextInjector}} implementation would take the key-value pairs out 
of the Locals and copy them into the LogEvent's CustomData every time something 
is logged.

> Injectable context properties
> -
>
> Key: LOG4J2-1010
> URL: https://issues.apache.org/jira/browse/LOG4J2-1010
> Project: Log4j 2
>  Issue Type: Improvement
>  Components: API
>Affects Versions: 2.2
>Reporter: Mikael Ståldal
> Attachments: properties.patch
>
>
> It would be useful to have a way to inject context properties into a 
> {{LogEvent}}, as an alternative to {{ThreadContext}}.
> In an asynchronous environment, using ThreadContext as currently implemented 
> is not so useful since JVM threads might not be coupled to the logical flow 
> of the application.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: log4j-dev-unsubscr...@logging.apache.org
For additional commands, e-mail: log4j-dev-h...@logging.apache.org

[jira] [Comment Edited] (LOG4J2-1010) Injectable context properties

[Remko Popma (JIRA)]

[ 
https://issues.apache.org/jira/browse/LOG4J2-1010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15346691#comment-15346691
 ] 

Remko Popma edited comment on LOG4J2-1010 at 6/23/16 4:20 PM:
--

I think there is still some confusion on how to use this.

The {{ContextInjector}} interface is not intended to be used by the 
application. It is only used internally by Log4j.

I propose that the Scala module provides a utility class similar to the 
log4j-api ThreadContext class, that allows users to put key-value pairs in the 
Finacle Context (or in Locals). _This_ is the API that applications would use. 
I imagine this utility would be used similarly to the log4j-api ThreadContext.

The {{ScalaContextInjector}} implementation would take the key-value pairs out 
of the Locals and copy them into the LogEvent's CustomData every time something 
is logged.


was (Author: rem...@yahoo.com):
I think there is still some confusion on how to use this.

This interface is not intended to be used by the application. It is used 
internally by Log4j.

I propose that in the Scala module, you provide a utility class similar to the 
log4j-api ThreadContext class, that allows users to put key-value pairs in the 
Finacle Context (or in Locals). _This_ is the API that applications would use.

The {{ScalaContextInjector}} implementation would take the key-value pairs out 
of the Locals and copy them into the LogEvent's CustomData every time something 
is logged.

> Injectable context properties
> -
>
> Key: LOG4J2-1010
> URL: https://issues.apache.org/jira/browse/LOG4J2-1010
> Project: Log4j 2
>  Issue Type: Improvement
>  Components: API
>Affects Versions: 2.2
>Reporter: Mikael Ståldal
> Attachments: properties.patch
>
>
> It would be useful to have a way to inject context properties into a 
> {{LogEvent}}, as an alternative to {{ThreadContext}}.
> In an asynchronous environment, using ThreadContext as currently implemented 
> is not so useful since JVM threads might not be coupled to the logical flow 
> of the application.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: log4j-dev-unsubscr...@logging.apache.org
For additional commands, e-mail: log4j-dev-h...@logging.apache.org

[jira] [Commented] (LOG4J2-1010) Injectable context properties

[Remko Popma (JIRA)]

[ 
https://issues.apache.org/jira/browse/LOG4J2-1010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15346691#comment-15346691
 ] 

Remko Popma commented on LOG4J2-1010:
-

I think there is still some confusion on how to use this.

This interface is not intended to be used by the application. It is used 
internally by Log4j.

I propose that in the Scala module, you provide a utility class similar to the 
log4j-api ThreadContext class, that allows users to put key-value pairs in the 
Finacle Context (or in Locals). _This_ is the API that applications would use.

The {{ScalaContextInjector}} implementation would take the key-value pairs out 
of the Locals and copy them into the LogEvent's CustomData every time something 
is logged.

> Injectable context properties
> -
>
> Key: LOG4J2-1010
> URL: https://issues.apache.org/jira/browse/LOG4J2-1010
> Project: Log4j 2
>  Issue Type: Improvement
>  Components: API
>Affects Versions: 2.2
>Reporter: Mikael Ståldal
> Attachments: properties.patch
>
>
> It would be useful to have a way to inject context properties into a 
> {{LogEvent}}, as an alternative to {{ThreadContext}}.
> In an asynchronous environment, using ThreadContext as currently implemented 
> is not so useful since JVM threads might not be coupled to the logical flow 
> of the application.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: log4j-dev-unsubscr...@logging.apache.org
For additional commands, e-mail: log4j-dev-h...@logging.apache.org

[jira] [Commented] (LOG4J2-1010) Injectable context properties

[Remko Popma (JIRA)]

[ 
https://issues.apache.org/jira/browse/LOG4J2-1010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15346681#comment-15346681
 ] 

Remko Popma commented on LOG4J2-1010:
-

Why? The whole point is to decouple LogEvent context data from the log4j-api 
ThreadContext. That is what you asked for...

> Injectable context properties
> -
>
> Key: LOG4J2-1010
> URL: https://issues.apache.org/jira/browse/LOG4J2-1010
> Project: Log4j 2
>  Issue Type: Improvement
>  Components: API
>Affects Versions: 2.2
>Reporter: Mikael Ståldal
> Attachments: properties.patch
>
>
> It would be useful to have a way to inject context properties into a 
> {{LogEvent}}, as an alternative to {{ThreadContext}}.
> In an asynchronous environment, using ThreadContext as currently implemented 
> is not so useful since JVM threads might not be coupled to the logical flow 
> of the application.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: log4j-dev-unsubscr...@logging.apache.org
For additional commands, e-mail: log4j-dev-h...@logging.apache.org

[jira] [Commented] (LOG4J2-1010) Injectable context properties

[JIRA]

[ 
https://issues.apache.org/jira/browse/LOG4J2-1010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15346677#comment-15346677
 ] 

Mikael Ståldal commented on LOG4J2-1010:


I think it's better to put this in log4j-api.

> Injectable context properties
> -
>
> Key: LOG4J2-1010
> URL: https://issues.apache.org/jira/browse/LOG4J2-1010
> Project: Log4j 2
>  Issue Type: Improvement
>  Components: API
>Affects Versions: 2.2
>Reporter: Mikael Ståldal
> Attachments: properties.patch
>
>
> It would be useful to have a way to inject context properties into a 
> {{LogEvent}}, as an alternative to {{ThreadContext}}.
> In an asynchronous environment, using ThreadContext as currently implemented 
> is not so useful since JVM threads might not be coupled to the logical flow 
> of the application.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: log4j-dev-unsubscr...@logging.apache.org
For additional commands, e-mail: log4j-dev-h...@logging.apache.org

[jira] [Comment Edited] (LOG4J2-1010) Injectable context properties

[Remko Popma (JIRA)]

[ 
https://issues.apache.org/jira/browse/LOG4J2-1010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15346670#comment-15346670
 ] 

Remko Popma edited comment on LOG4J2-1010 at 6/23/16 4:08 PM:
--

Ah! No, what I had in mind is to put this interface in the 
{{org.apache.logging.log4j.core.impl}} package, which is where the 
LogEventFactory is. The {{ContextInjector}} interface is intended to be used by 
objects that either create or initialize LogEvents, so this is a log4j-core 
concept.

It decouples the LogEvent context data from the log4j-api ThreadContext, which 
gives you the freedom to swap in context data from Scala Locals.


was (Author: rem...@yahoo.com):
Ah! No, what I had in mind is to put this interface in the 
{{org.apache.logging.log4j.core.impl}} package, which is where the 
LogEventFactory is.

> Injectable context properties
> -
>
> Key: LOG4J2-1010
> URL: https://issues.apache.org/jira/browse/LOG4J2-1010
> Project: Log4j 2
>  Issue Type: Improvement
>  Components: API
>Affects Versions: 2.2
>Reporter: Mikael Ståldal
> Attachments: properties.patch
>
>
> It would be useful to have a way to inject context properties into a 
> {{LogEvent}}, as an alternative to {{ThreadContext}}.
> In an asynchronous environment, using ThreadContext as currently implemented 
> is not so useful since JVM threads might not be coupled to the logical flow 
> of the application.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: log4j-dev-unsubscr...@logging.apache.org
For additional commands, e-mail: log4j-dev-h...@logging.apache.org

[jira] [Commented] (LOG4J2-1010) Injectable context properties

[Remko Popma (JIRA)]

[ 
https://issues.apache.org/jira/browse/LOG4J2-1010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15346670#comment-15346670
 ] 

Remko Popma commented on LOG4J2-1010:
-

Ah! No, what I had in mind is to put this interface in the 
{{org.apache.logging.log4j.core.impl}} package, which is where the 
LogEventFactory is.

> Injectable context properties
> -
>
> Key: LOG4J2-1010
> URL: https://issues.apache.org/jira/browse/LOG4J2-1010
> Project: Log4j 2
>  Issue Type: Improvement
>  Components: API
>Affects Versions: 2.2
>Reporter: Mikael Ståldal
> Attachments: properties.patch
>
>
> It would be useful to have a way to inject context properties into a 
> {{LogEvent}}, as an alternative to {{ThreadContext}}.
> In an asynchronous environment, using ThreadContext as currently implemented 
> is not so useful since JVM threads might not be coupled to the logical flow 
> of the application.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: log4j-dev-unsubscr...@logging.apache.org
For additional commands, e-mail: log4j-dev-h...@logging.apache.org

[jira] [Comment Edited] (LOG4J2-1010) Injectable context properties

[Remko Popma (JIRA)]

[ 
https://issues.apache.org/jira/browse/LOG4J2-1010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15346636#comment-15346636
 ] 

Remko Popma edited comment on LOG4J2-1010 at 6/23/16 4:02 PM:
--

Actually the {{List}} parameter is coming from the configuration (see 
[LoggerConfig.log()|https://github.com/apache/logging-log4j2/blob/master/log4j-core/src/main/java/org/apache/logging/log4j/core/config/LoggerConfig.java#L329].
 This list is merged with the ThreadContext map and the result is stored in the 
LogEvent. (So it is data that is consumed, not data to produce.)

Another reason I want to do it this way is to support primitive thread-local 
values in a garbage-free manner (see LOG4J2-1401 and LOG4J2-1349).

I don't see why this interface would be the wrong abstraction level. It makes 
initialization of the LogEvent's context data pluggable - I believe this is 
what you asked for.



was (Author: rem...@yahoo.com):
Actually the {{List}} parameter is coming from the configuration (see 
[LoggerConfig.log()|https://github.com/apache/logging-log4j2/blob/master/log4j-core/src/main/java/org/apache/logging/log4j/core/config/LoggerConfig.java#L329].
 This list is merged with the ThreadContext map and the result is stored in the 
LogEvent. (So it is data that is consumed, not data to produce.)

Another reason I want to do it this way is to support primitive thread-local 
values in a garbage-free manner (see LOG4J2-1401).

I don't see why this interface would be the wrong abstraction level.


> Injectable context properties
> -
>
> Key: LOG4J2-1010
> URL: https://issues.apache.org/jira/browse/LOG4J2-1010
> Project: Log4j 2
>  Issue Type: Improvement
>  Components: API
>Affects Versions: 2.2
>Reporter: Mikael Ståldal
> Attachments: properties.patch
>
>
> It would be useful to have a way to inject context properties into a 
> {{LogEvent}}, as an alternative to {{ThreadContext}}.
> In an asynchronous environment, using ThreadContext as currently implemented 
> is not so useful since JVM threads might not be coupled to the logical flow 
> of the application.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: log4j-dev-unsubscr...@logging.apache.org
For additional commands, e-mail: log4j-dev-h...@logging.apache.org

[jira] [Commented] (LOG4J2-1010) Injectable context properties

[JIRA]

[ 
https://issues.apache.org/jira/browse/LOG4J2-1010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15346655#comment-15346655
 ] 

Mikael Ståldal commented on LOG4J2-1010:


{{LogEvent}} is in log4j-core, and I guess we want to have this injector in 
log4j-api.

> Injectable context properties
> -
>
> Key: LOG4J2-1010
> URL: https://issues.apache.org/jira/browse/LOG4J2-1010
> Project: Log4j 2
>  Issue Type: Improvement
>  Components: API
>Affects Versions: 2.2
>Reporter: Mikael Ståldal
> Attachments: properties.patch
>
>
> It would be useful to have a way to inject context properties into a 
> {{LogEvent}}, as an alternative to {{ThreadContext}}.
> In an asynchronous environment, using ThreadContext as currently implemented 
> is not so useful since JVM threads might not be coupled to the logical flow 
> of the application.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: log4j-dev-unsubscr...@logging.apache.org
For additional commands, e-mail: log4j-dev-h...@logging.apache.org

[jira] [Commented] (LOG4J2-1010) Injectable context properties

[Matt Sicker (JIRA)]

[ 
https://issues.apache.org/jira/browse/LOG4J2-1010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15346651#comment-15346651
 ] 

Matt Sicker commented on LOG4J2-1010:
-

Here's an example of an asynchronous library and how they implement MDC in such 
an environment:

http://camel.apache.org/mdc-logging.html

> Injectable context properties
> -
>
> Key: LOG4J2-1010
> URL: https://issues.apache.org/jira/browse/LOG4J2-1010
> Project: Log4j 2
>  Issue Type: Improvement
>  Components: API
>Affects Versions: 2.2
>Reporter: Mikael Ståldal
> Attachments: properties.patch
>
>
> It would be useful to have a way to inject context properties into a 
> {{LogEvent}}, as an alternative to {{ThreadContext}}.
> In an asynchronous environment, using ThreadContext as currently implemented 
> is not so useful since JVM threads might not be coupled to the logical flow 
> of the application.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: log4j-dev-unsubscr...@logging.apache.org
For additional commands, e-mail: log4j-dev-h...@logging.apache.org

[jira] [Comment Edited] (LOG4J2-1010) Injectable context properties

[Remko Popma (JIRA)]

[ 
https://issues.apache.org/jira/browse/LOG4J2-1010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15346636#comment-15346636
 ] 

Remko Popma edited comment on LOG4J2-1010 at 6/23/16 3:47 PM:
--

Actually the {{List}} parameter is coming from the configuration (see 
[LoggerConfig.log()|https://github.com/apache/logging-log4j2/blob/master/log4j-core/src/main/java/org/apache/logging/log4j/core/config/LoggerConfig.java#L329].
 This list is merged with the ThreadContext map and the result is stored in the 
LogEvent. (So it is data that is consumed, not data to produce.)

Another reason I want to do it this way is to support primitive thread-local 
values in a garbage-free manner (see LOG4J2-1401).

I don't see why this interface would be the wrong abstraction level.



was (Author: rem...@yahoo.com):
Actually the {{List}} parameter is coming from the configuration (see 
[LoggerConfig.log()|https://github.com/apache/logging-log4j2/blob/master/log4j-core/src/main/java/org/apache/logging/log4j/core/config/LoggerConfig.java#L329].
 This list is merged with the ThreadContext map and the result is stored in the 
LogEvent.

Another reason I want to do it this way is to support primitive thread-local 
values in a garbage-free manner (see LOG4J2-1401).

I don't see why this interface would be the wrong abstraction level.


> Injectable context properties
> -
>
> Key: LOG4J2-1010
> URL: https://issues.apache.org/jira/browse/LOG4J2-1010
> Project: Log4j 2
>  Issue Type: Improvement
>  Components: API
>Affects Versions: 2.2
>Reporter: Mikael Ståldal
> Attachments: properties.patch
>
>
> It would be useful to have a way to inject context properties into a 
> {{LogEvent}}, as an alternative to {{ThreadContext}}.
> In an asynchronous environment, using ThreadContext as currently implemented 
> is not so useful since JVM threads might not be coupled to the logical flow 
> of the application.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: log4j-dev-unsubscr...@logging.apache.org
For additional commands, e-mail: log4j-dev-h...@logging.apache.org

[jira] [Commented] (LOG4J2-1010) Injectable context properties

[Remko Popma (JIRA)]

[ 
https://issues.apache.org/jira/browse/LOG4J2-1010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15346636#comment-15346636
 ] 

Remko Popma commented on LOG4J2-1010:
-

Actually the {{List}} parameter is coming from the configuration (see 
[LoggerConfig.log()|https://github.com/apache/logging-log4j2/blob/master/log4j-core/src/main/java/org/apache/logging/log4j/core/config/LoggerConfig.java#L329].
 This list is merged with the ThreadContext map and the result is stored in the 
LogEvent.

Another reason I want to do it this way is to support primitive thread-local 
values in a garbage-free manner (see LOG4J2-1401).

I don't see why this interface would be the wrong abstraction level.


> Injectable context properties
> -
>
> Key: LOG4J2-1010
> URL: https://issues.apache.org/jira/browse/LOG4J2-1010
> Project: Log4j 2
>  Issue Type: Improvement
>  Components: API
>Affects Versions: 2.2
>Reporter: Mikael Ståldal
> Attachments: properties.patch
>
>
> It would be useful to have a way to inject context properties into a 
> {{LogEvent}}, as an alternative to {{ThreadContext}}.
> In an asynchronous environment, using ThreadContext as currently implemented 
> is not so useful since JVM threads might not be coupled to the logical flow 
> of the application.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: log4j-dev-unsubscr...@logging.apache.org
For additional commands, e-mail: log4j-dev-h...@logging.apache.org

[jira] [Commented] (LOG4J2-1010) Injectable context properties

[JIRA]

[ 
https://issues.apache.org/jira/browse/LOG4J2-1010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15346594#comment-15346594
 ] 

Mikael Ståldal commented on LOG4J2-1010:


I think this is putting the abstraction level wrong. I am interested in 
providing the context properties, not putting them into the LogEvent.

I would prefer an interface like this:

{code}
/**
 * Provide context data for a LogEvent.
 * By default context data is obtained from the ThreadContext but implementers 
are free to do something different.
 *
 * @return  context data to add to the LogEvent
 */
interface ContextDataInjector {
List injectContextData();
}
{code}


> Injectable context properties
> -
>
> Key: LOG4J2-1010
> URL: https://issues.apache.org/jira/browse/LOG4J2-1010
> Project: Log4j 2
>  Issue Type: Improvement
>  Components: API
>Affects Versions: 2.2
>Reporter: Mikael Ståldal
> Attachments: properties.patch
>
>
> It would be useful to have a way to inject context properties into a 
> {{LogEvent}}, as an alternative to {{ThreadContext}}.
> In an asynchronous environment, using ThreadContext as currently implemented 
> is not so useful since JVM threads might not be coupled to the logical flow 
> of the application.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: log4j-dev-unsubscr...@logging.apache.org
For additional commands, e-mail: log4j-dev-h...@logging.apache.org

[jira] [Commented] (LOG4J2-1010) Injectable context properties

[Remko Popma (JIRA)]

[ 
https://issues.apache.org/jira/browse/LOG4J2-1010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15346564#comment-15346564
 ] 

Remko Popma commented on LOG4J2-1010:
-

In the Scala module, you would define a class, let's call it 
ScalaContextDataInjector, that implements ContextDataInjector.

In the implementation of the {{injectContextData(List props, LogEvent 
evt)}} method, this class would obtain key-value pairs from the Scala Context 
or Locals, and put those in the LogEvent's CustomData.


There would be some configuration option to tell {{ContextDataInjectorFactory}} 
to return {{ScalaContextDataInjector}} objects instead of the default 
{{DefaultContextDataInjector}} that Log4j would normally use.

> Injectable context properties
> -
>
> Key: LOG4J2-1010
> URL: https://issues.apache.org/jira/browse/LOG4J2-1010
> Project: Log4j 2
>  Issue Type: Improvement
>  Components: API
>Affects Versions: 2.2
>Reporter: Mikael Ståldal
> Attachments: properties.patch
>
>
> It would be useful to have a way to inject context properties into a 
> {{LogEvent}}, as an alternative to {{ThreadContext}}.
> In an asynchronous environment, using ThreadContext as currently implemented 
> is not so useful since JVM threads might not be coupled to the logical flow 
> of the application.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: log4j-dev-unsubscr...@logging.apache.org
For additional commands, e-mail: log4j-dev-h...@logging.apache.org

[jira] [Comment Edited] (LOG4J2-1010) Injectable context properties

[Remko Popma (JIRA)]

[ 
https://issues.apache.org/jira/browse/LOG4J2-1010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15346564#comment-15346564
 ] 

Remko Popma edited comment on LOG4J2-1010 at 6/23/16 3:04 PM:
--

In the Scala module, you would define a class, let's call it 
ScalaContextDataInjector, that implements ContextDataInjector.

In the implementation of the {{injectContextData(List props, LogEvent 
evt)}} method, this class would obtain key-value pairs from the Scala Context 
or Locals, and put those in the LogEvent's CustomData.


There would be some configuration option to tell {{ContextDataInjectorFactory}} 
to return {{ScalaContextDataInjector}} objects instead of the default 
{{DefaultContextDataInjector}} that Log4j would normally use.

In Scala applications, users would set key-value pairs in the Scala Context or 
Locals, similar to how in a Java application one would put key-value pairs in 
the ThreadContext. You may want to provide a utility class for this in the 
log4j-scala module.


was (Author: rem...@yahoo.com):
In the Scala module, you would define a class, let's call it 
ScalaContextDataInjector, that implements ContextDataInjector.

In the implementation of the {{injectContextData(List props, LogEvent 
evt)}} method, this class would obtain key-value pairs from the Scala Context 
or Locals, and put those in the LogEvent's CustomData.


There would be some configuration option to tell {{ContextDataInjectorFactory}} 
to return {{ScalaContextDataInjector}} objects instead of the default 
{{DefaultContextDataInjector}} that Log4j would normally use.

In your Scala application, you would set key-value pairs in the Scala Context 
or Locals, similar to how in a Java application one would put key-value pairs 
in the ThreadContext.

> Injectable context properties
> -
>
> Key: LOG4J2-1010
> URL: https://issues.apache.org/jira/browse/LOG4J2-1010
> Project: Log4j 2
>  Issue Type: Improvement
>  Components: API
>Affects Versions: 2.2
>Reporter: Mikael Ståldal
> Attachments: properties.patch
>
>
> It would be useful to have a way to inject context properties into a 
> {{LogEvent}}, as an alternative to {{ThreadContext}}.
> In an asynchronous environment, using ThreadContext as currently implemented 
> is not so useful since JVM threads might not be coupled to the logical flow 
> of the application.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: log4j-dev-unsubscr...@logging.apache.org
For additional commands, e-mail: log4j-dev-h...@logging.apache.org

[jira] [Comment Edited] (LOG4J2-1010) Injectable context properties

[Remko Popma (JIRA)]

[ 
https://issues.apache.org/jira/browse/LOG4J2-1010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15346564#comment-15346564
 ] 

Remko Popma edited comment on LOG4J2-1010 at 6/23/16 3:02 PM:
--

In the Scala module, you would define a class, let's call it 
ScalaContextDataInjector, that implements ContextDataInjector.

In the implementation of the {{injectContextData(List props, LogEvent 
evt)}} method, this class would obtain key-value pairs from the Scala Context 
or Locals, and put those in the LogEvent's CustomData.


There would be some configuration option to tell {{ContextDataInjectorFactory}} 
to return {{ScalaContextDataInjector}} objects instead of the default 
{{DefaultContextDataInjector}} that Log4j would normally use.

In your Scala application, you would set key-value pairs in the Scala Context 
or Locals, similar to how in a Java application one would put key-value pairs 
in the ThreadContext.


was (Author: rem...@yahoo.com):
In the Scala module, you would define a class, let's call it 
ScalaContextDataInjector, that implements ContextDataInjector.

In the implementation of the {{injectContextData(List props, LogEvent 
evt)}} method, this class would obtain key-value pairs from the Scala Context 
or Locals, and put those in the LogEvent's CustomData.


There would be some configuration option to tell {{ContextDataInjectorFactory}} 
to return {{ScalaContextDataInjector}} objects instead of the default 
{{DefaultContextDataInjector}} that Log4j would normally use.

> Injectable context properties
> -
>
> Key: LOG4J2-1010
> URL: https://issues.apache.org/jira/browse/LOG4J2-1010
> Project: Log4j 2
>  Issue Type: Improvement
>  Components: API
>Affects Versions: 2.2
>Reporter: Mikael Ståldal
> Attachments: properties.patch
>
>
> It would be useful to have a way to inject context properties into a 
> {{LogEvent}}, as an alternative to {{ThreadContext}}.
> In an asynchronous environment, using ThreadContext as currently implemented 
> is not so useful since JVM threads might not be coupled to the logical flow 
> of the application.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: log4j-dev-unsubscr...@logging.apache.org
For additional commands, e-mail: log4j-dev-h...@logging.apache.org

[jira] [Commented] (LOG4J2-1010) Injectable context properties

[JIRA]

[ 
https://issues.apache.org/jira/browse/LOG4J2-1010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15346517#comment-15346517
 ] 

Mikael Ståldal commented on LOG4J2-1010:


Can you give an example of how I am supposed to use this?

> Injectable context properties
> -
>
> Key: LOG4J2-1010
> URL: https://issues.apache.org/jira/browse/LOG4J2-1010
> Project: Log4j 2
>  Issue Type: Improvement
>  Components: API
>Affects Versions: 2.2
>Reporter: Mikael Ståldal
> Attachments: properties.patch
>
>
> It would be useful to have a way to inject context properties into a 
> {{LogEvent}}, as an alternative to {{ThreadContext}}.
> In an asynchronous environment, using ThreadContext as currently implemented 
> is not so useful since JVM threads might not be coupled to the logical flow 
> of the application.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: log4j-dev-unsubscr...@logging.apache.org
For additional commands, e-mail: log4j-dev-h...@logging.apache.org

[jira] [Commented] (LOG4J2-1010) Injectable context properties

[Remko Popma (JIRA)]

[ 
https://issues.apache.org/jira/browse/LOG4J2-1010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15346419#comment-15346419
 ] 

Remko Popma commented on LOG4J2-1010:
-

The best place to inject context map data into the LogEvent is probably the 
place where the LogEvent is created or (in the case of reused mutable log 
events) initialized. 

Side note: Doing this will allow users to inject any kind of data into the 
LogEvent (not just Strings), and this data can come from anywhere (not 
necessarily from ThreadContext). See also LOG4J2-1349, which proposes to make 
the LogEvent internal data structure more general to support this. The below 
assumes that LogEvents will use the {{CustomData}} data structure 
proposed in LOG4J2-1349 instead of the current {{Map}}.

One idea is to introduce an interface like this:

{code}
/**
 * Updates the specified LogEvent's CustomData with context values.
 * By default context data is obtained from the ThreadContext but implementers 
are free to do something different.
 *
 * @param properties Properties from the configuration to be added to the log 
event
 * @event the log event whose CustomData to initialize
 */
interface ContextDataInjector {
void injectContextData(List properties, LogEvent event);
}

// Default implementation that copies data from ThreadContext.
// The Scala version of this will likely use Local to get Context key-value 
pairs.
class DefaultContextDataInjector implements ContextDataInjector {
public void injectContextData(List properties, LogEvent event) {
Map map = Log4jLogEvent.createMap(properties);
event.getCustomData().clear();
for (String key : map.keySet()) {
event.getCustomData().put(key, map.get(key));
}
}
}

// example usage in DefaultLogEventFactory
public class DefaultLogEventFactory implements LogEventFactory {
private ContextDataInjector injector = ContextDataInjectorFactory.create();

public LogEvent createEvent(final String loggerName, final Marker marker,
final String fqcn, final Level level, final 
Message data,
final List properties, final 
Throwable t) {
LogEvent result = new Log4jLogEvent(loggerName, marker, fqcn, level, 
data, t);
injector.injectContextData(properties, result);
return result;
}
}
{code}

> Injectable context properties
> -
>
> Key: LOG4J2-1010
> URL: https://issues.apache.org/jira/browse/LOG4J2-1010
> Project: Log4j 2
>  Issue Type: Improvement
>  Components: API
>Affects Versions: 2.2
>Reporter: Mikael Ståldal
> Attachments: properties.patch
>
>
> It would be useful to have a way to inject context properties into a 
> {{LogEvent}}, as an alternative to {{ThreadContext}}.
> In an asynchronous environment, using ThreadContext as currently implemented 
> is not so useful since JVM threads might not be coupled to the logical flow 
> of the application.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: log4j-dev-unsubscr...@logging.apache.org
For additional commands, e-mail: log4j-dev-h...@logging.apache.org

[jira] [Commented] (LOG4J2-1010) Injectable context properties

[JIRA]

[ 
https://issues.apache.org/jira/browse/LOG4J2-1010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15346359#comment-15346359
 ] 

Mikael Ståldal commented on LOG4J2-1010:


OK, I have changed the title and description of the issue.

My idea is not that the application should explicitly pass in the context in 
each logging call, I assume you have some kind of wrapper around the Log4j API. 
But that will not work with third-party code, as you correctly point out.

Other ideas of injection mechanisms are welcome.


> Injectable context properties
> -
>
> Key: LOG4J2-1010
> URL: https://issues.apache.org/jira/browse/LOG4J2-1010
> Project: Log4j 2
>  Issue Type: Improvement
>  Components: API
>Affects Versions: 2.2
>Reporter: Mikael Ståldal
> Attachments: properties.patch
>
>
> It would be useful to have a way to inject context properties into a 
> {{LogEvent}}, as an alternative to {{ThreadContext}}.
> In an asynchronous environment, using ThreadContext as currently implemented 
> is not so useful since JVM threads might not be coupled to the logical flow 
> of the application.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: log4j-dev-unsubscr...@logging.apache.org
For additional commands, e-mail: log4j-dev-h...@logging.apache.org

[jira] [Updated] (LOG4J2-1010) Injectable context properties

[JIRA]

 [ 
https://issues.apache.org/jira/browse/LOG4J2-1010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mikael Ståldal updated LOG4J2-1010:
---
Description: 
It would be useful to have a way to inject context properties into a 
{{LogEvent}}, as an alternative to {{ThreadContext}}.

In an asynchronous environment, using ThreadContext as currently implemented is 
not so useful since JVM threads might not be coupled to the logical flow of the 
application.

  was:
It would be useful to have some logging methods in the Logger interface to set 
ThreadContext values for a single log message only.

In an asynchronous environment, using ThreadContext as currently defined is not 
so useful since JVM threads might not be coupled to the logical flow of the 
application.

Summary: Injectable context properties  (was: Possibility to set 
ThreadContext values in calls to Logger method)

> Injectable context properties
> -
>
> Key: LOG4J2-1010
> URL: https://issues.apache.org/jira/browse/LOG4J2-1010
> Project: Log4j 2
>  Issue Type: Improvement
>  Components: API
>Affects Versions: 2.2
>Reporter: Mikael Ståldal
> Attachments: properties.patch
>
>
> It would be useful to have a way to inject context properties into a 
> {{LogEvent}}, as an alternative to {{ThreadContext}}.
> In an asynchronous environment, using ThreadContext as currently implemented 
> is not so useful since JVM threads might not be coupled to the logical flow 
> of the application.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: log4j-dev-unsubscr...@logging.apache.org
For additional commands, e-mail: log4j-dev-h...@logging.apache.org

[jira] [Commented] (LOG4J2-1430) Add optional support for Conversant DisruptorBlockingQueue in AsyncAppender

[Anthony Maire (JIRA)]

[ 
https://issues.apache.org/jira/browse/LOG4J2-1430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15346328#comment-15346328
 ] 

Anthony Maire commented on LOG4J2-1430:
---

I had a look on the ResponseTimeTest, it's exactly what I was thinking about 
for a good latency benchmark. I will try to find some time to play with it


> Add optional support for Conversant DisruptorBlockingQueue in AsyncAppender
> ---
>
> Key: LOG4J2-1430
> URL: https://issues.apache.org/jira/browse/LOG4J2-1430
> Project: Log4j 2
>  Issue Type: New Feature
>  Components: Appenders
>Affects Versions: 2.6.1
>Reporter: Matt Sicker
>Assignee: Matt Sicker
> Fix For: 2.7
>
> Attachments: AsyncAppenderPerf01.txt, 
> log4j2-1430-jctools-tmp-patch.txt
>
>
> [Conversant Disruptor|https://github.com/conversant/disruptor] works as an 
> implementation of BlockingQueue that is much faster than ArrayBlockingQueue. 
> I did some benchmarks earlier and found it to be a bit faster:
> h3. AsyncAppender/ArrayBlockingQueue
> {code}
> Benchmark   Mode  Samples 
>Score   Error  Units
> o.a.l.l.p.j.AsyncAppenderLog4j2Benchmark.throughput10Paramsthrpt   20 
>  1101267.173 ± 17583.204  ops/s
> o.a.l.l.p.j.AsyncAppenderLog4j2Benchmark.throughput11Paramsthrpt   20 
>  1128269.255 ± 12188.910  ops/s
> o.a.l.l.p.j.AsyncAppenderLog4j2Benchmark.throughput1Param  thrpt   20 
>  1525470.805 ± 56515.933  ops/s
> o.a.l.l.p.j.AsyncAppenderLog4j2Benchmark.throughput2Params thrpt   20 
>  1789434.196 ± 42733.475  ops/s
> o.a.l.l.p.j.AsyncAppenderLog4j2Benchmark.throughput3Params thrpt   20 
>  1803276.278 ± 34938.176  ops/s
> o.a.l.l.p.j.AsyncAppenderLog4j2Benchmark.throughput4Params thrpt   20 
>  1468550.776 ± 26402.286  ops/s
> o.a.l.l.p.j.AsyncAppenderLog4j2Benchmark.throughput5Params thrpt   20 
>  1322304.349 ± 22417.997  ops/s
> o.a.l.l.p.j.AsyncAppenderLog4j2Benchmark.throughput6Params thrpt   20 
>  1179756.489 ± 16502.276  ops/s
> o.a.l.l.p.j.AsyncAppenderLog4j2Benchmark.throughput7Params thrpt   20 
>  1324660.677 ± 18893.944  ops/s
> o.a.l.l.p.j.AsyncAppenderLog4j2Benchmark.throughput8Params thrpt   20 
>  1309365.962 ± 19602.489  ops/s
> o.a.l.l.p.j.AsyncAppenderLog4j2Benchmark.throughput9Params thrpt   20 
>  1422144.180 ± 20815.042  ops/s
> o.a.l.l.p.j.AsyncAppenderLog4j2Benchmark.throughputSimple  thrpt   20 
>  1247862.372 ± 18300.764  ops/s
> {code}
> h3. AsyncAppender/DisruptorBlockingQueue
> {code}
> Benchmark   Mode  Samples 
>ScoreError  Units
> o.a.l.l.p.j.AsyncAppenderLog4j2Benchmark.throughput10Paramsthrpt   20 
>  3704735.586 ±  59766.253  ops/s
> o.a.l.l.p.j.AsyncAppenderLog4j2Benchmark.throughput11Paramsthrpt   20 
>  3622175.410 ±  31975.353  ops/s
> o.a.l.l.p.j.AsyncAppenderLog4j2Benchmark.throughput1Param  thrpt   20 
>  6862480.428 ± 121473.276  ops/s
> o.a.l.l.p.j.AsyncAppenderLog4j2Benchmark.throughput2Params thrpt   20 
>  6193288.988 ±  93545.144  ops/s
> o.a.l.l.p.j.AsyncAppenderLog4j2Benchmark.throughput3Params thrpt   20 
>  5715621.712 ± 131878.581  ops/s
> o.a.l.l.p.j.AsyncAppenderLog4j2Benchmark.throughput4Params thrpt   20 
>  5745187.005 ± 213854.016  ops/s
> o.a.l.l.p.j.AsyncAppenderLog4j2Benchmark.throughput5Params thrpt   20 
>  5307137.396 ±  88135.709  ops/s
> o.a.l.l.p.j.AsyncAppenderLog4j2Benchmark.throughput6Params thrpt   20 
>  4953015.419 ±  72100.403  ops/s
> o.a.l.l.p.j.AsyncAppenderLog4j2Benchmark.throughput7Params thrpt   20 
>  4833836.418 ±  52919.314  ops/s
> o.a.l.l.p.j.AsyncAppenderLog4j2Benchmark.throughput8Params thrpt   20 
>  4353791.507 ±  79047.812  ops/s
> o.a.l.l.p.j.AsyncAppenderLog4j2Benchmark.throughput9Params thrpt   20 
>  4136761.624 ±  67804.253  ops/s
> o.a.l.l.p.j.AsyncAppenderLog4j2Benchmark.throughputSimple  thrpt   20 
>  6719456.722 ± 187433.301  ops/s
> {code}
> h3. AsyncLogger
> {code}
> BenchmarkMode  Samples
>  ScoreError  Units
> o.a.l.l.p.j.AsyncLoggersBenchmark.throughput10Paramsthrpt   20   
> 5075883.371 ± 180465.316  ops/s
> o.a.l.l.p.j.AsyncLoggersBenchmark.throughput11Paramsthrpt   20   
> 4867362.030 ± 193909.465  ops/s
> o.a.l.l.p.j.AsyncLoggersBenchmark.throughput1Param  thrpt   20  
> 10294733.024 ± 226536.965  ops/s
> o.a.l.l.p.j.AsyncLoggersBenchmark.throughput2Params thrpt   20   
> 9021650.667 ± 351102.255  ops/s
> o.a.l.l.p.j.AsyncLoggersBenchmark.throughput3Params thrpt   20   
> 8079337.905 ± 

[jira] [Updated] (LOG4J2-1401) Support changing the log level for all messages related to some domain object

[Remko Popma (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LOG4J2-1401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Remko Popma updated LOG4J2-1401:

Description: 
Use case:

We have a low-garbage trading system. When a production incident occurs we want 
to temporarily increase visibility into what the system is doing by increasing 
the log level. However, we want to do this selectively: we want the ability to 
turn on detailed logging only for certain orders, for certain instruments, for 
a certain user, or for some other attributes.

The system processes many messages, say in the order of hundreds of thousands 
of messages per second. When a message is processed it passes through many 
components of the system, and when each of these components do logging, the 
message context information like order ID, instrument etc. should be available 
so the log event can be filtered in the normal case and enabled in case of a 
production incident.

We envision this "verbose logging" filtering to be switched on and off manually 
by an operations team. In addition to manual operation, it would be nice if the 
original verbosity can be restored automatically when some condition no longer 
holds. For example, log at some verbose level 
* for some period of time (5 minutes)
* for some fixed number of events (10,000 log messages)
* any other user-specified condition 

There are two problems to solve:
# efficient and garbage-free log event "tagging"
# dynamic and garbage-free filtering

*Problem 1: Log Event "Tagging"*

Considerations:
* Because of the volume of events we want the mechanism for tagging log events 
to be garbage-free. 
* Our order IDs and instrument IDs etc are modeled as primitives ({{long}} and 
{{int}} values).

Current options in Log4j for "tagging" a log event:
* *ThreadContext map*. (This _almost_ does what we need.) Data in this map is 
copied into each Log Event, and downstream components can query the data with 
the {{LogEvent.getContextMap() : Map}} method. The drawback of 
the ThreadContext API is that values must be Strings. We would like to be able 
to store {{long}} primitives in the ThreadContext map. We can also take this 
opportunity to provide a garbage-free version of the ThreadContext 
(LOG4J2-1349) and the LogEvent properties map.

* *Direct user access to the LogEvent properties map*. LOG4J2-1010 suggests 
changing the Logger API to accomplish this, but there may be alternatives.
* Update: the idea raised in LOG4J2-1010 to make initialization of the 
LogEvent's {{contextMap}} data pluggable is relevant for this ticket also: It 
means that context data can come from anywhere, not necessarily only from Log4j 
API classes like ThreadContext. Applications can have their own custom 
ThreadContext-like class with data that is copied into every log event, without 
requiring changes to the Log4j API.

* *Markers*: (Not a good option, mentioned here for completeness.) 
String-based, name attribute only (no key-value), hierarchical (may have 
parents). Current implementation caches all Markers forever, with an option to 
clear the whole cache. Unclear how to use this mechanism to tag a log event 
with key-value pairs like order ID, user ID and/or product ID.
* Update: one way to work around this is to create a KeyValueMarker 
implementation of the Marker interface. A single instance could contain 
multiple tags (key-value pairs of arbitrary type).  Instances cannot be cached 
by name in the MarkerManager, but instead I imagine we can use a pool of 
KeyValueMarkers to reuse (so we would need to call release() or something when 
the log event is fully processed). This feels hacky though...

*Problem 2: Dynamic and garbage-free filtering*

It looks like installing a global filter on the Configuration is a good way to 
accomplish this. It is straightforward to do this with existing Log4j APIs:

{code}
LoggerContext context = (org.apache.logging.log4j.core.LoggerContext) 
LogManager.getContext(false);
context.addFilter(ourFilter);
// when done, call context.removeFilter(ourFilter)
{code}

Installing and removing a filter is rare and it is okay to create temporary 
objects when doing this. Once the filter is installed, it should not create 
temporary objects during steady state processing.

For the filter implementation, something like 
[ThreadContextMapFilter|https://logging.apache.org/log4j/2.x/manual/filters.html#ThreadContextMapFilter]
  is very close to what we need.

To do: 
* {{ThreadContextMapFilter}} is not garbage-free: it creates a new iterator 
every time the {{filter()}} method is called.
* Auto-removal of the Filter after some time has passed or some number of 
events have been processed. It may be a useful addition to Log4j to provide a 
{{CompositeFilter}} that automatically removes itself from a {{Configuration}} 
when some condition is satisfied.


  was:
During system operations, it is commonly desirable to 

[jira] [Commented] (LOG4J2-1349) Garbage-free ThreadContext map and stack

[Remko Popma (JIRA)]

[ 
https://issues.apache.org/jira/browse/LOG4J2-1349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15346312#comment-15346312
 ] 

Remko Popma commented on LOG4J2-1349:
-

The current {{ThreadContext}} map implementation is a copy-on-write data 
structure. This is efficient because the number of _reads_ vastly outnumbers 
the number of _writes_. Every time a message is logged, the current 
ThreadContext map is passed to the LogEvent as is. This is safe because this 
map is immutable: it is replaced, not modified, when the user calls 
ThreadContext.put(key, somevalue).

The drawback is that a copy-on-write data structure is not garbage-free. 

A garbage-free ThreadContext map would have to be a mutable data structure. To 
ensure that LogEvents can safely be passed off to other threads, the LogEvent 
cannot have a direct reference to the ThreadContext map. Instead, the data from 
the ThreadContext map would need to be copied into the LogEvent's context map.

For this to be garbage-free, LogEvent needs a different data structure to carry 
context map data. Currently LogEvent implementations use a 
{{java.util.Map}} for this. The JDK Map is not an easy data 
structure to make garbage-free. It would also be nice to have the ability in 
LogEvent to carry data of any type.

One idea is to introduce a small interface that is general enough to be 
map-like but can be implemented in a garbage-free manner. 

LogEvent implementations would have an instance of this interface instead of 
the current {{java.util.Map contextMap}} attribute.

Something like this:
{code}
interface CustomData {
/** Called to implement {@link LogEvent#getContextMap()}. */
Map asMap();

/** Put key-value pair into the table.
Remove key if value is null. */
void put(K key, V value);

/** Returns the value for the specified key. */
V getValue(K key);

/** Number of key-value pairs. */
int size();

/** Removes all key-value pairs. */
void clear();

// Instead of Iterators, use an index-based approach.

/** Returns the index of the specified key. */
int indexOfKey(K key);

/** Returns the i-th key. */
K getKey(int i);

/** Returns the value for the i-th key. */
V getValueAt(int i);
}
{code}

The LogEvent interface would have an additional method {{getCustomData() : 
CustomData}} that gives downstream components direct access to the new 
data structure. Existing downstream components would still be able to call 
{{logEvent.getContextMap()}} to get a {{Map}} view of the 
context map data and this would work as expected.

> Garbage-free ThreadContext map and stack
> 
>
> Key: LOG4J2-1349
> URL: https://issues.apache.org/jira/browse/LOG4J2-1349
> Project: Log4j 2
>  Issue Type: Improvement
>  Components: API
>Affects Versions: 2.5
>Reporter: Remko Popma
> Fix For: 2.7
>
>
> The current ThreadContext map and stack implementations allocate temporary 
> objects. This ticket is to investigate and track the work for alternative 
> implementations that are garbage-free.
> Both DefaultThreadContextMap and DefaultThreadContextStack are copy-on-write 
> data structures: each modification replaces the ThreadLocal object with a 
> modified copy. The advantage of this approach is that there is no need to 
> make a copy for each LogEvent.
> Also, DefaultThreadContextMap uses a JDK map, the JDK collections tend to 
> allocate a lot of temporary objects.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: log4j-dev-unsubscr...@logging.apache.org
For additional commands, e-mail: log4j-dev-h...@logging.apache.org

[jira] [Commented] (LOG4J2-1010) Possibility to set ThreadContext values in calls to Logger method

[Greg Thomas (JIRA)]

[ 
https://issues.apache.org/jira/browse/LOG4J2-1010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15346283#comment-15346283
 ] 

Greg Thomas commented on LOG4J2-1010:
-

> void logMessage(String fqcn, Level level, Marker marker, Message message, 
> Throwable t, Map contextMap);

My issue with this solution is the client code must explicitly pass in the 
context. 

For my particular use case, we call out to the Apache HttpClient library. This 
logs using Apache Commons Logging, which I can re-route through to log4j2 using 
the log4j-jcl adapter. This means I can set the ThreadContext up front, call 
the libraries I wish, and those libraries will happily log with the right 
ThreadContext even if they know nothing about it. And, in the case of Apache 
HttpClient, even if their underlying logging framework has no concept of a 
ThreadContext/MDC/NDC.

I appreciate that this would solve the problem as phrased by this issue ("set 
ThreadContext values in calls to Logger method"), but I think the issue as 
currently phrased is highlighting a single solution rather than identifying the 
underlying problem.

> Possibility to set ThreadContext values in calls to Logger method
> -
>
> Key: LOG4J2-1010
> URL: https://issues.apache.org/jira/browse/LOG4J2-1010
> Project: Log4j 2
>  Issue Type: Improvement
>  Components: API
>Affects Versions: 2.2
>Reporter: Mikael Ståldal
> Attachments: properties.patch
>
>
> It would be useful to have some logging methods in the Logger interface to 
> set ThreadContext values for a single log message only.
> In an asynchronous environment, using ThreadContext as currently defined is 
> not so useful since JVM threads might not be coupled to the logical flow of 
> the application.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: log4j-dev-unsubscr...@logging.apache.org
For additional commands, e-mail: log4j-dev-h...@logging.apache.org

[jira] [Commented] (LOG4J2-1010) Possibility to set ThreadContext values in calls to Logger method

[JIRA]

[ 
https://issues.apache.org/jira/browse/LOG4J2-1010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15346054#comment-15346054
 ] 

Mikael Ståldal commented on LOG4J2-1010:


That might work. However, Twitter Finagle have already implemented all this for 
you with {{Local}} and {{Context}}, without requiring the application to 
explicitly pass the state around like in your example. I would like to leverage 
that.

And the structure of the code is not like in your example, it is a chain of 
futures (Twitter or Scala Future, or 
[CompletableFuture|http://docs.oracle.com/javase/8/docs/api/java/util/concurrent/CompletableFuture.html]
 in Java 8).

> Possibility to set ThreadContext values in calls to Logger method
> -
>
> Key: LOG4J2-1010
> URL: https://issues.apache.org/jira/browse/LOG4J2-1010
> Project: Log4j 2
>  Issue Type: Improvement
>  Components: API
>Affects Versions: 2.2
>Reporter: Mikael Ståldal
> Attachments: properties.patch
>
>
> It would be useful to have some logging methods in the Logger interface to 
> set ThreadContext values for a single log message only.
> In an asynchronous environment, using ThreadContext as currently defined is 
> not so useful since JVM threads might not be coupled to the logical flow of 
> the application.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: log4j-dev-unsubscr...@logging.apache.org
For additional commands, e-mail: log4j-dev-h...@logging.apache.org

[jira] [Comment Edited] (LOG4J2-1010) Possibility to set ThreadContext values in calls to Logger method

[JIRA]

[ 
https://issues.apache.org/jira/browse/LOG4J2-1010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15346054#comment-15346054
 ] 

Mikael Ståldal edited comment on LOG4J2-1010 at 6/23/16 8:35 AM:
-

That might work. However, Twitter Finagle have already implemented all this for 
you with {{Local}} and {{Context}}, without requiring the application to 
explicitly pass the state around like in your example. I would like to leverage 
that.

And the structure of the code is not like in your example, it is a chain of 
futures (Twitter or Scala Future, or 
[CompletableFuture|http://docs.oracle.com/javase/8/docs/api/java/util/concurrent/CompletableFuture.html]
 in Java 8). The application does not manage Threads explicitly.


was (Author: mikaelstaldal):
That might work. However, Twitter Finagle have already implemented all this for 
you with {{Local}} and {{Context}}, without requiring the application to 
explicitly pass the state around like in your example. I would like to leverage 
that.

And the structure of the code is not like in your example, it is a chain of 
futures (Twitter or Scala Future, or 
[CompletableFuture|http://docs.oracle.com/javase/8/docs/api/java/util/concurrent/CompletableFuture.html]
 in Java 8).

> Possibility to set ThreadContext values in calls to Logger method
> -
>
> Key: LOG4J2-1010
> URL: https://issues.apache.org/jira/browse/LOG4J2-1010
> Project: Log4j 2
>  Issue Type: Improvement
>  Components: API
>Affects Versions: 2.2
>Reporter: Mikael Ståldal
> Attachments: properties.patch
>
>
> It would be useful to have some logging methods in the Logger interface to 
> set ThreadContext values for a single log message only.
> In an asynchronous environment, using ThreadContext as currently defined is 
> not so useful since JVM threads might not be coupled to the logical flow of 
> the application.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: log4j-dev-unsubscr...@logging.apache.org
For additional commands, e-mail: log4j-dev-h...@logging.apache.org

[jira] [Commented] (LOG4J2-1010) Possibility to set ThreadContext values in calls to Logger method

[JIRA]

[ 
https://issues.apache.org/jira/browse/LOG4J2-1010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15346041#comment-15346041
 ] 

Mikael Ståldal commented on LOG4J2-1010:


I am actually using 
[Local|https://twitter.github.io/util/docs/#com.twitter.util.Local], which is 
the low level mechanism (similar to Java's ThreadLocal) used by Finagle to 
implement Contexts. I currently have a custom Scala API wrapper which does this 
in a clumsy way.

But this only works for Finagle. With other asynchronous frameworks (such as 
Play Framework) you would have to do it differently.

A pluggable way to inject context properties to the LogEvent is exactly what I 
want.

My intention with adding arguments to logging calls was not that the 
application would used it directly, it was that some API wrapper (such as the 
Scala API) can use it for injection. But if we can come up with some other way 
of doing the injection, then that's fine.

However, I am not satisfied with the two other proposals that came up earlier 
in this discussion. Custom messages is not good since I want to leverage 
existing Layouts which already uses context properties. Configuration 
properties is not good since I don't what to have to specify all possible 
context property keys in my configuration file.

My idea of injection mechanism is to add a parameter to 
{{ExtendedLogger.logMessage}}:

{code}
 /**
 * Always logs a message at the specified level. It is the responsibility 
of the caller to ensure the specified
 * level is enabled.
 * 
 * @param fqcn The fully qualified class name of the logger entry point, 
used to determine the caller class and
 *method when location information needs to be logged.
 * @param level The logging Level to check.
 * @param marker A Marker or null.
 * @param message The Message.
 * @param t the exception to log, including its stack trace.
 * @param contextMap contextMap to include in the log event, or null.
 */
void logMessage(String fqcn, Level level, Marker marker, Message message, 
Throwable t, Map contextMap);
{code}

I am open for other ideas.



> Possibility to set ThreadContext values in calls to Logger method
> -
>
> Key: LOG4J2-1010
> URL: https://issues.apache.org/jira/browse/LOG4J2-1010
> Project: Log4j 2
>  Issue Type: Improvement
>  Components: API
>Affects Versions: 2.2
>Reporter: Mikael Ståldal
> Attachments: properties.patch
>
>
> It would be useful to have some logging methods in the Logger interface to 
> set ThreadContext values for a single log message only.
> In an asynchronous environment, using ThreadContext as currently defined is 
> not so useful since JVM threads might not be coupled to the logical flow of 
> the application.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: log4j-dev-unsubscr...@logging.apache.org
For additional commands, e-mail: log4j-dev-h...@logging.apache.org

[jira] [Commented] (LOG4J2-1010) Possibility to set ThreadContext values in calls to Logger method

[Greg Thomas (JIRA)]

[ 
https://issues.apache.org/jira/browse/LOG4J2-1010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15346009#comment-15346009
 ] 

Greg Thomas commented on LOG4J2-1010:
-

On the assumption that (generally) a thread that is processing a request, or 
even part of a request, may wish to log more than once, how about enhancing 
ThreadContext with a state getter and setter; the state is essentially a 
combination of {{getImmutableStack()}} and {{getImmutableContext()}}; you could 
retrieve the current state of the ThreadContext at any point in time, and then 
use that current ThreadContext to that state at any other point to set the 
state to that point; giving something like ...

{code}
final ThreadContext.State state = ThreadContext.getState()
new Thread(new Runnable() {
public void run() {
ThreadContext.setState(state);
logger.debug("Starting processing");
...
logger.debug("Ending processing");
ThreadContext.clear();
}
}).start();
{code}

(OK, so that example could more easily be implemented using 
{{isThreadContextMapInheritable}}, but the above could be extended to use 
thread pools, unlike {{isThreadContextMapInheritable}}.

> Possibility to set ThreadContext values in calls to Logger method
> -
>
> Key: LOG4J2-1010
> URL: https://issues.apache.org/jira/browse/LOG4J2-1010
> Project: Log4j 2
>  Issue Type: Improvement
>  Components: API
>Affects Versions: 2.2
>Reporter: Mikael Ståldal
> Attachments: properties.patch
>
>
> It would be useful to have some logging methods in the Logger interface to 
> set ThreadContext values for a single log message only.
> In an asynchronous environment, using ThreadContext as currently defined is 
> not so useful since JVM threads might not be coupled to the logical flow of 
> the application.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: log4j-dev-unsubscr...@logging.apache.org
For additional commands, e-mail: log4j-dev-h...@logging.apache.org