[jira] [Commented] (LOG4J2-1397) Support ByteBufferLayout
[ https://issues.apache.org/jira/browse/LOG4J2-1397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15338354#comment-15338354 ] Remko Popma commented on LOG4J2-1397: - For this to be garbage-free, LOG4J2-1438 is required. > Support ByteBufferLayout > > > Key: LOG4J2-1397 > URL: https://issues.apache.org/jira/browse/LOG4J2-1397 > Project: Log4j 2 > Issue Type: New Feature > Components: Layouts >Affects Versions: 2.6 >Reporter: Remko Popma > > As requested by Kirk Pepperdine on the Mechanical Sympathy [mailing > list|https://groups.google.com/d/msg/mechanical-sympathy/klefjRqlpQE/-1WF59IGAwAJ]: > Support a new Layout that takes ObjectMessages containing a ByteBuffer > payload and write them to the appender without modification. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: log4j-dev-unsubscr...@logging.apache.org For additional commands, e-mail: log4j-dev-h...@logging.apache.org
[jira] [Commented] (LOG4J2-1397) Support ByteBufferLayout
[ https://issues.apache.org/jira/browse/LOG4J2-1397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15326130#comment-15326130 ] Remko Popma commented on LOG4J2-1397: - I created LOG4J2-1424 for a more general garbage-free async message transfer mechanism. > Support ByteBufferLayout > > > Key: LOG4J2-1397 > URL: https://issues.apache.org/jira/browse/LOG4J2-1397 > Project: Log4j 2 > Issue Type: New Feature > Components: Layouts >Affects Versions: 2.6 >Reporter: Remko Popma > > As requested by Kirk Pepperdine on the Mechanical Sympathy [mailing > list|https://groups.google.com/d/msg/mechanical-sympathy/klefjRqlpQE/-1WF59IGAwAJ]: > Support a new Layout that takes ObjectMessages containing a ByteBuffer > payload and write them to the appender without modification. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: log4j-dev-unsubscr...@logging.apache.org For additional commands, e-mail: log4j-dev-h...@logging.apache.org
[jira] [Commented] (LOG4J2-1397) Support ByteBufferLayout
[ https://issues.apache.org/jira/browse/LOG4J2-1397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15323632#comment-15323632 ] Remko Popma commented on LOG4J2-1397: - I have been thinking about doing something like this in the context of other feature requests. I still haven't figured out how I can make this work though. Assume that RingBufferLogEvent has an extra attribute that we want the user to be able to populate. How do we expose this attribute? Because at the moment the user calls the logger method, the ringbuffer slot has not been selected yet. I need to think about this more, because it looks like this would require a new API with something like a claim() method that would return the RingBufferLogEvent that the user can then modify and publish. > Support ByteBufferLayout > > > Key: LOG4J2-1397 > URL: https://issues.apache.org/jira/browse/LOG4J2-1397 > Project: Log4j 2 > Issue Type: New Feature > Components: Layouts >Affects Versions: 2.6 >Reporter: Remko Popma > > As requested by Kirk Pepperdine on the Mechanical Sympathy [mailing > list|https://groups.google.com/d/msg/mechanical-sympathy/klefjRqlpQE/-1WF59IGAwAJ]: > Support a new Layout that takes ObjectMessages containing a ByteBuffer > payload and write them to the appender without modification. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: log4j-dev-unsubscr...@logging.apache.org For additional commands, e-mail: log4j-dev-h...@logging.apache.org
[jira] [Commented] (LOG4J2-1397) Support ByteBufferLayout
[ https://issues.apache.org/jira/browse/LOG4J2-1397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15322196#comment-15322196 ] Anthony Maire commented on LOG4J2-1397: --- I agree that this is a narrow use case Another possible approach is to let the user inject it's implementation of the ringbuffer slot factory (and provide the method to transfer data from the application thread to the ringbuffer slot, then from the slot to the background thread). Does it seems OK to you ? Basically all use cases where the user want to access parameters at the appender level (background formatting, custom filters ...) are potentially broken with async logger if these parameters are mutable. And it will almost always be narrow use cases. I think that it can be a good solution to let the user a way to copy the piece of data that need to be copied for its use case > Support ByteBufferLayout > > > Key: LOG4J2-1397 > URL: https://issues.apache.org/jira/browse/LOG4J2-1397 > Project: Log4j 2 > Issue Type: New Feature > Components: Layouts >Affects Versions: 2.6 >Reporter: Remko Popma > > As requested by Kirk Pepperdine on the Mechanical Sympathy [mailing > list|https://groups.google.com/d/msg/mechanical-sympathy/klefjRqlpQE/-1WF59IGAwAJ]: > Support a new Layout that takes ObjectMessages containing a ByteBuffer > payload and write them to the appender without modification. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: log4j-dev-unsubscr...@logging.apache.org For additional commands, e-mail: log4j-dev-h...@logging.apache.org
[jira] [Commented] (LOG4J2-1397) Support ByteBufferLayout
[
https://issues.apache.org/jira/browse/LOG4J2-1397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15322176#comment-15322176
]
Mikael Ståldal commented on LOG4J2-1397:
I think we should either do this properly, or not at all.
I don't see the point in just adding a {{ByteBufferLayout}} to log4j-core,
since that will be a half-baked solution. Users of Log4j can use their own
{{ByteBufferLayout}} if they really want to.
> Support ByteBufferLayout
>
>
> Key: LOG4J2-1397
> URL: https://issues.apache.org/jira/browse/LOG4J2-1397
> Project: Log4j 2
> Issue Type: New Feature
> Components: Layouts
>Affects Versions: 2.6
>Reporter: Remko Popma
>
> As requested by Kirk Pepperdine on the Mechanical Sympathy [mailing
> list|https://groups.google.com/d/msg/mechanical-sympathy/klefjRqlpQE/-1WF59IGAwAJ]:
> Support a new Layout that takes ObjectMessages containing a ByteBuffer
> payload and write them to the appender without modification.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: log4j-dev-unsubscr...@logging.apache.org
For additional commands, e-mail: log4j-dev-h...@logging.apache.org
[jira] [Commented] (LOG4J2-1397) Support ByteBufferLayout
[ https://issues.apache.org/jira/browse/LOG4J2-1397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15322161#comment-15322161 ] Anthony Maire commented on LOG4J2-1397: --- It just doesn't work because of different orderings in the preallocated array and the ringbuffer. Let's take a simple example: - 2 threads - a ringbuffer with 2 slots So we need a preallocated array with 4 buffers - Thread A get sequence #1 from the atomic counter, then get interrupted by the system scheduler before doing the copy - Thread B is still logging, it get sequence #2, #3, #4 from the counter, pushed 2 events to the async logger (so the ringbuffer is full) and is blocked trying to push the third - Async Logger processed the first event submitted by thread B - Thread B can push another event (with preallocated buffer sequence #4) - Thread B wants to log another event, it get sequence #5 for the preallocated array - Thread A is resumed by the scheduler => Data race, both thread will copy data in the same preAllocated buffer > Support ByteBufferLayout > > > Key: LOG4J2-1397 > URL: https://issues.apache.org/jira/browse/LOG4J2-1397 > Project: Log4j 2 > Issue Type: New Feature > Components: Layouts >Affects Versions: 2.6 >Reporter: Remko Popma > > As requested by Kirk Pepperdine on the Mechanical Sympathy [mailing > list|https://groups.google.com/d/msg/mechanical-sympathy/klefjRqlpQE/-1WF59IGAwAJ]: > Support a new Layout that takes ObjectMessages containing a ByteBuffer > payload and write them to the appender without modification. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: log4j-dev-unsubscr...@logging.apache.org For additional commands, e-mail: log4j-dev-h...@logging.apache.org
[jira] [Commented] (LOG4J2-1397) Support ByteBufferLayout
[ https://issues.apache.org/jira/browse/LOG4J2-1397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15322149#comment-15322149 ] Remko Popma commented on LOG4J2-1397: - It is probably possible to do this in Log4j, but at the moment the use case seems (to me) too narrow to pull into the logging framework. It may be that the more generic LOG4J2-1305 will result in some mechanism to deal with this, but we need to do more analysis. > Support ByteBufferLayout > > > Key: LOG4J2-1397 > URL: https://issues.apache.org/jira/browse/LOG4J2-1397 > Project: Log4j 2 > Issue Type: New Feature > Components: Layouts >Affects Versions: 2.6 >Reporter: Remko Popma > > As requested by Kirk Pepperdine on the Mechanical Sympathy [mailing > list|https://groups.google.com/d/msg/mechanical-sympathy/klefjRqlpQE/-1WF59IGAwAJ]: > Support a new Layout that takes ObjectMessages containing a ByteBuffer > payload and write them to the appender without modification. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: log4j-dev-unsubscr...@logging.apache.org For additional commands, e-mail: log4j-dev-h...@logging.apache.org
[jira] [Commented] (LOG4J2-1397) Support ByteBufferLayout
[ https://issues.apache.org/jira/browse/LOG4J2-1397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15322124#comment-15322124 ] Mikael Ståldal commented on LOG4J2-1397: Wouldn't it be nicer if this copying was done by the logging framework, to alleviative the application code from this task? > Support ByteBufferLayout > > > Key: LOG4J2-1397 > URL: https://issues.apache.org/jira/browse/LOG4J2-1397 > Project: Log4j 2 > Issue Type: New Feature > Components: Layouts >Affects Versions: 2.6 >Reporter: Remko Popma > > As requested by Kirk Pepperdine on the Mechanical Sympathy [mailing > list|https://groups.google.com/d/msg/mechanical-sympathy/klefjRqlpQE/-1WF59IGAwAJ]: > Support a new Layout that takes ObjectMessages containing a ByteBuffer > payload and write them to the appender without modification. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: log4j-dev-unsubscr...@logging.apache.org For additional commands, e-mail: log4j-dev-h...@logging.apache.org
[jira] [Commented] (LOG4J2-1397) Support ByteBufferLayout
[
https://issues.apache.org/jira/browse/LOG4J2-1397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15322120#comment-15322120
]
Remko Popma commented on LOG4J2-1397:
-
In that case you need to use an AtomicLong instead of a primitive long. The
preAllocated array is not modified during the life of the program so is safe to
use from multiple threads, as long as each buffer is only accessed by a single
thread, which the AtomicLong counter guarantees:
{code}
ByteBuffer[] preAllocated = ...
AtomicLong counter = new AtomicLong();
private void onMessage(ByteBuffer data) {
...
long current = counter.incrementAndGet();
ByteBuffer copy = preAllocated[MASK & current];
}
{code}
I was wrong about the number of pre-allocated buffers necessary: you need
{{LoggerRingBufferSize + ThreadCount}}. That way you will never overwrite a
pre-allocated buffer that has not been logged yet. When the AsyncLogger
RingBuffer is full the AsyncLogger will block until a RingBuffer slot becomes
available.
> Support ByteBufferLayout
>
>
> Key: LOG4J2-1397
> URL: https://issues.apache.org/jira/browse/LOG4J2-1397
> Project: Log4j 2
> Issue Type: New Feature
> Components: Layouts
>Affects Versions: 2.6
>Reporter: Remko Popma
>
> As requested by Kirk Pepperdine on the Mechanical Sympathy [mailing
> list|https://groups.google.com/d/msg/mechanical-sympathy/klefjRqlpQE/-1WF59IGAwAJ]:
> Support a new Layout that takes ObjectMessages containing a ByteBuffer
> payload and write them to the appender without modification.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: log4j-dev-unsubscr...@logging.apache.org
For additional commands, e-mail: log4j-dev-h...@logging.apache.org
[jira] [Commented] (LOG4J2-1397) Support ByteBufferLayout
[ https://issues.apache.org/jira/browse/LOG4J2-1397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15322038#comment-15322038 ] Anthony Maire commented on LOG4J2-1397: --- At first sight, this approach may work correctly in a single-threaded context, but I think you need 1 more preallocated buffer than the ringbuffer size (so need to use a modulo instead of a binary mask). However in our application, this code can be invoked by several threads. Since the order in which buffers are obtained from the pre-allocated pool might not be the same as the order they are in the ringbuffer, the application will then need a callback to know that a buffer has been released (i.e logged to disk). > Support ByteBufferLayout > > > Key: LOG4J2-1397 > URL: https://issues.apache.org/jira/browse/LOG4J2-1397 > Project: Log4j 2 > Issue Type: New Feature > Components: Layouts >Affects Versions: 2.6 >Reporter: Remko Popma > > As requested by Kirk Pepperdine on the Mechanical Sympathy [mailing > list|https://groups.google.com/d/msg/mechanical-sympathy/klefjRqlpQE/-1WF59IGAwAJ]: > Support a new Layout that takes ObjectMessages containing a ByteBuffer > payload and write them to the appender without modification. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: log4j-dev-unsubscr...@logging.apache.org For additional commands, e-mail: log4j-dev-h...@logging.apache.org
[jira] [Commented] (LOG4J2-1397) Support ByteBufferLayout
[
https://issues.apache.org/jira/browse/LOG4J2-1397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15321679#comment-15321679
]
Remko Popma commented on LOG4J2-1397:
-
The byte buffer should not be used by two threads simultaneously. This is a
valid concern and needs to be addressed either in the application or in the
logging framework.
How about doing this in the application:
* Pre-allocate a set of ByteBuffers in the application
* How many to pre-allocate? I would say one for each slot in the Log4j
AsyncLogger RingBuffer. By default this is 256*1024, which may be a lot for
your purposes. Let's say you change this to 32,768. Then also pre-allocate
32,768 ByteBuffers. The AsyncLogger RingBuffer size is a power of 2. Use the
same size for your pre-allocated array to allow you have a long counter that is
incremented indefinitely and get the current slot by masking the counter by
length-1.
* How large should each pre-allocated ByteBuffer be? That depends on the size
(max size) of the messages you are processing.
When you receive a message, obtain the next ByteBuffer from the pre-allocated
set, copy the data into it and log that pre-allocated ByteBuffer.
{code}
final static int COUNT = 32768; // use same size as Async Logger RingBuffer!
final static int MASK = COUNT - 1
ByteBuffer[] preAllocated = preAllocate(COUNT);
long counter = 0;
private void onMessage(ByteBuffer msg) {
if (messageLog.isInfoEnabled()) {
ByteBuffer copy = preAllocated[MASK & (counter++)];
copy.clear();
copy.put(buffer);
buffer.rewind();
copy.flip();
// messageLog.info("Message on {}: IN {}", connection, copy); // do not log
as text
messageLog.info(copy);
}
}
{code}
> Support ByteBufferLayout
>
>
> Key: LOG4J2-1397
> URL: https://issues.apache.org/jira/browse/LOG4J2-1397
> Project: Log4j 2
> Issue Type: New Feature
> Components: Layouts
>Affects Versions: 2.6
>Reporter: Remko Popma
>
> As requested by Kirk Pepperdine on the Mechanical Sympathy [mailing
> list|https://groups.google.com/d/msg/mechanical-sympathy/klefjRqlpQE/-1WF59IGAwAJ]:
> Support a new Layout that takes ObjectMessages containing a ByteBuffer
> payload and write them to the appender without modification.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: log4j-dev-unsubscr...@logging.apache.org
For additional commands, e-mail: log4j-dev-h...@logging.apache.org
[jira] [Commented] (LOG4J2-1397) Support ByteBufferLayout
[ https://issues.apache.org/jira/browse/LOG4J2-1397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15321285#comment-15321285 ] Anthony Maire commented on LOG4J2-1397: --- Basically, a layout that have a dependency on a mutable object will not be compatible with asynchronous logger/appender. A copy of the content has to be made. And to avoid excessive allocation, the copy should be reusable. If we want to safely use the buffer content without excessive allocation, we already have some solutions that use String (such as wrapping it in a StringBuilderFormattable that decode the buffer content into a StringBuilder). But this is slow (it takes a few extra microseconds for a message of a few hundreds bytes compared to bulk byte copy) If we want to use a ByteBufferLayout, that means that these few microseconds matters, so it's nearly 100% sure that asynchronous logging will be used. Without some byte copy support provided by the async logger, I'm afraid that this layout can't be really useful. > Support ByteBufferLayout > > > Key: LOG4J2-1397 > URL: https://issues.apache.org/jira/browse/LOG4J2-1397 > Project: Log4j 2 > Issue Type: New Feature > Components: Layouts >Affects Versions: 2.6 >Reporter: Remko Popma > > As requested by Kirk Pepperdine on the Mechanical Sympathy [mailing > list|https://groups.google.com/d/msg/mechanical-sympathy/klefjRqlpQE/-1WF59IGAwAJ]: > Support a new Layout that takes ObjectMessages containing a ByteBuffer > payload and write them to the appender without modification. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: log4j-dev-unsubscr...@logging.apache.org For additional commands, e-mail: log4j-dev-h...@logging.apache.org
[jira] [Commented] (LOG4J2-1397) Support ByteBufferLayout
[
https://issues.apache.org/jira/browse/LOG4J2-1397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15320744#comment-15320744
]
Mikael Ståldal commented on LOG4J2-1397:
So the async logger should copy the application's {{ByteBuffer}} into its own
{{ByteBuffer}} before returning from the logging call?
> Support ByteBufferLayout
>
>
> Key: LOG4J2-1397
> URL: https://issues.apache.org/jira/browse/LOG4J2-1397
> Project: Log4j 2
> Issue Type: New Feature
> Components: Layouts
>Affects Versions: 2.6
>Reporter: Remko Popma
>
> As requested by Kirk Pepperdine on the Mechanical Sympathy [mailing
> list|https://groups.google.com/d/msg/mechanical-sympathy/klefjRqlpQE/-1WF59IGAwAJ]:
> Support a new Layout that takes ObjectMessages containing a ByteBuffer
> payload and write them to the appender without modification.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: log4j-dev-unsubscr...@logging.apache.org
For additional commands, e-mail: log4j-dev-h...@logging.apache.org
[jira] [Commented] (LOG4J2-1397) Support ByteBufferLayout
[ https://issues.apache.org/jira/browse/LOG4J2-1397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15320181#comment-15320181 ] Mikael Ståldal commented on LOG4J2-1397: Thanks, it's helpful to get this kind of insight into the actual use case. > Support ByteBufferLayout > > > Key: LOG4J2-1397 > URL: https://issues.apache.org/jira/browse/LOG4J2-1397 > Project: Log4j 2 > Issue Type: New Feature > Components: Layouts >Affects Versions: 2.6 >Reporter: Remko Popma > > As requested by Kirk Pepperdine on the Mechanical Sympathy [mailing > list|https://groups.google.com/d/msg/mechanical-sympathy/klefjRqlpQE/-1WF59IGAwAJ]: > Support a new Layout that takes ObjectMessages containing a ByteBuffer > payload and write them to the appender without modification. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: log4j-dev-unsubscr...@logging.apache.org For additional commands, e-mail: log4j-dev-h...@logging.apache.org
[jira] [Commented] (LOG4J2-1397) Support ByteBufferLayout
[
https://issues.apache.org/jira/browse/LOG4J2-1397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15320175#comment-15320175
]
Anthony Maire commented on LOG4J2-1397:
---
Here is my usecase:
- The application is receiving FIX messages (Ascii-based protocol used in
financial services) at a rate from a few thousands to a hundred thousands
messages per second
- The application need to log every message received (mostly for legal and
auditing reasons)
- Latency is critical, so logging need to be asynchronous since the jitter
induced by kernel write back can be very high at this rate
Our (simplified) architecture is something like this :
{code}
public class Connection {
private SocketChannel channel;
private ByteBuffer buffer;
private MessageListener listener;
...
//called when the main loop involving a NIO Selector told us that there
is data to process for this socket
public void onDataReady(){
channel.read(buffer);
if(containsCompleteMessage(buffer)){
listener.onIncomingMessage(this, buffer);
}
}
public String toString(){
return "Connection [remoteAddress=" +
channel.getRemoteAddress() + "]";
}
}
public class MessageListener(){
private static final Logger messageLog =
LoggerFactory.getLogger("message.logger");
private static final Logger log =
LoggerFactory.getLogger(MessageListener.class);
private FIXMessageCodec codec;
...
public void onIncomingMessage(ByteBuffer buffer, Connection connection){
messageLog.info("Message on {}: IN {}", connection, buffer);
FIXMessage businessMessage = codec.decode(buffer);
buffer.clear();
log.debug("start business processing for {}", businessMessage);
doBusinessProcessing(businessMessage);
}
}
{code}
So we have a dedicated logger for these raw messages that we need to store on
disk.
Currently we are using logback through SLF4J, and the first line of
onIncomingMessage is replaced by something like this:
{code}
if(messageLog.isInfoEnabled()){
ByteBuffer copy = ByteBuffer.allocate(buffer.remaining());
copy.put(buffer);
buffer.rewind();
copy.flip();
messageLog.info("Message on {}: IN {}", connection, copy);
}
{code}
And we have some custom layout that will process the content of the ByteBuffer
in the background thread of the AsyncAppender.
Making this copy is not currently an issue since logback is already allocating
roughly 20 times the buffer's size per event, but since we are now looking to
use log4j for it's low allocation rate, we want to avoid that garbage.
> Support ByteBufferLayout
>
>
> Key: LOG4J2-1397
> URL: https://issues.apache.org/jira/browse/LOG4J2-1397
> Project: Log4j 2
> Issue Type: New Feature
> Components: Layouts
>Affects Versions: 2.6
>Reporter: Remko Popma
>
> As requested by Kirk Pepperdine on the Mechanical Sympathy [mailing
> list|https://groups.google.com/d/msg/mechanical-sympathy/klefjRqlpQE/-1WF59IGAwAJ]:
> Support a new Layout that takes ObjectMessages containing a ByteBuffer
> payload and write them to the appender without modification.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: log4j-dev-unsubscr...@logging.apache.org
For additional commands, e-mail: log4j-dev-h...@logging.apache.org
[jira] [Commented] (LOG4J2-1397) Support ByteBufferLayout
[ https://issues.apache.org/jira/browse/LOG4J2-1397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15320112#comment-15320112 ] Anthony Maire commented on LOG4J2-1397: --- The issue with TL is that we need to pass the copy between 2 threads through the disruptor. That means that there should be some support for swapping cached buffers in MutableLogEvent for example (same idea that the parameter swap that was made for LOG4J2-1342). Without this kind of support, we need either to create a defensive copy (and create garbage) or to have a shared pool of ByteBuffer (and create synchronization performance penalty) > Support ByteBufferLayout > > > Key: LOG4J2-1397 > URL: https://issues.apache.org/jira/browse/LOG4J2-1397 > Project: Log4j 2 > Issue Type: New Feature > Components: Layouts >Affects Versions: 2.6 >Reporter: Remko Popma > > As requested by Kirk Pepperdine on the Mechanical Sympathy [mailing > list|https://groups.google.com/d/msg/mechanical-sympathy/klefjRqlpQE/-1WF59IGAwAJ]: > Support a new Layout that takes ObjectMessages containing a ByteBuffer > payload and write them to the appender without modification. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: log4j-dev-unsubscr...@logging.apache.org For additional commands, e-mail: log4j-dev-h...@logging.apache.org
[jira] [Commented] (LOG4J2-1397) Support ByteBufferLayout
[
https://issues.apache.org/jira/browse/LOG4J2-1397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15318616#comment-15318616
]
Mikael Ståldal commented on LOG4J2-1397:
A ThreadLocal {{ByteBuffer}} maybe?
> Support ByteBufferLayout
>
>
> Key: LOG4J2-1397
> URL: https://issues.apache.org/jira/browse/LOG4J2-1397
> Project: Log4j 2
> Issue Type: New Feature
> Components: Layouts
>Affects Versions: 2.6
>Reporter: Remko Popma
>
> As requested by Kirk Pepperdine on the Mechanical Sympathy [mailing
> list|https://groups.google.com/d/msg/mechanical-sympathy/klefjRqlpQE/-1WF59IGAwAJ]:
> Support a new Layout that takes ObjectMessages containing a ByteBuffer
> payload and write them to the appender without modification.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: log4j-dev-unsubscr...@logging.apache.org
For additional commands, e-mail: log4j-dev-h...@logging.apache.org
[jira] [Commented] (LOG4J2-1397) Support ByteBufferLayout
[
https://issues.apache.org/jira/browse/LOG4J2-1397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15318607#comment-15318607
]
Mikael Ståldal commented on LOG4J2-1397:
I would like to see a more elaborate example of how this would be used from
application code, including what the {{getBuffer()}} method in Remko's example
actually does.
> Support ByteBufferLayout
>
>
> Key: LOG4J2-1397
> URL: https://issues.apache.org/jira/browse/LOG4J2-1397
> Project: Log4j 2
> Issue Type: New Feature
> Components: Layouts
>Affects Versions: 2.6
>Reporter: Remko Popma
>
> As requested by Kirk Pepperdine on the Mechanical Sympathy [mailing
> list|https://groups.google.com/d/msg/mechanical-sympathy/klefjRqlpQE/-1WF59IGAwAJ]:
> Support a new Layout that takes ObjectMessages containing a ByteBuffer
> payload and write them to the appender without modification.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: log4j-dev-unsubscr...@logging.apache.org
For additional commands, e-mail: log4j-dev-h...@logging.apache.org
[jira] [Commented] (LOG4J2-1397) Support ByteBufferLayout
[ https://issues.apache.org/jira/browse/LOG4J2-1397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15307652#comment-15307652 ] Remko Popma commented on LOG4J2-1397: - Linking to LOG4J2-1305 which allows binary logging of other attributes of the LogEvent in addition to just the log message payload. > Support ByteBufferLayout > > > Key: LOG4J2-1397 > URL: https://issues.apache.org/jira/browse/LOG4J2-1397 > Project: Log4j 2 > Issue Type: New Feature > Components: Layouts >Affects Versions: 2.6 >Reporter: Remko Popma > > As requested by Kirk Pepperdine on the Mechanical Sympathy [mailing > list|https://groups.google.com/d/msg/mechanical-sympathy/klefjRqlpQE/-1WF59IGAwAJ]: > Support a new Layout that takes ObjectMessages containing a ByteBuffer > payload and write them to the appender without modification. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: log4j-dev-unsubscr...@logging.apache.org For additional commands, e-mail: log4j-dev-h...@logging.apache.org
[jira] [Commented] (LOG4J2-1397) Support ByteBufferLayout
[
https://issues.apache.org/jira/browse/LOG4J2-1397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15306400#comment-15306400
]
Remko Popma commented on LOG4J2-1397:
-
I'm thinking simply something like this:
{code}
ByteBuffer binaryData = getBuffer();
logger.debug(binaryData);
{code}
> Support ByteBufferLayout
>
>
> Key: LOG4J2-1397
> URL: https://issues.apache.org/jira/browse/LOG4J2-1397
> Project: Log4j 2
> Issue Type: New Feature
> Components: Layouts
>Affects Versions: 2.6
>Reporter: Remko Popma
>
> As requested by Kirk Pepperdine on the Mechanical Sympathy [mailing
> list|https://groups.google.com/d/msg/mechanical-sympathy/klefjRqlpQE/-1WF59IGAwAJ]:
> Support a new Layout that takes ObjectMessages containing a ByteBuffer
> payload and write them to the appender without modification.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: log4j-dev-unsubscr...@logging.apache.org
For additional commands, e-mail: log4j-dev-h...@logging.apache.org
[jira] [Commented] (LOG4J2-1397) Support ByteBufferLayout
[ https://issues.apache.org/jira/browse/LOG4J2-1397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15306342#comment-15306342 ] Mikael Ståldal commented on LOG4J2-1397: How would one log to this layout from application code? > Support ByteBufferLayout > > > Key: LOG4J2-1397 > URL: https://issues.apache.org/jira/browse/LOG4J2-1397 > Project: Log4j 2 > Issue Type: New Feature > Components: Layouts >Affects Versions: 2.6 >Reporter: Remko Popma > > As requested by Kirk Pepperdine on the Mechanical Sympathy [mailing > list|https://groups.google.com/d/msg/mechanical-sympathy/klefjRqlpQE/-1WF59IGAwAJ]: > Support a new Layout that takes ObjectMessages containing a ByteBuffer > payload and write them to the appender without modification. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: log4j-dev-unsubscr...@logging.apache.org For additional commands, e-mail: log4j-dev-h...@logging.apache.org
[jira] [Commented] (LOG4J2-1397) Support ByteBufferLayout
[ https://issues.apache.org/jira/browse/LOG4J2-1397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15305883#comment-15305883 ] Remko Popma commented on LOG4J2-1397: - Example no-frills implementation: https://gist.github.com/remkop/715fcd6035c4f798c264e21450106399 > Support ByteBufferLayout > > > Key: LOG4J2-1397 > URL: https://issues.apache.org/jira/browse/LOG4J2-1397 > Project: Log4j 2 > Issue Type: New Feature > Components: Layouts >Affects Versions: 2.6 >Reporter: Remko Popma > > As requested by Kirk Pepperdine on the Mechanical Sympathy [mailing > list|https://groups.google.com/d/msg/mechanical-sympathy/klefjRqlpQE/-1WF59IGAwAJ]: > Support a new Layout that takes ObjectMessages containing a ByteBuffer > payload and write them to the appender without modification. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: log4j-dev-unsubscr...@logging.apache.org For additional commands, e-mail: log4j-dev-h...@logging.apache.org
