date:20170226

Yes, ByteBuffer is only useful for binary data. Using it for text would be
painful.

On Mon, Feb 27, 2017 at 2:55 Matt Sicker  wrote:

> Oh, alright, that makes sense. No need to change it, I was mainly curious
> since I'm working on some NIO stuff (from the other thread) and came to a
> point where I need to add a similar interface, though now I'm also seeing
> why a StringBuilder is used (I was trying to skip an intermediate step of
> formatting a log message by directly encoding the LogEvent into a
> ByteBuffer since this playground code doesn't have a need for complicated
> layouts, but ensuring the write buffer has enough space between each and
> every byte would be extremely tedious).
>
> On 26 February 2017 at 19:48, Remko Popma  wrote:
>
> Similarly to the way the drain() method gives *implementations *the
> flexibility to return a different ByteBuffer, I wanted to also give *callers
> *the flexibility to drain a different ByteBuffer. I don't have a concrete
> use case for it, but I imagined there might be cases where callers
> overflowed content into another buffer which would need to be drained
> separately.
>
> I don't want to modify the signatures of the ByteBufferDestination
> interface. It would break binary compatibility for no good reason.
>
>
> On Mon, Feb 27, 2017 at 10:33 AM, Matt Sicker  wrote:
>
> Since all our implementations of ByteBufferDestination return a shared
> ByteBuffer in getByteBuffer(), I don't see why it needs to be provided to
> the drain() method. drain() returns the buffer (or a new one in the case of
> MemoryMappedFileManager), and I don't see why an assumption could be made
> that the buffer you're draining is the exact one the destination already
> knows about. Is there a particular use case why this might not work?
>
> --
> Matt Sicker 
>
>
>
>
>
> --
> Matt Sicker 
>

[jira] [Commented] (LOG4J2-1824) (GC) Avoid allocating temporary objects in KafkaAppender

2017-02-26 Thread Matt Sicker (JIRA)


[ 
https://issues.apache.org/jira/browse/LOG4J2-1824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15885154#comment-15885154
 ] 

Matt Sicker commented on LOG4J2-1824:
-

Based on some mailing list discussions, it may be possible that the 
{{ByteBufferDestination.drain(ByteBuffer)}} method could be used here. 
Honestly, to properly implement this may require some interesting changes to 
kafka-clients (e.g., a ByteBufferDestination-like interface for writing 
directly to a message while it's being prepared to be sent over the network so 
we can buffer a log message without splitting it into multiple kafka messages). 
This is definitely going to require more research to handle properly.

It may be possible that GatheringByteChannel may be useful here.

> (GC) Avoid allocating temporary objects in KafkaAppender
> 
>
> Key: LOG4J2-1824
> URL: https://issues.apache.org/jira/browse/LOG4J2-1824
> Project: Log4j 2
>  Issue Type: Improvement
>  Components: Appenders
>Reporter: Matt Sicker
>
> Due to API limitations in kafka-clients, log messages must be encoded into 
> byte arrays rather than directly into ByteBuffers via 
> {{Encoder}}-enhanced layouts. I've opened up KAFKA-4802 to get 
> support on the client side for not converting the provided ByteBuffers back 
> into a byte array, but in the meantime, KafkaAppender and KafkaManager can be 
> enhanced to use the Encoder API.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: log4j-dev-unsubscr...@logging.apache.org
For additional commands, e-mail: log4j-dev-h...@logging.apache.org

Re: Experimenting with NIO

Sent too early!

mmap version:

https://github.com/jvz/nio-logger/blob/master/src/main/java/org/musigma/logging/appender/MemoryMappedFileAppender.java

I'm going to rerun the benchmarks. I think I'm getting closer to finding
some things we've missed in the NIO API (plus now I actually have a much
better understanding on how the hell channels and buffers work).

On 26 February 2017 at 23:10, Matt Sicker  wrote:

> Whoops, totally forgot some links. So here is a basic FileChannelAppender
> (though in log4j-core, this would likely be a Manager instead along with a
> zillion other features I'm not bothering to reimplement in a playground
> project):
>
> https://github.com/jvz/nio-logger/blob/master/src/main/
> java/org/musigma/logging/appender/FileChannelAppender.java
>
> I've also done similarly with the mmap file version:
>
>
> On 26 February 2017 at 23:08, Matt Sicker  wrote:
>
>> I'm testing out a simplified system that pretty much tests out how fast
>> different appenders and encoding styles can work. Log messages here do not
>> have any information besides a message and a timestamp. It's to investigate
>> what possible NIO optimizations are possible in Log4j 2.
>>
>> Anyways, I've fixed the performance of the FileChannelAppender, too. By
>> using the equivalent of ByteBufferDestination here (I called it Buffered> extends Buffer> for now), this not only provided buffering for free (well,
>> it's kind of the point), but by using direct allocation of a ByteBuffer,
>> the transfer from the encoded message in said buffer into the FileChannel
>> is a lot faster (in theory, the OS can bypass some things to make the
>> transfer faster or something; I'm not super clear on the details).
>>
>> One thing I've already discovered is that unless there's some reason not
>> to, we should be using ByteBuffer.allocateDirect(size) instead of
>> ByteBuffer.wrap(new byte[size]) when the underlying IO class uses
>> ByteBuffer instead of byte[]. It might improve performance of the
>> OutputStream-based appenders, too, but I haven't really tested this in the
>> full project.
>>
>> On 26 February 2017 at 22:53, Ralph Goers 
>> wrote:
>>
>>> I’m not sure what these numbers mean as I don’t know what you’ve changed
>>> and how they compare to what was there previously.
>>>
>>> Ralph
>>>
>>> On Feb 26, 2017, at 9:44 PM, Matt Sicker  wrote:
>>>
>>> Anyways, I managed to implement MemoryMappedFileAppender in the
>>> playground by basically copying it from log4j-core. Everything is still in
>>> a state of flux here, but without adding manual buffering to the
>>> FileChannel and RandomAccessFile appenders, here are my current results
>>> (the async channel one is still failing):
>>>
>>> BenchmarkMode  CntScore
>>>   Error  Units
>>> FileChannelAppenderBenchmark.logToFile  thrpt   10   306320.023
>>> ±  4911.376  ops/s
>>> FileOutputStreamAppenderBenchmark.logToFile thrpt   10  1311134.313
>>> ± 32814.148  ops/s
>>> MemoryMappedFileAppenderBenchmark.logToFile thrpt   10  1569499.970
>>> ± 47550.210  ops/s
>>> NioFileOutputStreamAppenderBenchmark.logToFile  thrpt   10  1414480.114
>>> ± 39527.202  ops/s
>>> RandomAccessFileAppenderBenchmark.logToFile thrpt   10   422874.583
>>> ±  5457.952  ops/s
>>>
>>> On 26 February 2017 at 21:39, Matt Sicker  wrote:
>>>
 I noticed that in MemoryMappedFileManager, if the MappedByteBuffer
 hasn't been fully written to when remap() is called, then the new
 MappedByteBuffer will include the unwritten area as well as an additional
 region of the file. Do you think this could cause issues with performance
 like we saw with the highly variable throughput in the appender? I'm
 guessing the idea here is to continue writing wherever we left off but
 while also providing a larger ByteBuffer for producers to use if the
 normally sized one is too small. But in that case, the producer can simply
 keep draining the buffer as it goes.

 On 26 February 2017 at 19:27, Remko Popma 
 wrote:

> Afonso, are you a robot?
>
> On Mon, Feb 27, 2017 at 8:44 AM, Afonso Murakami <
> murakam...@icloud.com> wrote:
>
>> That is great thank you very much for your help
>>
>> On Feb 26, 2017, at 3:36 PM, Matt Sicker  wrote:
>>
>> Afonso, are you on the right mailing list?
>>
>> Anyways, I've made more updates to my playground. Without
>> implementing manual buffering on top of RandomAccessFile or FileChannel 
>> (or
>> the async file channel, but that one has a bug in it right now causing 
>> OOM
>> errors occasionally which is probably due to a misunderstanding on how 
>> many
>> ByteBuffers I can queue up asynchronously before I'm supposed to stop and
>> wait), the Java 101 style of using new

Re: Experimenting with NIO

Whoops, totally forgot some links. So here is a basic FileChannelAppender
(though in log4j-core, this would likely be a Manager instead along with a
zillion other features I'm not bothering to reimplement in a playground
project):

https://github.com/jvz/nio-logger/blob/master/src/main/java/org/musigma/logging/appender/FileChannelAppender.java

I've also done similarly with the mmap file version:


On 26 February 2017 at 23:08, Matt Sicker  wrote:

> I'm testing out a simplified system that pretty much tests out how fast
> different appenders and encoding styles can work. Log messages here do not
> have any information besides a message and a timestamp. It's to investigate
> what possible NIO optimizations are possible in Log4j 2.
>
> Anyways, I've fixed the performance of the FileChannelAppender, too. By
> using the equivalent of ByteBufferDestination here (I called it Buffered extends Buffer> for now), this not only provided buffering for free (well,
> it's kind of the point), but by using direct allocation of a ByteBuffer,
> the transfer from the encoded message in said buffer into the FileChannel
> is a lot faster (in theory, the OS can bypass some things to make the
> transfer faster or something; I'm not super clear on the details).
>
> One thing I've already discovered is that unless there's some reason not
> to, we should be using ByteBuffer.allocateDirect(size) instead of
> ByteBuffer.wrap(new byte[size]) when the underlying IO class uses
> ByteBuffer instead of byte[]. It might improve performance of the
> OutputStream-based appenders, too, but I haven't really tested this in the
> full project.
>
> On 26 February 2017 at 22:53, Ralph Goers 
> wrote:
>
>> I’m not sure what these numbers mean as I don’t know what you’ve changed
>> and how they compare to what was there previously.
>>
>> Ralph
>>
>> On Feb 26, 2017, at 9:44 PM, Matt Sicker  wrote:
>>
>> Anyways, I managed to implement MemoryMappedFileAppender in the
>> playground by basically copying it from log4j-core. Everything is still in
>> a state of flux here, but without adding manual buffering to the
>> FileChannel and RandomAccessFile appenders, here are my current results
>> (the async channel one is still failing):
>>
>> BenchmarkMode  CntScore
>> Error  Units
>> FileChannelAppenderBenchmark.logToFile  thrpt   10   306320.023
>> ±  4911.376  ops/s
>> FileOutputStreamAppenderBenchmark.logToFile thrpt   10  1311134.313
>> ± 32814.148  ops/s
>> MemoryMappedFileAppenderBenchmark.logToFile thrpt   10  1569499.970
>> ± 47550.210  ops/s
>> NioFileOutputStreamAppenderBenchmark.logToFile  thrpt   10  1414480.114
>> ± 39527.202  ops/s
>> RandomAccessFileAppenderBenchmark.logToFile thrpt   10   422874.583
>> ±  5457.952  ops/s
>>
>> On 26 February 2017 at 21:39, Matt Sicker  wrote:
>>
>>> I noticed that in MemoryMappedFileManager, if the MappedByteBuffer
>>> hasn't been fully written to when remap() is called, then the new
>>> MappedByteBuffer will include the unwritten area as well as an additional
>>> region of the file. Do you think this could cause issues with performance
>>> like we saw with the highly variable throughput in the appender? I'm
>>> guessing the idea here is to continue writing wherever we left off but
>>> while also providing a larger ByteBuffer for producers to use if the
>>> normally sized one is too small. But in that case, the producer can simply
>>> keep draining the buffer as it goes.
>>>
>>> On 26 February 2017 at 19:27, Remko Popma  wrote:
>>>
 Afonso, are you a robot?

 On Mon, Feb 27, 2017 at 8:44 AM, Afonso Murakami  wrote:

> That is great thank you very much for your help
>
> On Feb 26, 2017, at 3:36 PM, Matt Sicker  wrote:
>
> Afonso, are you on the right mailing list?
>
> Anyways, I've made more updates to my playground. Without implementing
> manual buffering on top of RandomAccessFile or FileChannel (or the async
> file channel, but that one has a bug in it right now causing OOM errors
> occasionally which is probably due to a misunderstanding on how many
> ByteBuffers I can queue up asynchronously before I'm supposed to stop and
> wait), the Java 101 style of using new BufferedOutputStream(new
> FileOutputStream(file)) is currently the fastest appender I have. I'm also
> trying to implement a MappedByteBuffer version, but now I'm rediscovering
> the reason why we have a ByteBufferDestination interface (plus all I'm
> getting as a result are 2 GB files filled with nulls, so I'm doing
> something wrong here anyways).
>
> I was also able to fix some strange performance issues in the tests
> that were being caused by my Layout implementation, so I've swapped in a
> trivial one that assumes input

Re: Experimenting with NIO

I'm testing out a simplified system that pretty much tests out how fast
different appenders and encoding styles can work. Log messages here do not
have any information besides a message and a timestamp. It's to investigate
what possible NIO optimizations are possible in Log4j 2.

Anyways, I've fixed the performance of the FileChannelAppender, too. By
using the equivalent of ByteBufferDestination here (I called it Buffered for now), this not only provided buffering for free (well,
it's kind of the point), but by using direct allocation of a ByteBuffer,
the transfer from the encoded message in said buffer into the FileChannel
is a lot faster (in theory, the OS can bypass some things to make the
transfer faster or something; I'm not super clear on the details).

One thing I've already discovered is that unless there's some reason not
to, we should be using ByteBuffer.allocateDirect(size) instead of
ByteBuffer.wrap(new byte[size]) when the underlying IO class uses
ByteBuffer instead of byte[]. It might improve performance of the
OutputStream-based appenders, too, but I haven't really tested this in the
full project.

On 26 February 2017 at 22:53, Ralph Goers 
wrote:

> I’m not sure what these numbers mean as I don’t know what you’ve changed
> and how they compare to what was there previously.
>
> Ralph
>
> On Feb 26, 2017, at 9:44 PM, Matt Sicker  wrote:
>
> Anyways, I managed to implement MemoryMappedFileAppender in the playground
> by basically copying it from log4j-core. Everything is still in a state of
> flux here, but without adding manual buffering to the FileChannel and
> RandomAccessFile appenders, here are my current results (the async channel
> one is still failing):
>
> BenchmarkMode  CntScore
> Error  Units
> FileChannelAppenderBenchmark.logToFile  thrpt   10   306320.023 ±
>  4911.376  ops/s
> FileOutputStreamAppenderBenchmark.logToFile thrpt   10  1311134.313 ±
> 32814.148  ops/s
> MemoryMappedFileAppenderBenchmark.logToFile thrpt   10  1569499.970 ±
> 47550.210  ops/s
> NioFileOutputStreamAppenderBenchmark.logToFile  thrpt   10  1414480.114 ±
> 39527.202  ops/s
> RandomAccessFileAppenderBenchmark.logToFile thrpt   10   422874.583 ±
>  5457.952  ops/s
>
> On 26 February 2017 at 21:39, Matt Sicker  wrote:
>
>> I noticed that in MemoryMappedFileManager, if the MappedByteBuffer hasn't
>> been fully written to when remap() is called, then the new MappedByteBuffer
>> will include the unwritten area as well as an additional region of the
>> file. Do you think this could cause issues with performance like we saw
>> with the highly variable throughput in the appender? I'm guessing the idea
>> here is to continue writing wherever we left off but while also providing a
>> larger ByteBuffer for producers to use if the normally sized one is too
>> small. But in that case, the producer can simply keep draining the buffer
>> as it goes.
>>
>> On 26 February 2017 at 19:27, Remko Popma  wrote:
>>
>>> Afonso, are you a robot?
>>>
>>> On Mon, Feb 27, 2017 at 8:44 AM, Afonso Murakami 
>>> wrote:
>>>
 That is great thank you very much for your help

 On Feb 26, 2017, at 3:36 PM, Matt Sicker  wrote:

 Afonso, are you on the right mailing list?

 Anyways, I've made more updates to my playground. Without implementing
 manual buffering on top of RandomAccessFile or FileChannel (or the async
 file channel, but that one has a bug in it right now causing OOM errors
 occasionally which is probably due to a misunderstanding on how many
 ByteBuffers I can queue up asynchronously before I'm supposed to stop and
 wait), the Java 101 style of using new BufferedOutputStream(new
 FileOutputStream(file)) is currently the fastest appender I have. I'm also
 trying to implement a MappedByteBuffer version, but now I'm rediscovering
 the reason why we have a ByteBufferDestination interface (plus all I'm
 getting as a result are 2 GB files filled with nulls, so I'm doing
 something wrong here anyways).

 I was also able to fix some strange performance issues in the tests
 that were being caused by my Layout implementation, so I've swapped in a
 trivial one that assumes input is in ASCII and casts chars to bytes to
 encode them. In Java 9, we may be able to do something with the compacted
 strings thing they added for ISO-8859-1 strings.

 There are also some additional benchmarks added that I found
 interesting while comparing some approaches. For one, copying from a
 directly-allocated ByteBuffer into a byte[] (via ByteBuffer::get(byte[]))
 is almost twice as fast as copying from a heap-allocated ByteBuffer into a
 byte[]. Another interesting thing I found was that doing new
 Date().toString() is faster than

Re: Experimenting with NIO

I’m not sure what these numbers mean as I don’t know what you’ve changed and 
how they compare to what was there previously.

Ralph

> On Feb 26, 2017, at 9:44 PM, Matt Sicker  wrote:
> 
> Anyways, I managed to implement MemoryMappedFileAppender in the playground by 
> basically copying it from log4j-core. Everything is still in a state of flux 
> here, but without adding manual buffering to the FileChannel and 
> RandomAccessFile appenders, here are my current results (the async channel 
> one is still failing):
> 
> BenchmarkMode  CntScore   
> Error  Units
> FileChannelAppenderBenchmark.logToFile  thrpt   10   306320.023 ±  
> 4911.376  ops/s
> FileOutputStreamAppenderBenchmark.logToFile thrpt   10  1311134.313 ± 
> 32814.148  ops/s
> MemoryMappedFileAppenderBenchmark.logToFile thrpt   10  1569499.970 ± 
> 47550.210  ops/s
> NioFileOutputStreamAppenderBenchmark.logToFile  thrpt   10  1414480.114 ± 
> 39527.202  ops/s
> RandomAccessFileAppenderBenchmark.logToFile thrpt   10   422874.583 ±  
> 5457.952  ops/s
> 
> On 26 February 2017 at 21:39, Matt Sicker  > wrote:
> I noticed that in MemoryMappedFileManager, if the MappedByteBuffer hasn't 
> been fully written to when remap() is called, then the new MappedByteBuffer 
> will include the unwritten area as well as an additional region of the file. 
> Do you think this could cause issues with performance like we saw with the 
> highly variable throughput in the appender? I'm guessing the idea here is to 
> continue writing wherever we left off but while also providing a larger 
> ByteBuffer for producers to use if the normally sized one is too small. But 
> in that case, the producer can simply keep draining the buffer as it goes.
> 
> On 26 February 2017 at 19:27, Remko Popma  > wrote:
> Afonso, are you a robot?
> 
> On Mon, Feb 27, 2017 at 8:44 AM, Afonso Murakami  > wrote:
> That is great thank you very much for your help
> 
>> On Feb 26, 2017, at 3:36 PM, Matt Sicker > > wrote:
>> 
>> Afonso, are you on the right mailing list?
>> 
>> Anyways, I've made more updates to my playground. Without implementing 
>> manual buffering on top of RandomAccessFile or FileChannel (or the async 
>> file channel, but that one has a bug in it right now causing OOM errors 
>> occasionally which is probably due to a misunderstanding on how many 
>> ByteBuffers I can queue up asynchronously before I'm supposed to stop and 
>> wait), the Java 101 style of using new BufferedOutputStream(new 
>> FileOutputStream(file)) is currently the fastest appender I have. I'm also 
>> trying to implement a MappedByteBuffer version, but now I'm rediscovering 
>> the reason why we have a ByteBufferDestination interface (plus all I'm 
>> getting as a result are 2 GB files filled with nulls, so I'm doing something 
>> wrong here anyways).
>> 
>> I was also able to fix some strange performance issues in the tests that 
>> were being caused by my Layout implementation, so I've swapped in a trivial 
>> one that assumes input is in ASCII and casts chars to bytes to encode them. 
>> In Java 9, we may be able to do something with the compacted strings thing 
>> they added for ISO-8859-1 strings.
>> 
>> There are also some additional benchmarks added that I found interesting 
>> while comparing some approaches. For one, copying from a directly-allocated 
>> ByteBuffer into a byte[] (via ByteBuffer::get(byte[])) is almost twice as 
>> fast as copying from a heap-allocated ByteBuffer into a byte[]. Another 
>> interesting thing I found was that doing new Date().toString() is faster 
>> than Instant.now().toString().
>> 
>> On 26 February 2017 at 16:10, Afonso Murakami > > wrote:
>> Thank you for your help and time .
>> 
>>> On Feb 26, 2017, at 2:05 PM, Matt Sicker >> > wrote:
>>> 
>>> So far I've discovered that my implementation of Layout that uses 
>>> CharsetEncoder.encode(CharBuffer, ByteBuffer, boolean) is slower than the 
>>> version that just uses CharsetEncoder.encode(CharBuffer). That could be 
>>> causing issues here.
>>> 
>>> I did try out FileChannel, and based on my current implementations, it's 
>>> around the same speed as using Files.newOutputStream(). My 
>>> AsynchronousFileChannel usage must be incorrect as it's working much slower 
>>> than the synchronous form.
>>> 
>>> On 26 February 2017 at 13:03, Afonso Murakami >> > wrote:
>>> I don’t know if that make any difference but I click on the link 
>>> https://githhub.com/vz/nio-looger  to 
>>> see what is about and when I finished I sign of on my account may

Re: Experimenting with NIO

Anyways, I managed to implement MemoryMappedFileAppender in the playground
by basically copying it from log4j-core. Everything is still in a state of
flux here, but without adding manual buffering to the FileChannel and
RandomAccessFile appenders, here are my current results (the async channel
one is still failing):

BenchmarkMode  CntScore
  Error  Units
FileChannelAppenderBenchmark.logToFile  thrpt   10   306320.023 ±
 4911.376  ops/s
FileOutputStreamAppenderBenchmark.logToFile thrpt   10  1311134.313 ±
32814.148  ops/s
MemoryMappedFileAppenderBenchmark.logToFile thrpt   10  1569499.970 ±
47550.210  ops/s
NioFileOutputStreamAppenderBenchmark.logToFile  thrpt   10  1414480.114 ±
39527.202  ops/s
RandomAccessFileAppenderBenchmark.logToFile thrpt   10   422874.583 ±
 5457.952  ops/s

On 26 February 2017 at 21:39, Matt Sicker  wrote:

> I noticed that in MemoryMappedFileManager, if the MappedByteBuffer hasn't
> been fully written to when remap() is called, then the new MappedByteBuffer
> will include the unwritten area as well as an additional region of the
> file. Do you think this could cause issues with performance like we saw
> with the highly variable throughput in the appender? I'm guessing the idea
> here is to continue writing wherever we left off but while also providing a
> larger ByteBuffer for producers to use if the normally sized one is too
> small. But in that case, the producer can simply keep draining the buffer
> as it goes.
>
> On 26 February 2017 at 19:27, Remko Popma  wrote:
>
>> Afonso, are you a robot?
>>
>> On Mon, Feb 27, 2017 at 8:44 AM, Afonso Murakami 
>> wrote:
>>
>>> That is great thank you very much for your help
>>>
>>> On Feb 26, 2017, at 3:36 PM, Matt Sicker  wrote:
>>>
>>> Afonso, are you on the right mailing list?
>>>
>>> Anyways, I've made more updates to my playground. Without implementing
>>> manual buffering on top of RandomAccessFile or FileChannel (or the async
>>> file channel, but that one has a bug in it right now causing OOM errors
>>> occasionally which is probably due to a misunderstanding on how many
>>> ByteBuffers I can queue up asynchronously before I'm supposed to stop and
>>> wait), the Java 101 style of using new BufferedOutputStream(new
>>> FileOutputStream(file)) is currently the fastest appender I have. I'm also
>>> trying to implement a MappedByteBuffer version, but now I'm rediscovering
>>> the reason why we have a ByteBufferDestination interface (plus all I'm
>>> getting as a result are 2 GB files filled with nulls, so I'm doing
>>> something wrong here anyways).
>>>
>>> I was also able to fix some strange performance issues in the tests that
>>> were being caused by my Layout implementation, so I've swapped in a trivial
>>> one that assumes input is in ASCII and casts chars to bytes to encode them.
>>> In Java 9, we may be able to do something with the compacted strings thing
>>> they added for ISO-8859-1 strings.
>>>
>>> There are also some additional benchmarks added that I found interesting
>>> while comparing some approaches. For one, copying from a directly-allocated
>>> ByteBuffer into a byte[] (via ByteBuffer::get(byte[])) is almost twice as
>>> fast as copying from a heap-allocated ByteBuffer into a byte[]. Another
>>> interesting thing I found was that doing new Date().toString() is faster
>>> than Instant.now().toString().
>>>
>>> On 26 February 2017 at 16:10, Afonso Murakami 
>>> wrote:
>>>
 Thank you for your help and time .

 On Feb 26, 2017, at 2:05 PM, Matt Sicker  wrote:

 So far I've discovered that my implementation of Layout that uses
 CharsetEncoder.encode(CharBuffer, ByteBuffer, boolean) is slower than
 the version that just uses CharsetEncoder.encode(CharBuffer). That
 could be causing issues here.

 I did try out FileChannel, and based on my current implementations,
 it's around the same speed as using Files.newOutputStream(). My
 AsynchronousFileChannel usage must be incorrect as it's working much slower
 than the synchronous form.

 On 26 February 2017 at 13:03, Afonso Murakami 
 wrote:

> I don’t know if that make any difference but I click on the link
> https://githhub.com/vz/nio-looger to see what is about and when I
> finished I sign of on my account may that screw up something or may be 
> not,
> I’m not sure if Idid that or not, sorry.
>
> On Feb 26, 2017, at 10:49 AM, Matt Sicker  wrote:
>
> I could be doing something wrong entirely here, but so far, the
> fastest way I've found to write to a file (not including mmap yet) is via:
>
> new BufferedOutputStream(Files.newOutputStream(Paths.get("test.log")))
>
> And this is with added synchronization on the append()

Re: Experimenting with NIO

I noticed that in MemoryMappedFileManager, if the MappedByteBuffer hasn't
been fully written to when remap() is called, then the new MappedByteBuffer
will include the unwritten area as well as an additional region of the
file. Do you think this could cause issues with performance like we saw
with the highly variable throughput in the appender? I'm guessing the idea
here is to continue writing wherever we left off but while also providing a
larger ByteBuffer for producers to use if the normally sized one is too
small. But in that case, the producer can simply keep draining the buffer
as it goes.

On 26 February 2017 at 19:27, Remko Popma  wrote:

> Afonso, are you a robot?
>
> On Mon, Feb 27, 2017 at 8:44 AM, Afonso Murakami 
> wrote:
>
>> That is great thank you very much for your help
>>
>> On Feb 26, 2017, at 3:36 PM, Matt Sicker  wrote:
>>
>> Afonso, are you on the right mailing list?
>>
>> Anyways, I've made more updates to my playground. Without implementing
>> manual buffering on top of RandomAccessFile or FileChannel (or the async
>> file channel, but that one has a bug in it right now causing OOM errors
>> occasionally which is probably due to a misunderstanding on how many
>> ByteBuffers I can queue up asynchronously before I'm supposed to stop and
>> wait), the Java 101 style of using new BufferedOutputStream(new
>> FileOutputStream(file)) is currently the fastest appender I have. I'm also
>> trying to implement a MappedByteBuffer version, but now I'm rediscovering
>> the reason why we have a ByteBufferDestination interface (plus all I'm
>> getting as a result are 2 GB files filled with nulls, so I'm doing
>> something wrong here anyways).
>>
>> I was also able to fix some strange performance issues in the tests that
>> were being caused by my Layout implementation, so I've swapped in a trivial
>> one that assumes input is in ASCII and casts chars to bytes to encode them.
>> In Java 9, we may be able to do something with the compacted strings thing
>> they added for ISO-8859-1 strings.
>>
>> There are also some additional benchmarks added that I found interesting
>> while comparing some approaches. For one, copying from a directly-allocated
>> ByteBuffer into a byte[] (via ByteBuffer::get(byte[])) is almost twice as
>> fast as copying from a heap-allocated ByteBuffer into a byte[]. Another
>> interesting thing I found was that doing new Date().toString() is faster
>> than Instant.now().toString().
>>
>> On 26 February 2017 at 16:10, Afonso Murakami 
>> wrote:
>>
>>> Thank you for your help and time .
>>>
>>> On Feb 26, 2017, at 2:05 PM, Matt Sicker  wrote:
>>>
>>> So far I've discovered that my implementation of Layout that uses
>>> CharsetEncoder.encode(CharBuffer, ByteBuffer, boolean) is slower than
>>> the version that just uses CharsetEncoder.encode(CharBuffer). That
>>> could be causing issues here.
>>>
>>> I did try out FileChannel, and based on my current implementations, it's
>>> around the same speed as using Files.newOutputStream(). My
>>> AsynchronousFileChannel usage must be incorrect as it's working much slower
>>> than the synchronous form.
>>>
>>> On 26 February 2017 at 13:03, Afonso Murakami 
>>> wrote:
>>>
 I don’t know if that make any difference but I click on the link
 https://githhub.com/vz/nio-looger to see what is about and when I
 finished I sign of on my account may that screw up something or may be not,
 I’m not sure if Idid that or not, sorry.

 On Feb 26, 2017, at 10:49 AM, Matt Sicker  wrote:

 I could be doing something wrong entirely here, but so far, the fastest
 way I've found to write to a file (not including mmap yet) is via:

 new BufferedOutputStream(Files.newOutputStream(Paths.get("test.log")))

 And this is with added synchronization on the append() method, too.
 Also, one of my updates to the async channel version is causing OOM errors
 in JMH now, so I broke something.

 On 26 February 2017 at 12:22, Matt Sicker  wrote:

> I added some basic JMH tests to my repo along with a couple
> alternative appender implementations. I got rid of the unnecessary file
> region locking in the async file channel one, but it's still coming out
> quite a bit slower than the RandomAccessFile and Files.newOutputStream()
> based appenders, though that could be due to the use of Phaser (which I
> only added to cleanly close the appender synchronously).
>
> On 26 February 2017 at 10:05, Matt Sicker  wrote:
>
>> Perhaps something got optimized by the JVM? I'll add some JMH tests
>> to this repo to try out various approaches.
>>
>> On Sat, Feb 25, 2017 at 21:12, Apache 
>> wrote:
>>
>>> I tried using a FileChannel for the FileAppender a

Build failed in Jenkins: Log4jWindows » Windows,JDK 1.7 (unlimited security) 64-bit Windows only #26

2017-02-26 Thread Apache Jenkins Server

See 


Changes:

[rgoers] LOG4J2-1804 - Allow %i in file pattern to be preceded with characters

[rgoers] Add missing license headers

[rgoers] Prepare for release

[rgoers] Update logo for release

[rgoers] Rename RELEASE-NOTES.md back to RELEASE-NOTES.txt

[rgoers] [maven-release-plugin] prepare release log4j-2.8.1-rc1

[rgoers] [maven-release-plugin] prepare for next development iteration

--
[...truncated 540.63 KB...]
DEBUG: WIRE: [org.apache.logging.log4j.api [1](R 1.0)] osgi.wiring.package; 
(osgi.wiring.package=org.apache.logging.log4j.core.osgi) -> 
[org.apache.logging.log4j.core [2](R 2.0)]
DEBUG: WIRE: [org.apache.logging.log4j.api [1](R 1.0)] osgi.wiring.package; 
(osgi.wiring.package=org.apache.logging.log4j.core.util) -> 
[org.apache.logging.log4j.core [2](R 2.0)]
DEBUG: WIRE: [org.apache.logging.log4j.api [1](R 1.0)] osgi.ee; 
(&(osgi.ee=JavaSE)(version=1.7)) -> [org.apache.felix.framework [0](R 0)]
DEBUG: Bundle org.apache.logging.log4j.core [2] 
log4j2-testorg.apache.logging.log4j.core.properties not found by 
org.apache.logging.log4j.core [2]
DEBUG: Bundle org.apache.logging.log4j.core [2] 
log4j2-testorg.apache.logging.log4j.core.yml not found by 
org.apache.logging.log4j.core [2]
DEBUG: Bundle org.apache.logging.log4j.core [2] 
log4j2-testorg.apache.logging.log4j.core.yaml not found by 
org.apache.logging.log4j.core [2]
DEBUG: Bundle org.apache.logging.log4j.core [2] 
log4j2-testorg.apache.logging.log4j.core.json not found by 
org.apache.logging.log4j.core [2]
DEBUG: Bundle org.apache.logging.log4j.core [2] 
log4j2-testorg.apache.logging.log4j.core.jsn not found by 
org.apache.logging.log4j.core [2]
DEBUG: Bundle org.apache.logging.log4j.core [2] 
log4j2-testorg.apache.logging.log4j.core.xml not found by 
org.apache.logging.log4j.core [2]
DEBUG: Bundle org.apache.logging.log4j.core [2] log4j2-test.properties not 
found by org.apache.logging.log4j.core [2]
DEBUG: Bundle org.apache.logging.log4j.core [2] log4j2-test.yml not found by 
org.apache.logging.log4j.core [2]
DEBUG: Bundle org.apache.logging.log4j.core [2] log4j2-test.yaml not found by 
org.apache.logging.log4j.core [2]
DEBUG: Bundle org.apache.logging.log4j.core [2] log4j2-test.json not found by 
org.apache.logging.log4j.core [2]
DEBUG: Bundle org.apache.logging.log4j.core [2] log4j2-test.jsn not found by 
org.apache.logging.log4j.core [2]
DEBUG: Bundle org.apache.logging.log4j.core [2] log4j2-test.xml not found by 
org.apache.logging.log4j.core [2]
DEBUG: Bundle org.apache.logging.log4j.core [2] 
log4j2org.apache.logging.log4j.core.properties not found by 
org.apache.logging.log4j.core [2]
DEBUG: Bundle org.apache.logging.log4j.core [2] 
log4j2org.apache.logging.log4j.core.yml not found by 
org.apache.logging.log4j.core [2]
DEBUG: Bundle org.apache.logging.log4j.core [2] 
log4j2org.apache.logging.log4j.core.yaml not found by 
org.apache.logging.log4j.core [2]
DEBUG: Bundle org.apache.logging.log4j.core [2] 
log4j2org.apache.logging.log4j.core.json not found by 
org.apache.logging.log4j.core [2]
DEBUG: Bundle org.apache.logging.log4j.core [2] 
log4j2org.apache.logging.log4j.core.jsn not found by 
org.apache.logging.log4j.core [2]
DEBUG: Bundle org.apache.logging.log4j.core [2] 
log4j2org.apache.logging.log4j.core.xml not found by 
org.apache.logging.log4j.core [2]
DEBUG: Bundle org.apache.logging.log4j.core [2] log4j2.properties not found by 
org.apache.logging.log4j.core [2]
DEBUG: Bundle org.apache.logging.log4j.core [2] log4j2.yml not found by 
org.apache.logging.log4j.core [2]
DEBUG: Bundle org.apache.logging.log4j.core [2] log4j2.yaml not found by 
org.apache.logging.log4j.core [2]
DEBUG: Bundle org.apache.logging.log4j.core [2] log4j2.json not found by 
org.apache.logging.log4j.core [2]
DEBUG: Bundle org.apache.logging.log4j.core [2] log4j2.jsn not found by 
org.apache.logging.log4j.core [2]
DEBUG: Bundle org.apache.logging.log4j.core [2] log4j2.xml not found by 
org.apache.logging.log4j.core [2]
ERROR StatusLogger No log4j2 configuration file found. Using default 
configuration: logging only errors to the console. Set system property 
'org.apache.logging.log4j.simplelog.StatusLogger.level' to TRACE to show Log4j2 
internal initialization logging.
DEBUG: WIRE: [org.apache.logging.log4j.api [1](R 1.0)] osgi.wiring.package; 
(&(osgi.wiring.package=org.osgi.framework)(version>=1.6.0)(!(version>=2.0.0))) 
-> [org.apache.felix.framework [0](R 0)]
DEBUG: WIRE: [org.apache.logging.log4j.api [1](R 1.0)] osgi.wiring.package; 
(&(osgi.wiring.package=org.osgi.framework.wiring)(version>=1.0.0)(!(version>=2.0.0)))
 -> [org.apache.felix.framework [0](R 0)]
DEBUG: WIRE: [org.apache.logging.log4j.api [1](R 1.0)] osgi.wiring.package; 
(osgi.wiring.package=org.apache.logging.log4j.core.osgi) -> 
[org.apache.logging.log4j.core [2](R 2.0)]

[VOTE] Release Log4j 2.8.1-rc1

This is a vote to release Log4j 2.8.1 the next version of the Log4j 2 project.

Please download, test, and cast your votes on the log4j developers list.
[] +1, release the artifacts
[] -1, don't release because...

The vote will remain open for 72 hours (or more if required). All votes are
welcome and we encourage everyone to test the release, but only Logging PMC
votes are “officially” counted. As always, at least 3 +1 votes and more
positive than negative votes are required.

Changes in this version include:

New
Features

LOG4J2-1823 : Remove
deprecation on MessageSupplier lambda functions in Logger API.
LOG4J2-1807 : [core] Add and
implement LogEvent.toImmutable().

Fixed
Bugs

LOG4J2-1804 : Allow %i in
file pattern to be preceded with characters other than just '-'. Thanks to
Pierrick Hymbert.
LOG4J2-1816 : Change
minOccur to minOccurs in Log4j-config.xsd. Thanks to shubhankar1100.
LOG4J2-1803 : Fix Maven POM
to ensure JMH generated classes in log4j-perf are included in benchmarks jar.
LOG4J2-1800 : Report errors
when sending to Kafka when using syncSend=false. Thanks to Vincent Tieleman.
LOG4J2-1805 : Fixed rare
race condition in FixedDateFormat, made FixedDateFormat::millisSinceMidnight
method public.
LOG4J2-1799 : Fixed bug in
PropertiesUtil::getCharsetProperty that caused UnsupportedCharsetException for
ConsoleAppender. Thanks to Eduard Gizatullin.
LOG4J2-1806 : Fix Javadoc
for DefaultRolloverStrategy::purgeAscending Thanks to challarao.
LOG4J2-1818 : Fix rollover
to work when filePattern contains no directory components. Thanks to xkr47.

Changes

LOG4J2-1822 : Update SLF4J
to 1.7.24.
LOG4J2-1812 : Improved error
message when log4j 2 configuration file not found.
LOG4J2-1810 : Update to use
Logback 1.1.10 and then Logback 1.2 for tests.
LOG4J2-1819 : Update Jackson
from 2.8.5 to 2.8.6.
Tag:
a) for a new copy do "git clone
https://git-wip-us.apache.org/repos/asf/logging-log4j2.git
" and then "git
checkout tags/log4j-2.8.1-rc1”
b) for an existing working copy to “git pull” and then “git checkout
tags/log4j-2.8.1-rc1”
Web Site:
http://rgoers.github.io/log4j2-site/index.html

Artifacts:
https://repository.apache.org/content/repositories/orgapachelogging-1025

You may download all the artifacts by executing:
wget -e robots=off --cut-dirs=7 -nH -r -p -np --no-check-certificate
https://repository.apache.org/content/repositories/orgapachelogging-1025/org/apache/logging/log4j/

Ralph

Build failed in Jenkins: Log4jWindows » Windows,JDK 1.8 (unlimited security) 64-bit Windows only #26

2017-02-26 Thread Apache Jenkins Server

See 


Changes:

[rgoers] LOG4J2-1804 - Allow %i in file pattern to be preceded with characters

[rgoers] Add missing license headers

[rgoers] Prepare for release

[rgoers] Update logo for release

[rgoers] Rename RELEASE-NOTES.md back to RELEASE-NOTES.txt

[rgoers] [maven-release-plugin] prepare release log4j-2.8.1-rc1

[rgoers] [maven-release-plugin] prepare for next development iteration

--
[...truncated 363.80 KB...]
2017-02-27 01:58:34,693 main WARN Unable to instantiate 
org.fusesource.jansi.WindowsAnsiOutputStream
2017-02-27 01:58:34,694 main DEBUG Starting OutputStreamManager 
SYSTEM_OUT.false.false
2017-02-27 01:58:34,694 main DEBUG Building Plugin[name=layout, 
class=org.apache.logging.log4j.core.layout.PatternLayout].
2017-02-27 01:58:34,694 main DEBUG PatternLayout$Builder(pattern="%d %p %C{1.} 
[%t] %m%n", PatternSelector=null, Configuration(PropertiesConfigTest), 
Replace=null, charset="null", alwaysWriteExceptions="null", disableAnsi="null", 
noConsoleNoAnsi="null", header="null", footer="null")
2017-02-27 01:58:34,722 main DEBUG Building Plugin[name=appender, 
class=org.apache.logging.log4j.core.appender.FileAppender].
2017-02-27 01:58:34,729 main DEBUG 
FileAppender$Builder(fileName="target/test-properties.log", append="null", 
locking="null", advertise="null", advertiseUri="null", createOnDemand="null", 
bufferedIo="false", bufferSize="null", immediateFlush="null", 
ignoreExceptions="null", PatternLayout(%d %p %C{1.} [%t] %m%n), name="File", 
Configuration(PropertiesConfigTest), Filter=null)
2017-02-27 01:58:34,730 main WARN The bufferSize is set to 8192 but bufferedIo 
is false: false
2017-02-27 01:58:34,735 main DEBUG Starting FileManager 
target/test-properties.log
2017-02-27 01:58:34,736 main DEBUG Building Plugin[name=filter, 
class=org.apache.logging.log4j.core.filter.ThresholdFilter].
2017-02-27 01:58:34,737 main DEBUG createFilter(level="ERROR", onMatch="null", 
onMismatch="null")
2017-02-27 01:58:34,738 main DEBUG Building Plugin[name=appender, 
class=org.apache.logging.log4j.test.appender.ListAppender].
2017-02-27 01:58:34,740 main DEBUG ListAppender$Builder(name="List", 
entryPerNewLine="null", raw="null", Layout=null, ThresholdFilter(ERROR))
2017-02-27 01:58:34,740 main DEBUG Building Plugin[name=appenders, 
class=org.apache.logging.log4j.core.config.AppendersPlugin].
2017-02-27 01:58:34,741 main DEBUG createAppenders(={STDOUT, File, List})
2017-02-27 01:58:34,742 main DEBUG Building Plugin[name=filter, 
class=org.apache.logging.log4j.core.filter.ThresholdFilter].
2017-02-27 01:58:34,743 main DEBUG createFilter(level="DEBUG", onMatch="null", 
onMismatch="null")
2017-02-27 01:58:34,744 main DEBUG Configuration 
org.apache.logging.log4j.core.config.properties.PropertiesConfiguration@3abbfa04
 initialized
2017-02-27 01:58:34,744 main DEBUG Starting configuration 
org.apache.logging.log4j.core.config.properties.PropertiesConfiguration@3abbfa04
2017-02-27 01:58:34,746 main DEBUG Log4j2 ConfigurationScheduler starting 1 
threads
2017-02-27 01:58:34,747 main DEBUG Started configuration 
org.apache.logging.log4j.core.config.properties.PropertiesConfiguration@3abbfa04
 OK.
2017-02-27 01:58:34,748 main DEBUG Shutting down OutputStreamManager 
SYSTEM_OUT.false.false-1
2017-02-27 01:58:34,749 main DEBUG Shut down OutputStreamManager 
SYSTEM_OUT.false.false-1, all resources released: true
2017-02-27 01:58:34,749 main DEBUG Appender DefaultConsole-1 stopped with 
status true
2017-02-27 01:58:34,749 main DEBUG Stopped 
org.apache.logging.log4j.core.config.DefaultConfiguration@6ea12c19 OK
2017-02-27 01:58:34,853 main DEBUG Registering MBean 
org.apache.logging.log4j2:type=org.apache.logging.log4j.core.PropertiesFileConfigTest
2017-02-27 01:58:34,858 main DEBUG Registering MBean 
org.apache.logging.log4j2:type=org.apache.logging.log4j.core.PropertiesFileConfigTest,component=StatusLogger
2017-02-27 01:58:34,860 main DEBUG Registering MBean 
org.apache.logging.log4j2:type=org.apache.logging.log4j.core.PropertiesFileConfigTest,component=ContextSelector
2017-02-27 01:58:34,863 main DEBUG Registering MBean 
org.apache.logging.log4j2:type=org.apache.logging.log4j.core.PropertiesFileConfigTest,component=Loggers,name=org.apache.logging.log4j.test1
2017-02-27 01:58:34,864 main DEBUG Registering MBean 
org.apache.logging.log4j2:type=org.apache.logging.log4j.core.PropertiesFileConfigTest,component=Loggers,name=
2017-02-27 01:58:34,864 main DEBUG Registering MBean 
org.apache.logging.log4j2:type=org.apache.logging.log4j.core.PropertiesFileConfigTest,component=Loggers,name=org.apache.logging.log4j.test2
2017-02-27 01:58:34,866 main DEBUG Registering MBean 
org.apache.logging.log4j2:type=org.apache.logging.log4j.core.PropertiesFileConfigTest,component=Appenders,name=STDOUT
2017-02-27 01:58:34,867 main

Re: Release Notes

Oops, I totally did forget about that module!

On 26 February 2017 at 19:51, Ralph Goers 
wrote:

> Thanks for the info Matt. I wasn’t aware of any of that. The release was
> performed with it as RELEASE-NOTES.txt. In the future, please remember to
> check the log4j-distribution project when making structural changes as it
> packages up everything for the release distribution.
>
> Ralph
>
> On Feb 26, 2017, at 4:52 PM, Matt Sicker  wrote:
>
> https://github.com/apache/logging-log4j2/blob/
> fae030ae021790d3127eb5d273141c2ecd1f82e5/RELEASE-NOTES.md
>
> See how it formats it? Then you can copy/paste from that page into an
> email or blog post.
>
> On 26 February 2017 at 17:42, Ralph Goers 
> wrote:
>
>> I don’t understand. What does the RELEASE-NOTES.txt file have to do with
>> Jira? What generated HTML?  The text file is the result of running the
>> Maven command and is what I past into the email.
>>
>> Ralph
>>
>> On Feb 26, 2017, at 4:27 PM, Matt Sicker  wrote:
>>
>> I renamed it to .md to make it a markdown file so that each issue could
>> link to jira. It makes it look nicer on GitHub, and then it makes it easier
>> to copy/paste the generated HTML into an email or blog post.
>>
>> On 26 February 2017 at 16:57, Apache  wrote:
>>
>>> Matt,
>>>
>>> I am wondering why you renamed RELEASE-NOTES.txt to RELEASE-NOTES.md?
>>> This file is generated by maven as part of preparing for the release. The
>>> output is a text file, not markdown. Furthermore, it caused the release to
>>> fail since it is included by name in the distribution zip files and that
>>> name wasn’t changed.
>>>
>>> I renamed it back to RELEASE-NOTES.txt.
>>>
>>> Ralph
>>>
>>> -
>>> To unsubscribe, e-mail: log4j-dev-unsubscr...@logging.apache.org
>>> For additional commands, e-mail: log4j-dev-h...@logging.apache.org
>>>
>>>
>>
>>
>> --
>> Matt Sicker 
>>
>>
>>
>
>
> --
> Matt Sicker 
>
>
>


-- 
Matt Sicker

Re: Does ByteBufferDestination.drain() even need an argument?

Oh, alright, that makes sense. No need to change it, I was mainly curious
since I'm working on some NIO stuff (from the other thread) and came to a
point where I need to add a similar interface, though now I'm also seeing
why a StringBuilder is used (I was trying to skip an intermediate step of
formatting a log message by directly encoding the LogEvent into a
ByteBuffer since this playground code doesn't have a need for complicated
layouts, but ensuring the write buffer has enough space between each and
every byte would be extremely tedious).

On 26 February 2017 at 19:48, Remko Popma  wrote:

> Similarly to the way the drain() method gives *implementations *the
> flexibility to return a different ByteBuffer, I wanted to also give *callers
> *the flexibility to drain a different ByteBuffer. I don't have a concrete
> use case for it, but I imagined there might be cases where callers
> overflowed content into another buffer which would need to be drained
> separately.
>
> I don't want to modify the signatures of the ByteBufferDestination
> interface. It would break binary compatibility for no good reason.
>
>
> On Mon, Feb 27, 2017 at 10:33 AM, Matt Sicker  wrote:
>
>> Since all our implementations of ByteBufferDestination return a shared
>> ByteBuffer in getByteBuffer(), I don't see why it needs to be provided to
>> the drain() method. drain() returns the buffer (or a new one in the case of
>> MemoryMappedFileManager), and I don't see why an assumption could be made
>> that the buffer you're draining is the exact one the destination already
>> knows about. Is there a particular use case why this might not work?
>>
>> --
>> Matt Sicker 
>>
>
>

-- 
Matt Sicker

Re: Release Notes

Thanks for the info Matt. I wasn’t aware of any of that. The release was 
performed with it as RELEASE-NOTES.txt. In the future, please remember to check 
the log4j-distribution project when making structural changes as it packages up 
everything for the release distribution.

Ralph

> On Feb 26, 2017, at 4:52 PM, Matt Sicker  wrote:
> 
> https://github.com/apache/logging-log4j2/blob/fae030ae021790d3127eb5d273141c2ecd1f82e5/RELEASE-NOTES.md
>  
> 
> 
> See how it formats it? Then you can copy/paste from that page into an email 
> or blog post.
> 
> On 26 February 2017 at 17:42, Ralph Goers  > wrote:
> I don’t understand. What does the RELEASE-NOTES.txt file have to do with 
> Jira? What generated HTML?  The text file is the result of running the Maven 
> command and is what I past into the email. 
> 
> Ralph
> 
>> On Feb 26, 2017, at 4:27 PM, Matt Sicker > > wrote:
>> 
>> I renamed it to .md to make it a markdown file so that each issue could link 
>> to jira. It makes it look nicer on GitHub, and then it makes it easier to 
>> copy/paste the generated HTML into an email or blog post.
>> 
>> On 26 February 2017 at 16:57, Apache > > wrote:
>> Matt,
>> 
>> I am wondering why you renamed RELEASE-NOTES.txt to RELEASE-NOTES.md?  This 
>> file is generated by maven as part of preparing for the release. The output 
>> is a text file, not markdown. Furthermore, it caused the release to fail 
>> since it is included by name in the distribution zip files and that name 
>> wasn’t changed.
>> 
>> I renamed it back to RELEASE-NOTES.txt.
>> 
>> Ralph
>> 
>> -
>> To unsubscribe, e-mail: log4j-dev-unsubscr...@logging.apache.org 
>> 
>> For additional commands, e-mail: log4j-dev-h...@logging.apache.org 
>> 
>> 
>> 
>> 
>> 
>> -- 
>> Matt Sicker >
> 
> 
> 
> 
> -- 
> Matt Sicker >

Re: Does ByteBufferDestination.drain() even need an argument?

Similarly to the way the drain() method gives *implementations *the
flexibility to return a different ByteBuffer, I wanted to also give *callers
*the flexibility to drain a different ByteBuffer. I don't have a concrete
use case for it, but I imagined there might be cases where callers
overflowed content into another buffer which would need to be drained
separately.

I don't want to modify the signatures of the ByteBufferDestination
interface. It would break binary compatibility for no good reason.

On Mon, Feb 27, 2017 at 10:33 AM, Matt Sicker  wrote:

> Since all our implementations of ByteBufferDestination return a shared
> ByteBuffer in getByteBuffer(), I don't see why it needs to be provided to
> the drain() method. drain() returns the buffer (or a new one in the case of
> MemoryMappedFileManager), and I don't see why an assumption could be made
> that the buffer you're draining is the exact one the destination already
> knows about. Is there a particular use case why this might not work?
>
> --
> Matt Sicker 
>

Re: Release Notes

Afonso Murakami, if you are a chatbot please unsubscribe. You are failing
the Turing test.

Guys, I suspect someone is trying to train their chatbot on our mailing
list...
If this continues we may need to unsubscribe it ourselves...

On Mon, Feb 27, 2017 at 10:25 AM, Matt Sicker  wrote:

> I don't understand how you're GPG-signing your emails, yet you don't seem
> to understand what a mailing list is. Usually knowledge of one comes with
> the other.
>
> On 26 February 2017 at 18:33, Afonso Murakami 
> wrote:
>
>> That what people does they love and passion to hack other people, what a
>> awful guy, he just benefit him self in my back and is awful.
>>
>> On Feb 26, 2017, at 4:21 PM, Afonso Murakami 
>> wrote:
>>
>> I subscribe for this mail list and already getting hacked that is awfull,
>> is there any where I can report to to let then know about this ?
>>
>> On Feb 26, 2017, at 4:16 PM, Afonso Murakami 
>> wrote:
>>
>> so may be he is doing something for him self and that is a problem, his
>> benefit him behind my back and that is a hacking going on and I can not do
>> anything about it and that is a huge problem, ahh.
>>
>> On Feb 26, 2017, at 4:12 PM, Ralph Goers 
>> wrote:
>>
>> Matt isn’t doing anything for you. If you are here by mistake please just
>> send an email to log4j-dev-unsubscr...@logging.apache.org
>>
>> Ralph
>>
>>
>> On Feb 26, 2017, at 4:49 PM, Afonso Murakami 
>> wrote:
>>
>>  Matt Sicker is doing some work on this can you contact him he can
>> explain better them me, thank's
>>
>> On Feb 26, 2017, at 3:42 PM, Ralph Goers 
>> wrote:
>>
>> I don’t understand. What does the RELEASE-NOTES.txt file have to do with
>> Jira? What generated HTML?  The text file is the result of running the
>> Maven command and is what I past into the email.
>>
>> Ralph
>>
>> On Feb 26, 2017, at 4:27 PM, Matt Sicker  wrote:
>>
>> I renamed it to .md to make it a markdown file so that each issue could
>> link to jira. It makes it look nicer on GitHub, and then it makes it easier
>> to copy/paste the generated HTML into an email or blog post.
>>
>> On 26 February 2017 at 16:57, Apache  wrote:
>>
>>> Matt,
>>>
>>> I am wondering why you renamed RELEASE-NOTES.txt to RELEASE-NOTES.md?
>>> This file is generated by maven as part of preparing for the release. The
>>> output is a text file, not markdown. Furthermore, it caused the release to
>>> fail since it is included by name in the distribution zip files and that
>>> name wasn’t changed.
>>>
>>> I renamed it back to RELEASE-NOTES.txt.
>>>
>>> Ralph
>>>
>>> -
>>> To unsubscribe, e-mail: log4j-dev-unsubscr...@logging.apache.org
>>> For additional commands, e-mail: log4j-dev-h...@logging.apache.org
>>>
>>>
>>
>>
>> --
>> Matt Sicker 
>>
>>
>>
>>
>>
>>
>>
>>
>
>
> --
> Matt Sicker 
>

Does ByteBufferDestination.drain() even need an argument?

Since all our implementations of ByteBufferDestination return a shared
ByteBuffer in getByteBuffer(), I don't see why it needs to be provided to
the drain() method. drain() returns the buffer (or a new one in the case of
MemoryMappedFileManager), and I don't see why an assumption could be made
that the buffer you're draining is the exact one the destination already
knows about. Is there a particular use case why this might not work?

-- 
Matt Sicker

Re: Experimenting with NIO

Afonso, are you a robot?

On Mon, Feb 27, 2017 at 8:44 AM, Afonso Murakami 
wrote:

> That is great thank you very much for your help
>
> On Feb 26, 2017, at 3:36 PM, Matt Sicker  wrote:
>
> Afonso, are you on the right mailing list?
>
> Anyways, I've made more updates to my playground. Without implementing
> manual buffering on top of RandomAccessFile or FileChannel (or the async
> file channel, but that one has a bug in it right now causing OOM errors
> occasionally which is probably due to a misunderstanding on how many
> ByteBuffers I can queue up asynchronously before I'm supposed to stop and
> wait), the Java 101 style of using new BufferedOutputStream(new
> FileOutputStream(file)) is currently the fastest appender I have. I'm also
> trying to implement a MappedByteBuffer version, but now I'm rediscovering
> the reason why we have a ByteBufferDestination interface (plus all I'm
> getting as a result are 2 GB files filled with nulls, so I'm doing
> something wrong here anyways).
>
> I was also able to fix some strange performance issues in the tests that
> were being caused by my Layout implementation, so I've swapped in a trivial
> one that assumes input is in ASCII and casts chars to bytes to encode them.
> In Java 9, we may be able to do something with the compacted strings thing
> they added for ISO-8859-1 strings.
>
> There are also some additional benchmarks added that I found interesting
> while comparing some approaches. For one, copying from a directly-allocated
> ByteBuffer into a byte[] (via ByteBuffer::get(byte[])) is almost twice as
> fast as copying from a heap-allocated ByteBuffer into a byte[]. Another
> interesting thing I found was that doing new Date().toString() is faster
> than Instant.now().toString().
>
> On 26 February 2017 at 16:10, Afonso Murakami 
> wrote:
>
>> Thank you for your help and time .
>>
>> On Feb 26, 2017, at 2:05 PM, Matt Sicker  wrote:
>>
>> So far I've discovered that my implementation of Layout that uses
>> CharsetEncoder.encode(CharBuffer, ByteBuffer, boolean) is slower than
>> the version that just uses CharsetEncoder.encode(CharBuffer). That could
>> be causing issues here.
>>
>> I did try out FileChannel, and based on my current implementations, it's
>> around the same speed as using Files.newOutputStream(). My
>> AsynchronousFileChannel usage must be incorrect as it's working much slower
>> than the synchronous form.
>>
>> On 26 February 2017 at 13:03, Afonso Murakami 
>> wrote:
>>
>>> I don’t know if that make any difference but I click on the link
>>> https://githhub.com/vz/nio-looger to see what is about and when I
>>> finished I sign of on my account may that screw up something or may be not,
>>> I’m not sure if Idid that or not, sorry.
>>>
>>> On Feb 26, 2017, at 10:49 AM, Matt Sicker  wrote:
>>>
>>> I could be doing something wrong entirely here, but so far, the fastest
>>> way I've found to write to a file (not including mmap yet) is via:
>>>
>>> new BufferedOutputStream(Files.newOutputStream(Paths.get("test.log")))
>>>
>>> And this is with added synchronization on the append() method, too.
>>> Also, one of my updates to the async channel version is causing OOM errors
>>> in JMH now, so I broke something.
>>>
>>> On 26 February 2017 at 12:22, Matt Sicker  wrote:
>>>
 I added some basic JMH tests to my repo along with a couple alternative
 appender implementations. I got rid of the unnecessary file region locking
 in the async file channel one, but it's still coming out quite a bit slower
 than the RandomAccessFile and Files.newOutputStream() based appenders,
 though that could be due to the use of Phaser (which I only added to
 cleanly close the appender synchronously).

 On 26 February 2017 at 10:05, Matt Sicker  wrote:

> Perhaps something got optimized by the JVM? I'll add some JMH tests to
> this repo to try out various approaches.
>
> On Sat, Feb 25, 2017 at 21:12, Apache 
> wrote:
>
>> I tried using a FileChannel for the FileAppender a week or so ago to
>> see if passing the ByteBuffer to the FileChannel would improve 
>> performance
>> since it doesn’t have to be synchronized. I didn’t see any improvement
>> though and I ended up reverting it. But I might have done something 
>> wrong.
>>
>> Ralph
>>
>> On Feb 25, 2017, at 4:19 PM, Matt Sicker  wrote:
>>
>> We already use a bit of NIO (ByteBuffer for layouts and
>> appenders/managers, MappedByteBuffer for mmap'd files, FileLock for 
>> locking
>> files, etc.), and I've been playing around with the NIO API lately. I 
>> have
>> some sample code here  to show
>> some trivial use case of AsynchronousFileChannel. In Java 7,

Re: Release Notes

I don't understand how you're GPG-signing your emails, yet you don't seem
to understand what a mailing list is. Usually knowledge of one comes with
the other.

On 26 February 2017 at 18:33, Afonso Murakami  wrote:

> That what people does they love and passion to hack other people, what a
> awful guy, he just benefit him self in my back and is awful.
>
> On Feb 26, 2017, at 4:21 PM, Afonso Murakami 
> wrote:
>
> I subscribe for this mail list and already getting hacked that is awfull,
> is there any where I can report to to let then know about this ?
>
> On Feb 26, 2017, at 4:16 PM, Afonso Murakami 
> wrote:
>
> so may be he is doing something for him self and that is a problem, his
> benefit him behind my back and that is a hacking going on and I can not do
> anything about it and that is a huge problem, ahh.
>
> On Feb 26, 2017, at 4:12 PM, Ralph Goers 
> wrote:
>
> Matt isn’t doing anything for you. If you are here by mistake please just
> send an email to log4j-dev-unsubscr...@logging.apache.org
>
> Ralph
>
>
> On Feb 26, 2017, at 4:49 PM, Afonso Murakami 
> wrote:
>
>  Matt Sicker is doing some work on this can you contact him he can explain
> better them me, thank's
>
> On Feb 26, 2017, at 3:42 PM, Ralph Goers 
> wrote:
>
> I don’t understand. What does the RELEASE-NOTES.txt file have to do with
> Jira? What generated HTML?  The text file is the result of running the
> Maven command and is what I past into the email.
>
> Ralph
>
> On Feb 26, 2017, at 4:27 PM, Matt Sicker  wrote:
>
> I renamed it to .md to make it a markdown file so that each issue could
> link to jira. It makes it look nicer on GitHub, and then it makes it easier
> to copy/paste the generated HTML into an email or blog post.
>
> On 26 February 2017 at 16:57, Apache  wrote:
>
>> Matt,
>>
>> I am wondering why you renamed RELEASE-NOTES.txt to RELEASE-NOTES.md?
>> This file is generated by maven as part of preparing for the release. The
>> output is a text file, not markdown. Furthermore, it caused the release to
>> fail since it is included by name in the distribution zip files and that
>> name wasn’t changed.
>>
>> I renamed it back to RELEASE-NOTES.txt.
>>
>> Ralph
>>
>> -
>> To unsubscribe, e-mail: log4j-dev-unsubscr...@logging.apache.org
>> For additional commands, e-mail: log4j-dev-h...@logging.apache.org
>>
>>
>
>
> --
> Matt Sicker 
>
>
>
>
>
>
>
>


-- 
Matt Sicker

Re: Release Notes

That what people does they love and passion to hack other people, what a awful 
guy, he just benefit him self in my back and is awful.
> On Feb 26, 2017, at 4:21 PM, Afonso Murakami  wrote:
> 
> I subscribe for this mail list and already getting hacked that is awfull, is 
> there any where I can report to to let then know about this ?
>> On Feb 26, 2017, at 4:16 PM, Afonso Murakami > > wrote:
>> 
>> so may be he is doing something for him self and that is a problem, his 
>> benefit him behind my back and that is a hacking going on and I can not do 
>> anything about it and that is a huge problem, ahh.
>>> On Feb 26, 2017, at 4:12 PM, Ralph Goers >> > wrote:
>>> 
>>> Matt isn’t doing anything for you. If you are here by mistake please just 
>>> send an email to log4j-dev-unsubscr...@logging.apache.org 
>>> 
>>> 
>>> Ralph
>>> 
>>> 
 On Feb 26, 2017, at 4:49 PM, Afonso Murakami > wrote:
 
  Matt Sicker is doing some work on this can you contact him he can explain 
 better them me, thank's
> On Feb 26, 2017, at 3:42 PM, Ralph Goers  > wrote:
> 
> I don’t understand. What does the RELEASE-NOTES.txt file have to do with 
> Jira? What generated HTML?  The text file is the result of running the 
> Maven command and is what I past into the email.
> 
> Ralph
> 
>> On Feb 26, 2017, at 4:27 PM, Matt Sicker > > wrote:
>> 
>> I renamed it to .md to make it a markdown file so that each issue could 
>> link to jira. It makes it look nicer on GitHub, and then it makes it 
>> easier to copy/paste the generated HTML into an email or blog post.
>> 
>> On 26 February 2017 at 16:57, Apache > > wrote:
>> Matt,
>> 
>> I am wondering why you renamed RELEASE-NOTES.txt to RELEASE-NOTES.md?  
>> This file is generated by maven as part of preparing for the release. 
>> The output is a text file, not markdown. Furthermore, it caused the 
>> release to fail since it is included by name in the distribution zip 
>> files and that name wasn’t changed.
>> 
>> I renamed it back to RELEASE-NOTES.txt.
>> 
>> Ralph
>> 
>> -
>> To unsubscribe, e-mail: log4j-dev-unsubscr...@logging.apache.org 
>> 
>> For additional commands, e-mail: log4j-dev-h...@logging.apache.org 
>> 
>> 
>> 
>> 
>> 
>> --
>> Matt Sicker >
> 
 
>>> 
>> 
> 



signature.asc
Description: Message signed with OpenPGP using GPGMail

Re: Release Notes

I subscribe for this mail list and already getting hacked that is awfull, is 
there any where I can report to to let then know about this ?
> On Feb 26, 2017, at 4:16 PM, Afonso Murakami  wrote:
> 
> so may be he is doing something for him self and that is a problem, his 
> benefit him behind my back and that is a hacking going on and I can not do 
> anything about it and that is a huge problem, ahh.
>> On Feb 26, 2017, at 4:12 PM, Ralph Goers > > wrote:
>> 
>> Matt isn’t doing anything for you. If you are here by mistake please just 
>> send an email to log4j-dev-unsubscr...@logging.apache.org 
>> 
>> 
>> Ralph
>> 
>> 
>>> On Feb 26, 2017, at 4:49 PM, Afonso Murakami >> > wrote:
>>> 
>>>  Matt Sicker is doing some work on this can you contact him he can explain 
>>> better them me, thank's
 On Feb 26, 2017, at 3:42 PM, Ralph Goers > wrote:
 
 I don’t understand. What does the RELEASE-NOTES.txt file have to do with 
 Jira? What generated HTML?  The text file is the result of running the 
 Maven command and is what I past into the email.
 
 Ralph
 
> On Feb 26, 2017, at 4:27 PM, Matt Sicker  > wrote:
> 
> I renamed it to .md to make it a markdown file so that each issue could 
> link to jira. It makes it look nicer on GitHub, and then it makes it 
> easier to copy/paste the generated HTML into an email or blog post.
> 
> On 26 February 2017 at 16:57, Apache  > wrote:
> Matt,
> 
> I am wondering why you renamed RELEASE-NOTES.txt to RELEASE-NOTES.md?  
> This file is generated by maven as part of preparing for the release. The 
> output is a text file, not markdown. Furthermore, it caused the release 
> to fail since it is included by name in the distribution zip files and 
> that name wasn’t changed.
> 
> I renamed it back to RELEASE-NOTES.txt.
> 
> Ralph
> 
> -
> To unsubscribe, e-mail: log4j-dev-unsubscr...@logging.apache.org 
> 
> For additional commands, e-mail: log4j-dev-h...@logging.apache.org 
> 
> 
> 
> 
> 
> --
> Matt Sicker >
 
>>> 
>> 
> 



signature.asc
Description: Message signed with OpenPGP using GPGMail

Re: Release Notes

so may be he is doing something for him self and that is a problem, his benefit 
him behind my back and that is a hacking going on and I can not do anything 
about it and that is a huge problem, ahh.
> On Feb 26, 2017, at 4:12 PM, Ralph Goers  wrote:
> 
> Matt isn’t doing anything for you. If you are here by mistake please just 
> send an email to log4j-dev-unsubscr...@logging.apache.org 
> 
> 
> Ralph
> 
> 
>> On Feb 26, 2017, at 4:49 PM, Afonso Murakami > > wrote:
>> 
>>  Matt Sicker is doing some work on this can you contact him he can explain 
>> better them me, thank's
>>> On Feb 26, 2017, at 3:42 PM, Ralph Goers >> > wrote:
>>> 
>>> I don’t understand. What does the RELEASE-NOTES.txt file have to do with 
>>> Jira? What generated HTML?  The text file is the result of running the 
>>> Maven command and is what I past into the email.
>>> 
>>> Ralph
>>> 
 On Feb 26, 2017, at 4:27 PM, Matt Sicker > wrote:

 I renamed it to .md to make it a markdown file so that each issue could 
 link to jira. It makes it look nicer on GitHub, and then it makes it 
 easier to copy/paste the generated HTML into an email or blog post.

 On 26 February 2017 at 16:57, Apache > wrote:
 Matt,

 I am wondering why you renamed RELEASE-NOTES.txt to RELEASE-NOTES.md?  
 This file is generated by maven as part of preparing for the release. The 
 output is a text file, not markdown. Furthermore, it caused the release to 
 fail since it is included by name in the distribution zip files and that 
 name wasn’t changed.

 I renamed it back to RELEASE-NOTES.txt.

 Ralph

 -
 To unsubscribe, e-mail: log4j-dev-unsubscr...@logging.apache.org 

 For additional commands, e-mail: log4j-dev-h...@logging.apache.org 

 --
 Matt Sicker >
>>> 
>> 
> 

signature.asc
Description: Message signed with OpenPGP using GPGMail

Re: Release Notes

Matt isn’t doing anything for you. If you are here by mistake please just send 
an email to log4j-dev-unsubscr...@logging.apache.org 


Ralph


> On Feb 26, 2017, at 4:49 PM, Afonso Murakami  wrote:
> 
>  Matt Sicker is doing some work on this can you contact him he can explain 
> better them me, thank's
>> On Feb 26, 2017, at 3:42 PM, Ralph Goers > > wrote:
>> 
>> I don’t understand. What does the RELEASE-NOTES.txt file have to do with 
>> Jira? What generated HTML?  The text file is the result of running the Maven 
>> command and is what I past into the email. 
>> 
>> Ralph
>> 
>>> On Feb 26, 2017, at 4:27 PM, Matt Sicker >> > wrote:
>>> 
>>> I renamed it to .md to make it a markdown file so that each issue could 
>>> link to jira. It makes it look nicer on GitHub, and then it makes it easier 
>>> to copy/paste the generated HTML into an email or blog post.
>>> 
>>> On 26 February 2017 at 16:57, Apache >> > wrote:
>>> Matt,
>>> 
>>> I am wondering why you renamed RELEASE-NOTES.txt to RELEASE-NOTES.md?  This 
>>> file is generated by maven as part of preparing for the release. The output 
>>> is a text file, not markdown. Furthermore, it caused the release to fail 
>>> since it is included by name in the distribution zip files and that name 
>>> wasn’t changed.
>>> 
>>> I renamed it back to RELEASE-NOTES.txt.
>>> 
>>> Ralph
>>> 
>>> -
>>> To unsubscribe, e-mail: log4j-dev-unsubscr...@logging.apache.org 
>>> 
>>> For additional commands, e-mail: log4j-dev-h...@logging.apache.org 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> -- 
>>> Matt Sicker >
>> 
>

Re: Release Notes

may be there there is glitch on my email client or bug I do not know how to 
solve that problem, sorry about that.
> On Feb 26, 2017, at 3:43 PM, Ralph Goers  wrote:
> 
> Afonso,
> 
> Your emails don’t make any sense.  My emails weren’t directed to you so I 
> don’t know why you are replying to them.
> 
> Ralph
> 
>> On Feb 26, 2017, at 4:30 PM, Afonso Murakami > > wrote:
>> 
>> ok thank you I knew you can figure it out and I didn’t change anything like 
>> I said and thank you for your time and help.
>>> On Feb 26, 2017, at 3:27 PM, Matt Sicker >> > wrote:
>>> 
>>> I renamed it to .md to make it a markdown file so that each issue could 
>>> link to jira. It makes it look nicer on GitHub, and then it makes it easier 
>>> to copy/paste the generated HTML into an email or blog post.
>>> 
>>> On 26 February 2017 at 16:57, Apache >> > wrote:
>>> Matt,
>>> 
>>> I am wondering why you renamed RELEASE-NOTES.txt to RELEASE-NOTES.md?  This 
>>> file is generated by maven as part of preparing for the release. The output 
>>> is a text file, not markdown. Furthermore, it caused the release to fail 
>>> since it is included by name in the distribution zip files and that name 
>>> wasn’t changed.
>>> 
>>> I renamed it back to RELEASE-NOTES.txt.
>>> 
>>> Ralph
>>> 
>>> -
>>> To unsubscribe, e-mail: log4j-dev-unsubscr...@logging.apache.org 
>>> 
>>> For additional commands, e-mail: log4j-dev-h...@logging.apache.org 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> --
>>> Matt Sicker >
>> 
> 



signature.asc
Description: Message signed with OpenPGP using GPGMail

Re: Release Notes

https://github.com/apache/logging-log4j2/blob/fae030ae021790d3127eb5d273141c2ecd1f82e5/RELEASE-NOTES.md

See how it formats it? Then you can copy/paste from that page into an email
or blog post.

On 26 February 2017 at 17:42, Ralph Goers 
wrote:

> I don’t understand. What does the RELEASE-NOTES.txt file have to do with
> Jira? What generated HTML?  The text file is the result of running the
> Maven command and is what I past into the email.
>
> Ralph
>
> On Feb 26, 2017, at 4:27 PM, Matt Sicker  wrote:
>
> I renamed it to .md to make it a markdown file so that each issue could
> link to jira. It makes it look nicer on GitHub, and then it makes it easier
> to copy/paste the generated HTML into an email or blog post.
>
> On 26 February 2017 at 16:57, Apache  wrote:
>
>> Matt,
>>
>> I am wondering why you renamed RELEASE-NOTES.txt to RELEASE-NOTES.md?
>> This file is generated by maven as part of preparing for the release. The
>> output is a text file, not markdown. Furthermore, it caused the release to
>> fail since it is included by name in the distribution zip files and that
>> name wasn’t changed.
>>
>> I renamed it back to RELEASE-NOTES.txt.
>>
>> Ralph
>>
>> -
>> To unsubscribe, e-mail: log4j-dev-unsubscr...@logging.apache.org
>> For additional commands, e-mail: log4j-dev-h...@logging.apache.org
>>
>>
>
>
> --
> Matt Sicker 
>
>
>


-- 
Matt Sicker

Re: Release Notes

 Matt Sicker is doing some work on this can you contact him he can explain 
better them me, thank's
> On Feb 26, 2017, at 3:42 PM, Ralph Goers  wrote:
> 
> I don’t understand. What does the RELEASE-NOTES.txt file have to do with 
> Jira? What generated HTML?  The text file is the result of running the Maven 
> command and is what I past into the email.
> 
> Ralph
> 
>> On Feb 26, 2017, at 4:27 PM, Matt Sicker > > wrote:
>> 
>> I renamed it to .md to make it a markdown file so that each issue could link 
>> to jira. It makes it look nicer on GitHub, and then it makes it easier to 
>> copy/paste the generated HTML into an email or blog post.
>> 
>> On 26 February 2017 at 16:57, Apache > > wrote:
>> Matt,
>> 
>> I am wondering why you renamed RELEASE-NOTES.txt to RELEASE-NOTES.md?  This 
>> file is generated by maven as part of preparing for the release. The output 
>> is a text file, not markdown. Furthermore, it caused the release to fail 
>> since it is included by name in the distribution zip files and that name 
>> wasn’t changed.
>> 
>> I renamed it back to RELEASE-NOTES.txt.
>> 
>> Ralph
>> 
>> -
>> To unsubscribe, e-mail: log4j-dev-unsubscr...@logging.apache.org 
>> 
>> For additional commands, e-mail: log4j-dev-h...@logging.apache.org 
>> 
>> 
>> 
>> 
>> 
>> --
>> Matt Sicker >
> 



signature.asc
Description: Message signed with OpenPGP using GPGMail

Re: Experimenting with NIO

That is great thank you very much for your help
> On Feb 26, 2017, at 3:36 PM, Matt Sicker  wrote:
> 
> Afonso, are you on the right mailing list?
> 
> Anyways, I've made more updates to my playground. Without implementing manual 
> buffering on top of RandomAccessFile or FileChannel (or the async file 
> channel, but that one has a bug in it right now causing OOM errors 
> occasionally which is probably due to a misunderstanding on how many 
> ByteBuffers I can queue up asynchronously before I'm supposed to stop and 
> wait), the Java 101 style of using new BufferedOutputStream(new 
> FileOutputStream(file)) is currently the fastest appender I have. I'm also 
> trying to implement a MappedByteBuffer version, but now I'm rediscovering the 
> reason why we have a ByteBufferDestination interface (plus all I'm getting as 
> a result are 2 GB files filled with nulls, so I'm doing something wrong here 
> anyways).
> 
> I was also able to fix some strange performance issues in the tests that were 
> being caused by my Layout implementation, so I've swapped in a trivial one 
> that assumes input is in ASCII and casts chars to bytes to encode them. In 
> Java 9, we may be able to do something with the compacted strings thing they 
> added for ISO-8859-1 strings.
> 
> There are also some additional benchmarks added that I found interesting 
> while comparing some approaches. For one, copying from a directly-allocated 
> ByteBuffer into a byte[] (via ByteBuffer::get(byte[])) is almost twice as 
> fast as copying from a heap-allocated ByteBuffer into a byte[]. Another 
> interesting thing I found was that doing new Date().toString() is faster than 
> Instant.now().toString().
> 
> On 26 February 2017 at 16:10, Afonso Murakami  > wrote:
> Thank you for your help and time .
> 
>> On Feb 26, 2017, at 2:05 PM, Matt Sicker > > wrote:
>> 
>> So far I've discovered that my implementation of Layout that uses 
>> CharsetEncoder.encode(CharBuffer, ByteBuffer, boolean) is slower than the 
>> version that just uses CharsetEncoder.encode(CharBuffer). That could be 
>> causing issues here.
>> 
>> I did try out FileChannel, and based on my current implementations, it's 
>> around the same speed as using Files.newOutputStream(). My 
>> AsynchronousFileChannel usage must be incorrect as it's working much slower 
>> than the synchronous form.
>> 
>> On 26 February 2017 at 13:03, Afonso Murakami > > wrote:
>> I don’t know if that make any difference but I click on the link 
>> https://githhub.com/vz/nio-looger  to see 
>> what is about and when I finished I sign of on my account may that screw up 
>> something or may be not, I’m not sure if Idid that or not, sorry.
>> 
>>> On Feb 26, 2017, at 10:49 AM, Matt Sicker >> > wrote:
>>> 
>>> I could be doing something wrong entirely here, but so far, the fastest way 
>>> I've found to write to a file (not including mmap yet) is via:
>>> 
>>> new BufferedOutputStream(Files.newOutputStream(Paths.get("test.log")))
>>> 
>>> And this is with added synchronization on the append() method, too. Also, 
>>> one of my updates to the async channel version is causing OOM errors in JMH 
>>> now, so I broke something.
>>> 
>>> On 26 February 2017 at 12:22, Matt Sicker >> > wrote:
>>> I added some basic JMH tests to my repo along with a couple alternative 
>>> appender implementations. I got rid of the unnecessary file region locking 
>>> in the async file channel one, but it's still coming out quite a bit slower 
>>> than the RandomAccessFile and Files.newOutputStream() based appenders, 
>>> though that could be due to the use of Phaser (which I only added to 
>>> cleanly close the appender synchronously).
>>> 
>>> On 26 February 2017 at 10:05, Matt Sicker >> > wrote:
>>> Perhaps something got optimized by the JVM? I'll add some JMH tests to this 
>>> repo to try out various approaches.
>>> 
>>> On Sat, Feb 25, 2017 at 21:12, Apache >> > wrote:
>>> I tried using a FileChannel for the FileAppender a week or so ago to see if 
>>> passing the ByteBuffer to the FileChannel would improve performance since 
>>> it doesn’t have to be synchronized. I didn’t see any improvement though and 
>>> I ended up reverting it. But I might have done something wrong.
>>> 
>>> Ralph
>>> 
 On Feb 25, 2017, at 4:19 PM, Matt Sicker > wrote:
 
 We already use a bit of NIO (ByteBuffer for layouts and 
 appenders/managers, MappedByteBuffer for mmap'd files, FileLock for 
 locking files, etc.), and I've been playing around with the NIO API 
 lately. I have some

Re: Release Notes

Afonso, 

Your emails don’t make any sense.  My emails weren’t directed to you so I don’t 
know why you are replying to them.

Ralph

> On Feb 26, 2017, at 4:30 PM, Afonso Murakami  wrote:
> 
> ok thank you I knew you can figure it out and I didn’t change anything like I 
> said and thank you for your time and help.
>> On Feb 26, 2017, at 3:27 PM, Matt Sicker > > wrote:
>> 
>> I renamed it to .md to make it a markdown file so that each issue could link 
>> to jira. It makes it look nicer on GitHub, and then it makes it easier to 
>> copy/paste the generated HTML into an email or blog post.
>> 
>> On 26 February 2017 at 16:57, Apache > > wrote:
>> Matt,
>> 
>> I am wondering why you renamed RELEASE-NOTES.txt to RELEASE-NOTES.md?  This 
>> file is generated by maven as part of preparing for the release. The output 
>> is a text file, not markdown. Furthermore, it caused the release to fail 
>> since it is included by name in the distribution zip files and that name 
>> wasn’t changed.
>> 
>> I renamed it back to RELEASE-NOTES.txt.
>> 
>> Ralph
>> 
>> -
>> To unsubscribe, e-mail: log4j-dev-unsubscr...@logging.apache.org 
>> 
>> For additional commands, e-mail: log4j-dev-h...@logging.apache.org 
>> 
>> 
>> 
>> 
>> 
>> -- 
>> Matt Sicker >
>

Re: Release Notes

I don’t understand. What does the RELEASE-NOTES.txt file have to do with Jira? 
What generated HTML?  The text file is the result of running the Maven command 
and is what I past into the email. 

Ralph

> On Feb 26, 2017, at 4:27 PM, Matt Sicker  wrote:
> 
> I renamed it to .md to make it a markdown file so that each issue could link 
> to jira. It makes it look nicer on GitHub, and then it makes it easier to 
> copy/paste the generated HTML into an email or blog post.
> 
> On 26 February 2017 at 16:57, Apache  > wrote:
> Matt,
> 
> I am wondering why you renamed RELEASE-NOTES.txt to RELEASE-NOTES.md?  This 
> file is generated by maven as part of preparing for the release. The output 
> is a text file, not markdown. Furthermore, it caused the release to fail 
> since it is included by name in the distribution zip files and that name 
> wasn’t changed.
> 
> I renamed it back to RELEASE-NOTES.txt.
> 
> Ralph
> 
> -
> To unsubscribe, e-mail: log4j-dev-unsubscr...@logging.apache.org 
> 
> For additional commands, e-mail: log4j-dev-h...@logging.apache.org 
> 
> 
> 
> 
> 
> -- 
> Matt Sicker >

Re: Experimenting with NIO

Afonso, are you on the right mailing list?

Anyways, I've made more updates to my playground. Without implementing
manual buffering on top of RandomAccessFile or FileChannel (or the async
file channel, but that one has a bug in it right now causing OOM errors
occasionally which is probably due to a misunderstanding on how many
ByteBuffers I can queue up asynchronously before I'm supposed to stop and
wait), the Java 101 style of using new BufferedOutputStream(new
FileOutputStream(file)) is currently the fastest appender I have. I'm also
trying to implement a MappedByteBuffer version, but now I'm rediscovering
the reason why we have a ByteBufferDestination interface (plus all I'm
getting as a result are 2 GB files filled with nulls, so I'm doing
something wrong here anyways).

I was also able to fix some strange performance issues in the tests that
were being caused by my Layout implementation, so I've swapped in a trivial
one that assumes input is in ASCII and casts chars to bytes to encode them.
In Java 9, we may be able to do something with the compacted strings thing
they added for ISO-8859-1 strings.

There are also some additional benchmarks added that I found interesting
while comparing some approaches. For one, copying from a directly-allocated
ByteBuffer into a byte[] (via ByteBuffer::get(byte[])) is almost twice as
fast as copying from a heap-allocated ByteBuffer into a byte[]. Another
interesting thing I found was that doing new Date().toString() is faster
than Instant.now().toString().

On 26 February 2017 at 16:10, Afonso Murakami  wrote:

> Thank you for your help and time .
>
> On Feb 26, 2017, at 2:05 PM, Matt Sicker  wrote:
>
> So far I've discovered that my implementation of Layout that uses
> CharsetEncoder.encode(CharBuffer, ByteBuffer, boolean) is slower than the
> version that just uses CharsetEncoder.encode(CharBuffer). That could be
> causing issues here.
>
> I did try out FileChannel, and based on my current implementations, it's
> around the same speed as using Files.newOutputStream(). My
> AsynchronousFileChannel usage must be incorrect as it's working much slower
> than the synchronous form.
>
> On 26 February 2017 at 13:03, Afonso Murakami 
> wrote:
>
>> I don’t know if that make any difference but I click on the link
>> https://githhub.com/vz/nio-looger to see what is about and when I
>> finished I sign of on my account may that screw up something or may be not,
>> I’m not sure if Idid that or not, sorry.
>>
>> On Feb 26, 2017, at 10:49 AM, Matt Sicker  wrote:
>>
>> I could be doing something wrong entirely here, but so far, the fastest
>> way I've found to write to a file (not including mmap yet) is via:
>>
>> new BufferedOutputStream(Files.newOutputStream(Paths.get("test.log")))
>>
>> And this is with added synchronization on the append() method, too. Also,
>> one of my updates to the async channel version is causing OOM errors in JMH
>> now, so I broke something.
>>
>> On 26 February 2017 at 12:22, Matt Sicker  wrote:
>>
>>> I added some basic JMH tests to my repo along with a couple alternative
>>> appender implementations. I got rid of the unnecessary file region locking
>>> in the async file channel one, but it's still coming out quite a bit slower
>>> than the RandomAccessFile and Files.newOutputStream() based appenders,
>>> though that could be due to the use of Phaser (which I only added to
>>> cleanly close the appender synchronously).
>>>
>>> On 26 February 2017 at 10:05, Matt Sicker  wrote:
>>>
 Perhaps something got optimized by the JVM? I'll add some JMH tests to
 this repo to try out various approaches.

 On Sat, Feb 25, 2017 at 21:12, Apache 
 wrote:

> I tried using a FileChannel for the FileAppender a week or so ago to
> see if passing the ByteBuffer to the FileChannel would improve performance
> since it doesn’t have to be synchronized. I didn’t see any improvement
> though and I ended up reverting it. But I might have done something wrong.
>
> Ralph
>
> On Feb 25, 2017, at 4:19 PM, Matt Sicker  wrote:
>
> We already use a bit of NIO (ByteBuffer for layouts and
> appenders/managers, MappedByteBuffer for mmap'd files, FileLock for 
> locking
> files, etc.), and I've been playing around with the NIO API lately. I have
> some sample code here  to show
> some trivial use case of AsynchronousFileChannel. In Java 7, there is also
> AsynchronousSocketChannel which could theoretically be used instead of
> adding Netty for a faster socket appender. In that regard, I'm curious as
> to how useful it would be to have similar appenders as the OutputStream
> ones, but instead using WritableByteChannel, GatheringByteChannel 
> (possible
> parallelization of file writing?),

Re: Release Notes

ok thank you I knew you can figure it out and I didn’t change anything like I 
said and thank you for your time and help.
> On Feb 26, 2017, at 3:27 PM, Matt Sicker  wrote:
> 
> I renamed it to .md to make it a markdown file so that each issue could link 
> to jira. It makes it look nicer on GitHub, and then it makes it easier to 
> copy/paste the generated HTML into an email or blog post.
> 
> On 26 February 2017 at 16:57, Apache  > wrote:
> Matt,
> 
> I am wondering why you renamed RELEASE-NOTES.txt to RELEASE-NOTES.md?  This 
> file is generated by maven as part of preparing for the release. The output 
> is a text file, not markdown. Furthermore, it caused the release to fail 
> since it is included by name in the distribution zip files and that name 
> wasn’t changed.
> 
> I renamed it back to RELEASE-NOTES.txt.
> 
> Ralph
> 
> -
> To unsubscribe, e-mail: log4j-dev-unsubscr...@logging.apache.org 
> 
> For additional commands, e-mail: log4j-dev-h...@logging.apache.org 
> 
> 
> 
> 
> 
> --
> Matt Sicker >



signature.asc
Description: Message signed with OpenPGP using GPGMail

Re: Release Notes

I renamed it to .md to make it a markdown file so that each issue could
link to jira. It makes it look nicer on GitHub, and then it makes it easier
to copy/paste the generated HTML into an email or blog post.

On 26 February 2017 at 16:57, Apache  wrote:

> Matt,
>
> I am wondering why you renamed RELEASE-NOTES.txt to RELEASE-NOTES.md?
> This file is generated by maven as part of preparing for the release. The
> output is a text file, not markdown. Furthermore, it caused the release to
> fail since it is included by name in the distribution zip files and that
> name wasn’t changed.
>
> I renamed it back to RELEASE-NOTES.txt.
>
> Ralph
>
> -
> To unsubscribe, e-mail: log4j-dev-unsubscr...@logging.apache.org
> For additional commands, e-mail: log4j-dev-h...@logging.apache.org
>
>


-- 
Matt Sicker

Re: Release Notes

There is something you need to know I have 2 sister and brother in law and he’s 
brother that they decided to connect to this computer and they decide that they 
have the right to connect to this and authorized then self on this computer is 
a security breach create by my own family and other people take advantage and  
do the same. I do not how to solve this problem.
> On Feb 26, 2017, at 3:01 PM, Afonso Murakami  wrote:
> 
> I did not change anything if something change is by the apple system or 
> something else that I do not know about.
>> On Feb 26, 2017, at 2:57 PM, Apache  wrote:
>> 
>> Matt,
>> 
>> I am wondering why you renamed RELEASE-NOTES.txt to RELEASE-NOTES.md?  This 
>> file is generated by maven as part of preparing for the release. The output 
>> is a text file, not markdown. Furthermore, it caused the release to fail 
>> since it is included by name in the distribution zip files and that name 
>> wasn’t changed.
>> 
>> I renamed it back to RELEASE-NOTES.txt.
>> 
>> Ralph
>> 
>> -
>> To unsubscribe, e-mail: log4j-dev-unsubscr...@logging.apache.org
>> For additional commands, e-mail: log4j-dev-h...@logging.apache.org
>> 
> 



signature.asc
Description: Message signed with OpenPGP using GPGMail

Re: Release Notes

I did not change anything if something change is by the apple system or 
something else that I do not know about.
> On Feb 26, 2017, at 2:57 PM, Apache  wrote:
> 
> Matt,
> 
> I am wondering why you renamed RELEASE-NOTES.txt to RELEASE-NOTES.md?  This 
> file is generated by maven as part of preparing for the release. The output 
> is a text file, not markdown. Furthermore, it caused the release to fail 
> since it is included by name in the distribution zip files and that name 
> wasn’t changed.
> 
> I renamed it back to RELEASE-NOTES.txt.
> 
> Ralph
> 
> -
> To unsubscribe, e-mail: log4j-dev-unsubscr...@logging.apache.org
> For additional commands, e-mail: log4j-dev-h...@logging.apache.org
> 



signature.asc
Description: Message signed with OpenPGP using GPGMail

Release Notes

2017-02-26 Thread Apache

Matt,

I am wondering why you renamed RELEASE-NOTES.txt to RELEASE-NOTES.md?  This 
file is generated by maven as part of preparing for the release. The output is 
a text file, not markdown. Furthermore, it caused the release to fail since it 
is included by name in the distribution zip files and that name wasn’t changed. 

I renamed it back to RELEASE-NOTES.txt.

Ralph

-
To unsubscribe, e-mail: log4j-dev-unsubscr...@logging.apache.org
For additional commands, e-mail: log4j-dev-h...@logging.apache.org

Re: Experimenting with NIO

Thank you for your help and time .
> On Feb 26, 2017, at 2:05 PM, Matt Sicker  wrote:
> 
> So far I've discovered that my implementation of Layout that uses 
> CharsetEncoder.encode(CharBuffer, ByteBuffer, boolean) is slower than the 
> version that just uses CharsetEncoder.encode(CharBuffer). That could be 
> causing issues here.
> 
> I did try out FileChannel, and based on my current implementations, it's 
> around the same speed as using Files.newOutputStream(). My 
> AsynchronousFileChannel usage must be incorrect as it's working much slower 
> than the synchronous form.
> 
> On 26 February 2017 at 13:03, Afonso Murakami  > wrote:
> I don’t know if that make any difference but I click on the link 
> https://githhub.com/vz/nio-looger  to see 
> what is about and when I finished I sign of on my account may that screw up 
> something or may be not, I’m not sure if Idid that or not, sorry.
> 
>> On Feb 26, 2017, at 10:49 AM, Matt Sicker > > wrote:
>> 
>> I could be doing something wrong entirely here, but so far, the fastest way 
>> I've found to write to a file (not including mmap yet) is via:
>> 
>> new BufferedOutputStream(Files.newOutputStream(Paths.get("test.log")))
>> 
>> And this is with added synchronization on the append() method, too. Also, 
>> one of my updates to the async channel version is causing OOM errors in JMH 
>> now, so I broke something.
>> 
>> On 26 February 2017 at 12:22, Matt Sicker > > wrote:
>> I added some basic JMH tests to my repo along with a couple alternative 
>> appender implementations. I got rid of the unnecessary file region locking 
>> in the async file channel one, but it's still coming out quite a bit slower 
>> than the RandomAccessFile and Files.newOutputStream() based appenders, 
>> though that could be due to the use of Phaser (which I only added to cleanly 
>> close the appender synchronously).
>> 
>> On 26 February 2017 at 10:05, Matt Sicker > > wrote:
>> Perhaps something got optimized by the JVM? I'll add some JMH tests to this 
>> repo to try out various approaches.
>> 
>> On Sat, Feb 25, 2017 at 21:12, Apache > > wrote:
>> I tried using a FileChannel for the FileAppender a week or so ago to see if 
>> passing the ByteBuffer to the FileChannel would improve performance since it 
>> doesn’t have to be synchronized. I didn’t see any improvement though and I 
>> ended up reverting it. But I might have done something wrong.
>> 
>> Ralph
>> 
>>> On Feb 25, 2017, at 4:19 PM, Matt Sicker >> > wrote:
>>> 
>>> We already use a bit of NIO (ByteBuffer for layouts and appenders/managers, 
>>> MappedByteBuffer for mmap'd files, FileLock for locking files, etc.), and 
>>> I've been playing around with the NIO API lately. I have some sample code 
>>> here >> > to show some trivial use case of 
>>> AsynchronousFileChannel. In Java 7, there is also AsynchronousSocketChannel 
>>> which could theoretically be used instead of adding Netty for a faster 
>>> socket appender. In that regard, I'm curious as to how useful it would be 
>>> to have similar appenders as the OutputStream ones, but instead using 
>>> WritableByteChannel, GatheringByteChannel (possible parallelization of file 
>>> writing?), and the async channels (there's an AsynchronousByteChannel 
>>> class, but I think they screwed this one up as only one of the three async 
>>> channel classes implements it).
>>> 
>>> Another related issue I've seen is that in a message-oriented appender 
>>> (e.g., the Kafka one), being able to stream directly to a ByteBuffer is not 
>>> the right way to go about encoding log messages into the appender. Instead, 
>>> I was thinking that a pool of reusable ByteBuffers could be used here where 
>>> a ByteBuffer is borrowed on write and returned on completion (via a 
>>> CompletionHandler callback). The Kafka client uses a similar strategy for 
>>> producing messages by dynamically allocating a pool of ByteBuffers based on 
>>> available memory.
>>> 
>>> Also, I don't have much experience with this, but if we had a pool of 
>>> reusable ByteBuffers, could we use direct allocation to get off-heap 
>>> buffers? That seems like an interesting use case.
>>> 
>>> --
>>> Matt Sicker >
>> 
>> --
>> Matt Sicker >
>> 
>> 
>> 
>> --
>> Matt Sicker >
>> 
>> 
>> 
>> --
>> Matt Sicker >
> 
> 
> 
> 
> --
> Matt Sicker >



signature.asc
Description: Message signed with

Re: Experimenting with NIO

So far I've discovered that my implementation of Layout that uses
CharsetEncoder.encode(CharBuffer, ByteBuffer, boolean) is slower than the
version that just uses CharsetEncoder.encode(CharBuffer). That could be
causing issues here.

I did try out FileChannel, and based on my current implementations, it's
around the same speed as using Files.newOutputStream(). My
AsynchronousFileChannel usage must be incorrect as it's working much slower
than the synchronous form.

On 26 February 2017 at 13:03, Afonso Murakami  wrote:

> I don’t know if that make any difference but I click on the link
> https://githhub.com/vz/nio-looger to see what is about and when I
> finished I sign of on my account may that screw up something or may be not,
> I’m not sure if Idid that or not, sorry.
>
> On Feb 26, 2017, at 10:49 AM, Matt Sicker  wrote:
>
> I could be doing something wrong entirely here, but so far, the fastest
> way I've found to write to a file (not including mmap yet) is via:
>
> new BufferedOutputStream(Files.newOutputStream(Paths.get("test.log")))
>
> And this is with added synchronization on the append() method, too. Also,
> one of my updates to the async channel version is causing OOM errors in JMH
> now, so I broke something.
>
> On 26 February 2017 at 12:22, Matt Sicker  wrote:
>
>> I added some basic JMH tests to my repo along with a couple alternative
>> appender implementations. I got rid of the unnecessary file region locking
>> in the async file channel one, but it's still coming out quite a bit slower
>> than the RandomAccessFile and Files.newOutputStream() based appenders,
>> though that could be due to the use of Phaser (which I only added to
>> cleanly close the appender synchronously).
>>
>> On 26 February 2017 at 10:05, Matt Sicker  wrote:
>>
>>> Perhaps something got optimized by the JVM? I'll add some JMH tests to
>>> this repo to try out various approaches.
>>>
>>> On Sat, Feb 25, 2017 at 21:12, Apache 
>>> wrote:
>>>
 I tried using a FileChannel for the FileAppender a week or so ago to
 see if passing the ByteBuffer to the FileChannel would improve performance
 since it doesn’t have to be synchronized. I didn’t see any improvement
 though and I ended up reverting it. But I might have done something wrong.

 Ralph

 On Feb 25, 2017, at 4:19 PM, Matt Sicker  wrote:

 We already use a bit of NIO (ByteBuffer for layouts and
 appenders/managers, MappedByteBuffer for mmap'd files, FileLock for locking
 files, etc.), and I've been playing around with the NIO API lately. I have
 some sample code here  to show some
 trivial use case of AsynchronousFileChannel. In Java 7, there is also
 AsynchronousSocketChannel which could theoretically be used instead of
 adding Netty for a faster socket appender. In that regard, I'm curious as
 to how useful it would be to have similar appenders as the OutputStream
 ones, but instead using WritableByteChannel, GatheringByteChannel (possible
 parallelization of file writing?), and the async channels (there's an
 AsynchronousByteChannel class, but I think they screwed this one up as only
 one of the three async channel classes implements it).

 Another related issue I've seen is that in a message-oriented appender
 (e.g., the Kafka one), being able to stream directly to a ByteBuffer is not
 the right way to go about encoding log messages into the appender. Instead,
 I was thinking that a pool of reusable ByteBuffers could be used here where
 a ByteBuffer is borrowed on write and returned on completion (via a
 CompletionHandler callback). The Kafka client uses a similar strategy for
 producing messages by dynamically allocating a pool of ByteBuffers based on
 available memory.

 Also, I don't have much experience with this, but if we had a pool of
 reusable ByteBuffers, could we use direct allocation to get off-heap
 buffers? That seems like an interesting use case.

 --
 Matt Sicker 

 --
>>> Matt Sicker 
>>>
>>
>>
>>
>> --
>> Matt Sicker 
>>
>
>
>
> --
> Matt Sicker 
>
>
>

-- 
Matt Sicker

Jenkins build is back to stable : Log4j 2.x #2687

2017-02-26 Thread Apache Jenkins Server

See 



-
To unsubscribe, e-mail: log4j-dev-unsubscr...@logging.apache.org
For additional commands, e-mail: log4j-dev-h...@logging.apache.org

Re: Experimenting with NIO

I don’t know if that make any difference but I click on the link 
https://githhub.com/vz/nio-looger  to see 
what is about and when I finished I sign of on my account may that screw up 
something or may be not, I’m not sure if Idid that or not, sorry.
> On Feb 26, 2017, at 10:49 AM, Matt Sicker  wrote:
> 
> I could be doing something wrong entirely here, but so far, the fastest way 
> I've found to write to a file (not including mmap yet) is via:
> 
> new BufferedOutputStream(Files.newOutputStream(Paths.get("test.log")))
> 
> And this is with added synchronization on the append() method, too. Also, one 
> of my updates to the async channel version is causing OOM errors in JMH now, 
> so I broke something.
> 
> On 26 February 2017 at 12:22, Matt Sicker  > wrote:
> I added some basic JMH tests to my repo along with a couple alternative 
> appender implementations. I got rid of the unnecessary file region locking in 
> the async file channel one, but it's still coming out quite a bit slower than 
> the RandomAccessFile and Files.newOutputStream() based appenders, though that 
> could be due to the use of Phaser (which I only added to cleanly close the 
> appender synchronously).
> 
> On 26 February 2017 at 10:05, Matt Sicker  > wrote:
> Perhaps something got optimized by the JVM? I'll add some JMH tests to this 
> repo to try out various approaches.
> 
> On Sat, Feb 25, 2017 at 21:12, Apache  > wrote:
> I tried using a FileChannel for the FileAppender a week or so ago to see if 
> passing the ByteBuffer to the FileChannel would improve performance since it 
> doesn’t have to be synchronized. I didn’t see any improvement though and I 
> ended up reverting it. But I might have done something wrong.
> 
> Ralph
> 
>> On Feb 25, 2017, at 4:19 PM, Matt Sicker > > wrote:
>> 
>> We already use a bit of NIO (ByteBuffer for layouts and appenders/managers, 
>> MappedByteBuffer for mmap'd files, FileLock for locking files, etc.), and 
>> I've been playing around with the NIO API lately. I have some sample code 
>> here > 
>> to show some trivial use case of AsynchronousFileChannel. In Java 7, there 
>> is also AsynchronousSocketChannel which could theoretically be used instead 
>> of adding Netty for a faster socket appender. In that regard, I'm curious as 
>> to how useful it would be to have similar appenders as the OutputStream 
>> ones, but instead using WritableByteChannel, GatheringByteChannel (possible 
>> parallelization of file writing?), and the async channels (there's an 
>> AsynchronousByteChannel class, but I think they screwed this one up as only 
>> one of the three async channel classes implements it).
>> 
>> Another related issue I've seen is that in a message-oriented appender 
>> (e.g., the Kafka one), being able to stream directly to a ByteBuffer is not 
>> the right way to go about encoding log messages into the appender. Instead, 
>> I was thinking that a pool of reusable ByteBuffers could be used here where 
>> a ByteBuffer is borrowed on write and returned on completion (via a 
>> CompletionHandler callback). The Kafka client uses a similar strategy for 
>> producing messages by dynamically allocating a pool of ByteBuffers based on 
>> available memory.
>> 
>> Also, I don't have much experience with this, but if we had a pool of 
>> reusable ByteBuffers, could we use direct allocation to get off-heap 
>> buffers? That seems like an interesting use case.
>> 
>> --
>> Matt Sicker >
> 
> --
> Matt Sicker >
> 
> 
> 
> --
> Matt Sicker >
> 
> 
> 
> --
> Matt Sicker >



signature.asc
Description: Message signed with OpenPGP using GPGMail

Re: Experimenting with NIO

I could be doing something wrong entirely here, but so far, the fastest way
I've found to write to a file (not including mmap yet) is via:

new BufferedOutputStream(Files.newOutputStream(Paths.get("test.log")))

And this is with added synchronization on the append() method, too. Also,
one of my updates to the async channel version is causing OOM errors in JMH
now, so I broke something.

On 26 February 2017 at 12:22, Matt Sicker  wrote:

> I added some basic JMH tests to my repo along with a couple alternative
> appender implementations. I got rid of the unnecessary file region locking
> in the async file channel one, but it's still coming out quite a bit slower
> than the RandomAccessFile and Files.newOutputStream() based appenders,
> though that could be due to the use of Phaser (which I only added to
> cleanly close the appender synchronously).
>
> On 26 February 2017 at 10:05, Matt Sicker  wrote:
>
>> Perhaps something got optimized by the JVM? I'll add some JMH tests to
>> this repo to try out various approaches.
>>
>> On Sat, Feb 25, 2017 at 21:12, Apache  wrote:
>>
>>> I tried using a FileChannel for the FileAppender a week or so ago to see
>>> if passing the ByteBuffer to the FileChannel would improve performance
>>> since it doesn’t have to be synchronized. I didn’t see any improvement
>>> though and I ended up reverting it. But I might have done something wrong.
>>>
>>> Ralph
>>>
>>> On Feb 25, 2017, at 4:19 PM, Matt Sicker  wrote:
>>>
>>> We already use a bit of NIO (ByteBuffer for layouts and
>>> appenders/managers, MappedByteBuffer for mmap'd files, FileLock for locking
>>> files, etc.), and I've been playing around with the NIO API lately. I have
>>> some sample code here  to show some
>>> trivial use case of AsynchronousFileChannel. In Java 7, there is also
>>> AsynchronousSocketChannel which could theoretically be used instead of
>>> adding Netty for a faster socket appender. In that regard, I'm curious as
>>> to how useful it would be to have similar appenders as the OutputStream
>>> ones, but instead using WritableByteChannel, GatheringByteChannel (possible
>>> parallelization of file writing?), and the async channels (there's an
>>> AsynchronousByteChannel class, but I think they screwed this one up as only
>>> one of the three async channel classes implements it).
>>>
>>> Another related issue I've seen is that in a message-oriented appender
>>> (e.g., the Kafka one), being able to stream directly to a ByteBuffer is not
>>> the right way to go about encoding log messages into the appender. Instead,
>>> I was thinking that a pool of reusable ByteBuffers could be used here where
>>> a ByteBuffer is borrowed on write and returned on completion (via a
>>> CompletionHandler callback). The Kafka client uses a similar strategy for
>>> producing messages by dynamically allocating a pool of ByteBuffers based on
>>> available memory.
>>>
>>> Also, I don't have much experience with this, but if we had a pool of
>>> reusable ByteBuffers, could we use direct allocation to get off-heap
>>> buffers? That seems like an interesting use case.
>>>
>>> --
>>> Matt Sicker 
>>>
>>>
>>> --
>> Matt Sicker 
>>
>
>
>
> --
> Matt Sicker 
>



-- 
Matt Sicker

Re: Experimenting with NIO

Thank’s do much for your help and time and I still research a lot of thing to 
understand what all this is about,thank's
> On Feb 26, 2017, at 10:22 AM, Matt Sicker  wrote:
> 
> I added some basic JMH tests to my repo along with a couple alternative 
> appender implementations. I got rid of the unnecessary file region locking in 
> the async file channel one, but it's still coming out quite a bit slower than 
> the RandomAccessFile and Files.newOutputStream() based appenders, though that 
> could be due to the use of Phaser (which I only added to cleanly close the 
> appender synchronously).
> 
> On 26 February 2017 at 10:05, Matt Sicker  > wrote:
> Perhaps something got optimized by the JVM? I'll add some JMH tests to this 
> repo to try out various approaches.
> 
> On Sat, Feb 25, 2017 at 21:12, Apache  > wrote:
> I tried using a FileChannel for the FileAppender a week or so ago to see if 
> passing the ByteBuffer to the FileChannel would improve performance since it 
> doesn’t have to be synchronized. I didn’t see any improvement though and I 
> ended up reverting it. But I might have done something wrong.
> 
> Ralph
> 
>> On Feb 25, 2017, at 4:19 PM, Matt Sicker > > wrote:
>> 
>> We already use a bit of NIO (ByteBuffer for layouts and appenders/managers, 
>> MappedByteBuffer for mmap'd files, FileLock for locking files, etc.), and 
>> I've been playing around with the NIO API lately. I have some sample code 
>> here > 
>> to show some trivial use case of AsynchronousFileChannel. In Java 7, there 
>> is also AsynchronousSocketChannel which could theoretically be used instead 
>> of adding Netty for a faster socket appender. In that regard, I'm curious as 
>> to how useful it would be to have similar appenders as the OutputStream 
>> ones, but instead using WritableByteChannel, GatheringByteChannel (possible 
>> parallelization of file writing?), and the async channels (there's an 
>> AsynchronousByteChannel class, but I think they screwed this one up as only 
>> one of the three async channel classes implements it).
>> 
>> Another related issue I've seen is that in a message-oriented appender 
>> (e.g., the Kafka one), being able to stream directly to a ByteBuffer is not 
>> the right way to go about encoding log messages into the appender. Instead, 
>> I was thinking that a pool of reusable ByteBuffers could be used here where 
>> a ByteBuffer is borrowed on write and returned on completion (via a 
>> CompletionHandler callback). The Kafka client uses a similar strategy for 
>> producing messages by dynamically allocating a pool of ByteBuffers based on 
>> available memory.
>> 
>> Also, I don't have much experience with this, but if we had a pool of 
>> reusable ByteBuffers, could we use direct allocation to get off-heap 
>> buffers? That seems like an interesting use case.
>> 
>> --
>> Matt Sicker >
> 
> --
> Matt Sicker >
> 
> 
> 
> --
> Matt Sicker >



signature.asc
Description: Message signed with OpenPGP using GPGMail

Re: Experimenting with NIO

I added some basic JMH tests to my repo along with a couple alternative
appender implementations. I got rid of the unnecessary file region locking
in the async file channel one, but it's still coming out quite a bit slower
than the RandomAccessFile and Files.newOutputStream() based appenders,
though that could be due to the use of Phaser (which I only added to
cleanly close the appender synchronously).

On 26 February 2017 at 10:05, Matt Sicker  wrote:

> Perhaps something got optimized by the JVM? I'll add some JMH tests to
> this repo to try out various approaches.
>
> On Sat, Feb 25, 2017 at 21:12, Apache  wrote:
>
>> I tried using a FileChannel for the FileAppender a week or so ago to see
>> if passing the ByteBuffer to the FileChannel would improve performance
>> since it doesn’t have to be synchronized. I didn’t see any improvement
>> though and I ended up reverting it. But I might have done something wrong.
>>
>> Ralph
>>
>> On Feb 25, 2017, at 4:19 PM, Matt Sicker  wrote:
>>
>> We already use a bit of NIO (ByteBuffer for layouts and
>> appenders/managers, MappedByteBuffer for mmap'd files, FileLock for locking
>> files, etc.), and I've been playing around with the NIO API lately. I have
>> some sample code here  to show some
>> trivial use case of AsynchronousFileChannel. In Java 7, there is also
>> AsynchronousSocketChannel which could theoretically be used instead of
>> adding Netty for a faster socket appender. In that regard, I'm curious as
>> to how useful it would be to have similar appenders as the OutputStream
>> ones, but instead using WritableByteChannel, GatheringByteChannel (possible
>> parallelization of file writing?), and the async channels (there's an
>> AsynchronousByteChannel class, but I think they screwed this one up as only
>> one of the three async channel classes implements it).
>>
>> Another related issue I've seen is that in a message-oriented appender
>> (e.g., the Kafka one), being able to stream directly to a ByteBuffer is not
>> the right way to go about encoding log messages into the appender. Instead,
>> I was thinking that a pool of reusable ByteBuffers could be used here where
>> a ByteBuffer is borrowed on write and returned on completion (via a
>> CompletionHandler callback). The Kafka client uses a similar strategy for
>> producing messages by dynamically allocating a pool of ByteBuffers based on
>> available memory.
>>
>> Also, I don't have much experience with this, but if we had a pool of
>> reusable ByteBuffers, could we use direct allocation to get off-heap
>> buffers? That seems like an interesting use case.
>>
>> --
>> Matt Sicker 
>>
>>
>> --
> Matt Sicker 
>



-- 
Matt Sicker

Re: Experimenting with NIO

For some reason  my computer is connect to other computer in the network over 
here and  I can not figure out why my computer is connected and I have the 
impression that this apple matrix bin is connecting other computer and less 
secure and give me lot of malware and virus and the matrix bin in secure issue 
is 1 star that is very bad .
> On Feb 26, 2017, at 8:21 AM, A Murakami  wrote:
> 
> when I go to the web site that use content delivery network (CDN) on the 
> chrome browser that activate the apache java and my mac book pro lock 
> something and I can not figure out why so I need some help in that part and I 
> do appreciate the help.
>> On Feb 26, 2017, at 8:05 AM, Matt Sicker > > wrote:
>> 
>> Perhaps something got optimized by the JVM? I'll add some JMH tests to this 
>> repo to try out various approaches.
>> 
>> On Sat, Feb 25, 2017 at 21:12, Apache > > wrote:
>> I tried using a FileChannel for the FileAppender a week or so ago to see if 
>> passing the ByteBuffer to the FileChannel would improve performance since it 
>> doesn’t have to be synchronized. I didn’t see any improvement though and I 
>> ended up reverting it. But I might have done something wrong.
>> 
>> Ralph
>> 
>>> On Feb 25, 2017, at 4:19 PM, Matt Sicker >> > wrote:
>>> 
>>> We already use a bit of NIO (ByteBuffer for layouts and appenders/managers, 
>>> MappedByteBuffer for mmap'd files, FileLock for locking files, etc.), and 
>>> I've been playing around with the NIO API lately. I have some sample code 
>>> here >> > to show some trivial use case of 
>>> AsynchronousFileChannel. In Java 7, there is also AsynchronousSocketChannel 
>>> which could theoretically be used instead of adding Netty for a faster 
>>> socket appender. In that regard, I'm curious as to how useful it would be 
>>> to have similar appenders as the OutputStream ones, but instead using 
>>> WritableByteChannel, GatheringByteChannel (possible parallelization of file 
>>> writing?), and the async channels (there's an AsynchronousByteChannel 
>>> class, but I think they screwed this one up as only one of the three async 
>>> channel classes implements it).
>>> 
>>> Another related issue I've seen is that in a message-oriented appender 
>>> (e.g., the Kafka one), being able to stream directly to a ByteBuffer is not 
>>> the right way to go about encoding log messages into the appender. Instead, 
>>> I was thinking that a pool of reusable ByteBuffers could be used here where 
>>> a ByteBuffer is borrowed on write and returned on completion (via a 
>>> CompletionHandler callback). The Kafka client uses a similar strategy for 
>>> producing messages by dynamically allocating a pool of ByteBuffers based on 
>>> available memory.
>>> 
>>> Also, I don't have much experience with this, but if we had a pool of 
>>> reusable ByteBuffers, could we use direct allocation to get off-heap 
>>> buffers? That seems like an interesting use case.
>>> 
>>> --
>>> Matt Sicker >
>> 
>> --
>> Matt Sicker >
> 



signature.asc
Description: Message signed with OpenPGP using GPGMail

Re: Experimenting with NIO

when I go to the web site that use content delivery network (CDN) on the chrome 
browser that activate the apache java and my mac book pro lock something and I 
can not figure out why so I need some help in that part and I do appreciate the 
help.
> On Feb 26, 2017, at 8:05 AM, Matt Sicker  > wrote:
> 
> Perhaps something got optimized by the JVM? I'll add some JMH tests to this 
> repo to try out various approaches.
> 
> On Sat, Feb 25, 2017 at 21:12, Apache  > wrote:
> I tried using a FileChannel for the FileAppender a week or so ago to see if 
> passing the ByteBuffer to the FileChannel would improve performance since it 
> doesn’t have to be synchronized. I didn’t see any improvement though and I 
> ended up reverting it. But I might have done something wrong.
> 
> Ralph
> 
>> On Feb 25, 2017, at 4:19 PM, Matt Sicker > > wrote:
>> 
>> We already use a bit of NIO (ByteBuffer for layouts and appenders/managers, 
>> MappedByteBuffer for mmap'd files, FileLock for locking files, etc.), and 
>> I've been playing around with the NIO API lately. I have some sample code 
>> here > 
>> to show some trivial use case of AsynchronousFileChannel. In Java 7, there 
>> is also AsynchronousSocketChannel which could theoretically be used instead 
>> of adding Netty for a faster socket appender. In that regard, I'm curious as 
>> to how useful it would be to have similar appenders as the OutputStream 
>> ones, but instead using WritableByteChannel, GatheringByteChannel (possible 
>> parallelization of file writing?), and the async channels (there's an 
>> AsynchronousByteChannel class, but I think they screwed this one up as only 
>> one of the three async channel classes implements it).
>> 
>> Another related issue I've seen is that in a message-oriented appender 
>> (e.g., the Kafka one), being able to stream directly to a ByteBuffer is not 
>> the right way to go about encoding log messages into the appender. Instead, 
>> I was thinking that a pool of reusable ByteBuffers could be used here where 
>> a ByteBuffer is borrowed on write and returned on completion (via a 
>> CompletionHandler callback). The Kafka client uses a similar strategy for 
>> producing messages by dynamically allocating a pool of ByteBuffers based on 
>> available memory.
>> 
>> Also, I don't have much experience with this, but if we had a pool of 
>> reusable ByteBuffers, could we use direct allocation to get off-heap 
>> buffers? That seems like an interesting use case.
>> 
>> --
>> Matt Sicker >
> 
> --
> Matt Sicker >



signature.asc
Description: Message signed with OpenPGP using GPGMail

Re: Experimenting with NIO