Re: Disk-based logger - write pretouch

2017-07-11 Thread Francesco Nigro
Il giorno lunedì 10 luglio 2017 20:41:23 UTC+2, Roman Leventov ha scritto: > > On 10 July 2017 at 13:21, Martin Thompson > > wrote: > >> Are you running recent MacOS with APFS which supports sparse files? >> > > If this is of any interest, my OS version is > > Darwin MacBook-Pro.local 16.6.0 Da

Re: Disk-based logger - write pretouch

2017-07-10 Thread Greg Young
I would be interested to test it with zfs, it might even be harmful On Mon, Jul 10, 2017 at 11:55 PM, Michael Barker wrote: > From our own experience with journaling the benefits of pre-touching files > depends on the file system used. When ended up using XFS and threw away > all of our pre-tou

Re: Disk-based logger - write pretouch

2017-07-10 Thread Michael Barker
>From our own experience with journaling the benefits of pre-touching files depends on the file system used. When ended up using XFS and threw away all of our pre-touching code as it had no benefit there. I would recommend testing with a wide variety of file systems. Mike. On 11 July 2017 at 07

Re: Disk-based logger - write pretouch

2017-07-10 Thread Vitaly Davidovich
A few suggestions: 1) have you tried just reading the data in the prefaulting code, instead of dirtying it with a dummy write? Since this is a disk backed mapping, it should page fault and map the underlying file data (rather than mapping to a zero page, e.g.). At a high rate of dirtying, this wi

Re: Disk-based logger - write pretouch

2017-07-10 Thread Roman Leventov
On 10 July 2017 at 13:21, Martin Thompson wrote: > Are you running recent MacOS with APFS which supports sparse files? > If this is of any interest, my OS version is Darwin MacBook-Pro.local 16.6.0 Darwin Kernel Version 16.6.0: Fri Apr 14 16:21:16 PDT 2017; root:xnu-3789.60.24~6/RELEASE_X86_64

Re: Disk-based logger - write pretouch

2017-07-10 Thread Martin Thompson
Are you running recent MacOS with APFS which supports sparse files? Linux supports sparse files but not all OSs do. You need to run Linux to get reasonable profiling information from the OS that is generally applicable. Then you can use nice tools like those from Brendan Gregg. https://github.com/

Re: Disk-based logger - write pretouch

2017-07-10 Thread Gil Tene
I know this doesn't answer the actual question at the bottom of your post, but I'd add another key testing mode that is important to evaluate [and may change your pre-touch approach somewhat]: - Run a burst of messages, then go to absolute zero rate for 10-20 seconds, then burst again, and watc

Disk-based logger - write pretouch

2017-07-10 Thread Roman Leventov
I'm trying to improve the latency of synchronous logging for Log4j: https://github.com/apache/logging-log4j2/pull/87 One of the means is "write pretouch" of the mapped file contents: https://github.com/leventov/logging-log4j2/blob/ 6d1c0e6b9a73b46ba7dd28ff853694f2012873c0/log4j-core/src/ main/java