Are you running recent MacOS with APFS which supports sparse files? Linux supports sparse files but not all OSs do. You need to run Linux to get reasonable profiling information from the OS that is generally applicable. Then you can use nice tools like those from Brendan Gregg.
https://github.com/brendangregg/perf-tools When considering an algorithm for pre-touch ahead it is often useful to consider rate to determine who far ahead you should be pre-touching. When pre-touching then it can be better with a positional write of a byte, i.e. use https://docs.oracle.com/javase/7/docs/api/java/nio/channels/FileChannel.html#write(java.nio.ByteBuffer,%20long) which should map to pwrite() on Linux. This will be a safepoint whereas the mapped buffer write will not be. You can avoid corruption by gating on the completion of this operation. Get the GC logs and compare GC time to application stopped time. On 10 July 2017 at 17:56, Roman Leventov <[email protected]> wrote: > I'm trying to improve the latency of synchronous logging for Log4j: > https://github.com/apache/logging-log4j2/pull/87 > > One of the means is "write pretouch" of the mapped file contents: > https://github.com/leventov/logging-log4j2/blob/6d > 1c0e6b9a73b46ba7dd28ff853694f2012873c0/log4j-core/src/main/ > java/org/apache/logging/log4j/core/appender/UnsafeMappedByte > BufferWriteTouch.java . Currently it is done as the following: when > logger hits the 1st MB of the file, a task is scheduled to write-pretouch > file contents between 1MB and 2MB marks (i. e. the 2nd MB) in a background > thread. When logger starts to write in the 2nd MB, a task for > write-pretouch of the 3rd MB is scheduled, and so on, i. e. at 1 MB blocks. > > Some testing is done on 4-core MBP, in 3 logging threads: > - new version looks great at 2k messages/sec/thread: https:// > cdn.rawgit.com/leventov/eee84006a53dfb8d2acaebfe126bb499/raw/ > 805c548d1f41e49e67b6815dde84d92be0568012/load2k.html (and pretouch > actually contributes a lot to this result, without pretouch the new version > is just a little better than the old) > - decent at 20k: https://cdn.rawgit.com/leventov/ > eee84006a53dfb8d2acaebfe126bb499/raw/805c548d1f41e49e67b6815dde84d9 > 2be0568012/load20k.html > - but suddenly much worse at 100k messages/sec/thread, than the old > version: https://cdn.rawgit.com/leventov/eee84006a53dfb8d2acaebfe126bb4 > 99/raw/805c548d1f41e49e67b6815dde84d92be0568012/load100k.html > > I tried different approaches to pretouch, but it consistency incurs > extremely long pauses, when running at "pretty extreme" message rates. Only > not doing pretouch at all allows to avoid those pauses. > > Does anybody have a theory what's going on here? Is it more likely the > feature of consumer-grade hardware, or the pretouch approach falls in bad > inference with the MM implementation in OS X? > > -- > You received this message because you are subscribed to the Google Groups > "mechanical-sympathy" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "mechanical-sympathy" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
