You can keep a pre-allocated BiConsumer (or multiple ThreadLocal ones) to avoid allocating a new consumer for each event. (I think that’s what we do in Log4j.)
Remko. (Shameless plug) Every java main() method deserves http://picocli.info > On Oct 25, 2018, at 15:50, Volkan Yazıcı <[email protected]> wrote: > > One thing I am still struggling is how to iterate over ContextData and > ContextStack without any garbage. > ContextData#forEach() necessitates instantiation of a new > BiConsumer<String, Object>. > And likewise, ContextStack necessitates instantiation of a new Iterator for > traversal. > Am I missing something? Is there any other way to traverse these > collections without generating garbage? > >> On Tue, Oct 23, 2018 at 2:28 AM Remko Popma <[email protected]> wrote: >> >> Looks very nice! >> >> The non garbage-free implementation is in the toSerializable(LogEvent) >> method. As you point out, this often ends in a call to >> StringBuilder.toString(). >> >> The garbage-free implementation is in the encode(LogEvent, >> ByteBufferDestination) method. This reuses a ThreadLocal StringBuilder. >> >> Be aware of edge cases like occasional very large messages that result in >> the reusable StringBuilder taking up a lot of memory and never releasing >> it. (We trim if some configurable max size is exceeded.) >> You get some of this for free by extending AbstractStringLayout. >> >> I took a brief look at the BENCHMARK.txt file but didn’t have time to look >> in detail. A graph comparison would be very cool! >> >> Good luck, keep us posted! >> >> (Shameless plug) Every java main() method deserves http://picocli.info >> >>> On Oct 23, 2018, at 6:40, Volkan Yazıcı <[email protected]> wrote: >>> >>> Hello, >>> >>> I am working on the next release of log4j2-logstash-layout >>> <https://github.com/vy/log4j2-logstash-layout/tree/json-generator>, >> which >>> is an alternative to the default JsonLayout and allows custom JSON >> schemas. >>> There I achieved to get up to 5x speed up compared to JsonLayout[1] and I >>> want to stretch this further by reducing the GC load. Though it is really >>> difficult given Layout interface (practically) requires an output of type >>> String. I examined the internals of other gc-free layouts, e.g., Pattern, >>> Gelf, etc. Though each ends up making a call to StringBuilder#toString(), >>> which is not gc-free, to the best of my knowledge. Would anyone mind >>> shedding some light onto the issue, please? >>> >>> Best. >>> >>> [1] Please feel free to check JMH benchmarks >>> < >> https://github.com/vy/log4j2-logstash-layout/blob/json-generator/layout/src/test/perf/com/vlkan/log4j2/logstash/layout/LogstashLayoutBenchmarkState.java >>> >>> and their results >>> < >> https://github.com/vy/log4j2-logstash-layout/blob/json-generator/BENCHMARK.txt >>> . >>> Maybe there I am misconfiguring JsonLayout to cause it under-perform. >>
