Just FYI, at linkedin we encountered this issue many times. And one of our coworker recently wrote a blog posted about this issue: https://engineering.linkedin.com/blog/2016/02/eliminating-large-jvm-gc-pauses-caused-by-background-io-traffic .
Thanks, -Tao On Wed, Feb 10, 2016 at 1:04 PM, Srinivas Ramakrishna <ysr1...@gmail.com> wrote: > We've run into similar issues at Twitter, and are readying a patch (re # > (4) of Thomas' mail) for buffering in-memory > and writing to disk/file out-of-line wrt safepoints or loggers. The patch > is under test and review internally at this time, > and we hope to publish it for review on Open JDK once internal test and > review are complete. > > Thomas, does unified logging include GC and will it be done outside of > safepoints? In that case, we can use the same > code rather than spinning code specifically for gc logging (as I am doing > in our patch for this). I must confess I haven't > looked at the unified logging jep or its state in the JDK 9 repo. > > thanks. > -- ramki > > On Wed, Feb 10, 2016 at 9:12 AM, Gustav Åkesson < > gustav.r.akes...@gmail.com> wrote: > >> Hi, >> >> Well, AFAIK the loggning IS part of STW cycle. I had troubles with this >> before, which results in GCs with high systime. Since then I always print >> the logs to RAM disk, which solves the problem. >> >> Best Regards, >> Gustav Åkesson >> Den 9 feb 2016 18:46 skrev "Tao Feng" <fengta...@gmail.com>: >> >>> Hi Jenny, Thomas, >>> >>> Thanks a lot for your reply. Indeed we observe high real time in gc >>> which we correlate with high write syscall time. This makes us doubt >>> whether GC logging is part of cycle. >>> >>> Thanks, >>> -Tao >>> >>> On Tue, Feb 9, 2016 at 2:11 AM, Thomas Schatzl < >>> thomas.scha...@oracle.com> wrote: >>> >>>> Hi, >>>> >>>> On Mon, 2016-02-08 at 22:35 -0800, Yu Zhang wrote: >>>> > Tao, >>>> > >>>> > GC logging is not part of GC STW pause. Though we may see application >>>> > being stopped due to gc logging, if there are a lot of I/O activities >>>> > on the system. >>>> > The gc log file is opened by the following >>>> > _fd = open(file_name, O_WRONLY | O_CREAT | O_TRUNC, 0666); >>>> > >>>> > Thanks, >>>> > Jenny >>>> > On 2/4/2016 11:02 PM, Tao Feng wrote: >>>> > > Hi, >>>> > > >>>> > > I have a basic question out of curiosity: is GC logging part of GC >>>> > > cycle? If yes, is it a blocking IO or non-blocking IO? And it would >>>> > > be good if someone could help point out the code base about the GC >>>> > > logging part as I have difficult to identify the code location. >>>> >>>> I think any logging enabled by -Xlog opens the file in asynchronous >>>> write mode, output is buffered. There is no deliberate fsync'ing going >>>> on (again, afaik). >>>> >>>> However, lower levels (OS, file system implementations) might impose >>>> additional synchronization. If a different process saturates I/O to the >>>> disk you want to write to at the same time, the behavior might not be >>>> optimal. :) >>>> >>>> See e.g. http://mail.openjdk.java.net/pipermail/hotspot-gc-use/2014-Apr >>>> il/001835.html >>>> <http://mail.openjdk.java.net/pipermail/hotspot-gc-use/2014-April/001835.html> >>>> . >>>> >>>> If you noticed a problem because of that, there are the following >>>> options: >>>> >>>> 1) use a separate disk for the logging (throwing the "tmpfs" keyword >>>> into this bucket) >>>> >>>> 2) use JFR. Afaik its events are written asynchronously using separate >>>> threads. That might or might not be a problem if you saturate I/O >>>> bandwidth of your target disk for a long time (go to option 1 in this >>>> case) >>>> >>>> 3) experiment with different FS implementations that do less blocking. >>>> >>>> 4) provide a patch to improve this situation :-) >>>> >>>> Thanks, >>>> Thomas >>>> >>>> >>>> >>> >>> _______________________________________________ >>> hotspot-gc-use mailing list >>> hotspot-gc-use@openjdk.java.net >>> http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use >>> >>> >> _______________________________________________ >> hotspot-gc-use mailing list >> hotspot-gc-use@openjdk.java.net >> http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use >> >> >
_______________________________________________ hotspot-gc-use mailing list hotspot-gc-use@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use
