William,
Thanks for the update. Gladly surprised as well.
One explanation is those humongous objects have pointers to the young
gen. Now since we moved them to young gen by increasing region size, we
do not need to update those pointers. In the tip.log, the update
remember set and object copy time is not evenly distributed among the
threads. Maybe g1 is having difficulties with those special kind of
object arrays.
Can you get a heap dump? I am curious what kind of objects those are.
Thanks
Jenny
On 09/08/2016 08:35 AM, William Good wrote:
Hi Jenny,
Attached are logs for runs with 8m and 16m region sizes. This is with
the old version of the application (before the akka patch). I was
pleasantly surprised by the 16m run and tried it a few times, but it
seems these results are reliable.
Thanks,
William
On Tue, Sep 6, 2016 at 6:25 PM, yu.zh...@oracle.com <yu.zh...@oracle.com> wrote:
William,
While this might not help the termination time, I hope adjusting the region
size can get rid of some of the humongous objects.
It seems the humongous objects are filling old gen, about ~5g. The full gc
can clean 10g. Can you try increase region size to 8m or 16m?
Thanks
Jenny
On 09/05/2016 03:02 AM, William Good wrote:
Hi Thomas,
Attached is a log file from a run with this morning's tip. I don't see
any change unfortunately.
I'll try with jdk9 tip later today since you mention there are more fixes
there.
configure summary looked like this, I think everything here is in order:
Configuration summary:
* Debug level: release
* JDK variant: normal
* JVM variants: server
* OpenJDK target: OS: linux, CPU architecture: x86, address length: 64
Tools summary:
* Boot JDK: java version "1.8.0_101" Java(TM) SE Runtime
Environment (build 1.8.0_101-b13) Java HotSpot(TM) 64-Bit Server VM
(build 25.101-b13, mixed mode) (at /var/tmp/jdk1.8.0_101)
* C Compiler: gcc (GCC) 4.8.5 20150623 (Red Hat-4) version 4.8.5
(at /usr/bin/gcc)
* C++ Compiler: g++ (GCC) 4.8.5 20150623 (Red Hat-4) version 4.8.5
(at /usr/bin/g++)
Best,
William
On Fri, Sep 2, 2016 at 4:01 PM, Thomas Schatzl
<thomas.scha...@oracle.com> wrote:
Hi,
On Fri, 2016-09-02 at 15:08 +0200, William Good wrote:
Thomas,
More than happy to build and test with a patched JDK. Let me know
what I need to do, at least what sources to grab (I think I can build
it once I've got them).
I learned this morning that my belief that significant tenuring
wasn't taking place was possibly wrong. Our problem seems to have
disappeared with an akka upgrade [1] and my guess for the relevant
fix [2] indicates that unintended tenuring was occurring at some
point (teh tenuring has never been significant enough for us to
notice). However as long as I'm unable to reproduce with the new
version I'm happy to continue testing using the older version I know
to reproduce, as I don't think this G1 behavior is intended.
this bug report makes me tend to believe that the suggested fix (for
JDK-8152438) will actually fix the issue. Let me explain:
Every G1 thread has two work queues, one fixed size public one where
others can steal from, and one resizable one that is private. Work
(references) is first put into the public one, and then if it is full,
into the private one.
Threads first process their private buffers, so that others can steal
from the public one while they are working on it.
Due to some conditions it can happen that the public queues are already
completely empty, while one thread is still busy for a long time with
its private one. I.e. the work in the private queue can be so that it
never generates more work in the public queue that others can steal and
continue work from. So they wait.
That can cause this high termination time. Of course this situation can
occur multiple times during a GC, so that every thread gets his fair
share of waiting :)
That mentioned fix has been pushed into the http://hg.openjdk.java.net/
jdk8u/jdk8u-dev/ repository. It should be a matter of pulling it.
The README-builds file in the repo has build instructions.
It would be really nice if we could track down your problem to this
issue, or at least significantly improve it. (JDK9 has more significant
patches in that area iirc, but this one is probably the most
important).
Thanks,
Thomas
_______________________________________________
hotspot-gc-use mailing list
hotspot-gc-use@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use
