Serguei & Nezih,
I've looked at the crash log and it seems like nm->method() is NULL,
which caused the crash.
I am not familiar with the JVMTI internals or nmethods, so I am just
guessing here -- is it legal for nm->method() to be NULL? Or, is it
possible that someone has cleared nm->method() before it's processed by
post_compiled_method_load?
Details from the crash log:
=================================
Native frames: (J=compiled Java code, A=aot compiled Java code,
j=interpreted, Vv=VM code, C=native code)
V [libjvm.so+0x99e365]
JvmtiExport::post_compiled_method_load(nmethod*)+0x275
V [libjvm.so+0x9a6614] JvmtiDeferredEvent::post()+0x44
V [libjvm.so+0xc2c507] ServiceThread::service_thread_entry(JavaThread*,
Thread*)+0x337
V [libjvm.so+0xceee58] JavaThread::thread_main_inner()+0xd8
V [libjvm.so+0xb6de12] thread_native_entry(Thread*)+0xf2
C [libpthread.so.0+0x7e25] start_thread+0xc5
...
R14=0x0000000000000000 is an unknown value
=================================
Here's a disassembly of JvmtiExport::post_compiled_method_load from JDK
9.0.1
http://hg.openjdk.java.net/jdk9/jdk9/hotspot/file/b756e7a2ec33/src/share/vm/prims/jvmtiExport.cpp
2051 HandleMark hm(thread);
0x00007ffff67dd239 <+329>: mov %rax,-0x140(%rbp)
0x00007ffff67dd240 <+336>: callq 0x7ffff66149d0
<HandleMark::initialize(Thread*)>
2054 jvmtiCompiledMethodLoadInlineRecord* inlinerecord =
create_inline_record(nm);
0x00007ffff67dd245 <+341>: mov %r12,%rdi
0x00007ffff67dd248 <+344>: callq 0x7ffff67d4090
<create_inline_record(nmethod*)>
2055 // Pass inlining information through the void pointer
2056 JvmtiCompiledMethodLoadEventMark jem(thread, nm, inlinerecord);
0x00007ffff67dd24d <+349>: mov %rax,-0x1a8(%rbp)
0x00007ffff67dd254 <+356>: mov 0x70(%r12),%rax
0x00007ffff67dd259 <+361>: mov %rbx,-0x178(%rbp)
0x00007ffff67dd260 <+368>: test %rax,%rax
0x00007ffff67dd263 <+371>: mov %rax,-0x180(%rbp)
0x00007ffff67dd26a <+378>: je 0x7ffff67dd282
<JvmtiExport::post_compiled_method_load(nmethod*)+402>
0x00007ffff67dd26c <+380>: mov %rax,-0xa0(%rbp)
0x00007ffff67dd273 <+387>: mov 0x160(%rbx),%rdi
0x00007ffff67dd27a <+394>: mov %r13,%rsi
0x00007ffff67dd27d <+397>: callq 0x7ffff671d5f0
<GrowableArray<Metadata*>::append(Metadata* const&)>
0x00007ffff67dd282 <+402>: lea 0x218(%rbx),%rax
0x00007ffff67dd289 <+409>: mov %rbx,-0xf0(%rbp)
0x00007ffff67dd290 <+416>: movl $0x0,-0xe0(%rbp)
0x00007ffff67dd29a <+426>: mov %rax,-0xe8(%rbp)
0x00007ffff67dd2a1 <+433>: mov 0x3c8(%rbx),%rax
0x00007ffff67dd2a8 <+440>: test %rax,%rax
0x00007ffff67dd2ab <+443>: je 0x7ffff67dd2b6
<JvmtiExport::post_compiled_method_load(nmethod*)+454>
0x00007ffff67dd2ad <+445>: mov 0x10(%rax),%eax
0x00007ffff67dd2b0 <+448>: mov %eax,-0xe0(%rbp)
0x00007ffff67dd2b6 <+454>: mov 0x38(%rbx),%r14
0x00007ffff67dd2ba <+458>: mov %rbx,%rdi
0x00007ffff67dd2bd <+461>: callq 0x7ffff6707d50
<JNIHandleBlock::allocate_block(Thread*)>
0x00007ffff67dd2c2 <+466>: lea 0x1f8(%rbx),%rdi
0x00007ffff67dd2c9 <+473>: mov %rbx,%rsi
0x00007ffff67dd2cc <+476>: mov %r14,0x118(%rax)
0x00007ffff67dd2d3 <+483>: mov %rax,0x38(%rbx)
0x00007ffff67dd2d7 <+487>: callq 0x7ffff6559410
<JavaFrameAnchor::make_walkable(JavaThread*)>
0x00007ffff67dd2dc <+492>: mov 0x1f0(%rbx),%rsi
0x00007ffff67dd2e3 <+499>: mov -0xf0(%rbp),%rdi
0x00007ffff67dd2ea <+506>: callq 0x7ffff6708520
<JNIHandles::make_local(Thread*, oopDesc*)>
0x00007ffff67dd2ef <+511>: mov -0x1a0(%rbp),%rsi
0x00007ffff67dd2f6 <+518>: mov %rax,-0xd8(%rbp)
0x00007ffff67dd2fd <+525>: lea -0x170(%rbp),%rax
0x00007ffff67dd304 <+532>: mov %rax,%rdi
0x00007ffff67dd307 <+535>: mov %rax,-0x1b0(%rbp)
// JvmtiMethodEventMark(JavaThread *thread, methodHandle method) :
// JvmtiThreadEventMark(thread),
234 _mid(to_jmethodID(method)) {}; <-- this line
0x00007ffff67dd30e <+542>: callq 0x7ffff66140a0
<methodHandle::methodHandle(methodHandle const&)>
0x00007ffff67dd313 <+547>: mov -0x170(%rbp),%r14
0x00007ffff67dd31a <+554>: movq $0x0,-0x98(%rbp)
0x00007ffff67dd325 <+565>: test %r14,%r14
0x00007ffff67dd328 <+568>: mov %r14,-0xa0(%rbp)
0x00007ffff67dd32f <+575>: je 0x7ffff67dd365
<JvmtiExport::post_compiled_method_load(nmethod*)+629>
0x00007ffff67dd331 <+577>: data16 lea 0x946ca7(%rip),%rdi
# 0x7ffff7123fe0
0x00007ffff67dd339 <+585>: data16 data16 callq 0x7ffff60b11d0
<__tls_get_addr@plt>
0x00007ffff67dd341 <+593>: lea -0x188(%rbp),%rsi
0x00007ffff67dd348 <+600>: mov %r14,-0x188(%rbp)
0x00007ffff67dd34f <+607>: mov (%rax),%rax
0x00007ffff67dd352 <+610>: mov %rax,-0x98(%rbp)
0x00007ffff67dd359 <+617>: mov 0x160(%rax),%rdi
0x00007ffff67dd360 <+624>: callq 0x7ffff671d5f0
<GrowableArray<Metadata*>::append(Metadata* const&)>
>>> CRASH %r14 == 0x0
198 jmethodID to_jmethodID(methodHandle method) { return
method->jmethod_id(); }
0x00007ffff67dd365 <+629>: mov 0x8(%r14),%rax
Hope this helps
- Ioi
On 5/8/18 4:51 PM, serguei.spit...@oracle.com wrote:
Hi Nezih,
Crashing in the JvmtiExport::post_compiled_method_load function
does not prove itself that it is a VM issue.
And there are mo specific details in the log that help to understand this.
Filing a bug with this log is almost useless as it would not help to
get to the root cause.
Also, there is still a pretty big chance the real problem is in the agent.
My advice is to get as more details about this crash as possible.
The best you can do is to provide a stand alone test case.
Even if the issue is in the agent it will help to figure it out.
Thanks,
Serguei
On 5/8/18 16:38, nezih yigitbasi wrote:
Hi Serguei,
We don't have a repro unfortunately. We have seen this only once so
far in our production environment.
Do you think this can be an agent issue? I thought like if it was an
agent issue the stack trace would show it. The stack trace, however,
ends in the VM code -- the crash happens in the
"JvmtiExport::post_compiled_method_load" method.
Thanks,
Nezih
2018-05-08 16:34 GMT-07:00 serguei.spit...@oracle.com
<mailto:serguei.spit...@oracle.com> <serguei.spit...@oracle.com
<mailto:serguei.spit...@oracle.com>>:
Hi Nezih,
You error file with log does not help much.
Could you, please, reproduce this issue with the debug or
fastdebug build?
Also, a standalone test case is better to provide to
increase chances the issue to be investigated and fixed.
In general, a JVMTI agent has to be written very carefully as it
can easily cause the VM to crash.
So, it makes sense to look at the agent implementation first to
make sure it does not do anything wrong.
It is what we normally do first and why a test case that
demonstrates the problem is needed.
Thanks,
Serguei
On 5/8/18 16:00, nezih yigitbasi wrote:
Hi,
First of all, sorry for bringing up a crash issue in this dev
mailing list, but the crash report submission page
(https://bugreport.java.com/bugreport/crash.jsp
<https://bugreport.java.com/bugreport/crash.jsp>) doesn't list
Java 9 in the release drop down, so I couldn't report it there.
We recently got a crash with Java 9.0.1+11 with an interesting
stack ending at "JvmtiExport::post_compiled_method_load()"
(entire error file is here
<https://gist.github.com/nezihyigitbasi/52a58698cc9acfcab21c69d00bd0cef2>).
A google search didn't end up with much info, so I just wanted
to check with this mailing list to see whether anyone has any
ideas to investigate this further.
Thanks,
Nezih
