Hi, I was recently playing around with the OpenJDK build in the bloody repo and noticed that pstack was only able to resolve Java stack frames when I used the 32-bit binary (/usr/bin/i86/pstack):
fa00f402 * java/lang/Thread.sleep(J)V+0 fa00367b * org/apache/tomcat/util/net/JIoEndpoint$AsyncTimeout.run()V+13 (line 294) But the 64-bit pstack binary (/usr/bin/amd64/pstack) just prints question marks instead of class and method names: fa0f08a6 ???????? (85f4a28, 0, 858a778, 0, 0, 0) fa0fae7c ???????? (0, a4d, 0, 4c3, 0, 9ca8ab8f) Looking at the code in illumos's pstack.c it seems like pstack looks for a library called "libjvm.so" loaded into the target Java process, then looks for a library called "libjvm_db.so" in the same directory as libjvm.so and opens it with dlopen(3). libjvm_db.so provides pstack with the jvm-specific knowledge it needs to read java stack frames from the target process. When you are using a 64-bit pstack to debug a 32-bit java process it uses a special 64-bit version of the libjvm_db.so library stored in a sub directory called "64", the directory structure looks like this: /usr/java/lib/i386/server/libjvm.so /usr/java/lib/i386/server/libjvm_db.so /usr/java/lib/i386/server/64/libjvm_db.so Note that this 64-bit libjvm_db.so is still meant for debugging a 32-bit JVM, it's just compiled as a 64-bit library so that a 64-bit pstack process is capable of using it. The problem seems to be that the 64-bit version of libjvm_db.so in the runtime/java package in bloody right now is actually a 32-bit library: $ file /usr/java/lib/i386/server/64/libjvm_db.so /usr/java/lib/i386/server/64/libjvm_db.so: ELF 32-bit ... So the call to dlopen(3) from a 64-bit pstack process fails. The 32-bit Java 7 JDK distributed by Oracle does not have this problem: $ file /usr/java/jre/lib/i386/server/libjvm_db.so /usr/java/jre/lib/i386/server/libjvm_db.so: ELF 32-bit $ file /usr/java/jre/lib/i386/server/64/libjvm_db.so /usr/java/jre/lib/i386/server/64/libjvm_db.so: ELF 64-bit I was able to use both 64-bit and 32-bit pstack to see the Java stack frames of 32-bit Java processes running with Oracle's binaries. I downloaded and started building OpenJDK myself mimicing what it looks like the omni-build repository scripts do (I didn't run the scripts themselves). I let the build run until the 64-bit library was created and confirmed that it was in fact a 64-bit library: $ file ./build/solaris/solaris_i486_compiler2/product/libjvm_db.so ./build/solaris/solaris_i486_compiler2/product/libjvm_db.so: ELF 32-bit $ file ./build/solaris/solaris_i486_compiler2/product/64/libjvm_db.so ./build/solaris/solaris_i486_compiler2/product/64/libjvm_db.so: ELF 64-bit This seems like there is a problem with the way the 64-bit libjvm_db.so files currently in the runtime/java package were built, but I'm not familiar with how OmniTI builds those packages. Could it be they were built on a 32-bit system or something? Otherwise there might be some difference between OpenJDK and whatever repo Oracle bases their builds on? Thanks, Chris _______________________________________________ OmniOS-discuss mailing list OmniOS-discuss@lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss
