Here is what's happening: - The VM sets the signal mask for all threads (except the internal VMThread) to mask out SIGQUIT using pthread_sigmask(). So the process will still respond to SIGQUIT, but only one thread. - The verifier code calls setjmp() to save the calling context. On OS X this also saves the signal mask. - The example code causes a verification error somewhere which causes the verifier to call longjmp(). - longjmp will restore the signal mask using sigprocmask() which sets the signal mask for the _process_. - Now the process has a signal mask that masks out SIGQUIT and attach stops working.
This only happens on OS X because setjmp/longjmp does not save and restore the signal mask on other platforms. There are functions _setjmp/_longjmp on OS X to skip the signal mask save/restore and I think this is what we need to use in the verifier (also need to check other uses of setjmp/longjmp in the JDK). I have filed bug 8023720 for this. Thanks again for the report and reproducer. /Staffan On 23 aug 2013, at 14:44, Staffan Larsen <staffan.lar...@oracle.com> wrote: > Thanks for the excellent bugreport! > > I've had a quick look today but haven't yet figured out what the problem is. > What happens is that somehow the process becomes immune to the SIGQUIT that > jstack sends to it when it wants to attach. Instead of running jstack, one > can also do "kill -QUIT <pid>" and with this code no thread stacks are > printed (as they normally will be). > > I'll continue looking, > /Staffan > > On 23 aug 2013, at 07:20, Martin Traverso <mtrave...@gmail.com> wrote: > >> I have a program that causes jstack and jcmd to fail to attach to the JVM on >> OS X. Unfortunately, it uses a third-party library, so I'm not sure what's >> the magic incantation that causes this to happen. >> >> I wrote a small test case using this library that reproduces the problem. >> You can find the code here: https://github.com/martint/jstackissue. >> >> I've seen the problem on both 1.7.0_25-b15 and 1.8.0-ea-b100 on OS X, but >> not on Java 6. It doesn't seem to happen on Linux, either. >> >> One interesting data point is that if I run jstack before the code makes the >> call to the library, it works, and continues to work even after that call is >> complete. On the other hand, if run jstack for the first time *after* the >> call to the library, it won't work at all. >> >> Any ideas? >> >> Thanks! >> Martin >
