Re: RFR(XS): 8218947: jdb threads command should print threadID in decimal, not hex

[serguei . spitsyn]

Hi Chris,

It looks good in general.

One question:
 It seems, the methods toHex() and fromHex() are no longer used and can 
be removed now.


Thanks,
Serguei


On 2/15/19 5:08 PM, Chris Plummer wrote:

Hi,

Please review the updated webrev:

http://cr.openjdk.java.net/~cjplummer/8218947/webrev.02/
https://bugs.openjdk.java.net/browse/JDK-8218947

I removed "hex" from "object description and hex id" and removed the 
java.lang. prefix from java.lang.Long references. There was no need to 
update the man page since it no longer references the "threads" 
command. The entire "Basic jdb Commands" section is gone.


Also, it turns out one test was failing. Not sure how I missed it. 
I've been sitting on this change for months and never noticed it 
before. Maybe I didn't run the right set of tests.


Anyway, the test (somewhat erroneously) searches the entire output 
line for the first thing that looks like a hex number, and assumes it 
is the thread id. If there was something that looked like a hex number 
in the class name, it would fail. I changed it to grab the first token 
after the classname.


BTW, I also learned that in nsk/share/jdb/Jdb.java there is a 
getThreadIds() method that this test uses (along with about a dozen 
other tests). It returns a String[] containing all the thread ids. It 
finds the thread ID in a way similar to what I did. It first skips the 
class name and then grabs everything up to the first space (my code 
goes up to the first whitespace character).


thanks,

Chris

On 2/13/19 7:37 PM, Chris Plummer wrote:

Hi,

Please review the following:

http://cr.openjdk.java.net/~cjplummer/8218947/webrev
https://bugs.openjdk.java.net/browse/JDK-8218947

Tested by running the following on all supported platforms:

open/test/hotspot/jtreg/vmTestbase/nsk/jdb
open/test/jdk/com/sun/jdi

thanks,

Chris

Re: RFR(XS): 8218947: jdb threads command should print threadID in decimal, not hex

[Chris Plummer]

Hi,

Please review the updated webrev:

http://cr.openjdk.java.net/~cjplummer/8218947/webrev.02/
https://bugs.openjdk.java.net/browse/JDK-8218947

I removed "hex" from "object description and hex id" and removed the 
java.lang. prefix from java.lang.Long references. There was no need to 
update the man page since it no longer references the "threads" command. 
The entire "Basic jdb Commands" section is gone.


Also, it turns out one test was failing. Not sure how I missed it. I've 
been sitting on this change for months and never noticed it before. 
Maybe I didn't run the right set of tests.


Anyway, the test (somewhat erroneously) searches the entire output line 
for the first thing that looks like a hex number, and assumes it is the 
thread id. If there was something that looked like a hex number in the 
class name, it would fail. I changed it to grab the first token after 
the classname.


BTW, I also learned that in nsk/share/jdb/Jdb.java there is a 
getThreadIds() method that this test uses (along with about a dozen 
other tests). It returns a String[] containing all the thread ids. It 
finds the thread ID in a way similar to what I did. It first skips the 
class name and then grabs everything up to the first space (my code goes 
up to the first whitespace character).


thanks,

Chris

On 2/13/19 7:37 PM, Chris Plummer wrote:

Hi,

Please review the following:

http://cr.openjdk.java.net/~cjplummer/8218947/webrev
https://bugs.openjdk.java.net/browse/JDK-8218947

Tested by running the following on all supported platforms:

open/test/hotspot/jtreg/vmTestbase/nsk/jdb
open/test/jdk/com/sun/jdi

thanks,

Chris

Re: RFR/C: 8218922: SA: Enable best-effort implementation of live regions iteration for ZGC

[Stefan Karlsson]

Testing showed that the re-enabling of the retiring of TLABs was broken. 
This has been fixed with this patch:


http://cr.openjdk.java.net/~stefank/8218922/webrev.03.delta
http://cr.openjdk.java.net/~stefank/8218922/webrev.03

During mark start we change what's considered the "good" view of the 
heap, and if -XX:+ZUnmapBadViews is turned on we actually unmap the 
previous good view. This happens before TLABs are retired and filler 
objects get written. This used to work when we didn't create filler 
objects for TLABs, but know that we do, we need to ensure that the TLAB 
pointers have the correct colors, otherwise we'll end up crashing when 
-XX:+ZUnmapBadViews are used.


With this fix, the patches passes tier1,tier2, and tier3 testing.

Thanks,
StefanK

On 2019-02-13 15:52, Stefan Karlsson wrote:

Hi all,

Please review / comment on this patch to enable a best-effort live 
heap region iteration implementation in ZGC.


http://cr.openjdk.java.net/~stefank/8218922/webrev.01/
https://bugs.openjdk.java.net/browse/JDK-8218922

The SA has functionally that relies on live heap region information 
from the GCs. This is problematic when dead objects are left in the 
heap, and their classes have been unloaded.


Because of this ZGC has so far not implemented this feature. However, 
we could provide a best-effort implementation that most likely will 
work if classes are not unloaded (or class unloading is turned off), 
and otherwise might fail to fully parse and report all live heap regions.


For active, non-relocating pages the patch simply returns [start, top) 
and for pages being actively relocated, it reports regions containing 
the non-forwarded objects, the other objects are either dead or could 
be found in one of the to-regions.


With this patch I'm able to get heap histograms with ZGC.

Maybe this is enough to enable a bit more SA debugging capabilities 
when running with ZGC? What do you think, should we bring in this change?


To be able to implement this more cleanly I've restructured the live 
region collection, and pushed GC specific code into the specific GCs. 
There are some extra usage of generics to make the code a bit easier 
to read and develop.


Thanks,
StefanK

Re: RFR(S): 8218941: jdb should support a dbgtrace command that acts the same as the dbgtrace command line option

[Chris Plummer]

I followed up on this just to make sure. It turns out we still do some 
localizations, but they are handled automatically near the end of the 
release cycle. So I think the changes are good as-is.


thanks,

Chris

On 2/14/19 6:56 AM, Chris Plummer wrote:
Do you mean for the non-english locales? If I understood correctly 
what I was told a while back, we don't localize anymore, so it 
shouldn't be necessary.


Chris

On 2/14/19 2:11 AM, gary.ad...@oracle.com wrote:
Do you need placeholder entries in the translated TTYResources 
message files?


On 2/13/19 9:43 PM, Chris Plummer wrote:

Hi,

Please review the following:

http://cr.openjdk.java.net/~cjplummer/8218941/webrev
https://bugs.openjdk.java.net/browse/JDK-8218941

Tested by running the following on all supported platforms:

open/test/hotspot/jtreg/vmTestbase/nsk/jdb
open/test/jdk/com/sun/jdi

thanks,

Chris

Re: RFR: JDK-8149461: jmap kills process if non-java pid is specified in the command line

[Chris Plummer]

  
  
Is anyone setup to try this with
  docker? I know initially jvm's running on the same host OS but in
  different docker containers would not show up when requesting a
  list of JVMs, but could be attached to when a specific PID was
  provided. I believe all these issues are now fixed, but would be
  good to double check that this change is not interfering in any
  way.


Chris



On 2/15/19 10:39 AM, Gary Adams wrote:


  
  It isn't pretty, but it's functional : "-f to force
  communication. ...
  
  � Revised webrev: http://cr.openjdk.java.net/~gadams/8149461/webrev.02/
  
  On 2/15/19, 11:57 AM, Daniel D. Daugherty wrote:
  

Yes. That's the direction I was thinking about.
  Don't know about the '-F' versus '-F ',
  but
  that a cmd line option parsing detail.
  
  Dan
  

On 2/15/19 11:37 AM, Gary Adams
  wrote:


  
  Here's a quick dirty "-F" that gets past
  the "-XX:-UsePerfData" setting for jcmd.
  Need to follow up on docs and usage 
  for the other commands.
  
  Is this the direction you were thinking?
  
  diff --git
a/src/jdk.jcmd/share/classes/sun/tools/common/ProcessArgumentMatcher.javab/src/jdk.jcmd/share/classes/sun/tools/common/ProcessArgumentMatcher.java
---
a/src/jdk.jcmd/share/classes/sun/tools/common/ProcessArgumentMatcher.java
+++
b/src/jdk.jcmd/share/classes/sun/tools/common/ProcessArgumentMatcher.java
@@ -48,16 +48,27 @@
�public class ProcessArgumentMatcher {
 private String matchClass;
 private String singlePid;
+��� private static boolean bypassPid;
+��� private long pid;
�
 public ProcessArgumentMatcher(String pidArg) {
 if (pidArg == null || pidArg.isEmpty()) {
 throw new IllegalArgumentException("Pid string
is invalid");
 }
 if (pidArg.charAt(0) == '-') {
+��� if (pidArg.charAt(1) == 'F') {
+��� // Allow -F to bypass the pid
check
+��� pid = Long.parseLong(pidArg.substring(2));
+��� if (pid != 0) {
+��� singlePid = String.valueOf(pid);
+��� }
+��� bypassPid = true;
+��� } else {
 throw new
IllegalArgumentException("Unrecognized " + pidArg);
 }
+��� }
 try {
-��� long pid = Long.parseLong(pidArg);
+��� pid = Long.parseLong(pidArg);
 if (pid != 0) {
 singlePid = String.valueOf(pid);
 }
@@ -170,4 +181,21 @@
 public Collection getVirtualMachinePids()
{
 return this.getVirtualMachinePids(null);
 }
+
+� // Check that pid matches a known running Java
process
+� public static boolean checkJavaPid(String pid) {
+� // Skip the perf data pid visibility check if
"-F" requested.
+� if (bypassPid) {
+� return true;
�}
+� List l =
VirtualMachine.list();
+� boolean found = false;
+� for (VirtualMachineDescriptor vmd: l) {
+� if (vmd.id().equals(pid)) {
+� found = true;
+� break;
+� }
+� }
+� return found;
+� }
+}
  On 2/15/19, 10:24 AM, Daniel D. Daugherty wrote:
  

On 2/15/19 8:44 AM, Gary Adams wrote:

  
  Confirmed
  � -XX:-UsePerfData
  
  prevents visibility to jps, but allows direct access via
  pid.
  
  The new check would block access before the attach is
  attempted.
  
  May be best to close this bug as "will not fix".
  Requires a valid Java process pid.

�
  Or make it a best effort solution. The idea that a jmap on
  a non-Java
  PID could kill that PID violates the "do no harm"
  principle for an
  observer tool.
  
  Of what I have seen on the thread so far, I like these:
  
  - make the PID 

Re: RFR/C: 8218922: SA: Enable best-effort implementation of live regions iteration for ZGC

[Stefan Karlsson]

Hi Kevin,

On 2019-02-15 13:14, Kevin Walls wrote:

Hi Stefan,

This is great - if I want to try it out, what order do the changes 
apply, does the 8218746 webrev apply to create ZExternalBitMap.java 
and then the 8218922 webrev?

This is my patch queue of outgoing patches:
 zSA.fixVMObjectFactory.8218731
 zSA.fixPhysical.8218732
 zSA.heapUsedCapacity.8218733
 zSA.resolveOopHandle.8218734
 zSA.bitMapsSegmented.8218743
 zSA.bitMapsZ.8218746
 zSA.pagesIterate.refactor.8219003
 zSA.pagesIterate.8218922.Z
 zSA.turnOnHprof.8218970
 zSA.turnOnTests.8218978

I've created one big webrev over all these changes:
 https://cr.openjdk.java.net/~stefank/zgc/zSABitMapsAndLiveRegions/webrev/

StefanK



Thanks
Kevin


On 14/02/2019 17:12, Stefan Karlsson wrote:

Hi again,

I've separated the live regions iteration refactoring into this patch:
https://cr.openjdk.java.net/~stefank/8219003/webrev.01/

And use this RFE for the ZGC specific parts:
https://cr.openjdk.java.net/~stefank/8218922/webrev.02/

Thanks,
StefanK

On 2019-02-14 14:39, Stefan Karlsson wrote:

Hi Yasumasa,

On 2019-02-14 14:11, Yasumasa Suenaga wrote:

Hi Stefan,

Maybe this is enough to enable a bit more SA debugging 
capabilities when

running with ZGC? What do you think, should we bring in this change?


I think it should be brought this in.
I filed same issue as JDK-8207843, but I've not yet worked.
So I will close it as duplicate of your change.



To be able to implement this more cleanly I've restructured the live
region collection, and pushed GC specific code into the specific GCs.
There are some extra usage of generics to make the code a bit 
easier to

read and develop.


IMHO refactoring for live region collection and ZGC related changes 
should

be separated. What do you think?


Yes. I think that would be good. I'll separate this out into two 
changes.





Your change looks good to me.
BTW, did you check `jhsdb jmap --binaryheap` with this change?



Yes, when testing this I ran all tests in serviceability/sa and 
manually tested the command above.


While testing this I also had this patch applied to enable SA hprof 
for ZGC:

http://cr.openjdk.java.net/~stefank/8218970/webrev.01/

And used this patch to turn off the ZGC filtering in the tests:
http://cr.openjdk.java.net/~stefank/8218978/webrev.01/

I'm currently rerunning the tests to see that the latest changes 
didn't break anything.


Thanks,
StefanK


Thanks,

Yasumasa



On 2019/02/13 23:52, Stefan Karlsson wrote:

Hi all,

Please review / comment on this patch to enable a best-effort live 
heap

region iteration implementation in ZGC.

http://cr.openjdk.java.net/~stefank/8218922/webrev.01/
https://bugs.openjdk.java.net/browse/JDK-8218922

The SA has functionally that relies on live heap region 
information from
the GCs. This is problematic when dead objects are left in the 
heap, and

their classes have been unloaded.

Because of this ZGC has so far not implemented this feature. 
However, we
could provide a best-effort implementation that most likely will 
work if

classes are not unloaded (or class unloading is turned off), and
otherwise might fail to fully parse and report all live heap regions.

For active, non-relocating pages the patch simply returns [start, 
top)

and for pages being actively relocated, it reports regions containing
the non-forwarded objects, the other objects are either dead or 
could be

found in one of the to-regions.

With this patch I'm able to get heap histograms with ZGC.

Maybe this is enough to enable a bit more SA debugging 
capabilities when

running with ZGC? What do you think, should we bring in this change?

To be able to implement this more cleanly I've restructured the live
region collection, and pushed GC specific code into the specific GCs.
There are some extra usage of generics to make the code a bit 
easier to

read and develop.

Thanks,
StefanK

Re: RFR: JDK-8149461: jmap kills process if non-java pid is specified in the command line

[Gary Adams]

It isn't pretty, but it's functional : "-f to force communication. ...

  Revised webrev: http://cr.openjdk.java.net/~gadams/8149461/webrev.02/

On 2/15/19, 11:57 AM, Daniel D. Daugherty wrote:

Yes. That's the direction I was thinking about.
Don't know about the '-F' versus '-F ', but
that a cmd line option parsing detail.

Dan


On 2/15/19 11:37 AM, Gary Adams wrote:

Here's a quick dirty "-F" that gets past
the "-XX:-UsePerfData" setting for jcmd.
Need to follow up on docs and usage
for the other commands.

Is this the direction you were thinking?

diff --git 
a/src/jdk.jcmd/share/classes/sun/tools/common/ProcessArgumentMatcher.javab/src/jdk.jcmd/share/classes/sun/tools/common/ProcessArgumentMatcher.java
--- 
a/src/jdk.jcmd/share/classes/sun/tools/common/ProcessArgumentMatcher.java
+++ 
b/src/jdk.jcmd/share/classes/sun/tools/common/ProcessArgumentMatcher.java

@@ -48,16 +48,27 @@
 public class ProcessArgumentMatcher {
 private String matchClass;
 private String singlePid;
+private static boolean bypassPid;
+private long pid;

 public ProcessArgumentMatcher(String pidArg) {
 if (pidArg == null || pidArg.isEmpty()) {
 throw new IllegalArgumentException("Pid string is invalid");
 }
 if (pidArg.charAt(0) == '-') {
+if (pidArg.charAt(1) == 'F') {
+// Allow -F to bypass the pid check
+pid = Long.parseLong(pidArg.substring(2));
+if (pid != 0) {
+singlePid = String.valueOf(pid);
+}
+bypassPid = true;
+} else {
 throw new IllegalArgumentException("Unrecognized " + 
pidArg);

 }
+}
 try {
-long pid = Long.parseLong(pidArg);
+pid = Long.parseLong(pidArg);
 if (pid != 0) {
 singlePid = String.valueOf(pid);
 }
@@ -170,4 +181,21 @@
 public Collection getVirtualMachinePids() {
 return this.getVirtualMachinePids(null);
 }
+
+  // Check that pid matches a known running Java process
+  public static boolean checkJavaPid(String pid) {
+  // Skip the perf data pid visibility check if "-F" 
requested.

+  if (bypassPid) {
+  return true;
 }
+  List l = VirtualMachine.list();
+  boolean found = false;
+  for (VirtualMachineDescriptor vmd: l) {
+  if (vmd.id().equals(pid)) {
+  found = true;
+  break;
+  }
+  }
+  return found;
+  }
+}
On 2/15/19, 10:24 AM, Daniel D. Daugherty wrote:

On 2/15/19 8:44 AM, Gary Adams wrote:

Confirmed
  -XX:-UsePerfData

prevents visibility to jps, but allows direct access via pid.

The new check would block access before the attach is attempted.

May be best to close this bug as "will not fix".
Requires a valid Java process pid.


Or make it a best effort solution. The idea that a jmap on a non-Java
PID could kill that PID violates the "do no harm" principle for an
observer tool.

Of what I have seen on the thread so far, I like these:

- make the PID check on the specified PID only (should be pretty fast)
- add a force option that tries to attach to the PID regardless
  of what the sanity check says; that will solve the -XX:-UsePerfData
  problem.

Writing/updating tests for this is going to be "interesting".

Dan





On 2/15/19, 8:29 AM, Bernd Eckenfels wrote:
I wonder, instead of listing all VMs, would it be better to check 
only the target PID?


BTW speaking of hs_perf files: is it supposed to work without 
-XX:-UserPerfData also?


Gruss
Bernd

Gruss
Bernd
--
http://bernd.eckenfels.net

*Von:* Gary Adams 
*Gesendet:* Freitag, Februar 15, 2019 2:19 PM
*An:* gary.ad...@oracle.com
*Cc:* Bernd Eckenfels; OpenJDK Serviceability
*Betreff:* Re: RFR: JDK-8149461: jmap kills process if non-java 
pid is specified in the command line

On a linux system with 1 Java process and 500 non-Java processes,
/tmp is not tmpfs mounted, 20 runs average 255.5 ms stddev 9.32.

JDK-8176828 is the issue that enabled perfmemory visibility once 
the vm is in live mode.


For containers that are configured without a shared view of /tmp, 
it may be necessary

to include a override of the pid check.

On 2/15/19, 7:29 AM, Gary Adams wrote:

I'll get some measurements on some local systems so we have a
specific idea about the overhead of the pid check.
I imagine in most production environments the /tmp tmpfs
is memory only set of checks.

One of the "not all vms are recognized" was fixed recently.
When starting a libjdwp session with server=y and suspend=y,
the vm was not recognized until a debugger was attached.

I'm not opposed to adding a force option to skip the valid pid
check. It might be better effort fixing the hung vm case if we
have a concrete failure to investigate.

On 2/15/19, 6:47 AM, Bernd Eckenfels 

Re: RFR: JDK-8149461: jmap kills process if non-java pid is specified in the command line

[Daniel D. Daugherty]

Yes. That's the direction I was thinking about.
Don't know about the '-F' versus '-F ', but
that a cmd line option parsing detail.

Dan


On 2/15/19 11:37 AM, Gary Adams wrote:

Here's a quick dirty "-F" that gets past
the "-XX:-UsePerfData" setting for jcmd.
Need to follow up on docs and usage
for the other commands.

Is this the direction you were thinking?

diff --git 
a/src/jdk.jcmd/share/classes/sun/tools/common/ProcessArgumentMatcher.javab/src/jdk.jcmd/share/classes/sun/tools/common/ProcessArgumentMatcher.java
--- 
a/src/jdk.jcmd/share/classes/sun/tools/common/ProcessArgumentMatcher.java
+++ 
b/src/jdk.jcmd/share/classes/sun/tools/common/ProcessArgumentMatcher.java

@@ -48,16 +48,27 @@
 public class ProcessArgumentMatcher {
 private String matchClass;
 private String singlePid;
+    private static boolean bypassPid;
+    private long pid;

 public ProcessArgumentMatcher(String pidArg) {
 if (pidArg == null || pidArg.isEmpty()) {
 throw new IllegalArgumentException("Pid string is invalid");
 }
 if (pidArg.charAt(0) == '-') {
+    if (pidArg.charAt(1) == 'F') {
+    // Allow -F to bypass the pid check
+    pid = Long.parseLong(pidArg.substring(2));
+    if (pid != 0) {
+    singlePid = String.valueOf(pid);
+    }
+    bypassPid = true;
+    } else {
 throw new IllegalArgumentException("Unrecognized " + pidArg);
 }
+    }
 try {
-    long pid = Long.parseLong(pidArg);
+    pid = Long.parseLong(pidArg);
 if (pid != 0) {
 singlePid = String.valueOf(pid);
 }
@@ -170,4 +181,21 @@
 public Collection getVirtualMachinePids() {
 return this.getVirtualMachinePids(null);
 }
+
+  // Check that pid matches a known running Java process
+  public static boolean checkJavaPid(String pid) {
+  // Skip the perf data pid visibility check if "-F" 
requested.

+  if (bypassPid) {
+  return true;
 }
+  List l = VirtualMachine.list();
+  boolean found = false;
+  for (VirtualMachineDescriptor vmd: l) {
+  if (vmd.id().equals(pid)) {
+  found = true;
+  break;
+  }
+  }
+  return found;
+  }
+}
On 2/15/19, 10:24 AM, Daniel D. Daugherty wrote:

On 2/15/19 8:44 AM, Gary Adams wrote:

Confirmed
  -XX:-UsePerfData

prevents visibility to jps, but allows direct access via pid.

The new check would block access before the attach is attempted.

May be best to close this bug as "will not fix".
Requires a valid Java process pid.


Or make it a best effort solution. The idea that a jmap on a non-Java
PID could kill that PID violates the "do no harm" principle for an
observer tool.

Of what I have seen on the thread so far, I like these:

- make the PID check on the specified PID only (should be pretty fast)
- add a force option that tries to attach to the PID regardless
  of what the sanity check says; that will solve the -XX:-UsePerfData
  problem.

Writing/updating tests for this is going to be "interesting".

Dan





On 2/15/19, 8:29 AM, Bernd Eckenfels wrote:
I wonder, instead of listing all VMs, would it be better to check 
only the target PID?


BTW speaking of hs_perf files: is it supposed to work without 
-XX:-UserPerfData also?


Gruss
Bernd

Gruss
Bernd
--
http://bernd.eckenfels.net

*Von:* Gary Adams 
*Gesendet:* Freitag, Februar 15, 2019 2:19 PM
*An:* gary.ad...@oracle.com
*Cc:* Bernd Eckenfels; OpenJDK Serviceability
*Betreff:* Re: RFR: JDK-8149461: jmap kills process if non-java pid 
is specified in the command line

On a linux system with 1 Java process and 500 non-Java processes,
/tmp is not tmpfs mounted, 20 runs average 255.5 ms stddev 9.32.

JDK-8176828 is the issue that enabled perfmemory visibility once 
the vm is in live mode.


For containers that are configured without a shared view of /tmp, 
it may be necessary

to include a override of the pid check.

On 2/15/19, 7:29 AM, Gary Adams wrote:

I'll get some measurements on some local systems so we have a
specific idea about the overhead of the pid check.
I imagine in most production environments the /tmp tmpfs
is memory only set of checks.

One of the "not all vms are recognized" was fixed recently.
When starting a libjdwp session with server=y and suspend=y,
the vm was not recognized until a debugger was attached.

I'm not opposed to adding a force option to skip the valid pid
check. It might be better effort fixing the hung vm case if we
have a concrete failure to investigate.

On 2/15/19, 6:47 AM, Bernd Eckenfels wrote:

Hello,

I see possible issues here, not sure if they still exist but I 
wanted to mention them:


the list-vm function might be slow on a loaded system (as it is a 
complex function). It’s not 

Re: RFR: JDK-8149461: jmap kills process if non-java pid is specified in the command line

[Gary Adams]

Here's a quick dirty "-F" that gets past
the "-XX:-UsePerfData" setting for jcmd.
Need to follow up on docs and usage
for the other commands.

Is this the direction you were thinking?

diff --git 
a/src/jdk.jcmd/share/classes/sun/tools/common/ProcessArgumentMatcher.java b/src/jdk.jcmd/share/classes/sun/tools/common/ProcessArgumentMatcher.java
--- 
a/src/jdk.jcmd/share/classes/sun/tools/common/ProcessArgumentMatcher.java
+++ 
b/src/jdk.jcmd/share/classes/sun/tools/common/ProcessArgumentMatcher.java

@@ -48,16 +48,27 @@
 public class ProcessArgumentMatcher {
 private String matchClass;
 private String singlePid;
+private static boolean bypassPid;
+private long pid;

 public ProcessArgumentMatcher(String pidArg) {
 if (pidArg == null || pidArg.isEmpty()) {
 throw new IllegalArgumentException("Pid string is invalid");
 }
 if (pidArg.charAt(0) == '-') {
+if (pidArg.charAt(1) == 'F') {
+// Allow -F to bypass the pid check
+pid = Long.parseLong(pidArg.substring(2));
+if (pid != 0) {
+singlePid = String.valueOf(pid);
+}
+bypassPid = true;
+} else {
 throw new IllegalArgumentException("Unrecognized " + pidArg);
 }
+}
 try {
-long pid = Long.parseLong(pidArg);
+pid = Long.parseLong(pidArg);
 if (pid != 0) {
 singlePid = String.valueOf(pid);
 }
@@ -170,4 +181,21 @@
 public Collection getVirtualMachinePids() {
 return this.getVirtualMachinePids(null);
 }
+
+  // Check that pid matches a known running Java process
+  public static boolean checkJavaPid(String pid) {
+  // Skip the perf data pid visibility check if "-F" 
requested.

+  if (bypassPid) {
+  return true;
 }
+  List l = VirtualMachine.list();
+  boolean found = false;
+  for (VirtualMachineDescriptor vmd: l) {
+  if (vmd.id().equals(pid)) {
+  found = true;
+  break;
+  }
+  }
+  return found;
+  }
+}
On 2/15/19, 10:24 AM, Daniel D. Daugherty wrote:

On 2/15/19 8:44 AM, Gary Adams wrote:

Confirmed
  -XX:-UsePerfData

prevents visibility to jps, but allows direct access via pid.

The new check would block access before the attach is attempted.

May be best to close this bug as "will not fix".
Requires a valid Java process pid.


Or make it a best effort solution. The idea that a jmap on a non-Java
PID could kill that PID violates the "do no harm" principle for an
observer tool.

Of what I have seen on the thread so far, I like these:

- make the PID check on the specified PID only (should be pretty fast)
- add a force option that tries to attach to the PID regardless
  of what the sanity check says; that will solve the -XX:-UsePerfData
  problem.

Writing/updating tests for this is going to be "interesting".

Dan





On 2/15/19, 8:29 AM, Bernd Eckenfels wrote:
I wonder, instead of listing all VMs, would it be better to check 
only the target PID?


BTW speaking of hs_perf files: is it supposed to work without 
-XX:-UserPerfData also?


Gruss
Bernd

Gruss
Bernd
--
http://bernd.eckenfels.net

*Von:* Gary Adams 
*Gesendet:* Freitag, Februar 15, 2019 2:19 PM
*An:* gary.ad...@oracle.com
*Cc:* Bernd Eckenfels; OpenJDK Serviceability
*Betreff:* Re: RFR: JDK-8149461: jmap kills process if non-java pid 
is specified in the command line

On a linux system with 1 Java process and 500 non-Java processes,
/tmp is not tmpfs mounted, 20 runs average 255.5 ms stddev 9.32.

JDK-8176828 is the issue that enabled perfmemory visibility once the 
vm is in live mode.


For containers that are configured without a shared view of /tmp, it 
may be necessary

to include a override of the pid check.

On 2/15/19, 7:29 AM, Gary Adams wrote:

I'll get some measurements on some local systems so we have a
specific idea about the overhead of the pid check.
I imagine in most production environments the /tmp tmpfs
is memory only set of checks.

One of the "not all vms are recognized" was fixed recently.
When starting a libjdwp session with server=y and suspend=y,
the vm was not recognized until a debugger was attached.

I'm not opposed to adding a force option to skip the valid pid
check. It might be better effort fixing the hung vm case if we
have a concrete failure to investigate.

On 2/15/19, 6:47 AM, Bernd Eckenfels wrote:

Hello,

I see possible issues here, not sure if they still exist but I 
wanted to mention them:


the list-vm function might be slow on a loaded system (as it is a 
complex function). It’s not the best Situation if your diagnostic 
attempts are slow down in such a situation.


Also in the past not all VMs might be recognized by the list 
function, a more targeted 

Re: RFR: JDK-8149461: jmap kills process if non-java pid is specified in the command line

[Daniel D. Daugherty]

On 2/15/19 8:44 AM, Gary Adams wrote:

Confirmed
  -XX:-UsePerfData

prevents visibility to jps, but allows direct access via pid.

The new check would block access before the attach is attempted.

May be best to close this bug as "will not fix".
Requires a valid Java process pid.


Or make it a best effort solution. The idea that a jmap on a non-Java
PID could kill that PID violates the "do no harm" principle for an
observer tool.

Of what I have seen on the thread so far, I like these:

- make the PID check on the specified PID only (should be pretty fast)
- add a force option that tries to attach to the PID regardless
  of what the sanity check says; that will solve the -XX:-UsePerfData
  problem.

Writing/updating tests for this is going to be "interesting".

Dan





On 2/15/19, 8:29 AM, Bernd Eckenfels wrote:
I wonder, instead of listing all VMs, would it be better to check 
only the target PID?


BTW speaking of hs_perf files: is it supposed to work without 
-XX:-UserPerfData also?


Gruss
Bernd

Gruss
Bernd
--
http://bernd.eckenfels.net

*Von:* Gary Adams 
*Gesendet:* Freitag, Februar 15, 2019 2:19 PM
*An:* gary.ad...@oracle.com
*Cc:* Bernd Eckenfels; OpenJDK Serviceability
*Betreff:* Re: RFR: JDK-8149461: jmap kills process if non-java pid 
is specified in the command line

On a linux system with 1 Java process and 500 non-Java processes,
/tmp is not tmpfs mounted, 20 runs average 255.5 ms stddev 9.32.

JDK-8176828 is the issue that enabled perfmemory visibility once the 
vm is in live mode.


For containers that are configured without a shared view of /tmp, it 
may be necessary

to include a override of the pid check.

On 2/15/19, 7:29 AM, Gary Adams wrote:

I'll get some measurements on some local systems so we have a
specific idea about the overhead of the pid check.
I imagine in most production environments the /tmp tmpfs
is memory only set of checks.

One of the "not all vms are recognized" was fixed recently.
When starting a libjdwp session with server=y and suspend=y,
the vm was not recognized until a debugger was attached.

I'm not opposed to adding a force option to skip the valid pid
check. It might be better effort fixing the hung vm case if we
have a concrete failure to investigate.

On 2/15/19, 6:47 AM, Bernd Eckenfels wrote:

Hello,

I see possible issues here, not sure if they still exist but I 
wanted to mention them:


the list-vm function might be slow on a loaded system (as it is a 
complex function). It’s not the best Situation if your diagnostic 
attempts are slow down in such a situation.


Also in the past not all VMs might be recognized by the list 
function, a more targeted attach could still succeed. Is that 
addressed since the container-PID changes? In both cases a force 
option would help.


Gruss
Bernd

Gruss
Bernd
--
http://bernd.eckenfels.net

*Von:* serviceability-dev 
 im Auftrag von 
gary.ad...@oracle.com

*Gesendet:* Freitag, Februar 15, 2019 12:07 PM
*An:* Thomas Stüfe; David Holmes; Chris Plummer
*Cc:* OpenJDK Serviceability
*Betreff:* Re: RFR: JDK-8149461: jmap kills process if non-java pid 
is specified in the command line

Let me clarify a few things about the proposed fix.

The VirtualMachine.list() mechanism is based on
Java processes that are up and running.
During VM startup when agent libraries are loaded,
the fact is recorded in the filesystem that a Java process
is eligible for an attach request.

This is a much smaller list than scanning for all the
running processes and works across all supported
platforms. It also doesn't rely on fragile command line
parsing to determine a Java launcher is used.

I believe most of the reported hang situations
are not for the first level information for pid and
command line requests. I believe the hangs are due
to the second level requests that actually attach
to the process and issue a command to the running
Java process.

The overhead for the pid check should be relatively small.
In a standalone command for a one time check at startup
with 10's of Java processes. I know the documentation
states the user is responsible for verifying with jps
that a Java process pid or vmid is provided. But the cost
of human error can be a terminated process.

Selection by name also has some drawbacks when multiple
copies of a command are running at the same time.

Running "jcmd 0 ..." is the documented way to run a command on
all running Java processes.

May I count you as a reviewer on the current changeset?

On 2/15/19 3:54 AM, Thomas Stüfe wrote:



On Fri, Feb 15, 2019 at 3:26 AM David Holmes 
mailto:david.hol...@oracle.com>> wrote:


Gary,

What is the overhead of doing the validation? How do we list
VMs? Do we
need to examine every running process to get the list of VMs?
Wouldn't
it be better to check the given process is a VM rather than
checking all
   

Re: RFR: JDK-8149461: jmap kills process if non-java pid is specified in the command line

[Thomas Stüfe]

Could we not just use another signal? One less... quitty? I think SIGUSR2
may still be unused, but I may be mistaken. Or, one of the real time
signals (SIGRTMIN + x).

Other than that, I still think that knowing the pid there could be platform
dependent ways of verifying the process, e.g. checking that the process has
a libjvm.so module loaded (/proc//maps). Doing this for just one pid
should be cheap.

Cheers, Thomas




On Fri, Feb 15, 2019 at 2:46 PM Gary Adams  wrote:

> Confirmed
>   -XX:-UsePerfData
>
> prevents visibility to jps, but allows direct access via pid.
>
> The new check would block access before the attach is attempted.
>
> May be best to close this bug as "will not fix".
> Requires a valid Java process pid.
>
> On 2/15/19, 8:29 AM, Bernd Eckenfels wrote:
>
> I wonder, instead of listing all VMs, would it be better to check only the
> target PID?
>
> BTW speaking of hs_perf files: is it supposed to work without
> -XX:-UserPerfData also?
>
> Gruss
> Bernd
>
> Gruss
> Bernd
> --
> http://bernd.eckenfels.net
>
> --
> *Von:* Gary Adams  
> *Gesendet:* Freitag, Februar 15, 2019 2:19 PM
> *An:* gary.ad...@oracle.com
> *Cc:* Bernd Eckenfels; OpenJDK Serviceability
> *Betreff:* Re: RFR: JDK-8149461: jmap kills process if non-java pid is
> specified in the command line
>
> On a linux system with 1 Java process and 500 non-Java processes,
> /tmp is not tmpfs mounted, 20 runs average 255.5 ms stddev 9.32.
>
> JDK-8176828 is the issue that enabled perfmemory visibility once the vm is
> in live mode.
>
> For containers that are configured without a shared view of /tmp, it may
> be necessary
> to include a override of the pid check.
>
> On 2/15/19, 7:29 AM, Gary Adams wrote:
>
> I'll get some measurements on some local systems so we have a
> specific idea about the overhead of the pid check.
> I imagine in most production environments the /tmp tmpfs
> is memory only set of checks.
>
> One of the "not all vms are recognized" was fixed recently.
> When starting a libjdwp session with server=y and suspend=y,
> the vm was not recognized until a debugger was attached.
>
> I'm not opposed to adding a force option to skip the valid pid
> check. It might be better effort fixing the hung vm case if we
> have a concrete failure to investigate.
>
> On 2/15/19, 6:47 AM, Bernd Eckenfels wrote:
>
> Hello,
>
> I see possible issues here, not sure if they still exist but I wanted to
> mention them:
>
> the list-vm function might be slow on a loaded system (as it is a complex
> function). It’s not the best Situation if your diagnostic attempts are slow
> down in such a situation.
>
> Also in the past not all VMs might be recognized by the list function, a
> more targeted attach could still succeed. Is that addressed since the
> container-PID changes? In both cases a force option would help.
>
> Gruss
> Bernd
>
> Gruss
> Bernd
> --
> http://bernd.eckenfels.net
>
> --
> *Von:* serviceability-dev 
>  im Auftrag von
> gary.ad...@oracle.com
> *Gesendet:* Freitag, Februar 15, 2019 12:07 PM
> *An:* Thomas Stüfe; David Holmes; Chris Plummer
> *Cc:* OpenJDK Serviceability
> *Betreff:* Re: RFR: JDK-8149461: jmap kills process if non-java pid is
> specified in the command line
>
> Let me clarify a few things about the proposed fix.
>
> The VirtualMachine.list() mechanism is based on
> Java processes that are up and running.
> During VM startup when agent libraries are loaded,
> the fact is recorded in the filesystem that a Java process
> is eligible for an attach request.
>
> This is a much smaller list than scanning for all the
> running processes and works across all supported
> platforms. It also doesn't rely on fragile command line
> parsing to determine a Java launcher is used.
>
> I believe most of the reported hang situations
> are not for the first level information for pid and
> command line requests. I believe the hangs are due
> to the second level requests that actually attach
> to the process and issue a command to the running
> Java process.
>
> The overhead for the pid check should be relatively small.
> In a standalone command for a one time check at startup
> with 10's of Java processes. I know the documentation
> states the user is responsible for verifying with jps
> that a Java process pid or vmid is provided. But the cost
> of human error can be a terminated process.
>
> Selection by name also has some drawbacks when multiple
> copies of a command are running at the same time.
>
> Running "jcmd 0 ..." is the documented way to run a command on
> all running Java processes.
>
> May I count you as a reviewer on the current changeset?
>
> On 2/15/19 3:54 AM, Thomas Stüfe wrote:
>
>
>
> On Fri, Feb 15, 2019 at 3:26 AM David Holmes 
> wrote:
>
>> Gary,
>>
>> What is the overhead of doing the validation? How do we list VMs? Do we
>> need to examine every running process to get the list of VMs? Wouldn't
>> it be better to check the given process is a VM rather 

Re: RFR: JDK-8149461: jmap kills process if non-java pid is specified in the command line

[Thomas Stüfe]

On Fri, Feb 15, 2019 at 3:19 PM Thomas Stüfe 
wrote:

> Could we not just use another signal? One less... quitty? I think SIGUSR2
> may still be unused, but I may be mistaken. Or, one of the real time
> signals (SIGRTMIN + x).
>
>
wait, that one would break downward compatibility. Still want to be able to
use new jcmds with older hotspots.


> Other than that, I still think that knowing the pid there could be
> platform dependent ways of verifying the process, e.g. checking that the
> process has a libjvm.so module loaded (/proc//maps). Doing this for
> just one pid should be cheap.
>
> Cheers, Thomas
>
>

>
>
> On Fri, Feb 15, 2019 at 2:46 PM Gary Adams  wrote:
>
>> Confirmed
>>   -XX:-UsePerfData
>>
>> prevents visibility to jps, but allows direct access via pid.
>>
>> The new check would block access before the attach is attempted.
>>
>> May be best to close this bug as "will not fix".
>> Requires a valid Java process pid.
>>
>> On 2/15/19, 8:29 AM, Bernd Eckenfels wrote:
>>
>> I wonder, instead of listing all VMs, would it be better to check only
>> the target PID?
>>
>> BTW speaking of hs_perf files: is it supposed to work without
>> -XX:-UserPerfData also?
>>
>> Gruss
>> Bernd
>>
>> Gruss
>> Bernd
>> --
>> http://bernd.eckenfels.net
>>
>> --
>> *Von:* Gary Adams  
>> *Gesendet:* Freitag, Februar 15, 2019 2:19 PM
>> *An:* gary.ad...@oracle.com
>> *Cc:* Bernd Eckenfels; OpenJDK Serviceability
>> *Betreff:* Re: RFR: JDK-8149461: jmap kills process if non-java pid is
>> specified in the command line
>>
>> On a linux system with 1 Java process and 500 non-Java processes,
>> /tmp is not tmpfs mounted, 20 runs average 255.5 ms stddev 9.32.
>>
>> JDK-8176828 is the issue that enabled perfmemory visibility once the vm
>> is in live mode.
>>
>> For containers that are configured without a shared view of /tmp, it may
>> be necessary
>> to include a override of the pid check.
>>
>> On 2/15/19, 7:29 AM, Gary Adams wrote:
>>
>> I'll get some measurements on some local systems so we have a
>> specific idea about the overhead of the pid check.
>> I imagine in most production environments the /tmp tmpfs
>> is memory only set of checks.
>>
>> One of the "not all vms are recognized" was fixed recently.
>> When starting a libjdwp session with server=y and suspend=y,
>> the vm was not recognized until a debugger was attached.
>>
>> I'm not opposed to adding a force option to skip the valid pid
>> check. It might be better effort fixing the hung vm case if we
>> have a concrete failure to investigate.
>>
>> On 2/15/19, 6:47 AM, Bernd Eckenfels wrote:
>>
>> Hello,
>>
>> I see possible issues here, not sure if they still exist but I wanted to
>> mention them:
>>
>> the list-vm function might be slow on a loaded system (as it is a complex
>> function). It’s not the best Situation if your diagnostic attempts are slow
>> down in such a situation.
>>
>> Also in the past not all VMs might be recognized by the list function, a
>> more targeted attach could still succeed. Is that addressed since the
>> container-PID changes? In both cases a force option would help.
>>
>> Gruss
>> Bernd
>>
>> Gruss
>> Bernd
>> --
>> http://bernd.eckenfels.net
>>
>> --
>> *Von:* serviceability-dev 
>>  im Auftrag von
>> gary.ad...@oracle.com
>> *Gesendet:* Freitag, Februar 15, 2019 12:07 PM
>> *An:* Thomas Stüfe; David Holmes; Chris Plummer
>> *Cc:* OpenJDK Serviceability
>> *Betreff:* Re: RFR: JDK-8149461: jmap kills process if non-java pid is
>> specified in the command line
>>
>> Let me clarify a few things about the proposed fix.
>>
>> The VirtualMachine.list() mechanism is based on
>> Java processes that are up and running.
>> During VM startup when agent libraries are loaded,
>> the fact is recorded in the filesystem that a Java process
>> is eligible for an attach request.
>>
>> This is a much smaller list than scanning for all the
>> running processes and works across all supported
>> platforms. It also doesn't rely on fragile command line
>> parsing to determine a Java launcher is used.
>>
>> I believe most of the reported hang situations
>> are not for the first level information for pid and
>> command line requests. I believe the hangs are due
>> to the second level requests that actually attach
>> to the process and issue a command to the running
>> Java process.
>>
>> The overhead for the pid check should be relatively small.
>> In a standalone command for a one time check at startup
>> with 10's of Java processes. I know the documentation
>> states the user is responsible for verifying with jps
>> that a Java process pid or vmid is provided. But the cost
>> of human error can be a terminated process.
>>
>> Selection by name also has some drawbacks when multiple
>> copies of a command are running at the same time.
>>
>> Running "jcmd 0 ..." is the documented way to run a command on
>> all running Java processes.
>>
>> May I count you as a reviewer on the current changeset?
>>
>> 

Re: RFR: JDK-8149461: jmap kills process if non-java pid is specified in the command line

[Gary Adams]

Confirmed
  -XX:-UsePerfData

prevents visibility to jps, but allows direct access via pid.

The new check would block access before the attach is attempted.

May be best to close this bug as "will not fix".
Requires a valid Java process pid.

On 2/15/19, 8:29 AM, Bernd Eckenfels wrote:
I wonder, instead of listing all VMs, would it be better to check only 
the target PID?


BTW speaking of hs_perf files: is it supposed to work without 
-XX:-UserPerfData also?


Gruss
Bernd

Gruss
Bernd
--
http://bernd.eckenfels.net

*Von:* Gary Adams 
*Gesendet:* Freitag, Februar 15, 2019 2:19 PM
*An:* gary.ad...@oracle.com
*Cc:* Bernd Eckenfels; OpenJDK Serviceability
*Betreff:* Re: RFR: JDK-8149461: jmap kills process if non-java pid is 
specified in the command line

On a linux system with 1 Java process and 500 non-Java processes,
/tmp is not tmpfs mounted, 20 runs average 255.5 ms stddev 9.32.

JDK-8176828 is the issue that enabled perfmemory visibility once the 
vm is in live mode.


For containers that are configured without a shared view of /tmp, it 
may be necessary

to include a override of the pid check.

On 2/15/19, 7:29 AM, Gary Adams wrote:

I'll get some measurements on some local systems so we have a
specific idea about the overhead of the pid check.
I imagine in most production environments the /tmp tmpfs
is memory only set of checks.

One of the "not all vms are recognized" was fixed recently.
When starting a libjdwp session with server=y and suspend=y,
the vm was not recognized until a debugger was attached.

I'm not opposed to adding a force option to skip the valid pid
check. It might be better effort fixing the hung vm case if we
have a concrete failure to investigate.

On 2/15/19, 6:47 AM, Bernd Eckenfels wrote:

Hello,

I see possible issues here, not sure if they still exist but I 
wanted to mention them:


the list-vm function might be slow on a loaded system (as it is a 
complex function). It’s not the best Situation if your diagnostic 
attempts are slow down in such a situation.


Also in the past not all VMs might be recognized by the list 
function, a more targeted attach could still succeed. Is that 
addressed since the container-PID changes? In both cases a force 
option would help.


Gruss
Bernd

Gruss
Bernd
--
http://bernd.eckenfels.net

*Von:* serviceability-dev 
 im Auftrag von 
gary.ad...@oracle.com

*Gesendet:* Freitag, Februar 15, 2019 12:07 PM
*An:* Thomas Stüfe; David Holmes; Chris Plummer
*Cc:* OpenJDK Serviceability
*Betreff:* Re: RFR: JDK-8149461: jmap kills process if non-java pid 
is specified in the command line

Let me clarify a few things about the proposed fix.

The VirtualMachine.list() mechanism is based on
Java processes that are up and running.
During VM startup when agent libraries are loaded,
the fact is recorded in the filesystem that a Java process
is eligible for an attach request.

This is a much smaller list than scanning for all the
running processes and works across all supported
platforms. It also doesn't rely on fragile command line
parsing to determine a Java launcher is used.

I believe most of the reported hang situations
are not for the first level information for pid and
command line requests. I believe the hangs are due
to the second level requests that actually attach
to the process and issue a command to the running
Java process.

The overhead for the pid check should be relatively small.
In a standalone command for a one time check at startup
with 10's of Java processes. I know the documentation
states the user is responsible for verifying with jps
that a Java process pid or vmid is provided. But the cost
of human error can be a terminated process.

Selection by name also has some drawbacks when multiple
copies of a command are running at the same time.

Running "jcmd 0 ..." is the documented way to run a command on
all running Java processes.

May I count you as a reviewer on the current changeset?

On 2/15/19 3:54 AM, Thomas Stüfe wrote:



On Fri, Feb 15, 2019 at 3:26 AM David Holmes 
mailto:david.hol...@oracle.com>> wrote:


Gary,

What is the overhead of doing the validation? How do we list
VMs? Do we
need to examine every running process to get the list of VMs?
Wouldn't
it be better to check the given process is a VM rather than
checking all
potential VM processes?


I think this is a valid point. Listing VMs is normally quick (just 
use jcmd without any parms) but I have seen this fail or hang 
rarely in situations where I then still was able to talk to my VM 
via pid. I never investigated this but I would not like to be 
unable to connect to my VM just because some rogue VM mailfunctions.


This would be an argument for the alternative I offered in my first 
answer - just use the proc file system or a similar mechanism to 
check only the pid you plan on sending 

Re: RFR: JDK-8149461: jmap kills process if non-java pid is specified in the command line

[David Holmes]

On 15/02/2019 11:28 pm, Gary Adams wrote:

On 2/15/19, 8:18 AM, David Holmes wrote:

On 15/02/2019 8:04 pm, gary.ad...@oracle.com wrote:

Let me clarify a few things about the proposed fix.

The VirtualMachine.list() mechanism is based on
Java processes that are up and running.
During VM startup when agent libraries are loaded,
the fact is recorded in the filesystem that a Java process
is eligible for an attach request.


I don't follow. The target VM is not started with an agent, so how is 
it recorded in the file system?

The VirtualMachine list is based on the /tmp/hsperfdata_/.
Perf data initialization is handled when the VM enters live mode.


I can run with -XX:-UsePerfdata and the VM will not appear in 
/tmp/hsperfdata_* but I can still run jmap against the pid.


David


A bug addressed last year fixed an issue with agent initialization and VM
visibility.



David
-


This is a much smaller list than scanning for all the
running processes and works across all supported
platforms. It also doesn't rely on fragile command line
parsing to determine a Java launcher is used.

I believe most of the reported hang situations
are not for the first level information for pid and
command line requests. I believe the hangs are due
to the second level requests that actually attach
to the process and issue a command to the running
Java process.

The overhead for the pid check should be relatively small.
In a standalone command for a one time check at startup
with 10's of Java processes. I know the documentation
states the user is responsible for verifying with jps
that a Java process pid or vmid is provided. But the cost
of human error can be a terminated process.

Selection by name also has some drawbacks when multiple
copies of a command are running at the same time.

Running "jcmd 0 ..." is the documented way to run a command on
all running Java processes.

May I count you as a reviewer on the current changeset?

On 2/15/19 3:54 AM, Thomas Stüfe wrote:



On Fri, Feb 15, 2019 at 3:26 AM David Holmes 
mailto:david.hol...@oracle.com>> wrote:


    Gary,

    What is the overhead of doing the validation? How do we list VMs?
    Do we
    need to examine every running process to get the list of VMs?
    Wouldn't
    it be better to check the given process is a VM rather than
    checking all
    potential VM processes?


I think this is a valid point. Listing VMs is normally quick (just 
use jcmd without any parms) but I have seen this fail or hang rarely 
in situations where I then still was able to talk to my VM via pid. 
I never investigated this but I would not like to be unable to 
connect to my VM just because some rogue VM mailfunctions.


This would be an argument for the alternative I offered in my first 
answer - just use the proc file system or a similar mechanism to 
check only the pid you plan on sending sigquit to...


    I think there is an onus of responsibility on the user to provide a
    correct pid here, rather than trying to make this foolproof.


Oh but it can happen quite easily. For example, by pulling up an 
older jcmd from your shell history and forgetting to modify the pid. 
Thank god for the command name argument option on jcmd, which I now 
use mostly.


We also had a customer which tried to find all VMs on his machine by 
attempting to attach with jcmd to every process, killing them left 
and right :)


... Thomas

    Thanks,
    David

    On 15/02/2019 5:12 am, Gary Adams wrote:
> The following commands present a similar kill process behavior:
> jcmd
> jinfo
> jmap
> jstack
>
> The following commands do not attach.
> jstat sun.jvmstat.monitor.MonitorException "not found"
> jps no pid arguments
>
> This update moves the checkJavaPid method into the
> common/ProcessArgumentsMatcher.java
> and applies the check before the pid is used for an attach
    operation.
>
>    Revised webrev:
    http://cr.openjdk.java.net/~gadams/8149461/webrev.01/
>
> On 2/14/19, 12:07 PM, Chris Plummer wrote:
>> Hi Gary,
>>
>> What about the other tools that attach to a user specified
    process. Do
>> they also have this issue?
>>
>> thanks,
>>
>> Chris
>>
>> On 2/14/19 8:35 AM, Gary Adams wrote:
>>> There is an old reported problem that using jmap on a process
    that is
>>> not running
>>> Java could cause the process to terminate. This is due to the
    SIGQUIT
>>> used
>>> when attaching to the process.
>>>
>>> It is a fairly simple operation to validate that the pid
    matches one
>>> of the known
>>> running Java processes using VirtualMachine.list().
>>>
>>>   Issue: https://bugs.openjdk.java.net/browse/JDK-8149461
>>>
>>> Proposed fix:
>>>
>>> diff --git a/src/jdk.jcmd/share/classes/sun/tools/jmap/JMap.java
>>> b/src/jdk.jcmd/share/classes/sun/tools/jmap/JMap.java
>>> --- a/src/jdk.jcmd/share/classes/sun/tools/jmap/JMap.java
>>> +++ b/src/jdk.jcmd/share/classes/sun/tools/jmap/JMap.java
>>> @@ -1,5 +1,5 @@
>>>  /*
>>> - * Copyright (c) 2005, 2018, Oracle and/or its 

Re: RFR: JDK-8149461: jmap kills process if non-java pid is specified in the command line

[Bernd Eckenfels]

I wonder, instead of listing all VMs, would it be better to check only the 
target PID?

BTW speaking of hs_perf files: is it supposed to work without -XX:-UserPerfData 
also?

Gruss
Bernd

Gruss
Bernd
--
http://bernd.eckenfels.net


Von: Gary Adams 
Gesendet: Freitag, Februar 15, 2019 2:19 PM
An: gary.ad...@oracle.com
Cc: Bernd Eckenfels; OpenJDK Serviceability
Betreff: Re: RFR: JDK-8149461: jmap kills process if non-java pid is specified 
in the command line

On a linux system with 1 Java process and 500 non-Java processes,
/tmp is not tmpfs mounted, 20 runs average 255.5 ms stddev 9.32.

JDK-8176828 is the issue that enabled perfmemory visibility once the vm is in 
live mode.

For containers that are configured without a shared view of /tmp, it may be 
necessary
to include a override of the pid check.

On 2/15/19, 7:29 AM, Gary Adams wrote:
I'll get some measurements on some local systems so we have a
specific idea about the overhead of the pid check.
I imagine in most production environments the /tmp tmpfs
is memory only set of checks.

One of the "not all vms are recognized" was fixed recently.
When starting a libjdwp session with server=y and suspend=y,
the vm was not recognized until a debugger was attached.

I'm not opposed to adding a force option to skip the valid pid
check. It might be better effort fixing the hung vm case if we
have a concrete failure to investigate.

On 2/15/19, 6:47 AM, Bernd Eckenfels wrote:
Hello,

I see possible issues here, not sure if they still exist but I wanted to 
mention them:

the list-vm function might be slow on a loaded system (as it is a complex 
function). It’s not the best Situation if your diagnostic attempts are slow 
down in such a situation.

Also in the past not all VMs might be recognized by the list function, a more 
targeted attach could still succeed. Is that addressed since the container-PID 
changes? In both cases a force option would help.

Gruss
Bernd

Gruss
Bernd
--
http://bernd.eckenfels.net


Von: serviceability-dev 

 im Auftrag von gary.ad...@oracle.com
Gesendet: Freitag, Februar 15, 2019 12:07 PM
An: Thomas Stüfe; David Holmes; Chris Plummer
Cc: OpenJDK Serviceability
Betreff: Re: RFR: JDK-8149461: jmap kills process if non-java pid is specified 
in the command line

Let me clarify a few things about the proposed fix.

The VirtualMachine.list() mechanism is based on
Java processes that are up and running.
During VM startup when agent libraries are loaded,
the fact is recorded in the filesystem that a Java process
is eligible for an attach request.

This is a much smaller list than scanning for all the
running processes and works across all supported
platforms. It also doesn't rely on fragile command line
parsing to determine a Java launcher is used.

I believe most of the reported hang situations
are not for the first level information for pid and
command line requests. I believe the hangs are due
to the second level requests that actually attach
to the process and issue a command to the running
Java process.

The overhead for the pid check should be relatively small.
In a standalone command for a one time check at startup
with 10's of Java processes. I know the documentation
states the user is responsible for verifying with jps
that a Java process pid or vmid is provided. But the cost
of human error can be a terminated process.

Selection by name also has some drawbacks when multiple
copies of a command are running at the same time.

Running "jcmd 0 ..." is the documented way to run a command on
all running Java processes.

May I count you as a reviewer on the current changeset?

On 2/15/19 3:54 AM, Thomas Stüfe wrote:


On Fri, Feb 15, 2019 at 3:26 AM David Holmes 
mailto:david.hol...@oracle.com>> wrote:
Gary,

What is the overhead of doing the validation? How do we list VMs? Do we
need to examine every running process to get the list of VMs? Wouldn't
it be better to check the given process is a VM rather than checking all
potential VM processes?


I think this is a valid point. Listing VMs is normally quick (just use jcmd 
without any parms) but I have seen this fail or hang rarely in situations where 
I then still was able to talk to my VM via pid. I never investigated this but I 
would not like to be unable to connect to my VM just because some rogue VM 
mailfunctions.

This would be an argument for the alternative I offered in my first answer - 
just use the proc file system or a similar mechanism to check only the pid you 
plan on sending sigquit to...

I think there is an onus of responsibility on the user to provide a
correct pid here, rather than trying to make this foolproof.


Oh but it can happen quite easily. For example, by pulling up an older jcmd 
from your shell history and forgetting to modify the pid. Thank god for the 
command name argument option on jcmd, which I now use 

Re: RFR: JDK-8149461: jmap kills process if non-java pid is specified in the command line

[Gary Adams]

On 2/15/19, 8:18 AM, David Holmes wrote:

On 15/02/2019 8:04 pm, gary.ad...@oracle.com wrote:

Let me clarify a few things about the proposed fix.

The VirtualMachine.list() mechanism is based on
Java processes that are up and running.
During VM startup when agent libraries are loaded,
the fact is recorded in the filesystem that a Java process
is eligible for an attach request.


I don't follow. The target VM is not started with an agent, so how is 
it recorded in the file system?

The VirtualMachine list is based on the /tmp/hsperfdata_/.
Perf data initialization is handled when the VM enters live mode.

A bug addressed last year fixed an issue with agent initialization and VM
visibility.



David
-


This is a much smaller list than scanning for all the
running processes and works across all supported
platforms. It also doesn't rely on fragile command line
parsing to determine a Java launcher is used.

I believe most of the reported hang situations
are not for the first level information for pid and
command line requests. I believe the hangs are due
to the second level requests that actually attach
to the process and issue a command to the running
Java process.

The overhead for the pid check should be relatively small.
In a standalone command for a one time check at startup
with 10's of Java processes. I know the documentation
states the user is responsible for verifying with jps
that a Java process pid or vmid is provided. But the cost
of human error can be a terminated process.

Selection by name also has some drawbacks when multiple
copies of a command are running at the same time.

Running "jcmd 0 ..." is the documented way to run a command on
all running Java processes.

May I count you as a reviewer on the current changeset?

On 2/15/19 3:54 AM, Thomas Stüfe wrote:



On Fri, Feb 15, 2019 at 3:26 AM David Holmes 
mailto:david.hol...@oracle.com>> wrote:


Gary,

What is the overhead of doing the validation? How do we list VMs?
Do we
need to examine every running process to get the list of VMs?
Wouldn't
it be better to check the given process is a VM rather than
checking all
potential VM processes?


I think this is a valid point. Listing VMs is normally quick (just 
use jcmd without any parms) but I have seen this fail or hang rarely 
in situations where I then still was able to talk to my VM via pid. 
I never investigated this but I would not like to be unable to 
connect to my VM just because some rogue VM mailfunctions.


This would be an argument for the alternative I offered in my first 
answer - just use the proc file system or a similar mechanism to 
check only the pid you plan on sending sigquit to...


I think there is an onus of responsibility on the user to provide a
correct pid here, rather than trying to make this foolproof.


Oh but it can happen quite easily. For example, by pulling up an 
older jcmd from your shell history and forgetting to modify the pid. 
Thank god for the command name argument option on jcmd, which I now 
use mostly.


We also had a customer which tried to find all VMs on his machine by 
attempting to attach with jcmd to every process, killing them left 
and right :)


... Thomas

Thanks,
David

On 15/02/2019 5:12 am, Gary Adams wrote:
> The following commands present a similar kill process behavior:
> jcmd
> jinfo
> jmap
> jstack
>
> The following commands do not attach.
> jstat sun.jvmstat.monitor.MonitorException "not found"
> jps no pid arguments
>
> This update moves the checkJavaPid method into the
> common/ProcessArgumentsMatcher.java
> and applies the check before the pid is used for an attach
operation.
>
>Revised webrev:
http://cr.openjdk.java.net/~gadams/8149461/webrev.01/
>
> On 2/14/19, 12:07 PM, Chris Plummer wrote:
>> Hi Gary,
>>
>> What about the other tools that attach to a user specified
process. Do
>> they also have this issue?
>>
>> thanks,
>>
>> Chris
>>
>> On 2/14/19 8:35 AM, Gary Adams wrote:
>>> There is an old reported problem that using jmap on a process
that is
>>> not running
>>> Java could cause the process to terminate. This is due to the
SIGQUIT
>>> used
>>> when attaching to the process.
>>>
>>> It is a fairly simple operation to validate that the pid
matches one
>>> of the known
>>> running Java processes using VirtualMachine.list().
>>>
>>>   Issue: https://bugs.openjdk.java.net/browse/JDK-8149461
>>>
>>> Proposed fix:
>>>
>>> diff --git a/src/jdk.jcmd/share/classes/sun/tools/jmap/JMap.java
>>> b/src/jdk.jcmd/share/classes/sun/tools/jmap/JMap.java
>>> --- a/src/jdk.jcmd/share/classes/sun/tools/jmap/JMap.java
>>> +++ b/src/jdk.jcmd/share/classes/sun/tools/jmap/JMap.java
>>> @@ -1,5 +1,5 @@
>>>  /*
>>> - * Copyright (c) 2005, 2018, Oracle and/or its affiliates. All
>>> rights reserved.
>>> + * Copyright (c) 2005, 2019, Oracle and/or its affiliates. All
>>> rights reserved.
>>>   * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR 

Re: RFR: JDK-8149461: jmap kills process if non-java pid is specified in the command line

[Gary Adams]

On a linux system with 1 Java process and 500 non-Java processes,
/tmp is not tmpfs mounted, 20 runs average 255.5 ms stddev 9.32.

JDK-8176828 is the issue that enabled perfmemory visibility once the vm 
is in live mode.


For containers that are configured without a shared view of /tmp, it may 
be necessary

to include a override of the pid check.

On 2/15/19, 7:29 AM, Gary Adams wrote:

I'll get some measurements on some local systems so we have a
specific idea about the overhead of the pid check.
I imagine in most production environments the /tmp tmpfs
is memory only set of checks.

One of the "not all vms are recognized" was fixed recently.
When starting a libjdwp session with server=y and suspend=y,
the vm was not recognized until a debugger was attached.

I'm not opposed to adding a force option to skip the valid pid
check. It might be better effort fixing the hung vm case if we
have a concrete failure to investigate.

On 2/15/19, 6:47 AM, Bernd Eckenfels wrote:

Hello,

I see possible issues here, not sure if they still exist but I wanted 
to mention them:


the list-vm function might be slow on a loaded system (as it is a 
complex function). It’s not the best Situation if your diagnostic 
attempts are slow down in such a situation.


Also in the past not all VMs might be recognized by the list 
function, a more targeted attach could still succeed. Is that 
addressed since the container-PID changes? In both cases a force 
option would help.


Gruss
Bernd

Gruss
Bernd
--
http://bernd.eckenfels.net

*Von:* serviceability-dev 
 im Auftrag von 
gary.ad...@oracle.com

*Gesendet:* Freitag, Februar 15, 2019 12:07 PM
*An:* Thomas Stüfe; David Holmes; Chris Plummer
*Cc:* OpenJDK Serviceability
*Betreff:* Re: RFR: JDK-8149461: jmap kills process if non-java pid 
is specified in the command line

Let me clarify a few things about the proposed fix.

The VirtualMachine.list() mechanism is based on
Java processes that are up and running.
During VM startup when agent libraries are loaded,
the fact is recorded in the filesystem that a Java process
is eligible for an attach request.

This is a much smaller list than scanning for all the
running processes and works across all supported
platforms. It also doesn't rely on fragile command line
parsing to determine a Java launcher is used.

I believe most of the reported hang situations
are not for the first level information for pid and
command line requests. I believe the hangs are due
to the second level requests that actually attach
to the process and issue a command to the running
Java process.

The overhead for the pid check should be relatively small.
In a standalone command for a one time check at startup
with 10's of Java processes. I know the documentation
states the user is responsible for verifying with jps
that a Java process pid or vmid is provided. But the cost
of human error can be a terminated process.

Selection by name also has some drawbacks when multiple
copies of a command are running at the same time.

Running "jcmd 0 ..." is the documented way to run a command on
all running Java processes.

May I count you as a reviewer on the current changeset?

On 2/15/19 3:54 AM, Thomas Stüfe wrote:



On Fri, Feb 15, 2019 at 3:26 AM David Holmes 
mailto:david.hol...@oracle.com>> wrote:


Gary,

What is the overhead of doing the validation? How do we list
VMs? Do we
need to examine every running process to get the list of VMs?
Wouldn't
it be better to check the given process is a VM rather than
checking all
potential VM processes?


I think this is a valid point. Listing VMs is normally quick (just 
use jcmd without any parms) but I have seen this fail or hang rarely 
in situations where I then still was able to talk to my VM via pid. 
I never investigated this but I would not like to be unable to 
connect to my VM just because some rogue VM mailfunctions.


This would be an argument for the alternative I offered in my first 
answer - just use the proc file system or a similar mechanism to 
check only the pid you plan on sending sigquit to...


I think there is an onus of responsibility on the user to provide a
correct pid here, rather than trying to make this foolproof.


Oh but it can happen quite easily. For example, by pulling up an 
older jcmd from your shell history and forgetting to modify the pid. 
Thank god for the command name argument option on jcmd, which I now 
use mostly.


We also had a customer which tried to find all VMs on his machine by 
attempting to attach with jcmd to every process, killing them left 
and right :)


... Thomas

Thanks,
David

On 15/02/2019 5:12 am, Gary Adams wrote:
> The following commands present a similar kill process behavior:
> jcmd
> jinfo
> jmap
> jstack
>
> The following commands do not attach.
> jstat sun.jvmstat.monitor.MonitorException 

Re: RFR: JDK-8149461: jmap kills process if non-java pid is specified in the command line

[David Holmes]

On 15/02/2019 8:04 pm, gary.ad...@oracle.com wrote:

Let me clarify a few things about the proposed fix.

The VirtualMachine.list() mechanism is based on
Java processes that are up and running.
During VM startup when agent libraries are loaded,
the fact is recorded in the filesystem that a Java process
is eligible for an attach request.


I don't follow. The target VM is not started with an agent, so how is it 
recorded in the file system?


David
-


This is a much smaller list than scanning for all the
running processes and works across all supported
platforms. It also doesn't rely on fragile command line
parsing to determine a Java launcher is used.

I believe most of the reported hang situations
are not for the first level information for pid and
command line requests. I believe the hangs are due
to the second level requests that actually attach
to the process and issue a command to the running
Java process.

The overhead for the pid check should be relatively small.
In a standalone command for a one time check at startup
with 10's of Java processes. I know the documentation
states the user is responsible for verifying with jps
that a Java process pid or vmid is provided. But the cost
of human error can be a terminated process.

Selection by name also has some drawbacks when multiple
copies of a command are running at the same time.

Running "jcmd 0 ..." is the documented way to run a command on
all running Java processes.

May I count you as a reviewer on the current changeset?

On 2/15/19 3:54 AM, Thomas Stüfe wrote:



On Fri, Feb 15, 2019 at 3:26 AM David Holmes > wrote:


Gary,

What is the overhead of doing the validation? How do we list VMs?
Do we
need to examine every running process to get the list of VMs?
Wouldn't
it be better to check the given process is a VM rather than
checking all
potential VM processes?


I think this is a valid point. Listing VMs is normally quick (just use 
jcmd without any parms) but I have seen this fail or hang rarely in 
situations where I then still was able to talk to my VM via pid. I 
never investigated this but I would not like to be unable to connect 
to my VM just because some rogue VM mailfunctions.


This would be an argument for the alternative I offered in my first 
answer - just use the proc file system or a similar mechanism to check 
only the pid you plan on sending sigquit to...


I think there is an onus of responsibility on the user to provide a
correct pid here, rather than trying to make this foolproof.


Oh but it can happen quite easily. For example, by pulling up an older 
jcmd from your shell history and forgetting to modify the pid. Thank 
god for the command name argument option on jcmd, which I now use mostly.


We also had a customer which tried to find all VMs on his machine by 
attempting to attach with jcmd to every process, killing them left and 
right :)


... Thomas

Thanks,
David

On 15/02/2019 5:12 am, Gary Adams wrote:
> The following commands present a similar kill process behavior:
>     jcmd
>     jinfo
>     jmap
>     jstack
>
> The following commands do not attach.
>     jstat sun.jvmstat.monitor.MonitorException "not found"
>     jps no pid arguments
>
> This update moves the checkJavaPid method into the
> common/ProcessArgumentsMatcher.java
> and applies the check before the pid is used for an attach
operation.
>
>    Revised webrev:
http://cr.openjdk.java.net/~gadams/8149461/webrev.01/
>
> On 2/14/19, 12:07 PM, Chris Plummer wrote:
>> Hi Gary,
>>
>> What about the other tools that attach to a user specified
process. Do
>> they also have this issue?
>>
>> thanks,
>>
>> Chris
>>
>> On 2/14/19 8:35 AM, Gary Adams wrote:
>>> There is an old reported problem that using jmap on a process
that is
>>> not running
>>> Java could cause the process to terminate. This is due to the
SIGQUIT
>>> used
>>> when attaching to the process.
>>>
>>> It is a fairly simple operation to validate that the pid
matches one
>>> of the known
>>> running Java processes using VirtualMachine.list().
>>>
>>>   Issue: https://bugs.openjdk.java.net/browse/JDK-8149461
>>>
>>> Proposed fix:
>>>
>>> diff --git a/src/jdk.jcmd/share/classes/sun/tools/jmap/JMap.java
>>> b/src/jdk.jcmd/share/classes/sun/tools/jmap/JMap.java
>>> --- a/src/jdk.jcmd/share/classes/sun/tools/jmap/JMap.java
>>> +++ b/src/jdk.jcmd/share/classes/sun/tools/jmap/JMap.java
>>> @@ -1,5 +1,5 @@
>>>  /*
>>> - * Copyright (c) 2005, 2018, Oracle and/or its affiliates. All
>>> rights reserved.
>>> + * Copyright (c) 2005, 2019, Oracle and/or its affiliates. All
>>> rights reserved.
>>>   * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
>>>   *
>>>   * This 

Re: RFR: JDK-8149461: jmap kills process if non-java pid is specified in the command line

[Gary Adams]

I'll get some measurements on some local systems so we have a
specific idea about the overhead of the pid check.
I imagine in most production environments the /tmp tmpfs
is memory only set of checks.

One of the "not all vms are recognized" was fixed recently.
When starting a libjdwp session with server=y and suspend=y,
the vm was not recognized until a debugger was attached.

I'm not opposed to adding a force option to skip the valid pid
check. It might be better effort fixing the hung vm case if we
have a concrete failure to investigate.

On 2/15/19, 6:47 AM, Bernd Eckenfels wrote:

Hello,

I see possible issues here, not sure if they still exist but I wanted 
to mention them:


the list-vm function might be slow on a loaded system (as it is a 
complex function). It’s not the best Situation if your diagnostic 
attempts are slow down in such a situation.


Also in the past not all VMs might be recognized by the list function, 
a more targeted attach could still succeed. Is that addressed since 
the container-PID changes? In both cases a force option would help.


Gruss
Bernd

Gruss
Bernd
--
http://bernd.eckenfels.net

*Von:* serviceability-dev 
 im Auftrag von 
gary.ad...@oracle.com

*Gesendet:* Freitag, Februar 15, 2019 12:07 PM
*An:* Thomas Stüfe; David Holmes; Chris Plummer
*Cc:* OpenJDK Serviceability
*Betreff:* Re: RFR: JDK-8149461: jmap kills process if non-java pid is 
specified in the command line

Let me clarify a few things about the proposed fix.

The VirtualMachine.list() mechanism is based on
Java processes that are up and running.
During VM startup when agent libraries are loaded,
the fact is recorded in the filesystem that a Java process
is eligible for an attach request.

This is a much smaller list than scanning for all the
running processes and works across all supported
platforms. It also doesn't rely on fragile command line
parsing to determine a Java launcher is used.

I believe most of the reported hang situations
are not for the first level information for pid and
command line requests. I believe the hangs are due
to the second level requests that actually attach
to the process and issue a command to the running
Java process.

The overhead for the pid check should be relatively small.
In a standalone command for a one time check at startup
with 10's of Java processes. I know the documentation
states the user is responsible for verifying with jps
that a Java process pid or vmid is provided. But the cost
of human error can be a terminated process.

Selection by name also has some drawbacks when multiple
copies of a command are running at the same time.

Running "jcmd 0 ..." is the documented way to run a command on
all running Java processes.

May I count you as a reviewer on the current changeset?

On 2/15/19 3:54 AM, Thomas Stüfe wrote:



On Fri, Feb 15, 2019 at 3:26 AM David Holmes > wrote:


Gary,

What is the overhead of doing the validation? How do we list VMs?
Do we
need to examine every running process to get the list of VMs?
Wouldn't
it be better to check the given process is a VM rather than
checking all
potential VM processes?


I think this is a valid point. Listing VMs is normally quick (just 
use jcmd without any parms) but I have seen this fail or hang rarely 
in situations where I then still was able to talk to my VM via pid. I 
never investigated this but I would not like to be unable to connect 
to my VM just because some rogue VM mailfunctions.


This would be an argument for the alternative I offered in my first 
answer - just use the proc file system or a similar mechanism to 
check only the pid you plan on sending sigquit to...


I think there is an onus of responsibility on the user to provide a
correct pid here, rather than trying to make this foolproof.


Oh but it can happen quite easily. For example, by pulling up an 
older jcmd from your shell history and forgetting to modify the pid. 
Thank god for the command name argument option on jcmd, which I now 
use mostly.


We also had a customer which tried to find all VMs on his machine by 
attempting to attach with jcmd to every process, killing them left 
and right :)


... Thomas

Thanks,
David

On 15/02/2019 5:12 am, Gary Adams wrote:
> The following commands present a similar kill process behavior:
> jcmd
> jinfo
> jmap
> jstack
>
> The following commands do not attach.
> jstat sun.jvmstat.monitor.MonitorException "not found"
> jps no pid arguments
>
> This update moves the checkJavaPid method into the
> common/ProcessArgumentsMatcher.java
> and applies the check before the pid is used for an attach
operation.
>
>Revised webrev:
http://cr.openjdk.java.net/~gadams/8149461/webrev.01/

>
> On 

Re: RFR/C: 8218922: SA: Enable best-effort implementation of live regions iteration for ZGC

[Kevin Walls]

Hi Stefan,

This is great - if I want to try it out, what order do the changes 
apply, does the 8218746 webrev apply to create ZExternalBitMap.java and 
then the 8218922 webrev?


Thanks
Kevin


On 14/02/2019 17:12, Stefan Karlsson wrote:

Hi again,

I've separated the live regions iteration refactoring into this patch:
https://cr.openjdk.java.net/~stefank/8219003/webrev.01/

And use this RFE for the ZGC specific parts:
https://cr.openjdk.java.net/~stefank/8218922/webrev.02/

Thanks,
StefanK

On 2019-02-14 14:39, Stefan Karlsson wrote:

Hi Yasumasa,

On 2019-02-14 14:11, Yasumasa Suenaga wrote:

Hi Stefan,

Maybe this is enough to enable a bit more SA debugging capabilities 
when

running with ZGC? What do you think, should we bring in this change?


I think it should be brought this in.
I filed same issue as JDK-8207843, but I've not yet worked.
So I will close it as duplicate of your change.



To be able to implement this more cleanly I've restructured the live
region collection, and pushed GC specific code into the specific GCs.
There are some extra usage of generics to make the code a bit 
easier to

read and develop.


IMHO refactoring for live region collection and ZGC related changes 
should

be separated. What do you think?


Yes. I think that would be good. I'll separate this out into two 
changes.





Your change looks good to me.
BTW, did you check `jhsdb jmap --binaryheap` with this change?



Yes, when testing this I ran all tests in serviceability/sa and 
manually tested the command above.


While testing this I also had this patch applied to enable SA hprof 
for ZGC:

http://cr.openjdk.java.net/~stefank/8218970/webrev.01/

And used this patch to turn off the ZGC filtering in the tests:
http://cr.openjdk.java.net/~stefank/8218978/webrev.01/

I'm currently rerunning the tests to see that the latest changes 
didn't break anything.


Thanks,
StefanK


Thanks,

Yasumasa



On 2019/02/13 23:52, Stefan Karlsson wrote:

Hi all,

Please review / comment on this patch to enable a best-effort live 
heap

region iteration implementation in ZGC.

http://cr.openjdk.java.net/~stefank/8218922/webrev.01/
https://bugs.openjdk.java.net/browse/JDK-8218922

The SA has functionally that relies on live heap region information 
from
the GCs. This is problematic when dead objects are left in the 
heap, and

their classes have been unloaded.

Because of this ZGC has so far not implemented this feature. 
However, we
could provide a best-effort implementation that most likely will 
work if

classes are not unloaded (or class unloading is turned off), and
otherwise might fail to fully parse and report all live heap regions.

For active, non-relocating pages the patch simply returns [start, top)
and for pages being actively relocated, it reports regions containing
the non-forwarded objects, the other objects are either dead or 
could be

found in one of the to-regions.

With this patch I'm able to get heap histograms with ZGC.

Maybe this is enough to enable a bit more SA debugging capabilities 
when

running with ZGC? What do you think, should we bring in this change?

To be able to implement this more cleanly I've restructured the live
region collection, and pushed GC specific code into the specific GCs.
There are some extra usage of generics to make the code a bit 
easier to

read and develop.

Thanks,
StefanK

Re: RFR: JDK-8149461: jmap kills process if non-java pid is specified in the command line

[Bernd Eckenfels]

Hello,

I see possible issues here, not sure if they still exist but I wanted to 
mention them:

the list-vm function might be slow on a loaded system (as it is a complex 
function). It’s not the best Situation if your diagnostic attempts are slow 
down in such a situation.

Also in the past not all VMs might be recognized by the list function, a more 
targeted attach could still succeed. Is that addressed since the container-PID 
changes? In both cases a force option would help.

Gruss
Bernd

Gruss
Bernd
--
http://bernd.eckenfels.net


Von: serviceability-dev  im 
Auftrag von gary.ad...@oracle.com
Gesendet: Freitag, Februar 15, 2019 12:07 PM
An: Thomas Stüfe; David Holmes; Chris Plummer
Cc: OpenJDK Serviceability
Betreff: Re: RFR: JDK-8149461: jmap kills process if non-java pid is specified 
in the command line

Let me clarify a few things about the proposed fix.

The VirtualMachine.list() mechanism is based on
Java processes that are up and running.
During VM startup when agent libraries are loaded,
the fact is recorded in the filesystem that a Java process
is eligible for an attach request.

This is a much smaller list than scanning for all the
running processes and works across all supported
platforms. It also doesn't rely on fragile command line
parsing to determine a Java launcher is used.

I believe most of the reported hang situations
are not for the first level information for pid and
command line requests. I believe the hangs are due
to the second level requests that actually attach
to the process and issue a command to the running
Java process.

The overhead for the pid check should be relatively small.
In a standalone command for a one time check at startup
with 10's of Java processes. I know the documentation
states the user is responsible for verifying with jps
that a Java process pid or vmid is provided. But the cost
of human error can be a terminated process.

Selection by name also has some drawbacks when multiple
copies of a command are running at the same time.

Running "jcmd 0 ..." is the documented way to run a command on
all running Java processes.

May I count you as a reviewer on the current changeset?

On 2/15/19 3:54 AM, Thomas Stüfe wrote:


On Fri, Feb 15, 2019 at 3:26 AM David Holmes 
mailto:david.hol...@oracle.com>> wrote:
Gary,

What is the overhead of doing the validation? How do we list VMs? Do we
need to examine every running process to get the list of VMs? Wouldn't
it be better to check the given process is a VM rather than checking all
potential VM processes?


I think this is a valid point. Listing VMs is normally quick (just use jcmd 
without any parms) but I have seen this fail or hang rarely in situations where 
I then still was able to talk to my VM via pid. I never investigated this but I 
would not like to be unable to connect to my VM just because some rogue VM 
mailfunctions.

This would be an argument for the alternative I offered in my first answer - 
just use the proc file system or a similar mechanism to check only the pid you 
plan on sending sigquit to...

I think there is an onus of responsibility on the user to provide a
correct pid here, rather than trying to make this foolproof.


Oh but it can happen quite easily. For example, by pulling up an older jcmd 
from your shell history and forgetting to modify the pid. Thank god for the 
command name argument option on jcmd, which I now use mostly.

We also had a customer which tried to find all VMs on his machine by attempting 
to attach with jcmd to every process, killing them left and right :)

... Thomas

Thanks,
David

On 15/02/2019 5:12 am, Gary Adams wrote:
> The following commands present a similar kill process behavior:
> jcmd
> jinfo
> jmap
> jstack
>
> The following commands do not attach.
> jstat sun.jvmstat.monitor.MonitorException "not found"
> jps no pid arguments
>
> This update moves the checkJavaPid method into the
> common/ProcessArgumentsMatcher.java
> and applies the check before the pid is used for an attach operation.
>
>Revised webrev: http://cr.openjdk.java.net/~gadams/8149461/webrev.01/
>
> On 2/14/19, 12:07 PM, Chris Plummer wrote:
>> Hi Gary,
>>
>> What about the other tools that attach to a user specified process. Do
>> they also have this issue?
>>
>> thanks,
>>
>> Chris
>>
>> On 2/14/19 8:35 AM, Gary Adams wrote:
>>> There is an old reported problem that using jmap on a process that is
>>> not running
>>> Java could cause the process to terminate. This is due to the SIGQUIT
>>> used
>>> when attaching to the process.
>>>
>>> It is a fairly simple operation to validate that the pid matches one
>>> of the known
>>> running Java processes using VirtualMachine.list().
>>>
>>>   Issue: https://bugs.openjdk.java.net/browse/JDK-8149461
>>>
>>> Proposed fix:
>>>
>>> diff --git a/src/jdk.jcmd/share/classes/sun/tools/jmap/JMap.java
>>> b/src/jdk.jcmd/share/classes/sun/tools/jmap/JMap.java
>>> --- 

RE: jstack mixed mode and jhsdb stack mixed mode can not get expected result(stack trace is incomplete) on latest ubuntu or fedora or RHEL 8 beta

[Shuai Gao]

Hello Sharath,

 

Thanks for your help.

 

Best Regards

 

From: Sharath Ballal 
Sent: 2019年2月15日 18:10
To: Shuai Gao ; serviceability-dev@openjdk.java.net
Subject: RE: jstack mixed mode and jhsdb stack mixed mode can not get expected 
result(stack trace is incomplete) on latest ubuntu or fedora or RHEL 8 beta

 

Ok Gao.  I will open another bug for your case.

 

Thanks,

Sharath

 

 

From: Shuai Gao 
Sent: Friday, February 15, 2019 3:35 PM
To: Sharath Ballal; HYPERLINK 
"mailto:serviceability-dev@openjdk.java.net"serviceability-dev@openjdk.java.net
Subject: RE: jstack mixed mode and jhsdb stack mixed mode can not get expected 
result(stack trace is incomplete) on latest ubuntu or fedora or RHEL 8 beta

 

Hello Sharath,

 

Thanks for your reply.

I am afraid the issue in BUG JDK—8219086 is not the same as mine.

In the bug, the output of jhsdb jstack prints some mixed stack trace but does 
not print all the threads.

Within my issue, the output of jhsdb jstack can print all the threads but can 
only print the top of the stack trace.

And this issue can always be reproduced on latest Ubuntu (18.04),fedora (29) 
and RHEL 8 Beta.

On the aforementioned OSs , The pstack can work well with java application. I 
am afraid there is something about compatibility between JDK and new Linux 
kernel or ptrace.

 

Best Regards

 

 

 

From: Sharath Ballal 
Sent: 2019年2月15日 17:46
To: Shuai Gao mailto:shuai@oracle.com"shuai@oracle.com>; 
HYPERLINK 
"mailto:serviceability-dev@openjdk.java.net"serviceability-dev@openjdk.java.net
Subject: RE: jstack mixed mode and jhsdb stack mixed mode can not get expected 
result(stack trace is incomplete) on latest ubuntu or fedora or RHEL 8 beta

 

Hi Gao,

 

I also observed this failure and opened HYPERLINK 
"https://bugs.openjdk.java.net/browse/JDK-8219086"JDK-8219086 for it.

 

 

Thanks,

Sharath

 

 

From: Shuai Gao 
Sent: Friday, February 15, 2019 12:15 PM
To: HYPERLINK 
"mailto:serviceability-dev@openjdk.java.net"serviceability-dev@openjdk.java.net
Subject: jstack mixed mode and jhsdb stack mixed mode can not get expected 
result(stack trace is incomplete) on latest ubuntu or fedora or RHEL 8 beta

 

Hello,

 

I tried “jstack -m PID”in latest openjdk 8 64-bit and “jhsdb jstack –mixed –pid 
PID” in latest openjdk 11 64-bit on latest Ubuntu(18.04) and latest fedora (29) 
and RHEL 8 Beta.

The stack traces returned are always incomplete like following output:

 

- 1479 -

"DestroyJavaVM" #17 prio=5 tid=0x00bae000 nid=0x5c7 waiting on 
condition [0x]

   java.lang.Thread.State: RUNNABLE

   JavaThread state: _thread_blocked

0x7f839224f9f3  __pthread_cond_wait + 0x243

- 1480 -

- 1481 -

"Reference Handler" #2 daemon prio=10 tid=0x00c36000 nid=0x5c9 waiting 
on condition [0x7f8376d15000]

   java.lang.Thread.State: RUNNABLE

   JavaThread state: _thread_blocked

0x7f839224f9f3  __pthread_cond_wait + 0x243

- 1482 -

"Finalizer" #3 daemon prio=8 tid=0x00c3a000 nid=0x5ca in Object.wait() 
[0x7f8376c14000]

   java.lang.Thread.State: WAITING (on object monitor)

   JavaThread state: _thread_blocked

0x7f839224f9f3  __pthread_cond_wait + 0x243

- 1483 -

"Signal Dispatcher" #4 daemon prio=9 tid=0x00c56800 nid=0x5cb runnable 
[0x]

   java.lang.Thread.State: RUNNABLE

   JavaThread state: _thread_blocked

0x7f83922526d6  do_futex_wait.constprop.1 + 0x36

 

The issue existed even after I disable the selinux. It can not be reproduced on 
oracle linux 7.4.

The kernel version on Ubuntu or fedora is 4.15. the kernel version on RHEL 8 
Beta is 4.18. the kernel version on Oracle linux 7.4 is 3.1.

I can not find any existed bug for now.

 

 

Is there any clue about this issue?

 

Any reply will be appreciated.

 

Best Regards

Gao Shuai from GCS

RE: jstack mixed mode and jhsdb stack mixed mode can not get expected result(stack trace is incomplete) on latest ubuntu or fedora or RHEL 8 beta

[Sharath Ballal]

Ok Gao.  I will open another bug for your case.

 

Thanks,

Sharath

 

 

From: Shuai Gao 
Sent: Friday, February 15, 2019 3:35 PM
To: Sharath Ballal; serviceability-dev@openjdk.java.net
Subject: RE: jstack mixed mode and jhsdb stack mixed mode can not get expected 
result(stack trace is incomplete) on latest ubuntu or fedora or RHEL 8 beta

 

Hello Sharath,

 

Thanks for your reply.

I am afraid the issue in BUG JDK—8219086 is not the same as mine.

In the bug, the output of jhsdb jstack prints some mixed stack trace but does 
not print all the threads.

Within my issue, the output of jhsdb jstack can print all the threads but can 
only print the top of the stack trace.

And this issue can always be reproduced on latest Ubuntu (18.04),fedora (29) 
and RHEL 8 Beta.

On the aforementioned OSs , The pstack can work well with java application. I 
am afraid there is something about compatibility between JDK and new Linux 
kernel or ptrace.

 

Best Regards

 

 

 

From: Sharath Ballal 
Sent: 2019年2月15日 17:46
To: Shuai Gao mailto:shuai@oracle.com"shuai@oracle.com>; 
HYPERLINK 
"mailto:serviceability-dev@openjdk.java.net"serviceability-dev@openjdk.java.net
Subject: RE: jstack mixed mode and jhsdb stack mixed mode can not get expected 
result(stack trace is incomplete) on latest ubuntu or fedora or RHEL 8 beta

 

Hi Gao,

 

I also observed this failure and opened HYPERLINK 
"https://bugs.openjdk.java.net/browse/JDK-8219086"JDK-8219086 for it.

 

 

Thanks,

Sharath

 

 

From: Shuai Gao 
Sent: Friday, February 15, 2019 12:15 PM
To: HYPERLINK 
"mailto:serviceability-dev@openjdk.java.net"serviceability-dev@openjdk.java.net
Subject: jstack mixed mode and jhsdb stack mixed mode can not get expected 
result(stack trace is incomplete) on latest ubuntu or fedora or RHEL 8 beta

 

Hello,

 

I tried “jstack -m PID”in latest openjdk 8 64-bit and “jhsdb jstack –mixed –pid 
PID” in latest openjdk 11 64-bit on latest Ubuntu(18.04) and latest fedora (29) 
and RHEL 8 Beta.

The stack traces returned are always incomplete like following output:

 

- 1479 -

"DestroyJavaVM" #17 prio=5 tid=0x00bae000 nid=0x5c7 waiting on 
condition [0x]

   java.lang.Thread.State: RUNNABLE

   JavaThread state: _thread_blocked

0x7f839224f9f3  __pthread_cond_wait + 0x243

- 1480 -

- 1481 -

"Reference Handler" #2 daemon prio=10 tid=0x00c36000 nid=0x5c9 waiting 
on condition [0x7f8376d15000]

   java.lang.Thread.State: RUNNABLE

   JavaThread state: _thread_blocked

0x7f839224f9f3  __pthread_cond_wait + 0x243

- 1482 -

"Finalizer" #3 daemon prio=8 tid=0x00c3a000 nid=0x5ca in Object.wait() 
[0x7f8376c14000]

   java.lang.Thread.State: WAITING (on object monitor)

   JavaThread state: _thread_blocked

0x7f839224f9f3  __pthread_cond_wait + 0x243

- 1483 -

"Signal Dispatcher" #4 daemon prio=9 tid=0x00c56800 nid=0x5cb runnable 
[0x]

   java.lang.Thread.State: RUNNABLE

   JavaThread state: _thread_blocked

0x7f83922526d6  do_futex_wait.constprop.1 + 0x36

 

The issue existed even after I disable the selinux. It can not be reproduced on 
oracle linux 7.4.

The kernel version on Ubuntu or fedora is 4.15. the kernel version on RHEL 8 
Beta is 4.18. the kernel version on Oracle linux 7.4 is 3.1.

I can not find any existed bug for now.

 

 

Is there any clue about this issue?

 

Any reply will be appreciated.

 

Best Regards

Gao Shuai from GCS

Re: RFR: JDK-8149461: jmap kills process if non-java pid is specified in the command line

[gary.ad...@oracle.com]

Let me clarify a few things about the proposed fix.

The VirtualMachine.list() mechanism is based on
Java processes that are up and running.
During VM startup when agent libraries are loaded,
the fact is recorded in the filesystem that a Java process
is eligible for an attach request.

This is a much smaller list than scanning for all the
running processes and works across all supported
platforms. It also doesn't rely on fragile command line
parsing to determine a Java launcher is used.

I believe most of the reported hang situations
are not for the first level information for pid and
command line requests. I believe the hangs are due
to the second level requests that actually attach
to the process and issue a command to the running
Java process.

The overhead for the pid check should be relatively small.
In a standalone command for a one time check at startup
with 10's of Java processes. I know the documentation
states the user is responsible for verifying with jps
that a Java process pid or vmid is provided. But the cost
of human error can be a terminated process.

Selection by name also has some drawbacks when multiple
copies of a command are running at the same time.

Running "jcmd 0 ..." is the documented way to run a command on
all running Java processes.

May I count you as a reviewer on the current changeset?

On 2/15/19 3:54 AM, Thomas Stüfe wrote:



On Fri, Feb 15, 2019 at 3:26 AM David Holmes > wrote:


Gary,

What is the overhead of doing the validation? How do we list VMs?
Do we
need to examine every running process to get the list of VMs?
Wouldn't
it be better to check the given process is a VM rather than
checking all
potential VM processes?


I think this is a valid point. Listing VMs is normally quick (just use 
jcmd without any parms) but I have seen this fail or hang rarely in 
situations where I then still was able to talk to my VM via pid. I 
never investigated this but I would not like to be unable to connect 
to my VM just because some rogue VM mailfunctions.


This would be an argument for the alternative I offered in my first 
answer - just use the proc file system or a similar mechanism to check 
only the pid you plan on sending sigquit to...


I think there is an onus of responsibility on the user to provide a
correct pid here, rather than trying to make this foolproof.


Oh but it can happen quite easily. For example, by pulling up an older 
jcmd from your shell history and forgetting to modify the pid. Thank 
god for the command name argument option on jcmd, which I now use mostly.


We also had a customer which tried to find all VMs on his machine by 
attempting to attach with jcmd to every process, killing them left and 
right :)


... Thomas

Thanks,
David

On 15/02/2019 5:12 am, Gary Adams wrote:
> The following commands present a similar kill process behavior:
>     jcmd
>     jinfo
>     jmap
>     jstack
>
> The following commands do not attach.
>     jstat sun.jvmstat.monitor.MonitorException "not found"
>     jps no pid arguments
>
> This update moves the checkJavaPid method into the
> common/ProcessArgumentsMatcher.java
> and applies the check before the pid is used for an attach
operation.
>
>    Revised webrev:
http://cr.openjdk.java.net/~gadams/8149461/webrev.01/
>
> On 2/14/19, 12:07 PM, Chris Plummer wrote:
>> Hi Gary,
>>
>> What about the other tools that attach to a user specified
process. Do
>> they also have this issue?
>>
>> thanks,
>>
>> Chris
>>
>> On 2/14/19 8:35 AM, Gary Adams wrote:
>>> There is an old reported problem that using jmap on a process
that is
>>> not running
>>> Java could cause the process to terminate. This is due to the
SIGQUIT
>>> used
>>> when attaching to the process.
>>>
>>> It is a fairly simple operation to validate that the pid
matches one
>>> of the known
>>> running Java processes using VirtualMachine.list().
>>>
>>>   Issue: https://bugs.openjdk.java.net/browse/JDK-8149461
>>>
>>> Proposed fix:
>>>
>>> diff --git a/src/jdk.jcmd/share/classes/sun/tools/jmap/JMap.java
>>> b/src/jdk.jcmd/share/classes/sun/tools/jmap/JMap.java
>>> --- a/src/jdk.jcmd/share/classes/sun/tools/jmap/JMap.java
>>> +++ b/src/jdk.jcmd/share/classes/sun/tools/jmap/JMap.java
>>> @@ -1,5 +1,5 @@
>>>  /*
>>> - * Copyright (c) 2005, 2018, Oracle and/or its affiliates. All
>>> rights reserved.
>>> + * Copyright (c) 2005, 2019, Oracle and/or its affiliates. All
>>> rights reserved.
>>>   * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
>>>   *
>>>   * This code is free software; you can redistribute it and/or
modify it
>>> @@ -30,6 +30,7 @@
>>>  import java.io.InputStream;
>>>  import java.io 

RE: jstack mixed mode and jhsdb stack mixed mode can not get expected result(stack trace is incomplete) on latest ubuntu or fedora or RHEL 8 beta

[Shuai Gao]

Hello Sharath,

 

Thanks for your reply.

I am afraid the issue in BUG JDK—8219086 is not the same as mine.

In the bug, the output of jhsdb jstack prints some mixed stack trace but does 
not print all the threads.

Within my issue, the output of jhsdb jstack can print all the threads but can 
only print the top of the stack trace.

And this issue can always be reproduced on latest Ubuntu (18.04),fedora (29) 
and RHEL 8 Beta.

On the aforementioned OSs , The pstack can work well with java application. I 
am afraid there is something about compatibility between JDK and new Linux 
kernel or ptrace.

 

Best Regards

 

 

 

From: Sharath Ballal 
Sent: 2019年2月15日 17:46
To: Shuai Gao ; serviceability-dev@openjdk.java.net
Subject: RE: jstack mixed mode and jhsdb stack mixed mode can not get expected 
result(stack trace is incomplete) on latest ubuntu or fedora or RHEL 8 beta

 

Hi Gao,

 

I also observed this failure and opened HYPERLINK 
"https://bugs.openjdk.java.net/browse/JDK-8219086"JDK-8219086 for it.

 

 

Thanks,

Sharath

 

 

From: Shuai Gao 
Sent: Friday, February 15, 2019 12:15 PM
To: HYPERLINK 
"mailto:serviceability-dev@openjdk.java.net"serviceability-dev@openjdk.java.net
Subject: jstack mixed mode and jhsdb stack mixed mode can not get expected 
result(stack trace is incomplete) on latest ubuntu or fedora or RHEL 8 beta

 

Hello,

 

I tried “jstack -m PID”in latest openjdk 8 64-bit and “jhsdb jstack –mixed –pid 
PID” in latest openjdk 11 64-bit on latest Ubuntu(18.04) and latest fedora (29) 
and RHEL 8 Beta.

The stack traces returned are always incomplete like following output:

 

- 1479 -

"DestroyJavaVM" #17 prio=5 tid=0x00bae000 nid=0x5c7 waiting on 
condition [0x]

   java.lang.Thread.State: RUNNABLE

   JavaThread state: _thread_blocked

0x7f839224f9f3  __pthread_cond_wait + 0x243

- 1480 -

- 1481 -

"Reference Handler" #2 daemon prio=10 tid=0x00c36000 nid=0x5c9 waiting 
on condition [0x7f8376d15000]

   java.lang.Thread.State: RUNNABLE

   JavaThread state: _thread_blocked

0x7f839224f9f3  __pthread_cond_wait + 0x243

- 1482 -

"Finalizer" #3 daemon prio=8 tid=0x00c3a000 nid=0x5ca in Object.wait() 
[0x7f8376c14000]

   java.lang.Thread.State: WAITING (on object monitor)

   JavaThread state: _thread_blocked

0x7f839224f9f3  __pthread_cond_wait + 0x243

- 1483 -

"Signal Dispatcher" #4 daemon prio=9 tid=0x00c56800 nid=0x5cb runnable 
[0x]

   java.lang.Thread.State: RUNNABLE

   JavaThread state: _thread_blocked

0x7f83922526d6  do_futex_wait.constprop.1 + 0x36

 

The issue existed even after I disable the selinux. It can not be reproduced on 
oracle linux 7.4.

The kernel version on Ubuntu or fedora is 4.15. the kernel version on RHEL 8 
Beta is 4.18. the kernel version on Oracle linux 7.4 is 3.1.

I can not find any existed bug for now.

 

 

Is there any clue about this issue?

 

Any reply will be appreciated.

 

Best Regards

Gao Shuai from GCS

RE: jstack mixed mode and jhsdb stack mixed mode can not get expected result(stack trace is incomplete) on latest ubuntu or fedora or RHEL 8 beta

[Sharath Ballal]

Hi Gao,

 

I also observed this failure and opened HYPERLINK 
"https://bugs.openjdk.java.net/browse/JDK-8219086"JDK-8219086 for it.

 

 

Thanks,

Sharath

 

 

From: Shuai Gao 
Sent: Friday, February 15, 2019 12:15 PM
To: serviceability-dev@openjdk.java.net
Subject: jstack mixed mode and jhsdb stack mixed mode can not get expected 
result(stack trace is incomplete) on latest ubuntu or fedora or RHEL 8 beta

 

Hello,

 

I tried “jstack -m PID”in latest openjdk 8 64-bit and “jhsdb jstack –mixed –pid 
PID” in latest openjdk 11 64-bit on latest Ubuntu(18.04) and latest fedora (29) 
and RHEL 8 Beta.

The stack traces returned are always incomplete like following output:

 

- 1479 -

"DestroyJavaVM" #17 prio=5 tid=0x00bae000 nid=0x5c7 waiting on 
condition [0x]

   java.lang.Thread.State: RUNNABLE

   JavaThread state: _thread_blocked

0x7f839224f9f3  __pthread_cond_wait + 0x243

- 1480 -

- 1481 -

"Reference Handler" #2 daemon prio=10 tid=0x00c36000 nid=0x5c9 waiting 
on condition [0x7f8376d15000]

   java.lang.Thread.State: RUNNABLE

   JavaThread state: _thread_blocked

0x7f839224f9f3  __pthread_cond_wait + 0x243

- 1482 -

"Finalizer" #3 daemon prio=8 tid=0x00c3a000 nid=0x5ca in Object.wait() 
[0x7f8376c14000]

   java.lang.Thread.State: WAITING (on object monitor)

   JavaThread state: _thread_blocked

0x7f839224f9f3  __pthread_cond_wait + 0x243

- 1483 -

"Signal Dispatcher" #4 daemon prio=9 tid=0x00c56800 nid=0x5cb runnable 
[0x]

   java.lang.Thread.State: RUNNABLE

   JavaThread state: _thread_blocked

0x7f83922526d6  do_futex_wait.constprop.1 + 0x36

 

The issue existed even after I disable the selinux. It can not be reproduced on 
oracle linux 7.4.

The kernel version on Ubuntu or fedora is 4.15. the kernel version on RHEL 8 
Beta is 4.18. the kernel version on Oracle linux 7.4 is 3.1.

I can not find any existed bug for now.

 

 

Is there any clue about this issue?

 

Any reply will be appreciated.

 

Best Regards

Gao Shuai from GCS

Re: RFR/C: 8218922: SA: Enable best-effort implementation of live regions iteration for ZGC

[Stefan Karlsson]

Thanks, Yasumasa!

StefanK

On 2019-02-15 01:54, Yasumasa Suenaga wrote:

Hi Stefan,

Both changes look good to me!


Thanks,

Yasumasa

2019年2月15日(金) 2:12 Stefan Karlsson :


Hi again,

I've separated the live regions iteration refactoring into this patch:
https://cr.openjdk.java.net/~stefank/8219003/webrev.01/

And use this RFE for the ZGC specific parts:
https://cr.openjdk.java.net/~stefank/8218922/webrev.02/

Thanks,
StefanK

On 2019-02-14 14:39, Stefan Karlsson wrote:

Hi Yasumasa,

On 2019-02-14 14:11, Yasumasa Suenaga wrote:

Hi Stefan,


Maybe this is enough to enable a bit more SA debugging capabilities
when
running with ZGC? What do you think, should we bring in this change?


I think it should be brought this in.
I filed same issue as JDK-8207843, but I've not yet worked.
So I will close it as duplicate of your change.



To be able to implement this more cleanly I've restructured the live
region collection, and pushed GC specific code into the specific GCs.
There are some extra usage of generics to make the code a bit easier to
read and develop.


IMHO refactoring for live region collection and ZGC related changes
should
be separated. What do you think?


Yes. I think that would be good. I'll separate this out into two changes.




Your change looks good to me.
BTW, did you check `jhsdb jmap --binaryheap` with this change?



Yes, when testing this I ran all tests in serviceability/sa and
manually tested the command above.

While testing this I also had this patch applied to enable SA hprof
for ZGC:
http://cr.openjdk.java.net/~stefank/8218970/webrev.01/

And used this patch to turn off the ZGC filtering in the tests:
http://cr.openjdk.java.net/~stefank/8218978/webrev.01/

I'm currently rerunning the tests to see that the latest changes
didn't break anything.

Thanks,
StefanK


Thanks,

Yasumasa



On 2019/02/13 23:52, Stefan Karlsson wrote:

Hi all,

Please review / comment on this patch to enable a best-effort live heap
region iteration implementation in ZGC.

http://cr.openjdk.java.net/~stefank/8218922/webrev.01/
https://bugs.openjdk.java.net/browse/JDK-8218922

The SA has functionally that relies on live heap region information
from
the GCs. This is problematic when dead objects are left in the heap,
and
their classes have been unloaded.

Because of this ZGC has so far not implemented this feature.
However, we
could provide a best-effort implementation that most likely will
work if
classes are not unloaded (or class unloading is turned off), and
otherwise might fail to fully parse and report all live heap regions.

For active, non-relocating pages the patch simply returns [start, top)
and for pages being actively relocated, it reports regions containing
the non-forwarded objects, the other objects are either dead or
could be
found in one of the to-regions.

With this patch I'm able to get heap histograms with ZGC.

Maybe this is enough to enable a bit more SA debugging capabilities
when
running with ZGC? What do you think, should we bring in this change?

To be able to implement this more cleanly I've restructured the live
region collection, and pushed GC specific code into the specific GCs.
There are some extra usage of generics to make the code a bit easier to
read and develop.

Thanks,
StefanK

Re: RFR: JDK-8149461: jmap kills process if non-java pid is specified in the command line

[Thomas Stüfe]

On Fri, Feb 15, 2019 at 3:26 AM David Holmes 
wrote:

> Gary,
>
> What is the overhead of doing the validation? How do we list VMs? Do we
> need to examine every running process to get the list of VMs? Wouldn't
> it be better to check the given process is a VM rather than checking all
> potential VM processes?
>
>
I think this is a valid point. Listing VMs is normally quick (just use jcmd
without any parms) but I have seen this fail or hang rarely in situations
where I then still was able to talk to my VM via pid. I never investigated
this but I would not like to be unable to connect to my VM just because
some rogue VM mailfunctions.

This would be an argument for the alternative I offered in my first answer
- just use the proc file system or a similar mechanism to check only the
pid you plan on sending sigquit to...


> I think there is an onus of responsibility on the user to provide a
> correct pid here, rather than trying to make this foolproof.
>
>
Oh but it can happen quite easily. For example, by pulling up an older jcmd
from your shell history and forgetting to modify the pid. Thank god for the
command name argument option on jcmd, which I now use mostly.

We also had a customer which tried to find all VMs on his machine by
attempting to attach with jcmd to every process, killing them left and
right :)

... Thomas


> Thanks,
> David
>
> On 15/02/2019 5:12 am, Gary Adams wrote:
> > The following commands present a similar kill process behavior:
> > jcmd
> > jinfo
> > jmap
> > jstack
> >
> > The following commands do not attach.
> > jstat sun.jvmstat.monitor.MonitorException "not found"
> > jps no pid arguments
> >
> > This update moves the checkJavaPid method into the
> > common/ProcessArgumentsMatcher.java
> > and applies the check before the pid is used for an attach operation.
> >
> >Revised webrev: http://cr.openjdk.java.net/~gadams/8149461/webrev.01/
> >
> > On 2/14/19, 12:07 PM, Chris Plummer wrote:
> >> Hi Gary,
> >>
> >> What about the other tools that attach to a user specified process. Do
> >> they also have this issue?
> >>
> >> thanks,
> >>
> >> Chris
> >>
> >> On 2/14/19 8:35 AM, Gary Adams wrote:
> >>> There is an old reported problem that using jmap on a process that is
> >>> not running
> >>> Java could cause the process to terminate. This is due to the SIGQUIT
> >>> used
> >>> when attaching to the process.
> >>>
> >>> It is a fairly simple operation to validate that the pid matches one
> >>> of the known
> >>> running Java processes using VirtualMachine.list().
> >>>
> >>>   Issue: https://bugs.openjdk.java.net/browse/JDK-8149461
> >>>
> >>> Proposed fix:
> >>>
> >>> diff --git a/src/jdk.jcmd/share/classes/sun/tools/jmap/JMap.java
> >>> b/src/jdk.jcmd/share/classes/sun/tools/jmap/JMap.java
> >>> --- a/src/jdk.jcmd/share/classes/sun/tools/jmap/JMap.java
> >>> +++ b/src/jdk.jcmd/share/classes/sun/tools/jmap/JMap.java
> >>> @@ -1,5 +1,5 @@
> >>>  /*
> >>> - * Copyright (c) 2005, 2018, Oracle and/or its affiliates. All
> >>> rights reserved.
> >>> + * Copyright (c) 2005, 2019, Oracle and/or its affiliates. All
> >>> rights reserved.
> >>>   * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
> >>>   *
> >>>   * This code is free software; you can redistribute it and/or modify
> it
> >>> @@ -30,6 +30,7 @@
> >>>  import java.io.InputStream;
> >>>  import java.io.UnsupportedEncodingException;
> >>>  import java.util.Collection;
> >>> +import java.util.List;
> >>>
> >>>  import com.sun.tools.attach.VirtualMachine;
> >>>  import com.sun.tools.attach.VirtualMachineDescriptor;
> >>> @@ -125,6 +126,10 @@
> >>>  private static void executeCommandForPid(String pid, String
> >>> command, Object ... args)
> >>>  throws AttachNotSupportedException, IOException,
> >>> UnsupportedEncodingException {
> >>> +// Before attaching, confirm that pid is a known Java process
> >>> +if (!checkJavaPid(pid)) {
> >>> +throw new AttachNotSupportedException();
> >>> +}
> >>>  VirtualMachine vm = VirtualMachine.attach(pid);
> >>>
> >>>  // Cast to HotSpotVirtualMachine as this is an
> >>> @@ -196,6 +201,19 @@
> >>>  executeCommandForPid(pid, "dumpheap", filename, liveopt);
> >>>  }
> >>>
> >>> +// Check that pid matches a known running Java process
> >>> +static boolean checkJavaPid(String pid) {
> >>> +List l = VirtualMachine.list();
> >>> +boolean found = false;
> >>> +for (VirtualMachineDescriptor vmd: l) {
> >>> +if (vmd.id().equals(pid)) {
> >>> +found = true;
> >>> +break;
> >>> +}
> >>> +}
> >>> +return found;
> >>> +}
> >>> +
> >>>  private static void checkForUnsupportedOptions(String[] args) {
> >>>  // Check arguments for -F, -m, and non-numeric value
> >>>  // and warn the user that SA is not supported anymore
> >>
> >>
> >
>