Hi Jc,
On 21/04/2019 8:15 am, Jean Christophe Beyler wrote:
Hi David,
Hopefully this gives more details :)
Basically, in the Java agent, we set up a SIGPROF handler and we get a
handler call at a set frequency rate (generally the profiling is for 30
seconds). The thread that is doing the signal handler can be a thread
from the JVM and as we have said in this thread, internal VM threads
can't get a JNIEnv or are visible to JVMTI anyway so we can't really get
the name.
And yet ...
Our current implementation, when faced with a thread like that, calls a
method like above that we added in the JVM to provide information about
what type of work was being done so that when we provide our profiles,
we can give that information to the user.
... they can call this method you added to the JVM? How do they do that?
And if they can do that why can't they get the name the same way? Or
call any existing query methods to piece together the information?
Cheers,
David
However for agents that cannot use the modified JVM, we cannot provide
this information it seems and basically we have to fallback to getting
the PC and just attributing it to libjvm.so (see an example here
https://github.com/GoogleCloudPlatform/cloud-profiler-java/blob/master/src/profiler.cc#L127)
So basically, in cases where we are in a signal handler, where we are
asynchronous and cannot do a lot, getting a side method to just say :
"hey is this a compiler thread, is this a GC thread, etc." would
actually be helpful information during profiling instead of the general
"well this many threads were in libjvm.so but we don't know what was
going on".
I've thought about using some other information such as
CompilationMXBean to try to assess what is going on during profiling but
it seems hard to correctly attribute that back into the actual profiles
and untangle the libjvm.so buccket.
Hopefully this makes sense,
Jc
On Sat, Apr 20, 2019 at 12:20 AM David Holmes <david.hol...@oracle.com
<mailto:david.hol...@oracle.com>> wrote:
On 20/04/2019 10:29 am, Jean Christophe Beyler wrote:
> Hi David,
>
> On Fri, Apr 19, 2019 at 6:49 PM David Holmes
<david.hol...@oracle.com <mailto:david.hol...@oracle.com>
> <mailto:david.hol...@oracle.com
<mailto:david.hol...@oracle.com>>> wrote:
>
> Hi Jc,
>
> On 20/04/2019 12:30 am, Jean Christophe Beyler wrote:
> > Problem is that if I don't have a jthread, I can't get the
name it
> > seems. Perhaps it could help if I gave more information:
> >
> > - In our JVM profiling mechanism, we have a SIGPROF (and maybe
> that's a
> > wrong approach :-)) that gets cycled across threads (some Java
> threads,
> > some are the other threads)
> > - It is the other threads that I'm interested here to
be able to
> > distinguish what they are in terms of of profiles
> >
> > Is there any way we could provide that (not in JVMTI then)?
> > - The only way I could imagine perhaps doing this would be
> perhaps
> > to have a set of other tools at the same time running
(either using
> > beans before/after or JFR) but this seems crude as well
(better than
> > nothing though)
> > - I wish there was a way to just be able to get a type
for those
> > internal frames while doing the actual SIGPROF handling
> >
> > FWIW, the method we expose basically is like this:
> > Thread* current_thread =
> ThreadLocalStorage::get_thread_async_safe();
>
> We have Thread::current_or_null_safe() for that.
>
>
> A decade old code might have rotted a bit (or been wrong from the
> start), I'll change this internal code :)
>
>
> > if (current_thread == NULL) {
> > return -1;
> > } else if (current_thread->is_Compiler_thread()) {
> > return _activity_compile;
> > } else if (current_thread->is_Java_thread()) {
> > return -1;
> > } else if (current_thread->is_GC_task_thread()) {
> > return _activity_gc;
> > } else if (current_thread->is_VM_thread()) {
> > VMThread* vm_thread = (VMThread*) current_thread;
> > VM_Operation* vm_op = vm_thread->vm_operation();
> > if (vm_op != NULL) {
> > switch (vm_op->type()) {
> > case VM_Operation::VMOp_GC_HeapInspection:
> > case VM_Operation::VMOp_GenCollectFull:
> > case VM_Operation::VMOp_GenCollectFullConcurrent:
> > case VM_Operation::VMOp_GenCollectForAllocation:
> > case VM_Operation::VMOp_ParallelGCFailedAllocation:
> > case VM_Operation::VMOp_ParallelGCSystemGC:
> > case VM_Operation::VMOp_CGC_Operation:
> > case VM_Operation::VMOp_CMS_Initial_Mark:
> > case VM_Operation::VMOp_CMS_Final_Remark:
> > case VM_Operation::VMOp_G1CollectFull:
> > case VM_Operation::VMOp_G1CollectForAllocation:
> > case VM_Operation::VMOp_G1IncCollectionPause:
> > return _activity_gc;
> > default:
> > break;
> > }
> > }
> > }
> > return _activity_other_vm;
> > }
>
> So it's not really the thread "type" but the logical
"activity". For
> "type" you'd just need a query version of
Thread::print_on_error (more
> or less).
>
>
> Not at all sure where you could put this - nor clear why you
need to
> put
> it somewhere: isn't this just something executed by your SIGPROF
> handler?
>
>
> Well problem is that I'm not in the JVM at the sigprof handler
level.
> I'm actually in the agent,
Not sure what you mean. I'm assuming you're sending a SIGPROF to each
thread and using the handler for profiling - no? Otherwise please
clarify what is happening in each thread.
> so basically from the agent's point of view,
> I don't really know what "Activity" I just stopped but would love to
> know. We added internally this change to figure it out but I'd
like to
> get it in the open-source so that all could use it and not just us
A "char* Thread::get_thread_type()" API might be useful (as I said a
query version of print_on_error(). But otherwise this seems something
peculiar to your agent so simply composing existing API calls - as you
outline - seems the appropriate way to deal with this.
Seems to me the VMThread is the problem here because you want to try
and
attribute the VM_operation to different "buckets". But given the
bulk of
the work is actually done by other threads (e.g. GC), and going forward
less and less will be done by the VMThread itself (e.g. async monitor
deflation), is it really worth trying to classify this at a finer level
than just "in the VM"?
Cheers,
David
> internally. Basically, like I said ,when using open-source
profilers,
> this would help divide up the "libjvm.so" bucket that a lot of
profilers
> are getting.
> Thanks for your insight as always,
> Jc
>
>
>
> David
>
> > It's crude but we believe it is effective to at least
"bucketize"
> the
> > internals while doing our profiling.
> >
> > Thanks for your input,
> > Jc
> >
> > On Fri, Apr 19, 2019 at 9:01 AM Alan Bateman
> <alan.bate...@oracle.com <mailto:alan.bate...@oracle.com>
<mailto:alan.bate...@oracle.com <mailto:alan.bate...@oracle.com>>
> > <mailto:alan.bate...@oracle.com
<mailto:alan.bate...@oracle.com>
> <mailto:alan.bate...@oracle.com
<mailto:alan.bate...@oracle.com>>>> wrote:
> >
> > On 19/04/2019 00:12, David Holmes wrote:
> > >
> > > I think it would be difficult to put something like
this
> in JVM TI
> > > given that the use of threads within the JVM are
purely an
> > > implementation detail and not standardized in any
way. And
> many of
> > > those threads are hidden from JVM TI anyway.
> > >
> > > The names of threads are the normal way to see what
"type"
> of thread
> > > you're dealing with.
> > Right, JVM TI only deals with "Java threads" (jthread
object) and
> > has no
> > knowledge about other threads. It might be possible to
use its
> > extension
> > mechanism to provide information about other threads
but it
> wouldn't be
> > interoperable with anything that use jtherad objects.
> >
> > -Alan
> >
> >
> >
> > --
> >
> > Thanks,
> > Jc
>
>
>
> --
>
> Thanks,
> Jc
--
Thanks,
Jc
