Re: Questions regarding Perl instrumentation
Rafael == Rafael Garcia-Suarez [EMAIL PROTECTED] writes:
Rafael Alexander Kolbasov wrote:
Solaris (*) has a dynamic instrumentation tool called DTrace. It allows
to
*dynamically* instrument any running application and get useful
information
about its behavior with zero (or almost zero) impact on the application
when it
is not instrumented.
This works fine with normal applications, but not so good with
interpteded
languages since C function names and data types have little to do with
the
structure of the interpreted language. I am playing with the idea of
adding some
hooks to Perl5 that will allow DTrace to be used for Perl programs (**).
As an
Rafael Maybe are you aware of
Rafael http://blogs.sun.com/roller/page/alanbur?entry=dtrace_and_perl
already ?
Thanks for the pointer! It seems that Alan already made a pretty good progress
on adding probes for subroutines. I am still curious whether it is possible to
use DTrace to be able to trace based on Perl line numbers. IMO this will add a
*tremendous* value.
initial experiment I changed Perl_runops_standard() function a bit and
was able
to use DTrace to trace start and end of each Perl OP evaluation. This is
not
Rafael You can also plug in your own runloop at compile time. See for
example
Rafael my module Runops::Switch on CPAN.
Cool! I'll take a look at it. Does it have the knowledge of the current file and
line numbers and when execution switches from line X to Y?
Rafael The line number isn't stored in all ops, but only in COPs. You then
get
Rafael it with the CopLINE macro, see cop.h.
How do I know whether I am dealing with COP? Also, looking at cop.h it seems
that getting file/lines with USE_ITHREADS is pretty cheap but otherwise can be
quite expensive. Is it a way to get file/line number information from the op
tree (may be not from every op) cheaply? For example, would it be reasonable
(performance-wise) to get line/file information once per Perl_runops_standard()
entry? Would it make any sense to do so?
In my prototype I changed Perl_runops_standard() to
int
Perl_runops_standard(pTHX)
{
OP* (CPERLscope(*op_addr))(pTHX) = PL_op-op_ppaddr;
/*
* Would be really cool to insert here something like
*
* DTRACE_PROBE2(perl__line, file_name, line_number);
*
*/
do {
DTRACE_PROBE1(perl, call__entry, op_addr);
PL_op = CALL_FPTR(op_addr)(aTHX);
DTRACE_PROBE1(perl, call__return, op_addr);
if (!PL_op)
break;
op_addr = PL_op-op_ppaddr;
PERL_ASYNC_CHECK();
} while (1);
TAINT_NOT;
return 0;
}
P.S. Please bear in mind that I am *really* ignorant in Perl internals.
- Alexander Kolbasov
Questions regarding Perl instrumentation
Dear Perl internals experts,
I am new to this list and to Perl internals, so please keep this in mind.
Solaris (*) has a dynamic instrumentation tool called DTrace. It allows to
*dynamically* instrument any running application and get useful information
about its behavior with zero (or almost zero) impact on the application when it
is not instrumented.
This works fine with normal applications, but not so good with interpteded
languages since C function names and data types have little to do with the
structure of the interpreted language. I am playing with the idea of adding some
hooks to Perl5 that will allow DTrace to be used for Perl programs (**). As an
initial experiment I changed Perl_runops_standard() function a bit and was able
to use DTrace to trace start and end of each Perl OP evaluation. This is not
very cool since there is no correlation between that and the Perl line numbers.
So the first question I have is
- Is it possible to get information about Perl script line number that the op
belongs to? I need this information from the call to
Perl_runops_standard(pTHX). In general, what is the good way (if any) to
access line number information?
A much more interesting thing is being able to trace Perl function calls. Seems
like pp_entersub() is the right place to look at. This seems like a rather
complex function that I don't quite understand. I hacked it a bit to get the
name of the called function by using GvNAME(CvGV(cv)) which is, probably, not
quite safe since I have not checked whether CvGV(cv) returns something non-NULL.
So I have the second question:
- What is the safe and efficient way (and place in pp_entersub()) to get the
string representing the function name?
- Can I get the function name from pp_leavesub()?
- Can I get line number information for the caller in pp_entersub()?
I am working with Perl 5.8.6.
Here is what I was able to trace from the simple perl script:
$x=1; $y=2;
sub foo() {$x+$y}
$z=foo();
1 | Perl_runops_standard:call-returnminiperl`Perl_pp_readline
1 | Perl_runops_standard:call-entry miniperl`Perl_pp_defined
1 | Perl_runops_standard:call-returnminiperl`Perl_pp_defined
1 | Perl_runops_standard:call-entry miniperl`Perl_pp_and
1 | Perl_runops_standard:call-returnminiperl`Perl_pp_and
1 | Perl_runops_standard:call-entry miniperl`Perl_pp_nextstate
1 | Perl_runops_standard:call-returnminiperl`Perl_pp_nextstate
1 | Perl_runops_standard:call-entry miniperl`Perl_pp_enterloop
1 | Perl_runops_standard:call-returnminiperl`Perl_pp_enterloop
1 | Perl_runops_standard:call-entry miniperl`Perl_pp_nextstate
1 | Perl_runops_standard:call-returnminiperl`Perl_pp_nextstate
1 | Perl_runops_standard:call-entry miniperl`Perl_pp_gvsv
1 | Perl_runops_standard:call-returnminiperl`Perl_pp_gvsv
1 | Perl_runops_standard:call-entry miniperl`Perl_pp_entereval
1 | Perl_runops_standard:call-returnminiperl`Perl_pp_entereval
1 | Perl_runops_standard:call-entry miniperl`Perl_pp_nextstate
1 | Perl_runops_standard:call-returnminiperl`Perl_pp_nextstate
1 | Perl_runops_standard:call-entry miniperl`Perl_pp_const
1 | Perl_runops_standard:call-returnminiperl`Perl_pp_const
1 | Perl_runops_standard:call-entry miniperl`Perl_pp_gvsv
1 | Perl_runops_standard:call-returnminiperl`Perl_pp_gvsv
1 | Perl_runops_standard:call-entry miniperl`Perl_pp_sassign
1 | Perl_runops_standard:call-returnminiperl`Perl_pp_sassign
1 | Perl_runops_standard:call-entry miniperl`Perl_pp_nextstate
1 | Perl_runops_standard:call-returnminiperl`Perl_pp_nextstate
1 | Perl_runops_standard:call-entry miniperl`Perl_pp_const
1 | Perl_runops_standard:call-returnminiperl`Perl_pp_const
1 | Perl_runops_standard:call-entry miniperl`Perl_pp_gvsv
1 | Perl_runops_standard:call-returnminiperl`Perl_pp_gvsv
1 | Perl_runops_standard:call-entry miniperl`Perl_pp_sassign
1 | Perl_runops_standard:call-returnminiperl`Perl_pp_sassign
1 | Perl_runops_standard:call-entry miniperl`Perl_pp_leaveeval
1 | Perl_runops_standard:call-returnminiperl`Perl_pp_leaveeval
1 | Perl_runops_standard:call-entry miniperl`Perl_pp_leaveloop
1 | Perl_runops_standard:call-returnminiperl`Perl_pp_leaveloop
1 | Perl_runops_standard:call-entry miniperl`Perl_pp_unstack
1 | Perl_runops_standard:call-returnminiperl`Perl_pp_unstack
1 | Perl_runops_standard:call-entry miniperl`Perl_pp_gvsv
1 | Perl_runops_standard:call-returnminiperl`Perl_pp_gvsv
1 | Perl_runops_standard:call-entry miniperl`Perl_pp_gv
1 | Perl_runops_standard:call-returnminiperl`Perl_pp_gv
1 | Perl_runops_standard:call-entry miniperl`Perl_pp_readline
1 | Perl_runops_standard:call-return
Re: Questions regarding Perl instrumentation
Alexander Kolbasov wrote: Solaris (*) has a dynamic instrumentation tool called DTrace. It allows to *dynamically* instrument any running application and get useful information about its behavior with zero (or almost zero) impact on the application when it is not instrumented. This works fine with normal applications, but not so good with interpteded languages since C function names and data types have little to do with the structure of the interpreted language. I am playing with the idea of adding some hooks to Perl5 that will allow DTrace to be used for Perl programs (**). As an Maybe are you aware of http://blogs.sun.com/roller/page/alanbur?entry=dtrace_and_perl already ? initial experiment I changed Perl_runops_standard() function a bit and was able to use DTrace to trace start and end of each Perl OP evaluation. This is not You can also plug in your own runloop at compile time. See for example my module Runops::Switch on CPAN. very cool since there is no correlation between that and the Perl line numbers. So the first question I have is - Is it possible to get information about Perl script line number that the op belongs to? I need this information from the call to Perl_runops_standard(pTHX). In general, what is the good way (if any) to access line number information? The line number isn't stored in all ops, but only in COPs. You then get it with the CopLINE macro, see cop.h. A much more interesting thing is being able to trace Perl function calls. Seems like pp_entersub() is the right place to look at. This seems like a rather complex function that I don't quite understand. I hacked it a bit to get the name of the called function by using GvNAME(CvGV(cv)) which is, probably, not quite safe since I have not checked whether CvGV(cv) returns something non-NULL. So I have the second question: - What is the safe and efficient way (and place in pp_entersub()) to get the string representing the function name? I can't completely answer this off hand, but consider a few cases : - exported functions. maybe use EGV instead of GV - test with XSUBs - some functions are autoloaded - generally, I look at how the warnings and error display such things as function names and infer from that - Can I get the function name from pp_leavesub()? IIRC the CV is on the stack (popped by POPSUB). - Can I get line number information for the caller in pp_entersub()? You can use PL_curcop to get the current COP. HTH. Your project looks very interesting.
Re: Questions regarding Perl instrumentation
Rafael Garcia-Suarez wrote: Maybe are you aware of http://blogs.sun.com/roller/page/alanbur?entry=dtrace_and_perl already ? As Rafael says, I've already done some work on this, documented at the link above. My intention is to put some generic macros into perl at the appropriate places. By default they woud be empty, in the case of Solaris you will be able to instantiate them as DTrace probes, on other platforms they could be whatever tracing system the platform supports. I've also spoken to Bryan, Mike and Adam about how to reduce the probe effect, I just haven't had the time to do the work. I'll be in MPK17 next week if you want to discuss it further. -- Alan Burlison --
Re: Questions regarding Perl instrumentation
Alan Burlison wrote: Rafael Garcia-Suarez wrote: Maybe are you aware of http://blogs.sun.com/roller/page/alanbur?entry=dtrace_and_perl already ? As Rafael says, I've already done some work on this, documented at the link above. My intention is to put some generic macros into perl at the appropriate places. By default they woud be empty, in the case of Solaris you will be able to instantiate them as DTrace probes, on other platforms they could be whatever tracing system the platform supports. I think it's worth investigating doing it in a module, providing an alternate runloop or only changing the pp functions for entersub and leavesub.
Re: Questions regarding Perl instrumentation
On Tue, Sep 13, 2005 at 10:02:38AM +0200, Rafael Garcia-Suarez wrote: Alan Burlison wrote: Rafael Garcia-Suarez wrote: Maybe are you aware of http://blogs.sun.com/roller/page/alanbur?entry=dtrace_and_perl already ? As Rafael says, I've already done some work on this, documented at the link above. My intention is to put some generic macros into perl at the appropriate places. By default they woud be empty, in the case of Solaris you will be able to instantiate them as DTrace probes, on other platforms they could be whatever tracing system the platform supports. I think it's worth investigating doing it in a module, providing an alternate runloop or only changing the pp functions for entersub and leavesub. From what I remember Alan telling me about what Dave had said, there are were a few more points other than entersub and leavesub that would need instrumentation. goto sub; was the most obscure, but I think that require was another. I see nothing wrong with doing it as a module, if that brings the benefit to older, installed, perls, but if the code changes are clean and compile time enabled, I think that once working and stable it would actually be appropriate for 5.8.x. (Given that the whole point of dtrace is to provide a way of instrumenting already running code, and it is already shipped in some very stable mission critical code, such as the Solaris kernel) Alan demonstrated dtrace to me with his perl prototype. It's impressive. ONE interface to instrument calls in perl space, the perl C code, the C runtime library, and the kernel. It's trivial to instrument particular kernel calls made only while you're in a particular perl subroutine. I want that, working threads debugging and valgrind on the same platform. Please. The pony can wait. Nicholas Clark
Re: Questions regarding Perl instrumentation
Nicholas Clark wrote: From what I remember Alan telling me about what Dave had said, there are were a few more points other than entersub and leavesub that would need instrumentation. goto sub; was the most obscure, but I think that require was another. What about the unusual ways of exiting a subroutine, e.g. with next ?
Re: Questions regarding Perl instrumentation
On Tue, Sep 13, 2005 at 11:14:07AM +0200, Rafael Garcia-Suarez wrote: Nicholas Clark wrote: From what I remember Alan telling me about what Dave had said, there are were a few more points other than entersub and leavesub that would need instrumentation. goto sub; was the most obscure, but I think that require was another. What about the unusual ways of exiting a subroutine, e.g. with next ? My suggestion to Alan was to hitch a ride on the PUSHSUB/POPSUB macros; that way you should be able to trap *all* occurances, including hand-rolled calls in XS (eg the call outs to Perl in List::Util) -- Emacs isn't a bad OS once you get used to it. It just lacks a decent editor.
Re: Questions regarding Perl instrumentation
Rafael Garcia-Suarez wrote: I think it's worth investigating doing it in a module, providing an alternate runloop or only changing the pp functions for entersub and leavesub. Putting probes around every op dispatch has a very noticeable effect, the reason for the per-sub probes was it provided a reasonable balance between observability and intrusiveness. The functions themselves aren't enough as the ENTERSUB and LEAVESUB macros are used in other places as well: # dtrace -l | grep perl 36464perl955libperl.so perl_run sub-entry 36465perl955libperl.so Perl_pp_sort sub-entry 36466perl955libperl.so Perl_pp_dbstate sub-entry 36467perl955libperl.so Perl_pp_entersub sub-entry 36468perl955libperl.so perl_run sub-return 36469perl955libperl.so Perl_pp_last sub-return 36470perl955libperl.soPerl_pp_return sub-return 36471perl955libperl.so Perl_dounwind sub-return 36472perl955libperl.soPerl_pp_leavesublv sub-return 36473perl955libperl.so Perl_pp_leavesub sub-return Dave Mitchell pointed out to me that we'll also need hooks in eval and require for example. I'd be more than happy to do it in a module, but I'm not clear how I would replace functions in libperl from a module. The whole point of dtrace is that it is lightweight enough to leave it in the code all the time. Loading an additional module normally requires that you stop/start the application, embedding the probes would mean you could enable the probes on the fly. -- Alan Burlison --
Re: Questions regarding Perl instrumentation
Alan Burlison wrote: I'd be more than happy to do it in a module, but I'm not clear how I would replace functions in libperl from a module. Your module could, for example, replace the op_ppaddr fields of the ops you want to intrument with your own DTrace-enabled implementation. (I note that every op has an op_ppaddr field, which is probably some memory overhead. However that also means that you can instrument only part of the optree) Or your module can replace the runloop and provide alternate implementations directly in it for the ops you want to instrument. The whole point of dtrace is that it is lightweight enough to leave it in the code all the time. Loading an additional module normally requires that you stop/start the application, embedding the probes would mean you could enable the probes on the fly. That's a significant point for enabling DTrace at compile time in perl.
Re: Questions regarding Perl instrumentation
On Tue, Sep 13, 2005 at 01:29:41PM +0200, Rafael Garcia-Suarez wrote: Alan Burlison wrote: I'd be more than happy to do it in a module, but I'm not clear how I would replace functions in libperl from a module. Your module could, for example, replace the op_ppaddr fields of the ops you want to intrument with your own DTrace-enabled implementation. (I note that every op has an op_ppaddr field, which is probably some memory overhead. However that also means that you can instrument only part of the optree) If you take this route remember that the optree is shared between threads which can make things even more interesting. Been there, done that, the wounds are still open. Or your module can replace the runloop and provide alternate implementations directly in it for the ops you want to instrument. That being a logical or. The whole point of dtrace is that it is lightweight enough to leave it in the code all the time. Loading an additional module normally requires that you stop/start the application, embedding the probes would mean you could enable the probes on the fly. That's a significant point for enabling DTrace at compile time in perl. If you can come up with a generic lightweight method of instrumenting subroutine entry and exit I can think of a few interesting applications and I'm sure others could think of many more. -- Paul Johnson - [EMAIL PROTECTED] http://www.pjcj.net
Re: Questions regarding Perl instrumentation
Rafael Garcia-Suarez wrote: Maybe are you aware of http://blogs.sun.com/roller/page/alanbur?entry=dtrace_and_perl already ? As Rafael says, I've already done some work on this, documented at the link above. My intention is to put some generic macros into perl at the appropriate places. By default they woud be empty, in the case of Solaris you will be able to instantiate them as DTrace probes, on other platforms they could be whatever tracing system the platform supports. I've also spoken to Bryan, Mike and Adam about how to reduce the probe effect, I just haven't had the time to do the work. I'll be in MPK17 next week if you want to discuss it further. Ok, now it is in good hands! - Alexander Kolbasov
