[issue4111] Add Systemtap/DTrace probes

[hai shi]

Change by hai shi :


--
pull_requests: +23865
pull_request: https://github.com/python/cpython/pull/25115

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4111] Add Systemtap/DTrace probes

[hai shi]

Change by hai shi :


--
pull_requests:  -23859

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4111] Add Systemtap/DTrace probes

[hai shi]

Change by hai shi :


--
nosy: +shihai1991
nosy_count: 25.0 -> 26.0
pull_requests: +23859
pull_request: https://github.com/python/cpython/pull/25115

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4111] Add Systemtap/DTrace probes

[cburroughs]

Changes by cburroughs :


--
nosy: +cburroughs

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4111] Add Systemtap/DTrace probes

[Dave Malcolm]

Dave Malcolm dmalc...@redhat.com added the comment:

Issue #13405 covers DTrace; I've taken the liberty of also opening issue #14776 
to cover SystemTap.  I hope that once one of these is in the tree it will be 
easier to get the other one in.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4111
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4111] Add Systemtap/DTrace probes

[Jesús Cea Avión]

Jesús Cea Avión j...@jcea.es added the comment:

This project continues in issue #13405.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4111
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4111] Add Systemtap/DTrace probes

[Jesús Cea Avión]

Changes by Jesús Cea Avión j...@jcea.es:


--
resolution:  - out of date
status: open - closed
superseder:  - Add DTrace probes

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4111
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4111] Add Systemtap/DTrace probes

[Jesús Cea Avión]

Jesús Cea Avión j...@jcea.es added the comment:

Anybody still working on this?.

We missed the 2.7 boat. DO NOT MISS THE 3.3 ONE!!! :-)

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4111
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4111] Add Systemtap/DTrace probes

[Brett Hoerner]

Changes by Brett Hoerner bretthoer...@gmail.com:


--
nosy:  -bretthoerner

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4111
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4111] Add Systemtap/DTrace probes

[Martin Marcher]

Changes by Martin Marcher mar...@marcher.name:


--
nosy: +serverhorror

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4111
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4111] Add Systemtap/DTrace probes

[anatoly techtonik]

anatoly techtonik techto...@gmail.com added the comment:

2011/4/6 Jesús Cea Avión rep...@bugs.python.org:

 Jesús Cea Avión j...@jcea.es added the comment:

 Some more references:

 Read the notes under the slides:
 https://dgl.cx/2011/01/dtrace-and-perl

 https://dgl.cx/dtrace

 http://dtrace.org/blogs/

 What do we need to unblock this?

Summarize 30+ page discussion in a new issue.
Blog about it on http://blog.python.org/
--
anatoly t.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4111
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4111] Add Systemtap/DTrace probes

[Dave Malcolm]

Dave Malcolm dmalc...@redhat.com added the comment:

jcea: I notice that on 2011-02-22 you made these changes:
  assignee: dmalcolm - dino.viehland
  nosy: +dino.viehland
  versions: +Python 3.3 -Python 3.2

Did you mean to change the assignee, or was this an accident?

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4111
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4111] Add Systemtap/DTrace probes

[Jesús Cea Avión]

Jesús Cea Avión j...@jcea.es added the comment:

Malcolm, it was a mistake. I only wanted to change the TARGET.

I assign the issue to you again.

Dino, I delete you from the nosy list now. Sorry for the inconvenience.

My excuses to both.

--
assignee: dino.viehland - dmalcolm
nosy:  -dino.viehland

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4111
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4111] Add Systemtap/DTrace probes

[Jesús Cea Avión]

Jesús Cea Avión j...@jcea.es added the comment:

Some more references:

Read the notes under the slides:
https://dgl.cx/2011/01/dtrace-and-perl

https://dgl.cx/dtrace

http://dtrace.org/blogs/

What do we need to unblock this?

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4111
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4111] Add Systemtap/DTrace probes

[Jesús Cea Avión]

Changes by Jesús Cea Avión j...@jcea.es:


--
assignee: dmalcolm - dino.viehland
nosy: +dino.viehland
versions: +Python 3.3 -Python 3.2

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4111
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4111] Add Systemtap/DTrace probes

[Jesús Cea Avión]

Jesús Cea Avión j...@jcea.es added the comment:

Some references to keep around:

http://src.opensolaris.org/source/xref/jds/spec-files/trunk/patches/
http://src.opensolaris.org/source/xref/jds/spec-files/trunk/patches/Python26-07-dtrace.diff
http://src.opensolaris.org/source/xref/jds/spec-files/trunk/patches/Python-07-dtrace.diff

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4111
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4111] Add Systemtap/DTrace probes

[John Levon]

John Levon movem...@users.sourceforge.net added the comment:

configure.in has:

AC_MSG_RESULT([$with_dtrace])
...
AC_MSG_RESULT($with_dtrace)

Why twice? It looks confusing.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4111
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4111] Add Systemtap/DTrace probes

[Kapil Thangavelu]

Changes by Kapil Thangavelu kapil.f...@gmail.com:


--
nosy: +hazmat

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4111
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4111] Add Systemtap/DTrace probes

[Jesús Cea Avión]

Jesús Cea Avión j...@jcea.es added the comment:

Dave, we need some kind of documentation, if we expect to ship this in Python 
3.2. The deadline is only 10-15 days away.

Could you write something able to be in the standard documentation?.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4111
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4111] Add Systemtap/DTrace probes

[Antoine Pitrou]

Antoine Pitrou pit...@free.fr added the comment:

For the record, replacing /dev/null with conftest.out in the configure test 
solves the detection problem (and allows Python to build cleanly). However, 
there is then a problem in test_systemtap (even when replacing stap with 
dtrace) since the syntax for scripts doesn't seem compatible.

In any case, the trivial fix for configure:

diff -r 777b171a63ae -r 1784ac25b52e configure
--- a/configure Tue Oct 26 18:50:46 2010 +0200
+++ b/configure Tue Oct 26 18:54:09 2010 +0200
@@ -9178,7 +9178,7 @@
 $as_echo $with_dtrace 6; }
 if test ! -z $with_dtrace
 then
-if dtrace -G -o /dev/null -s $srcdir/Include/pydtrace.d 2/dev/null
+if dtrace -G -o conftest.out -s $srcdir/Include/pydtrace.d 2/dev/null
 then
 
 $as_echo #define WITH_DTRACE 1 confdefs.h
diff -r 777b171a63ae -r 1784ac25b52e configure.in
--- a/configure.in  Tue Oct 26 18:50:46 2010 +0200
+++ b/configure.in  Tue Oct 26 18:54:09 2010 +0200
@@ -2466,7 +2466,7 @@
 AC_MSG_RESULT([$with_dtrace])
 if test ! -z $with_dtrace
 then
-if dtrace -G -o /dev/null -s $srcdir/Include/pydtrace.d 2/dev/null
+if dtrace -G -o conftest.out -s $srcdir/Include/pydtrace.d 2/dev/null
 then
AC_DEFINE(WITH_DTRACE, 1, 
 [Define if you want to compile in Dtrace support])

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4111
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4111] Add Systemtap/DTrace probes

[Antoine Pitrou]

Antoine Pitrou pit...@free.fr added the comment:

(my last message was about building on OpenSolaris)

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4111
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4111] Add Systemtap/DTrace probes

[Jesús Cea Avión]

Jesús Cea Avión j...@jcea.es added the comment:

I am trying to review this for 3.2, but I am having some issues. For instance, 
include/pydtrace.d is not present in the last patch.

Please, post a patch with all the required changes in the same (patch) file. 
Hurry, we are still on track for 3.2. :-).

Another question: I am not able to decide between Sun/Apple style, or breaking 
dtrace scripts compatibility completely. Anybody has an opinion about this?. Is 
this actually important?. Are there so many legacy dtrace scripts out there?.

--
assignee:  - dmalcolm

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4111
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4111] Add Systemtap/DTrace probes

[Alexander Belopolsky]

Alexander Belopolsky belopol...@users.sourceforge.net added the comment:

2010/10/25 Jesús Cea Avión rep...@bugs.python.org:
..
 Another question: I am not able to decide between Sun/Apple style, or 
 breaking dtrace scripts
 compatibility completely. Anybody has an opinion about this?. Is this 
 actually important?. Are
 there so many legacy dtrace scripts out there?.


I would say compatibility with Sun/Apple probes should not stand in
the way of implementing this in cpython.  Of course familiarity to
existing users is a consideration, but in cases where self-consistency
can be improved, I don't think we should be overly concerned about
legacy scripts.  I would estimate that 90% of future users will never
have used either Sun or Apple probes, 9% will have used Apple and 1%
Sun. (This is completely unscientific guess, of course.)

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4111
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4111] Add Systemtap/DTrace probes

[Jesús Cea Avión]

Jesús Cea Avión j...@jcea.es added the comment:

Malcolm, does your last patch address the performance issue?. Ideally, dtrace 
support should be compiled in by default, so performance issues are important.

Idealy, performance difference between compiling dtrace or not should be 
negligible. Until you actually use it, of course.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4111
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4111] Add Systemtap/DTrace probes

[Giampaolo Rodola']

Changes by Giampaolo Rodola' g.rod...@gmail.com:


--
nosy:  -giampaolo.rodola

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4111
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4111] Add Systemtap/DTrace probes

[Jesús Cea Avión]

Jesús Cea Avión j...@jcea.es added the comment:

We need some documentation, too.

Maybe a new chapter, or a new section in the debug chapter. Better the first, 
since this is not only for debugging, but for performance study too.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4111
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4111] Add Systemtap/DTrace probes

[Jesús Cea Avión]

Jesús Cea Avión j...@jcea.es added the comment:

Compiling the code WITHOUT dtrace support gives an error:


Undefined   first referenced
 symbol in file
__dtrace_python___function__entry   ./libpython3.2m.so
__dtraceenabled_python___function__return ./libpython3.2m.so
__dtrace_python___function__return  ./libpython3.2m.so
__dtraceenabled_python___function__entry ./libpython3.2m.so
ld: fatal: Symbol referencing errors. No output written to python
collect2: ld returned 1 exit status


--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4111
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4111] Add Systemtap/DTrace probes

[Jesús Cea Avión]

Jesús Cea Avión j...@jcea.es added the comment:

Compiling WITH dtrace... shows the same error :-

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4111
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4111] Add Systemtap/DTrace probes

[Jesús Cea Avión]

Jesús Cea Avión j...@jcea.es added the comment:

I am using Solaris 10, but the configure script detects apple implementation.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4111
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4111] Add Systemtap/DTrace probes

[Danek Duvall]

Changes by Danek Duvall duv...@comfychair.org:


--
nosy: +dhduvall

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4111
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4111] Add Systemtap/DTrace probes

[Frank Ch. Eigler]

Frank Ch. Eigler f...@elastic.org added the comment:

I believe the problem jcea is experiencing is with the
solaris (/linux?) branch of the configure.in:

if dtrace -G -o /dev/null -s $srcdir/Include/pydtrace.d 2/dev/null

It seems solaris doesn't like the -o /dev/null part.  Try
specifying some real temporary file name instead.

--
nosy: +fche

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4111
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4111] Add Systemtap/DTrace probes

[Dave Malcolm]

Dave Malcolm dmalc...@redhat.com added the comment:

Updated patch, removing the FIXMEs, and slightly reworking the test code.

I've wrapped the whole of get_frame_marker_info with a 
PyErr_Fetch/PyErr_Restore pair: the PyUnicode_AsUTF8String calls could fail 
with a MemoryError, and we don't want to confuse the regular exception handling 
within ceval.

I'm not sure how to write a unit test to test for this: perhaps we could 
corrupt the __code__ instance so that co_filename is not a PyUnicodeObject, 
leading to a TypeError within the function, but that's a readonly attribute.  
Any ideas?

I've also added a unit test for a non-ASCII script 
(Lib/test/systemtap_sample_☠.py), containing a non-ASCII identifier (文字化け).

The non-ASCII script name (Lib/test/systemtap_sample_☠.py) may be 
controversial: do we have anything like that in the source tree yet?  Is there 
any risk of messing up the build across platforms, or of impacting the Hg 
migration?

Still to-do:
  - address the Undefined symbol issues seen by jcea.
  - documentation
  - doublecheck performance

--
Added file: 
http://bugs.python.org/file19360/py3k-add-systemtap-2010-10-25-002.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4111
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4111] Add Systemtap/DTrace probes

[Antoine Pitrou]

Antoine Pitrou pit...@free.fr added the comment:

 I've wrapped the whole of get_frame_marker_info with a
 PyErr_Fetch/PyErr_Restore pair: the PyUnicode_AsUTF8String calls could
 fail with a MemoryError, and we don't want to confuse the regular
 exception handling within ceval.

If PyUnicode_AsUTF8String() is meant to encode a filename, you should
use PyUnicode_EncodeFSDefault() instead.

 The non-ASCII script name (Lib/test/systemtap_sample_☠.py) may be
 controversial: do we have anything like that in the source tree yet?
 Is there any risk of messing up the build across platforms, or of
 impacting the Hg migration?

It would be better to generate the sample dynamically, so that users
with an incompatible locale or filesystem aren't prevented from checking
out the source.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4111
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4111] Add Systemtap/DTrace probes

[Dave Malcolm]

Dave Malcolm dmalc...@redhat.com added the comment:

 It would be better to generate the sample dynamically, so that users
 with an incompatible locale or filesystem aren't prevented from checking
 out the source.
Thanks: am attaching updated patch: I've removed 
Lib/test/systemtap_sample_☠.py, and now generate a similarly-named file during 
the test, using test.support.TEST_FN and unlink

Still TODO:
  - address pitrou's concerns about PyUnicode_AsUTF8String from msg119580
  - address the Undefined symbol issues seen by jcea (msg119563 onwards)
  - documentation
  - doublecheck performance
  - perhaps add a systemtap tapset, and demo code using it (like I did in 
Fedora's python3 RPMs)
  - anything else I've missed :)

--
Added file: 
http://bugs.python.org/file19363/py3k-add-systemtap-2010-10-25-003.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4111
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4111] Add Systemtap/DTrace probes

[Dave Malcolm]

Dave Malcolm dmalc...@redhat.com added the comment:

I should note that I've only ever been testing this with SystemTap, on Linux.  
I don't have a box with DTrace, and I've never directly used it.  I wouldn't 
today be able to diagnose a buildbot failure related to DTrace (I believe I 
would with systemtap, fwiw).

Are there any DTrace experts around who would be willing to handle the DTrace 
side of this?  If not, would it be reasonable to make this issue be only 
explicitly about systemtap?  (e.g. perhaps change the configure argument 
accordingly?)  Alternatively, given that this bug originally started as an RFE 
about DTrace, should we split out systemtap as a separate RFE?

I'm sorry if I've muddied the waters by doing this.

For example, the only test coverage I've written (test_systemtap.py) checks for 
the presence of a stap executable, and skips the tests if it's not found.  
I'm not familiar enough with DTrace to create the same for DTrace.

FWIW (in response to IRC question): thread_indent is documented here:
http://sourceware.org/systemtap/SystemTap_Beginners_Guide/systemtapscript-handler.html#thread_indent

It looks like it may be systemtap-specific; however the only usage is within 
test_systemtap.py, guarded by the presence of a stap binary, skipping the 
tests if it is not found.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4111
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4111] Add Systemtap/DTrace probes

[Antoine Pitrou]

Antoine Pitrou pit...@free.fr added the comment:

 I've the same question as Jesús Cea Avión: what is needed to get this
 in 3.2?

Same as usual: someone to review, apply, commit. Why do you ask?

--
nosy: +pitrou

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4111
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4111] Add Systemtap/DTrace probes

[Ronald Oussoren]

Ronald Oussoren ronaldousso...@mac.com added the comment:

I've the same question as Jesús Cea Avión: what is needed to get this in 3.2?  
This would IMHO be a useful feature.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4111
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4111] Add Systemtap/DTrace probes

[Dave Malcolm]

Dave Malcolm dmalc...@redhat.com added the comment:

Updated version of patch attached (against py3k; r85426)

I've changed the names back to function__entry and function__return.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4111
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4111] Add Systemtap/DTrace probes

[Jesús Cea Avión]

Jesús Cea Avión j...@jcea.es added the comment:

What do we need to have this ready for Python 3.2?. The oportunity window is 
closing fast, and 3.3 is two years away...

So now or never...

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4111
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4111] Add Systemtap/DTrace probes

[Mark Wielaard]

Mark Wielaard m...@redhat.com added the comment:

  - renamed the probepoints:
  function__entry - frame__entry
  function__return - frame__exit
as I believe this better describes what these do

Are you sure you want to do this? You are right that it better describes the 
action that these probe points signify. But since previous versions of this 
patch have been used and applied people might be depending on the old names. It 
is not a very big issue, we can always write a tapset to translate the old to 
new names. But it is an interface break for anybody who might already have 
scripts around.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4111
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4111] Add Systemtap/DTrace probes

[anatoly techtonik]

anatoly techtonik techto...@gmail.com added the comment:

I don't understand the last request. Is there already a practice to depend on 
patches that were applied by people, but were not released, reviewed or 
documented in some PEP?

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4111
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4111] Add Systemtap/DTrace probes

[Mark Wielaard]

Mark Wielaard m...@redhat.com added the comment:

The original patch was created to be as close as possible to the support that 
Sun and Apple added to their python implementation for Solaris and MacOS. 
Changing the probe point names would make the current patch slightly different 
for scripts written against those. That isn't a major problem, and can be 
worked around in user scripts or a tapset that translates the old names to new 
names. But it is something to consider before changing the probe names.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4111
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4111] Add Systemtap/DTrace probes

[anatoly techtonik]

anatoly techtonik techto...@gmail.com added the comment:

So, it was released. But why they didn't send their patches here? We could 
review them, or we want Sun and Apple create Python specifications 
unilaterally? It would be best to contact them and notify about the change. Is 
it something a PSF should do?

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4111
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4111] Add Systemtap/DTrace probes

[John Levon]

John Levon movem...@users.sourceforge.net added the comment:

Anatoly, see above. We have done exactly that. (Honestly, when I first wrote 
the changes, I saw them as too hacky to ever be accepted upstream).

This name change seems frivolous and will indeed break existing scripts.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4111
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4111] Add Systemtap/DTrace probes

[Dave Malcolm]

Dave Malcolm dmalc...@redhat.com added the comment:

Updated py3k version of patch.

Changes:
  - renamed the probepoints:
  function__entry - frame__entry
  function__return - frame__exit
as I believe this better describes what these do
  - added a test suite: Lib/test/test_systemtap.py
  - reformatted to use 4-space indentation
  - slightly reworked the configure.in code

Some error-handling issue remain (see the FIXMEs in the code)

Tested and working on Fedora 13 x86_64, using:
  systemtap-1.3-1.fc13.x86_64
  kernel-2.6.33.3-84.fc13.x86_64
  kernel-devel-2.6.33.3-84.fc13.x86_64
  kernel-headers-2.6.33.3-84.fc13.x86_64
  kernel-debuginfo-2.6.33.3-84.fc13.x86_64
  kernel-debuginfo-common-x86_64-2.6.33.3-84.fc13.x86_64

--
Added file: http://bugs.python.org/file19037/py3k-add-systemtap-2010-09-27.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4111
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4111] Add Systemtap/DTrace probes

[anatoly techtonik]

Changes by anatoly techtonik techto...@gmail.com:


--
nosy: +techtonik

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4111
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4111] Add Systemtap/DTrace probes

[Giampaolo Rodola']

Changes by Giampaolo Rodola' g.rod...@gmail.com:


--
nosy: +giampaolo.rodola

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4111
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4111] Add Systemtap/DTrace probes

[Ronald Oussoren]

Ronald Oussoren ronaldousso...@mac.com added the comment:

On 19 Jul, 2010, at 1:07, Alexander Belopolsky wrote:

 
 Alexander Belopolsky belopol...@users.sourceforge.net added the comment:
 
 If any RedHat/Fedora people are tuned in, can you give us an update on 
 Systemtap/DTrace support in RedHat/Fedora python?  I understand that one of 
 the main problems with bringing this upstream was the lack of uniformity 
 between Apple and Sun approaches.  These days Apple and Sun(Oracle) are 
 increasingly unfriendly to open source, so with a third solution available 
 from RedHat, the choice seems obvious.

What do you mean by increasing unfriendly in Apple's case? I haven't noted any 
change from them in the last few years. Not that their (perceived) 
unfriendlyness is relevant.

I'm +1 on merging this feature having dtrace support is pretty convenient.

Ronald

--
Added file: http://bugs.python.org/file18058/smime.p7s

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4111
___

smime.p7s
Description: S/MIME cryptographic signature
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4111] Add Systemtap/DTrace probes

[Mark Lawrence]

Mark Lawrence breamore...@yahoo.co.uk added the comment:

Comments suggest this is wanted, can our build experts get this into 3.2?

--
nosy: +BreamoreBoy

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4111
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4111] Add Systemtap/DTrace probes

[Jesús Cea Avión]

Jesús Cea Avión j...@jcea.es added the comment:

+1!

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4111
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4111] Add Systemtap/DTrace probes

[Alexander Belopolsky]

Alexander Belopolsky belopol...@users.sourceforge.net added the comment:

If any RedHat/Fedora people are tuned in, can you give us an update on 
Systemtap/DTrace support in RedHat/Fedora python?  I understand that one of the 
main problems with bringing this upstream was the lack of uniformity between 
Apple and Sun approaches.  These days Apple and Sun(Oracle) are increasingly 
unfriendly to open source, so with a third solution available from RedHat, the 
choice seems obvious.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4111
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4111] Add Systemtap/DTrace probes

[Skip Montanaro]

Changes by Skip Montanaro s...@pobox.com:


--
assignee: skip.montanaro - 
nosy:  -skip.montanaro

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4111
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4111] Add Systemtap/DTrace probes

[Jesús Cea Avión]

Jesús Cea Avión j...@jcea.es added the comment:

Ping...

We already missed the 2.7 boat. Any hope for 3.2?

--
assignee:  - prescod
keywords: +needs review -patch
nosy: +prescod, skip.montanaro -Garen
priority: normal - 
versions: +Python 3.2 -Python 2.7, Python 3.1

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4111
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4111] Add Systemtap/DTrace probes

[R. David Murray]

Changes by R. David Murray rdmur...@bitdance.com:


--
assignee: prescod - 
keywords: +patch
nosy: +Garen -prescod, skip.montanaro
priority:  - normal
stage:  - patch review

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4111
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4111] Add Systemtap/DTrace probes

[Garen]

Changes by Garen garen.par...@gmail.com:


--
nosy: +Garen

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4111
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4111] Add Systemtap/DTrace probes

[Dave Malcolm]

Dave Malcolm dmalc...@redhat.com added the comment:

At Red Hat we've done some work on this feature.  I'm sorry for not updating 
this issue, I was swamped with tasks both pre-PyCon and during the event; I did 
show the following to various folks at PyCon; I attempted to find Skip at PyCon 
but was unsuccessful)

Unfortunately I have two further versions of this patch for you :(

High level summary
--
SystemTap is a similar tool for DTrace, but for Linux.

At Red Hat we've been looking at implementing this using SystemTap.  SystemTap 
makes some attempts to be at least somewhat compatible with DTrace, and we are 
able to get the DTrace support to build against SystemTap.

I measured the performance and found that some versions of the patch impose a 
significant performance penalty.

We have a version of the patch that works with SystemTap on Linux, and appears 
not to impose a performance penalty for the cases I think are important (see 
notes below)

I have also ported the patch to Python 3 (I pass the strings back to the probe 
support as UTF-8 encoded)

I hope that there's enough commonality between the SystemTap and DTrace patches 
to permit common parts to be committed to Python, perhaps eventually supporting 
both.  Having said that, I don't have any systems that can run DTrace.

I've chosen to build the upcoming Fedora 13's Python 2 and Python 3 rpms with 
this support enabled (I'm a Python guy, not a SystemTap guy, FWIW, but we have 
a strong in-house SystemTap team).  We've been tracking this downstream for 
Fedora, our detailed discussions are within this bug in our downstream tracker:
https://bugzilla.redhat.com/show_bug.cgi?id=545179
along with status:
  https://fedoraproject.org/wiki/Features/SystemtapStaticProbes#Python
and documentation:
  https://fedoraproject.org/wiki/Features/SystemtapStaticProbes#Python_2


Details
---
SystemTap supplies a dtrace executable implemented in Python, which mimics 
the dtrace binary.
$ which dtrace
/usr/bin/dtrace
$ file $(which dtrace)
/usr/bin/dtrace: python script text executable
$ dtrace --help
Usage /usr/bin/dtrace [--help] [-h | -G] [-C [-IPath]] -s File.d [-o File]
Where -h builds a systemtap header file from the .d file
  -C when used with -h, also run cpp preprocessor
  -o specifies an explicit output file name,
 the default for -G is file.o and -h is file.h
  -I when running cpp pass through this -I include Path
  -s specifies the name of the .d input file
  -G builds a stub file.o from file.d,
 which is required by some packages that use dtrace.

I'm attaching a version of the patch against svn trunk.

This adds a --with-dtrace configuration option (I filed issue 7997 to better 
document how to do this).

Tested on a Fedora 12 box and works, with:
systemtap-1.1-2.fc12.i686
systemtap-sdt-devel-1.1-1.fc12.i686
systemtap-runtime-1.1-2.fc12.i686
kernel-PAE-2.6.31.12-174.2.3.fc12.i686
kernel-PAE-devel-2.6.31.12-174.2.3.fc12.i686
kernel-PAE-debuginfo-2.6.31.12-174.2.3.fc12.i686
kernel-headers-2.6.31.12-174.2.3.fc12.i686
kernel-debuginfo-common-i686-2.6.31.12-174.2.3.fc12.i686
kernel-firmware-2.6.31.12-174.2.3.fc12.noarch

Upon building --with-dtrace (but actually with systemtap), the linked binary 
containing ceval.o should contain a probes section (the following is a 
statically-linked build, hence the probes are in ./python):

$ eu-readelf -S ./python | grep probes
[25] .probes  PROGBITS 081f9d28 1b0d28 54  0 WA 0   0  8

$ eu-readelf -x .probes python

Hex dump of section [25] '.probes', 84 bytes at offset 0x1b0d28:
  0x 66756e63 74696f6e 5f5f656e 74727900 function__entry.
  0x0010 50524231  289d1f08  PRB1(...
  0x0020 a50e0f08  66756e63 74696f6e function
  0x0030 5f5f7265 7475726e  50524231 __returnPRB1
  0x0040 509d1f08  1a0f0f08  P...
  0x0050 

The above contains locations of nop instructions within the binary.  
Systemtap generates kernel modules, and it uses this information to generate 
modules that monkeypatch the nop instruction at runtime to jump to code that 
gathers the pertinent information (filename, lineno etc) and returns that 
information to the kernel, and thus back to the user-space probe process.

Example of use
--
Here's an example of running regrtest whilst probing python to see function 
call and return during a run of the regression test suite (all in one shell 
command; obviously it would be saner to split the systemtap probe into a 
separate function):

[da...@brick trunk-stap]$ stap -v -e'probe python.function.entry = 
process(./python).mark(function__entry)
{
filename = user_string($arg1);
funcname = user_string($arg2);
lineno = $arg3;
}
probe python.function.return = process(./python).mark(function__return)
{
filename = user_string($arg1);
funcname = 

[issue4111] Add Systemtap/DTrace probes

[Dave Malcolm]

Dave Malcolm dmalc...@redhat.com added the comment:

Attached is the patch I'm currently applying in Fedora 13 to Python 3.

The patch is actually against the 3.1.1 tarball, rather than SVN; sorry 
(swamped with post-pycon tasks here), as that's what I've been testing this 
work against.

The patch is somewhat different to the Python 2 case, in that the filename and 
module name are stored internally by Python as unicode.  The markers convert 
these to UTF8 for the case when the markers are in use by an active probe.  
Unfortunately it's now possible for this to fail (e.g. low memory).

An example of using this for a non-Latin 1 script can be seen towards the end 
of:
https://fedoraproject.org/wiki/Features/SystemtapStaticProbes#Python_2

--
Added file: http://bugs.python.org/file16383/python-3.1.1-systemtap.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4111
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4111] Add Systemtap/DTrace probes

[Skip Montanaro]

Skip Montanaro s...@pobox.com added the comment:

Jesus can I ask if this (very useful) feature is on time for Python 2.7?

You can ask, but I suspect you'd be disappointed in the answer.  Do you have
time to look at the issue?  The biggest sticking point in my mind is coming
up with a uniform set of probes which would be supported by both Sun and
Apple.

Skip

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4111
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4111] Add Systemtap/DTrace probes

[Wilfredo Sanchez]

Changes by Wilfredo Sanchez wsanc...@wsanchez.net:


--
nosy: +wsanchez

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4111
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4111] Add Systemtap/DTrace probes

[Glyph Lefkowitz]

Changes by Glyph Lefkowitz gl...@divmod.com:


--
nosy: +glyph

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4111
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4111] Add Systemtap/DTrace probes

[Mark Wielaard]

Mark Wielaard m...@redhat.com added the comment:

I took the patch and tweaked it a little so that it works with systemtap
(at least for the function entry/exit probes). It is against 2.6.2 since
that was what was in the Fedora 12 src package I was using. Cleaned up
the configure checks a little, but otherwise it is almost exactly as is.

Also tracked in Fedora as 
https://bugzilla.redhat.com/show_bug.cgi?id=545179

--
components:  -Installation
title: Add DTrace probes - Add Systemtap/DTrace probes
Added file: http://bugs.python.org/file15503/stap-python.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4111
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com