[issue2281] Enhanced cPython profiler with high-resolution timer

[Inada Naoki]

Inada Naoki  added the comment:

I think https://bugs.python.org/issue36575 fixed this.

--
nosy: +inada.naoki
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2281] Enhanced cPython profiler with high-resolution timer

[Alexandre Vassalotti]

Changes by Alexandre Vassalotti alexan...@peadrop.com:


--
priority: normal - low
versions: +Python 3.5 -Python 3.2

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue2281
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2281] Enhanced cPython profiler with high-resolution timer

[Alexandre Vassalotti]

Changes by Alexandre Vassalotti alexan...@peadrop.com:


--
priority: normal - low
versions: +Python 3.5 -Python 3.2

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue2281
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2281] Enhanced cPython profiler with high-resolution timer

[Alexandre Vassalotti]

Changes by Alexandre Vassalotti alexan...@peadrop.com:


--
priority: normal - low
versions: +Python 3.5 -Python 3.2

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue2281
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2281] Enhanced cPython profiler with high-resolution timer

[Christian Heimes]

Changes by Christian Heimes li...@cheimes.de:


--
versions: +Python 3.4 -Python 3.2

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue2281
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2281] Enhanced cPython profiler with high-resolution timer

[Christian Heimes]

Changes by Christian Heimes li...@cheimes.de:


--
versions: +Python 3.4 -Python 3.2

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue2281
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2281] Enhanced cPython profiler with high-resolution timer

[Christian Heimes]

Changes by Christian Heimes li...@cheimes.de:


--
versions: +Python 3.4 -Python 3.2

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue2281
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2281] Enhanced cPython profiler with high-resolution timer

[Christian Heimes]

Changes by Christian Heimes li...@cheimes.de:


--
versions: +Python 3.4 -Python 3.2

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue2281
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2281] Enhanced cPython profiler with high-resolution timer

[Kristján Valur Jónsson]

Kristján Valur Jónsson added the comment:

There is another major issue: you have to synchronize the counter of all CPU 
cores. Windows XP didn't synchronize the counters, and it was a major issue 
on QueryPerformanceCounter.
http://support.microsoft.com/?id=896256

I don't understand why you keep bashing Windows for this.  It was actually a 
problem with CPU drivers.  We encountered this problem for a number of our 
players and the fix was to get the latest intel/amd drivers.  The issue you 
quote is a separate one.
This appears to be the correct one:  http://support.microsoft.com/kb/895980

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue2281
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2281] Enhanced cPython profiler with high-resolution timer

[STINNER Victor]

STINNER Victor added the comment:

Python 3.3 has a new time.perf_counter() function. It may be possible to expose 
this function in C and then use it in _lsprof. See the PEP for the list of 
(OS/hardware) clocks used by this function:

http://www.python.org/dev/peps/pep-0418/#time-perf-counter
http://www.python.org/dev/peps/pep-0418/#time-monotonic

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue2281
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2281] Enhanced cPython profiler with high-resolution timer

[STINNER Victor]

STINNER Victor added the comment:

Reading directly the CPU timestamp counter is not a good idea.

 One issue with using timestamp counters is that their resolution varies
 with CPU speed, which is not necessarily constant during a whole run
 (because of power management). On the other hand I'm not sure it's
 really a problem.

There is another major issue: you have to synchronize the counter of all CPU 
cores. Windows XP didn't synchronize the counters, and it was a major issue on 
QueryPerformanceCounter.
http://support.microsoft.com/?id=896256

It is really better to rely on the OS rather than querying directly the 
hardward. The OS (well, most OS except Windows) handles all these minor nits 
and provide the best possible resolution.

--

 Python 3.3 has a new time.perf_counter() function.
 It may be possible to expose this function in C ...

Oh, I now remember why I didn't do this before: when clock_gettime() function 
is used in perf_counter(), the program or library must be linked to librt on 
most Linux distro. It was easier to only handle this correctly for the time 
module. We can do something similar on the _lsprof module.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue2281
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2281] Enhanced cPython profiler with high-resolution timer

[Antoine Pitrou]

Changes by Antoine Pitrou pit...@free.fr:


--
nosy: +krisvale

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue2281
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2281] Enhanced cPython profiler with high-resolution timer

[Antoine Pitrou]

Changes by Antoine Pitrou pit...@free.fr:


--
assignee: pitrou - 
nosy: +belopolsky

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue2281
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2281] Enhanced cPython profiler with high-resolution timer

[Alexander Belopolsky]

Alexander Belopolsky belopol...@users.sourceforge.net added the comment:

Can someone post a diff against current py3k?   I would like to take a look, 
but the files attached to this issue seem to be more than a year old.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue2281
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2281] Enhanced cPython profiler with high-resolution timer

[Gregory P. Smith]

Changes by Gregory P. Smith g...@krypto.org:


--
assignee: gregory.p.smith - pitrou

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue2281
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2281] Enhanced cPython profiler with high-resolution timer

[Antoine Pitrou]

Antoine Pitrou pit...@free.fr added the comment:

Nir, would you be interested in looking at this?

--
nosy: +nirai
versions:  -Python 2.7

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue2281
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2281] Enhanced cPython profiler with high-resolution timer

[Antoine Pitrou]

Antoine Pitrou pit...@free.fr added the comment:

Regarding the calibrating loop when using ASM timestamp counters, there's one 
problem if the thread gets preempted in the middle of the loop: the calibration 
will return incorrect results. Perhaps a way of avoiding this would be to call 
nanosleep(1 ns) just before the loop, which should relinquish the current 
timeslice.

Another issue can appear when threads migrate from one CPU core to another, 
according to Wikipedia: “AMD processors up to the K8 core always incremented 
the time-stamp counter every clock cycle.[5] Thus, power management features 
were able to change the number of increments per second, and the values could 
get out of sync between different cores or processors in the same system.”

Perhaps using clock_gettime(CLOCK_REALTIME or CLOCK_MONOTONIC) is better than 
using raw ASM timestamp counters, when available.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue2281
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2281] Enhanced cPython profiler with high-resolution timer

[STINNER Victor]

Changes by STINNER Victor victor.stin...@haypocalc.com:


--
nosy: +haypo

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue2281
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2281] Enhanced cPython profiler with high-resolution timer

[Antoine Pitrou]

Antoine Pitrou pit...@free.fr added the comment:

Sorry, the calibration loop actually looks correct in case of preemption, since 
it simply divides the TSC delta by the elapsed physical time (and both continue 
increasing monotonously when another thread runs). It is still vulnerable to 
the thread migration problem, though.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue2281
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2281] Enhanced cPython profiler with high-resolution timer

[Nir Aides]

Nir Aides n...@winpdb.org added the comment:

 Nir, would you be interested in looking at this?

yes, I'll take a look, but will take me a few days.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue2281
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2281] Enhanced cPython profiler with high-resolution timer

[Antoine Pitrou]

Antoine Pitrou pit...@free.fr added the comment:

One issue with using timestamp counters is that their resolution varies
with CPU speed, which is not necessarily constant during a whole run
(because of power management). On the other hand I'm not sure it's
really a problem.

--
versions: +Python 3.2 -Python 3.1

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue2281
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2281] Enhanced cPython profiler with high-resolution timer

[Jean Brouwers]

Jean Brouwers mrje...@gmail.com added the comment:

Another thought on the hires timer to make the hires time and hires time 
units available as 2 other functions in the time module.

For example, function time.ticks() returns the hires time stamp as an 
int.  Function time.ticks2secs(t) converts a given number of ticks to 
seconds.

To avoid duplicating the hires time code in both the time and profile 
modules, it would be necessary to move the hpTimer and hpTimerUnit 
functions to some place inside the Python core accessible for the time 
and profile modules.  Perhaps to a new file, say Python/gethptime.c.

That new file can handle other platform-specific idiosyncrasies with 
respect to hires time.  In particular, it could implement a different 
(and better) way to determine the resolution of a hires tick, e.g. on 
Linux and BSD Unix.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue2281
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2281] Enhanced cPython profiler with high-resolution timer

[Jean Brouwers]

Jean Brouwers mrje...@gmail.com added the comment:

Here a small, standalone test program of the hires timer code from 
_lsprof4.c.

The results from 10 different builds are included in the comment.

--
Added file: http://bugs.python.org/file14006/hpTime.c

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue2281
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2281] Enhanced cPython profiler with high-resolution timer

[Jean Brouwers]

Jean Brouwers mrje...@gmail.com added the comment:

Both the hires_lsprof4.tgz and hpTime.c files have been updated with a 
change in the PowerPC asm code.

The previous implementation used a loop: label and that may cause 
duplicate symbols when compiled at optimization levels of -O3 and above 
with older GNU C compilers.

--
Added file: http://bugs.python.org/file14011/hires_lsprof4.tgz

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue2281
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2281] Enhanced cPython profiler with high-resolution timer

[Jean Brouwers]

Jean Brouwers mrje...@gmail.com added the comment:

Another test build and result has been added to the hpTime.c comment.

--
Added file: http://bugs.python.org/file14012/hpTime.c

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue2281
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2281] Enhanced cPython profiler with high-resolution timer

[Jean Brouwers]

Changes by Jean Brouwers mrje...@gmail.com:


Removed file: http://bugs.python.org/file14004/hires_lsprof4.tgz

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue2281
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2281] Enhanced cPython profiler with high-resolution timer

[Jean Brouwers]

Changes by Jean Brouwers mrje...@gmail.com:


Removed file: http://bugs.python.org/file14006/hpTime.c

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue2281
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2281] Enhanced cPython profiler with high-resolution timer

[Jean Brouwers]

Jean Brouwers mrje...@gmail.com added the comment:

Attached is yet another version of the Modules/_lsprof.c source, called 
_lsprof4.c.

This one adds support for the hires timer on 32- and 64-bit Intel and 
PowerPC processors iff compiled with GNU C.

With _lsprof4.c, the high precision profile is available on Linux, Mac 
OS X, Solaris, HP-UX and MS Windows.

Included is the forward diff with _lsprof.c rev 66700 from the trunk.  
The changes are only at the very top and in the initialization function 
at the bottom.

--
Added file: http://bugs.python.org/file14001/hires_lsprof4.tgz

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue2281
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2281] Enhanced cPython profiler with high-resolution timer

[Jean Brouwers]

Changes by Jean Brouwers mrje...@gmail.com:


Removed file: http://bugs.python.org/file14001/hires_lsprof4.tgz

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue2281
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2281] Enhanced cPython profiler with high-resolution timer

[Jean Brouwers]

Jean Brouwers mrje...@gmail.com added the comment:

There were two problems with _lsprof4.c.  Attached is the corrected file 
and the forward diff with _lsprof.c rev 6670.

--
Added file: http://bugs.python.org/file14004/hires_lsprof4.tgz

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue2281
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2281] Enhanced cPython profiler with high-resolution timer

[Antoine Pitrou]

Antoine Pitrou pit...@free.fr added the comment:

Interesting patch, but there are lots of spurious #if's and #ifdef's
which could be simplified. Also, some changes look slightly unrelated
(e.g. the switch from malloc/free to PyObject_Malloc/PyObject_Free).

--
nosy: +pitrou
stage:  - patch review
versions: +Python 3.1

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue2281
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2281] Enhanced cPython profiler with high-resolution timer

[Gregory P. Smith]

Changes by Gregory P. Smith g...@krypto.org:


--
assignee:  - gregory.p.smith
nosy: +gregory.p.smith

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue2281
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2281] Enhanced cPython profiler with high-resolution timer

[Jean Brouwers]

Changes by Jean Brouwers [EMAIL PROTECTED]:


--
versions: +Python 2.5.3 -Python 3.0

___
Python tracker [EMAIL PROTECTED]
http://bugs.python.org/issue2281
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2281] Enhanced cPython profiler with high-resolution timer

[Jean Brouwers]

Changes by Jean Brouwers [EMAIL PROTECTED]:


--
versions: +Python 2.7 -Python 2.5.3

___
Python tracker [EMAIL PROTECTED]
http://bugs.python.org/issue2281
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2281] Enhanced cPython profiler with high-resolution timer

[Jean Brouwers]

Jean Brouwers [EMAIL PROTECTED] added the comment:

Here are 2 forward diff files against _lspprof.c rev 59564.  One 
_lsprof2.6.diff for Python 2.6a1 and _lsprof3.0.diff for 3.0.

Added file: http://bugs.python.org/file9812/_lsprof2.6.diff

__
Tracker [EMAIL PROTECTED]
http://bugs.python.org/issue2281
__
___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2281] Enhanced cPython profiler with high-resolution timer

[Jean Brouwers]

Changes by Jean Brouwers [EMAIL PROTECTED]:


--
versions: +Python 3.0 -Python 2.5
Added file: http://bugs.python.org/file9813/_lsprof3.0.diff

__
Tracker [EMAIL PROTECTED]
http://bugs.python.org/issue2281
__
___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2281] Enhanced cPython profiler with high-resolution timer

[Alexandre Vassalotti]

Alexandre Vassalotti [EMAIL PROTECTED] added the comment:

I don't think this should be added to 2.5. Only bugfixes are admissible
to the backporting process (see PEP 6).

Finally, could you post the diff of your changes as described at
http://www.python.org/dev/patches/. Thanks!

--
components: +Extension Modules -None
keywords: +patch
nosy: +alexandre.vassalotti
priority:  - normal
type:  - performance

__
Tracker [EMAIL PROTECTED]
http://bugs.python.org/issue2281
__
___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2281] Enhanced cPython profiler with high-resolution timer

[Jean Brouwers]

Jean Brouwers [EMAIL PROTECTED] added the comment:

Attached are 2 Modules/_lsprof.c files, one for Python 2.5.2 and 2.6a1 
and the other for Python 3.0a3.  Discard the previously posted one.

Both contain the same enhancements to use the high-resolution timer 
where available.

These versions catch wrap around of the timer and clock and adjust 
accordingly.  In the hpTimerUnit function only and not for profile 
times.

Lastly, instead of malloc and free functions PyObject_MALLOC and 
PyObject_FREE are called making profiler memory usage the same as for 
other objects created.

Added file: http://bugs.python.org/file9668/hires_lsprof2.tgz

__
Tracker [EMAIL PROTECTED]
http://bugs.python.org/issue2281
__
___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2281] Enhanced cPython profiler with high-resolution timer

[Jean Brouwers]

New submission from Jean Brouwers [EMAIL PROTECTED]:

Attached is a modified version of the cPython profiler file 
Modules/_lsprof.c using a high-resolution timer where available.

The enhancement has been tested on 32- and 64-bit Linux (x86 and x86_64) 
and on 32-bit MacOS X Tiger (Intel) and Panther (PPC).

No other platforms have been tested but as before the profiler will 
fallback to using gettimeofday() on non-Windows version, except the 64-
bit PPC build will issue a compile-time warning.

--
files: hires_lsprof.tgz
messages: 63486
nosy: MrJean1
severity: normal
status: open
title: Enhanced cPython profiler with high-resolution timer
Added file: http://bugs.python.org/file9665/hires_lsprof.tgz

__
Tracker [EMAIL PROTECTED]
http://bugs.python.org/issue2281
__
___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2281] Enhanced cPython profiler with high-resolution timer

[Jean Brouwers]

Jean Brouwers [EMAIL PROTECTED] added the comment:

This enhancement applies to Python 2.5.2 only.

--
components: +None
versions: +Python 2.5

__
Tracker [EMAIL PROTECTED]
http://bugs.python.org/issue2281
__
___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com