Is there a Python profiler that preserves function arguments?
I'm profiling a Python function `foo()` that takes a single argument,
but that argument makes a huge difference in what the function
actually does.
Currently I'm using `cProfile`, which records every call to `foo()` as
if it was the same, preventing me from figuring out what's going on.
Is there a way to get `cProfile`, or any other Python profiler, to
preserve function call arguments, so when I look at the call stack
later (especially using a visualizer like SnakeViz) I can distinguish
between `foo('bar')` and `foo('qux')`?
P.S. arguably this is a code design issue: since `foo('bar')` and
`foo('qux')` do very different things, they should be distinctly-named
separate functions, like `foo_bar()` and `foo_qux()`. However, the
question is whether I can profile them as-is, without refactoring.
--
https://mail.python.org/mailman/listinfo/python-list
[issue744841] Python-Profiler bug: Bad call
Terje Wiesener added the comment:
This bug seems to have resurfaced in newer python versions.
I have tested the file attached in the original report (prof2.py) in python
2.6.6 and 2.7 (x86 versions) under Windows 7, and both give the same output:
c:\tempc:\Python27\python.exe prof2.py
type 'exceptions.ZeroDivisionError'
Exception AssertionError: AssertionError('Bad call', ('prof2.py', 19, 'h'),
frame object at 0x023880E0, frame object at 0x00586E18, frame object at
0x02388518, frame object at 0x02388248) in
bound method C.__del__ of __main__.C instance at 0x02342A80 ignored
5 function calls in 0.007 CPU seconds
Ordered by: standard name
ncalls tottime percall cumtime percall filename:lineno(function)
10.0000.0000.0070.007 string:1(module)
10.0010.0010.0010.001 prof2.py:11(g)
10.0060.0060.0070.007 prof2.py:19(h)
10.0000.0000.0000.000 prof2.py:7(f)
10.0000.0000.0070.007 profile:0(h())
00.000 0.000 profile:0(profiler)
--
nosy: +Terje.Wiesener
type: - performance
versions: +Python 2.7
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue744841
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
RE: Python profiler usage with objects
-Original Message- From: python-list-bounces+bsk16=case@python.org [mailto:python-list- bounces+bsk16=case@python.org] On Behalf Of rik Sent: Tuesday, June 29, 2010 10:52 PM To: python-list@python.org Subject: Re: Python profiler usage with objects harit harit.himanshu at gmail.com writes: Hi, I have a specific question regarding the usage of profiler. I am new to python programming I am trying to profile a function which I want to invoke as a class method, something like this import profile class Class: def doSomething(): do here .. def callMethod(): **self.doSomething()** instead of this I want to use **profile.run(self.doSomething())** but the profile.run expects the string inside it and I get error TypeError: exec: arg 1 must be a string, file, or code object Can somebody please help? Thank you Harit, i am OLD to python, and have used its profiler in the past. but i'm getting your same error: TypeError: exec: arg 1 must be a string, file, or code object on both Ubuntu with Python 2.6 and OSX with 2.4. with both cProfile and profile?! whether or not i specify a file for profile output!?! anybody else having trouble profiling? - rik Let's take this code as an example: def foo() : return None import profile profile.run(foo()) What does the profile.run call do? First thin it does is evaluate foo(), which returns None. So you're calling profile.run(None) There's nothing special about profile.run- you have to hand it something to execute, not something already executed. Try calling Profile.run(doSomething) # no parenthesis for doSomething. -- http://mail.python.org/mailman/listinfo/python-list -- http://mail.python.org/mailman/listinfo/python-list
Re: Python profiler usage with objects
Ben Kaplan bsk16 at case.edu writes: Let's take this code as an example: def foo() : return None import profile profile.run(foo()) What does the profile.run call do? First thin it does is evaluate foo(), which returns None. So you're calling profile.run(None) There's nothing special about profile.run- you have to hand it something to execute, not something already executed. Try calling Profile.run(doSomething) # no parenthesis for doSomething. -- http://mail.python.org/mailman/listinfo/python-list hi Ben, right: i have a top-level function main() that runs just fine. but i give it as an argument to cProfile.run() or profile.run(), it executes as expected, but then: File /var/folders/lu/luGJNSGwE0mO84R+YbcKpU+++TI/-Tmp-/python-439Ffi.py, line 544, in ? profile.run(main(seedFile),'/Data/tmp/fetchProfile_100629.profile') File /Library/Frameworks/Python.framework/Versions/2.4//lib/python2.4/profile.py, line 72, in run prof = prof.run(statement) File /Library/Frameworks/Python.framework/Versions/2.4//lib/python2.4/profile.py, line 448, in run return self.runctx(cmd, dict, dict) File /Library/Frameworks/Python.framework/Versions/2.4//lib/python2.4/profile.py, line 454, in runctx exec cmd in globals, locals TypeError: exec: arg 1 must be a string, file, or code object this example is from python2.4 on OSX, but the same code generates the same error on python2.6 on Ubuntu?! -- http://mail.python.org/mailman/listinfo/python-list
Re: Python profiler usage with objects
Ben Kaplan bsk16 at case.edu writes: First thin it does is evaluate foo(), which returns None. So you're calling profile.run(None) There's nothing special about profile.run- you have to hand it something to execute, not something already executed. Try calling Profile.run(doSomething) # no parenthesis for doSomething. i'm reading your message more carefully, and so tried calling profile.run(main), without any parens (and absorbing the argument into the function's body for this test). same error!? -- http://mail.python.org/mailman/listinfo/python-list
Re: Python profiler usage with objects
On 2010-06-30 06:39, rik wrote:
Ben Kaplan bsk16 at case.edu writes:
Let's take this code as an example:
def foo() :
return None
import profile
profile.run(foo())
What does the profile.run call do?
First thin it does is evaluate foo(), which returns None. So you're calling
profile.run(None)
There's nothing special about profile.run- you have to hand it something to
execute, not something already executed. Try calling
Profile.run(doSomething) # no parenthesis for doSomething.
--
http://mail.python.org/mailman/listinfo/python-list
hi Ben, right: i have a top-level function main() that runs just
fine. but i give it as an argument to cProfile.run() or profile.run(),
it executes as expected, but then:
File /var/folders/lu/luGJNSGwE0mO84R+YbcKpU+++TI/-Tmp-/python-439Ffi.py,
line 544, in ?
profile.run(main(seedFile),'/Data/tmp/fetchProfile_100629.profile')
Looks like the same problem. As Ben pointed out, you need to pass profile.run()
an executable
(e.g. the main() function itself) and not something that's already been
executed. Since you've already called main(seedFile), it's the return value
that is being passed to profile.run(), and that's why it's complaining...
Perhaps you meant:
profile.run('main(seedFile)', '/Data/tmp/whatever')
?
--
http://mail.python.org/mailman/listinfo/python-list
--
http://mail.python.org/mailman/listinfo/python-list
Re: Python profiler usage with objects
Ben Kaplan bsk16 at case.edu writes:
There's nothing special about profile.run- you have to hand it something to
execute, not something already executed. Try calling
Profile.run(doSomething) # no parenthesis for doSomething.
your hint and REREADING THE DOCUMENTATION made me realize it
was the QUOTE MARKS on the function call i was missing:
profile.run('main(seedFile)','profileOutFile')
Harit, perhaps this was your problem, too?
Ben, thanks for your help.
--
http://mail.python.org/mailman/listinfo/python-list
Python profiler usage with objects
Hi, I have a specific question regarding the usage of profiler. I am new to python programming I am trying to profile a function which I want to invoke as a class method, something like this import profile class Class: def doSomething(): do here .. def callMethod(): **self.doSomething()** instead of this I want to use **profile.run(self.doSomething())** but the profile.run expects the string inside it and I get error TypeError: exec: arg 1 must be a string, file, or code object Can somebody please help? Thank you -- http://mail.python.org/mailman/listinfo/python-list
Re: Python profiler usage with objects
harit harit.himanshu at gmail.com writes: Hi, I have a specific question regarding the usage of profiler. I am new to python programming I am trying to profile a function which I want to invoke as a class method, something like this import profile class Class: def doSomething(): do here .. def callMethod(): **self.doSomething()** instead of this I want to use **profile.run(self.doSomething())** but the profile.run expects the string inside it and I get error TypeError: exec: arg 1 must be a string, file, or code object Can somebody please help? Thank you Harit, i am OLD to python, and have used its profiler in the past. but i'm getting your same error: TypeError: exec: arg 1 must be a string, file, or code object on both Ubuntu with Python 2.6 and OSX with 2.4. with both cProfile and profile?! whether or not i specify a file for profile output!?! anybody else having trouble profiling? - rik -- http://mail.python.org/mailman/listinfo/python-list
ANN: Yappi(Yet Another Python Profiler) 0.51 released
Hi all, Yappi (Yet Another Python Profiler) 0.51 released. See the version highlights: * OPTIMIZATION:Use per-pit cpc for better accuracy in different timing_sample values. Now timing_sample is not linearly decreasing the timing accuracy for most of the applications tested. We reduced the runtime of the profiler from 3.2 secs to 0.7 by giving away ~%2 timing accuracy. This means yappi can now run 4x-7x times faster than the standart cProfile by increasing timing_sample values with very little accuracy defect. * Python 2.6.5 gives import yappi Dll load failed error. This is due to VC2008 compile. Unfortunately, we need to use VC Express Mingw is not working because of MSVCR dll files on 2.6.5 on some machines. * Use snprintf for stat handling code. There may be some buffer overflow situations. Use PyOS_snprintf for platform differences. (Thanks to Dave Peticolas) * Format stat STRINGs from left, instead of right. (Thanks to Dave Peticolas) * OPTIMIZATION:Move ctx calculation code to callback function. Optimize call_XXX functions as much as possible. * Move timing calculation code from header to the C file. * MACRO defining code in setup.py is broken for some compilers, use DEFINE_MACRO keyword. * OPTIMIZATION:Disable HASH table linked list swapping code, it is not adapting to the profiler nature very well. Rollback to simple hash- table-bucketing. See below for more and download: http://code.google.com/p/yappi/ Thanks, twitter.com/sumercip -- http://mail.python.org/mailman/listinfo/python-announce-list Support the Python Software Foundation: http://www.python.org/psf/donations/
ANN: yappi 0.1 beta : Yet Another Python Profiler
Hi all, After implementing a game server on which 100k people playing games per-day, it turns out to be that continuous and efficient profiling is key to improve an long-running applications like these. With this idea in mind, I am motivated to write a profiler. I am not a Python expert or even close to it but thanks to the ease of language and C API, I can come up with a source code profiler with multithreading support. Thanks for all the people here for answering my silly questions. I have tested the profiler in our game server for about 3 days without any issues but please use it at your own risk and note that this is still beta. Expect silly errors and if possible, help me to fix them:) Get more information: (thanks to Google Code) http://code.google.com/p/yappi/ Download: http://code.google.com/p/yappi/downloads/list Not to mention any comment/idea/curse is welcome. -- Sumer Cip -- http://mail.python.org/mailman/listinfo/python-list
ANN: yappi 0.1 beta : Yet Another Python Profiler
Hi all, After implementing a game server on which 100k people playing games per-day, it turns out to be that continuous and efficient profiling is key to improve an long-running applications like these. With this idea in mind, I am motivated to write a profiler. I am not a Python expert or even close to it but thanks to the ease of language and C API, I can come up with a source code profiler with multithreading support. Thanks for all the people here for answering my silly questions. I have tested the profiler in our game server for about 3 days without any issues but please use it at your own risk and note that this is still beta. Expect silly errors and if possible, help me to fix them:) Get more information: (thanks to Google Code) http://code.google.com/p/yappi/ Download: http://code.google.com/p/yappi/downloads/list Not to mention any comment/idea/curse is welcome. Sumer Cip -- http://mail.python.org/mailman/listinfo/python-list
[issue881261] Overflow in Python Profiler
Amaury Forgeot d'Arc amaur...@gmail.com added the comment: on POSIX, clock() indeed wraps around after some time. This was corrected with issue645894, and is available since python 2.5. -- nosy: +amaury.forgeotdarc resolution: - out of date status: open - closed superseder: - better timer resolution for profile.py ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue881261 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
Python Profiler GUI like Matlab?
Hi all, Is there any Python Profiler GUI like Matlab? I found the Matlab Profiler is very intuitive and easy to use. Best regards, Davy -- http://mail.python.org/mailman/listinfo/python-list
Re: Python Profiler GUI like Matlab?
On 7 Des, 02:58, Davy [EMAIL PROTECTED] wrote: Is there any Python Profiler GUI like Matlab? I found the Matlab Profiler is very intuitive and easy to use. There is a Python profiler. But is does not have a GUI. -- http://mail.python.org/mailman/listinfo/python-list
Python profiler and decorators
HiI was wondering about this while working on profiling my program using the profile module.Say we have the following:@decdef func: blah blah blahWhen profiling the whole program, what would the total time for func represent? The time spent just in the original function or the time spent spent in the function along with the execution of the decorator's code? Thanks -- http://mail.python.org/mailman/listinfo/python-list
Interpreting python profiler results
I'm working on a piece of software that uses PyQt and PyOpenGL for drawing acceleration. While profiling it to find opportunities to speed things up, I got this from the profiler: ncalls tottime percall cumtime percall filename:lineno(function) 12108/12084 55.3900.005 55.3900.005 :0(getattr) 29340.4300.0001.2400.000 files_mmp.py:344(_read_atom) 10.3100.310 59.240 59.240 :0(exec_loop) 29340.2100.0000.3100.000 chem.py:1774(best_atomtype_for_numbonds) 87370.1500.0000.1900.000 :0(map) 29440.1500.0000.4500.000 chem.py:288(__init__) more. I assume :0(getattr) means a getattr function in either PyQt or PyOpenGL. I grepped the sources of both looking for getattr, and it doesn't appear in PyQt but appears frequently in PyOpenGL. Does anybody know if I am correct in my conclusion that most of the program's time is being spent in some getattr function in PyOpenGL? Thanks Will Ware -- http://mail.python.org/mailman/listinfo/python-list
Re: Python profiler
look in the gc module ... Celine Dave wrote: Hello All, I am trying to find a profiler that can measure the memory usage in a Python program. I would like to gather some statistics about object usages. For example, I would like to be able to see how much time it takes to search for an item in a dict object, how many times it has to access the symbol table to retrieve a specific item, and things like that. Thanks, Dave __ Yahoo! Mail - PC Magazine Editors' Choice 2005 http://mail.yahoo.com -- http://mail.python.org/mailman/listinfo/python-list
Python profiler
Hello All,
I am trying to measure memory used in a Python
program. Also, I would like to gather some statistics
about object usages. For example, I would like to be
able to see how much time it takes to search for an
item in a dict object, how many times it has to access
the symbol table to retrieve a specific item, and
things like that. Furthermore, I would like to be able
to see how a symbol table is created, how many items
it has, how many times it is accessed during
execution, etc.
Python profiler gives timing information about
functions/methods. For example, it doesn't show how
many times a dictionary object is accessed to retrieve
a key or how much time it takes.
The dis module doesn't provide detailed information
either. For the following code,
dict = { 'a': 1, 'b': 2, 'c': 3, 'd': 4, 'e': 5}
print dict.has_key('e')
The dis module provides the following:
0 LOAD_GLOBAL0 (dict)
3 LOAD_ATTR1 (has_key)
6 LOAD_CONST 1 ('e')
9 CALL_FUNCTION 1
12 PRINT_ITEM
But this doesn't really help to see what's going on
with the symbol table.
Any help is greatly appreaciated. Thanks a lot!
Regards,
Dave
__
Yahoo! Mail - PC Magazine Editors' Choice 2005
http://mail.yahoo.com
--
http://mail.python.org/mailman/listinfo/python-list
Python profiler
Hello All, I am trying to find a profiler that can measure the memory usage in a Python program. I would like to gather some statistics about object usages. For example, I would like to be able to see how much time it takes to search for an item in a dict object, how many times it has to access the symbol table to retrieve a specific item, and things like that. Thanks, Dave __ Yahoo! Mail - PC Magazine Editors' Choice 2005 http://mail.yahoo.com -- http://mail.python.org/mailman/listinfo/python-list
Re: Python profiler
In message [EMAIL PROTECTED], Celine Dave [EMAIL PROTECTED] writes I am trying to find a profiler that can measure the memory usage in a Python program. I would like to gather some statistics about object usages. For Python Memory Validator. Apply for beta here: http://www.softwareverify.com/beta.php?product=PMVB000 example, I would like to be able to see how much time it takes to search for an item in a dict object, That is something for a performance profiler, not a memory profiler. Python Performance Validator will help you there. how many times it has to access the symbol table to retrieve a specific item, and things like that. Python Coverage Validator will help you there. Python Performance Validator will help you there. http://www.softwareverify.com Windows NT/2000/XP/and above(vista, etc) Stephen -- Stephen Kellett Object Media Limitedhttp://www.objmedia.demon.co.uk/software.html Computer Consultancy, Software Development Windows C++, Java, Assembler, Performance Analysis, Troubleshooting -- http://mail.python.org/mailman/listinfo/python-list
