Re: [Python-Dev] Updated PEP 454 (tracemalloc): no more metrics!

2013-10-30 Thread Victor Stinner
2013/10/31 Victor Stinner : > Log calls to the memory allocator > - > > A different approach is to log calls to ``malloc()``, ``realloc()`` and > ``free()`` functions. Calls can be logged into a file or send to another > computer through the network. Example of a log

Re: [Python-Dev] Updated PEP 454 (tracemalloc): no more metrics!

2013-10-30 Thread Victor Stinner
New update of the PEP combining various remarks: * Remove GroupedStats class and Snapshot.group_by(): replaced with a new Snapshot.statistics() method which combines all features * Rename reset() to clear_traces() and explain how to get traces before clearing traces * Snapshot.apply_filters() now

Re: [Python-Dev] Updated PEP 454 (tracemalloc): no more metrics!

2013-10-30 Thread Victor Stinner
> Snapshot > > > ``Snapshot(timestamp: datetime.datetime, traceback_limit: int, stats: > dict=None, traces: dict=None)`` class: > > Snapshot of statistics and traces of memory blocks allocated by > Python. > > ``apply_filters(filters)`` method: > > Apply filters on the ``traces

Re: [Python-Dev] Updated PEP 454 (tracemalloc): no more metrics!

2013-10-30 Thread Victor Stinner
2013/10/30 Kristján Valur Jónsson : > The point of a PEP is getting something into standard python. The command > line flag is also part of this. > Piggybacking a lightweight client/server data-gathering version of this on > top of the PEP > could be beneficial in that respect. In my opinion, y

Re: [Python-Dev] Updated PEP 454 (tracemalloc): no more metrics!

2013-10-30 Thread Kristján Valur Jónsson
> -Original Message- > From: Victor Stinner [mailto:victor.stin...@gmail.com] > Sent: 29. október 2013 21:30 > To: Kristján Valur Jónsson > Cc: python-dev > Subject: Re: [Python-Dev] Updated PEP 454 (tracemalloc): no more metrics! > tracemalloc maintains a dictio

Re: [Python-Dev] Updated PEP 454 (tracemalloc): no more metrics!

2013-10-29 Thread Victor Stinner
2013/10/26 Kristján Valur Jónsson : > In that case, how about adding a client/server feature? > > If you standardize the format, a minimal tracing client could write a log, > or send it to a socket, in a way that can be turned into a snapshot by a > corresponsing utility reading from a file or list

Re: [Python-Dev] Updated PEP 454 (tracemalloc): no more metrics!

2013-10-27 Thread Victor Stinner
Hum, timestamp and traceback_limit attributed of GroupedStats can also be removed, they are just of the same attribute of the Snapshot class and GroupedStats is created from Snapshot.group_by(). Le 27 oct. 2013 17:26, "Victor Stinner" a écrit : > * do you have a better suggestion for GroupedStats

Re: [Python-Dev] Updated PEP 454 (tracemalloc): no more metrics!

2013-10-27 Thread Victor Stinner
I updated the PEP according to different comments: * replace Snapshot.create() class method with take_snapshot() function * get_traces() now returns a list instead of a dict (remove addresses) * remove get_stats() * unknown frames are now stored as ("", 0) instead of (None, None) * remove get_obje

Re: [Python-Dev] Updated PEP 454 (tracemalloc): no more metrics!

2013-10-26 Thread Kristján Valur Jónsson
be a future addition… From: Python-Dev [mailto:python-dev-bounces+kristjan=ccpgames@python.org] On Behalf Of Victor Stinner Sent: 24. október 2013 16:45 To: python-dev Subject: Re: [Python-Dev] Updated PEP 454 (tracemalloc): no more metrics! > When I was looking for memory leaks in the re

Re: [Python-Dev] Updated PEP 454 (tracemalloc): no more metrics!

2013-10-24 Thread Charles-François Natali
2013/10/24 Kristján Valur Jónsson : > > Now, I would personally not truncate the stack, because I can afford the > memory, > but even if I would, for example, to hide a bunch of detail, I would want to > throw away > the _lower_ detals of the stack. It is unimportant to me to know if memory > w

Re: [Python-Dev] Updated PEP 454 (tracemalloc): no more metrics!

2013-10-24 Thread Victor Stinner
> When I was looking for memory leaks in the regex module I simply wrote all of the allocations, reallocations and deallocations to a log file and then parsed it afterwards using a Python script. Simple, but effective. He he, it's funny because you described exactly my first implementation of trac

Re: [Python-Dev] Updated PEP 454 (tracemalloc): no more metrics!

2013-10-24 Thread MRAB
On 24/10/2013 15:40, Kristján Valur Jónsson wrote: -Original Message- From: Nick Coghlan [mailto:ncogh...@gmail.com] Sent: 24. október 2013 12:44 To: Kristján Valur Jónsson Cc: Python Dev Subject: Re: [Python-Dev] Updated PEP 454 (tracemalloc): no more metrics! Not everything is a PC

Re: [Python-Dev] Updated PEP 454 (tracemalloc): no more metrics!

2013-10-24 Thread Kristján Valur Jónsson
> -Original Message- > From: Nick Coghlan [mailto:ncogh...@gmail.com] > Sent: 24. október 2013 12:44 > To: Kristján Valur Jónsson > Cc: Python Dev > Subject: Re: [Python-Dev] Updated PEP 454 (tracemalloc): no more metrics! > Not everything is a PC that you can just a

Re: [Python-Dev] Updated PEP 454 (tracemalloc): no more metrics!

2013-10-24 Thread Victor Stinner
2013/10/24 Kristján Valur Jónsson : >> Test 1. With the Python test suite, 467,738 traces limited to 1 frame: > ... >> I'm surprised: it's faster than the benchmark I ran some weeks ago. >> Maybe I optimized something? The most critical operation, taking a snapshot >> takes half a second, so it's e

Re: [Python-Dev] Updated PEP 454 (tracemalloc): no more metrics!

2013-10-24 Thread Nick Coghlan
On 24 October 2013 22:34, Kristján Valur Jónsson wrote: > I understand your desire for things to be fast, but let me just re-iterate my > view > that for this kind of jobs, performance is completely secondary. Memory > debugging and analysis is an off-line, laboratory task. In my opinion, > per

Re: [Python-Dev] Updated PEP 454 (tracemalloc): no more metrics!

2013-10-24 Thread Kristján Valur Jónsson
> -Original Message- > From: Victor Stinner [mailto:victor.stin...@gmail.com] > Sent: 24. október 2013 01:03 > To: Kristján Valur Jónsson > Cc: Python Dev > Subject: Re: [Python-Dev] Updated PEP 454 (tracemalloc): no more metrics! > > > The use case of get_

Re: [Python-Dev] Updated PEP 454 (tracemalloc): no more metrics!

2013-10-23 Thread Victor Stinner
Hi, 2013/10/23 Kristján Valur Jónsson : > This might be a good place to make some comments. > I have discussed some of this in private with Victor, but wanted to make them > here, for the record. Yes, I prefer to discuss the PEP on python-dev. It's nice to get more feedback, I expect to get a be

Re: [Python-Dev] Updated PEP 454 (tracemalloc): no more metrics!

2013-10-23 Thread Kristján Valur Jónsson
games@python.org] fyrir hönd Victor Stinner [victor.stin...@gmail.com] Sent: 23. október 2013 18:25 To: Python Dev Efni: [Python-Dev] Updated PEP 454 (tracemalloc): no more metrics! Hi, I was at the restaurant with Charles-François and Antoine yesterday to discuss the PEP 454 (tracemalloc).

[Python-Dev] Updated PEP 454 (tracemalloc): no more metrics!

2013-10-23 Thread Victor Stinner
Hi, I was at the restaurant with Charles-François and Antoine yesterday to discuss the PEP 454 (tracemalloc). They gave me a lot of advices to improve the PEP. Most remarks were request to remove code :-) I also improved surprising/strange APIs (like the infamous GroupedStats.compate_to(None)). H