Re: weakref, memory management and execution slow down in PyQt4

2014-09-09 Thread Terry Reedy
On 9/9/2014 11:34 AM, Michael Torrie wrote: On 09/08/2014 08:45 PM, kjs wrote: You're right, a dictionary can do everything I need and more. Actually I am wrong in suggesting a dictionary. A list or an array would probably be more appropriate. Thinking about it this morning, one additional r

Re: weakref, memory management and execution slow down in PyQt4

2014-09-09 Thread kjs
On September 9, 2014 8:57:02 AM PDT, Michael Torrie wrote: >On 09/09/2014 09:37 AM, Chris Angelico wrote: >> On Wed, Sep 10, 2014 at 1:32 AM, Michael Torrie >wrote: >>> Yes you're correct. It is the equivalent. But it always involves >>> lookup in the object's dictionary, which is big O order

Re: weakref, memory management and execution slow down in PyQt4

2014-09-09 Thread Michael Torrie
On 09/09/2014 09:37 AM, Chris Angelico wrote: > On Wed, Sep 10, 2014 at 1:32 AM, Michael Torrie wrote: >> Yes you're correct. It is the equivalent. But it always involves >> lookup in the object's dictionary, which is big O order O(n log n) >> complexity for each and every access. > > Where do

Re: weakref, memory management and execution slow down in PyQt4

2014-09-09 Thread Chris Angelico
On Wed, Sep 10, 2014 at 1:32 AM, Michael Torrie wrote: > Yes you're correct. It is the equivalent. But it always involves > lookup in the object's dictionary, which is big O order O(n log n) > complexity for each and every access. Where do you get that figure from? A CPython dictionary is imple

Re: weakref, memory management and execution slow down in PyQt4

2014-09-09 Thread Michael Torrie
On 09/08/2014 08:45 PM, kjs wrote: > You're right, a dictionary can do everything I need and more. Actually I am wrong in suggesting a dictionary. A list or an array would probably be more appropriate. Thinking about it this morning, one additional reason why getattr and setattr aren't appropria

Re: weakref, memory management and execution slow down in PyQt4

2014-09-09 Thread Michael Torrie
Reposting to list, instead of directly to kjs On 09/08/2014 08:45 PM, kjs wrote: > Thanks for the consideration Michael. If you do get the data, and are > able to run the code, let me know if you notice anything interesting. Yeah I don't think I'll be able to have the time to download a 3 GB file

Re: weakref, memory management and execution slow down in PyQt4

2014-09-09 Thread Mark Lawrence
On 09/09/2014 03:45, kjs wrote: You're right, a dictionary can do everything I need and more. This happened to be the first thing I thought of, and I didn't imagine it would be very expensive. I figured it was simply a different way of defining and retrieving a class variable. IE setattr(self, f

Re: weakref, memory management and execution slow down in PyQt4

2014-09-08 Thread kjs
Thanks for the consideration Michael. If you do get the data, and are able to run the code, let me know if you notice anything interesting. Michael Torrie: > On 09/07/2014 02:39 PM, kjs wrote: >> The code is minimal[0]. The only other widgets are a start button that >> fires off the plotting and

Re: weakref, memory management and execution slow down in PyQt4

2014-09-08 Thread Michael Torrie
On 09/07/2014 02:39 PM, kjs wrote: > The code is minimal[0]. The only other widgets are a start button that > fires off the plotting and a stop button that calls sys.exit(). Unfortunately there are no data files in your git repository so I can't run it. > > Lines 112-114 appear to be causing the

Re: weakref, memory management and execution slow down in PyQt4

2014-09-08 Thread kjs
Michael Torrie: > On 09/07/2014 01:11 PM, kjs wrote: >> Thanks for the advice. I commented out the graph generation and PyQt call >> > self.app.processEvents() >> >> where in the class __init__ >> > self.app = QtGui.QApplication(sys.argv) >> >> This stopped the weakref proliferation. All

Re: weakref, memory management and execution slow down in PyQt4

2014-09-07 Thread Michael Torrie
On 09/07/2014 01:11 PM, kjs wrote: > Thanks for the advice. I commented out the graph generation and PyQt call > self.app.processEvents() > > where in the class __init__ > self.app = QtGui.QApplication(sys.argv) > > This stopped the weakref proliferation. All other objects grow and >

Re: weakref, memory management and execution slow down in PyQt4

2014-09-07 Thread kjs
Antoine Pitrou: > kjs riseup.net> writes: >> >> I have come to believe that the growing number of weakrefs is slowing >> down execution. Is my analysis misguided? How can I introspect further? >> If the slowdown can be attributed to weakref escalation, what are some >> next steps? > > The way t

Re: weakref, memory management and execution slow down in PyQt4

2014-09-07 Thread Antoine Pitrou
kjs riseup.net> writes: > > I have come to believe that the growing number of weakrefs is slowing > down execution. Is my analysis misguided? How can I introspect further? > If the slowdown can be attributed to weakref escalation, what are some > next steps? The way to analyze this is to build s

weakref, memory management and execution slow down in PyQt4

2014-09-07 Thread kjs
I built a small application using PyQt4 and pyqtgraph to visualize some data. The app has 32 graphs that plot deques of size 512. The plots are updated when 200 ints are cycled through each deque. The plotting slows down in a linear manner with respect to time. In other words after cycling through