On Thu, Nov 18, 2010 at 1:11 PM, Caleb Constantine <cadamant...@gmail.com>wrote:
> Matplotlib Users:
>
> It seems matplotlib plotting has a relatively small memory leak. My
> experiments suggest it leaks between 5K and 8K bytes of RAM for ever plot
> redraw. For example, in one experiment, plotting the same buffer (so as to
> not
> allocate new memory) every second for a period of about 12 hours resulted
> in
> memory usage (physical RAM) increasing by approximately 223MB, which is
> about
> 5.3K per replot. The plotting code is:
>
> class PlotPanel(wx.Panel):
> def __init__(self, parent):
> wx.Panel.__init__(self, parent, wx.ID_ANY,
> style=wx.BORDER_THEME|wx.TAB_TRAVERSAL)
> self._figure = MplFigure(dpi=None)
> self._canvas = MplCanvas(self, -1, self._figure)
> self._axes = self._figure.add_subplot(1,1,1)
>
> sizer = wx.BoxSizer(wx.VERTICAL)
> sizer.Add(self._canvas, 1, wx.EXPAND|wx.TOP, 5)
> self.SetSizer(sizer)
>
> def draw(self, channel, seconds):
> self._axes.clear()
> self._axes.plot(channel, seconds)
> self._canvas.draw()
>
>
> `draw()` is called every second with the same `channels` and `seconds`
> numpy.array buffers.
>
> In my case, this leak, though relatively small, becomes a serious issue
> since
> my software often runs for long periods of time (days) plotting data
> streamed
> from a data acquisition unit.
>
> Any suggestions will help. Am I miss understanding something here? Maybe I
> need to call some obscure function to free memory, or something?
>
> My testing environment:
>
> * Windws XP SP3, Intel Core 2 Duo @ 2.33GHz, 1.96 GB RAM
> * Python 2.6.6 (r266:84297, Aug 24 2010, 18:46:32) [MSC v.1500 32 bit
> (Intel)] on win32
> * matplotlib version 1.0.0
> * numpy 1.4.1
> * wxPython version 2.8.11.0
>
> The complete test program follows.
>
> Thanks,
>
> Caleb
>
>
> from random import random
> from datetime import datetime
> import os
> import time
> import win32api
> import win32con
> import win32process
>
> import wx
> import numpy
>
> import matplotlib as mpl
> from matplotlib.figure import Figure as MplFigure
> from matplotlib.backends.backend_wxagg import FigureCanvasWxAgg as
> MplCanvas
>
> def get_process_memory_info(process_id):
> memory = {}
> process = None
> try:
> process = win32api.OpenProcess(
> win32con.PROCESS_QUERY_INFORMATION|win32con.PROCESS_VM_READ,
> False, process_id);
> if process is not None:
> return win32process.GetProcessMemoryInfo(process)
> finally:
> if process:
> win32api.CloseHandle(process)
> return memory
>
> meg = 1024.0 * 1024.0
>
> class PlotPanel(wx.Panel):
> def __init__(self, parent):
> wx.Panel.__init__(self, parent, wx.ID_ANY,
> style=wx.BORDER_THEME|wx.TAB_TRAVERSAL)
> self._figure = MplFigure(dpi=None)
> self._canvas = MplCanvas(self, -1, self._figure)
> self._axes = self._figure.add_subplot(1,1,1)
>
> sizer = wx.BoxSizer(wx.VERTICAL)
> sizer.Add(self._canvas, 1, wx.EXPAND|wx.TOP, 5)
> self.SetSizer(sizer)
>
> def draw(self, channel, seconds):
> self._axes.clear()
> self._axes.plot(channel, seconds)
> self._canvas.draw()
>
> class TestFrame(wx.Frame):
> def __init__(self, parent, id, title):
> wx.Frame.__init__(
> self, parent, id, title, wx.DefaultPosition, (600, 400))
>
> self.testDuration = 60 * 60 * 24
> self.startTime = 0
>
> self.channel = numpy.sin(numpy.arange(1000) * random())
> self.seconds = numpy.arange(len(self.channel))
>
> self.plotPanel = PlotPanel(self)
>
> sizer = wx.BoxSizer(wx.VERTICAL)
> sizer.Add(self.plotPanel, 1 ,wx.EXPAND)
> self.SetSizer(sizer)
>
> self._timer = wx.Timer(self)
> self.Bind(wx.EVT_TIMER, self._onTimer, self._timer)
> self._timer.Start(1000)
> print "starting memory: ",\
> get_process_memory_info(os.getpid())["WorkingSetSize"]/meg
>
> def _onTimer(self, evt):
> if self.startTime == 0:
> self.startTime = time.time()
>
> if (time.time() - self.startTime) >= self.testDuration:
> self._timer.Stop()
>
> self.plotPanel.draw(self.channel, self.seconds)
>
> t = datetime.now()
> memory = get_process_memory_info(os.getpid())
> print "time: {0}, working: {1:f}".format(
> t, memory["WorkingSetSize"]/meg)
>
> class MyApp(wx.App):
> def OnInit(self):
> frame = TestFrame(None, wx.ID_ANY, "Memory Leak")
> self.SetTopWindow(frame)
> frame.Show(True)
> return True
>
> if __name__ == '__main__':
> app = MyApp(0)
> app.MainLoop()
>
>
>
Caleb,
Interesting analysis. One possible source of a leak would be some sort of
dangling reference that still hangs around even though the plot objects have
been cleared. By the time of the matplotlib 1.0.0 release, we did seem to
clear out pretty much all of these, but it is possible there are still some
lurking about. We should probably run your script against the latest svn to
see how the results compare.
Another possibility might be related to numpy. However this is the draw
statement, so I don't know how much numpy is used in there. The latest
refactor work in numpy has revealed some memory leaks that have existed, so
who knows?
Might be interesting to try making equivalent versions of this script using
different backends, and different package versions to possibly isolate the
source of the memory leak.
Thanks for your observations,
Ben Root
------------------------------------------------------------------------------
Beautiful is writing same markup. Internet Explorer 9 supports
standards for HTML5, CSS3, SVG 1.1, ECMAScript5, and DOM L2 & L3.
Spend less time writing and rewriting code and more time creating great
experiences on the web. Be a part of the beta today
http://p.sf.net/sfu/msIE9-sfdev2dev
_______________________________________________
Matplotlib-users mailing list
Matplotlib-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/matplotlib-users
