On 2/2/2011 6:06 PM, Eric Firing wrote: > On 02/02/2011 03:08 PM, Robert Abiad wrote: >> On 2/2/2011 3:59 PM, Christoph Gohlke wrote: >>> On 2/2/2011 3:33 PM, Robert Abiad wrote: >>>> Hello All, >>>> >>>> I'm very new to python, so bear with me. >>>> >>>> I'd like to use python to do my image processing, but I'm running into >>>> behavior that doesn't make >>>> sense to me. I'm using Windows 7 Pro (64-bit) with 4 gigs of memory, >>>> python 2.6.6, and the newest >>>> versions of ipython, pyfits, matplotlib (1.0.1), numpy (1.5.1), scipy. >>>> I'm loading in a fits file >>>> that's 26 MB (~16 Mpixels). When I load my image in ImageJ, I can see >>>> memory usage go up by 50MB, >>>> but when I try displaying the image using imshow(), my memory usage goes >>>> up by around 500MB, each >>>> time. If I close the figure and replot it, imshow() crashes. I don't >>>> know if I'm doing something >>>> wrong, or if it's a new or known bug. I tried the same thing on Linux and >>>> got the same result. >>>> Here's a transcript. >>>> >>>> Welcome to pylab, a matplotlib-based Python environment. >>>> For more information, type 'help(pylab)'. >>>> >>>> In [1]: import pyfits >>>> >>>> In [2]: from Tkinter import * >>>> >>>> In [3]: import tkFileDialog >>>> >>>> In [4]: image=pyfits.getdata(tkFileDialog.askopenfilename()) >>>> >>>> In [5]: imshow(image) >>>> Out[5]:<matplotlib.image.AxesImage object at 0x03BCA170> >>>> >>>> In [6]: close() >>>> >>>> In [7]: imshow(image,origin='lower') >>>> Out[7]:<matplotlib.image.AxesImage object at 0x0440E170> >>>> >>>> In [8]: close() >>>> >>>> In [9]: imshow(image[100:3600,100:3600],origin='lower') >>>> Out[9]:<matplotlib.image.AxesImage object at 0x045D9FB0> >>>> >>>> In [10]: Exception in Tkinter callback >>>> Traceback (most recent call last): >>>> File "C:\app\Python2.6\lib\lib-tk\Tkinter.py", line 1410, in >>>> __call__ >>>> return self.func(*args) >>>> File "C:\app\Python2.6\lib\lib-tk\Tkinter.py", line 495, in callit >>>> func(*args) >>>> File >>>> "C:\app\Python2.6\lib\site-packages\matplotlib\backends\backend_tkagg.py", >>>> line 263, in >>>> idle_draw >>>> self.draw() >>>> File >>>> "C:\app\Python2.6\lib\site-packages\matplotlib\backends\backend_tkagg.py", >>>> line 248, in draw >>>> FigureCanvasAgg.draw(self) >>>> File >>>> "C:\app\Python2.6\lib\site-packages\matplotlib\backends\backend_agg.py", >>>> line 394, in draw >>>> self.figure.draw(self.renderer) >>>> File "C:\app\Python2.6\lib\site-packages\matplotlib\artist.py", >>>> line 55, in draw_wrapper >>>> draw(artist, renderer, *args, **kwargs) >>>> File "C:\app\Python2.6\lib\site-packages\matplotlib\figure.py", >>>> line 798, in draw >>>> func(*args) >>>> File "C:\app\Python2.6\lib\site-packages\matplotlib\artist.py", >>>> line 55, in draw_wrapper >>>> draw(artist, renderer, *args, **kwargs) >>>> File "C:\app\Python2.6\lib\site-packages\matplotlib\axes.py", line >>>> 1946, in draw >>>> a.draw(renderer) >>>> File "C:\app\Python2.6\lib\site-packages\matplotlib\artist.py", >>>> line 55, in draw_wrapper >>>> draw(artist, renderer, *args, **kwargs) >>>> File "C:\app\Python2.6\lib\site-packages\matplotlib\image.py", line >>>> 354, in draw >>>> im = self.make_image(renderer.get_image_magnification()) >>>> File "C:\app\Python2.6\lib\site-packages\matplotlib\image.py", line >>>> 569, in make_image >>>> transformed_viewLim) >>>> File "C:\app\Python2.6\lib\site-packages\matplotlib\image.py", line >>>> 201, in _get_unsampled_image >>>> x = self.to_rgba(self._A, self._alpha) >>>> File "C:\app\Python2.6\lib\site-packages\matplotlib\cm.py", line >>>> 193, in to_rgba >>>> x = self.norm(x) >>>> File "C:\app\Python2.6\lib\site-packages\matplotlib\colors.py", >>>> line 820, in __call__ >>>> result = (val-vmin) / (vmax-vmin) >>>> File "C:\app\Python2.6\lib\site-packages\numpy\ma\core.py", line >>>> 3673, in __div__ >>>> return divide(self, other) >>>> File "C:\app\Python2.6\lib\site-packages\numpy\ma\core.py", line >>>> 1077, in __call__ >>>> m |= filled(domain(da, db), True) >>>> File "C:\app\Python2.6\lib\site-packages\numpy\ma\core.py", line >>>> 772, in __call__ >>>> return umath.absolute(a) * self.tolerance>= umath.absolute(b) >>>> MemoryError >>>> >>>> >>>> Thanks for any help, >>>> -robert >>>> >>> >>> These are previous discussions on the issue: >>> >>> <http://www.mail-archive.com/matplotlib-users@lists.sourceforge.net/msg14727.html> >>> <http://www.mail-archive.com/matplotlib-users@lists.sourceforge.net/msg19815.html> >>> <http://www.mail-archive.com/matplotlib-users@lists.sourceforge.net/msg19614.html> >>> >>> Christoph >>> >> The first 2 discussions lead to suggestions of more memory on a 64-bit >> installation, but that >> doesn't seem like a great solution. I use other image processing software >> (ImageJ and IDL) and >> neither has any trouble with my images. As I mentioned ImageJ uses 1/10th >> the memory for the same >> display, or about 1 byte of memory for 1 byte of image. I think matplotlib >> should aim for the same. >> I also think it should free up memory when the image is closed, but >> maybe I'm not doing the right >> thing. Is there something else I should be doing to free up memory? >> >> Things are even worse with plot. >> >> I'll file a bug report as Benjamin suggests. > Please file it in the "enhancement" category, not as a "bug". The > difficulty that mpl has with images is the result of a design decision > long ago; as Christoph notes, mpl works primarily with float64 rgba, > which is horribly memory inefficient, but is very general and works fine > so long as the image is not too large. uint8 is already used in some > circumstances. I don't know how hard it would be to simply switch to > uint8 for images, or to make that an option, perhaps for all color > specification. It may involve getting into arcana of the extension code > and the agg library, which is a daunting prospect for most of us, > certainly for me. I agree entirely that a major reduction in memory > usage would be good; patches, or a mpl branch to achieve that, are welcome. > > Eric I'll put it in as an enhancement, but I'm still unsure if there is a bug in there as well. Is there something I should be doing to clear memory after the first figure is closed other than close()? I don't understand why memory usage grows each time I replot, but I'm pretty sure it isn't desireable behavior. As I mentioned, this effect is worse with plot.
So is this a bug or improper usage? As for how to implement a fix for memory usage, I'll let you folks figure that out. But it seems that if I want a grayscale image, I could reduce memory usage by 4 if matplotlib could turn rgba into intensity. -robert ------------------------------------------------------------------------------ Special Offer-- Download ArcSight Logger for FREE (a $49 USD value)! Finally, a world-class log management solution at an even better price-free! Download using promo code Free_Logger_4_Dev2Dev. Offer expires February 28th, so secure your free ArcSight Logger TODAY! http://p.sf.net/sfu/arcsight-sfd2d _______________________________________________ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users
