On 2/2/2011 8:22 PM, Benjamin Root wrote: > On Wed, Feb 2, 2011 at 8:06 PM, Eric Firing <efir...@hawaii.edu > <mailto:efir...@hawaii.edu>> wrote: > > On 02/02/2011 03:08 PM, Robert Abiad wrote: > > On 2/2/2011 3:59 PM, Christoph Gohlke wrote: > > > On 2/2/2011 3:33 PM, Robert Abiad wrote: > > >> Hello All, > > >> > > >> I'm very new to python, so bear with me. > > >> > > >> I'd like to use python to do my image processing, but I'm > running into behavior that doesn't make > > >> sense to me. I'm using Windows 7 Pro (64-bit) with 4 gigs of > memory, python 2.6.6, and the newest > > >> versions of ipython, pyfits, matplotlib (1.0.1), numpy (1.5.1), > scipy. I'm loading in a fits file > > >> that's 26 MB (~16 Mpixels). When I load my image in ImageJ, I > can see memory usage go up by 50MB, > > >> but when I try displaying the image using imshow(), my memory > usage goes up by around 500MB, each > > >> time. If I close the figure and replot it, imshow() crashes. I > don't know if I'm doing something > > >> wrong, or if it's a new or known bug. I tried the same thing on > Linux and got the same result. > > >> Here's a transcript. > > >> > > >> Welcome to pylab, a matplotlib-based Python environment. > > >> For more information, type 'help(pylab)'. > > >> > > >> In [1]: import pyfits > > >> > > >> In [2]: from Tkinter import * > > >> > > >> In [3]: import tkFileDialog > > >> > > >> In [4]: image=pyfits.getdata(tkFileDialog.askopenfilename()) > > >> > > >> In [5]: imshow(image) > > >> Out[5]:<matplotlib.image.AxesImage object at 0x03BCA170> > > >> > > >> In [6]: close() > > >> > > >> In [7]: imshow(image,origin='lower') > > >> Out[7]:<matplotlib.image.AxesImage object at 0x0440E170> > > >> > > >> In [8]: close() > > >> > > >> In [9]: imshow(image[100:3600,100:3600],origin='lower') > > >> Out[9]:<matplotlib.image.AxesImage object at 0x045D9FB0> > > >> > > >> In [10]: Exception in Tkinter callback > > >> Traceback (most recent call last): > > >> File "C:\app\Python2.6\lib\lib-tk\Tkinter.py", line 1410, > in __call__ > > >> return self.func(*args) > > >> File "C:\app\Python2.6\lib\lib-tk\Tkinter.py", line 495, > in callit > > >> func(*args) > > >> File > "C:\app\Python2.6\lib\site-packages\matplotlib\backends\backend_tkagg.py", > line 263, in > > >> idle_draw > > >> self.draw() > > >> File > "C:\app\Python2.6\lib\site-packages\matplotlib\backends\backend_tkagg.py", > line 248, in draw > > >> FigureCanvasAgg.draw(self) > > >> File > "C:\app\Python2.6\lib\site-packages\matplotlib\backends\backend_agg.py", > line 394, in draw > > >> self.figure.draw(self.renderer) > > >> File > "C:\app\Python2.6\lib\site-packages\matplotlib\artist.py", line 55, > in draw_wrapper > > >> draw(artist, renderer, *args, **kwargs) > > >> File > "C:\app\Python2.6\lib\site-packages\matplotlib\figure.py", line 798, > in draw > > >> func(*args) > > >> File > "C:\app\Python2.6\lib\site-packages\matplotlib\artist.py", line 55, > in draw_wrapper > > >> draw(artist, renderer, *args, **kwargs) > > >> File > "C:\app\Python2.6\lib\site-packages\matplotlib\axes.py", line 1946, > in draw > > >> a.draw(renderer) > > >> File > "C:\app\Python2.6\lib\site-packages\matplotlib\artist.py", line 55, > in draw_wrapper > > >> draw(artist, renderer, *args, **kwargs) > > >> File > "C:\app\Python2.6\lib\site-packages\matplotlib\image.py", line 354, > in draw > > >> im = self.make_image(renderer.get_image_magnification()) > > >> File > "C:\app\Python2.6\lib\site-packages\matplotlib\image.py", line 569, > in make_image > > >> transformed_viewLim) > > >> File > "C:\app\Python2.6\lib\site-packages\matplotlib\image.py", line 201, > in _get_unsampled_image > > >> x = self.to_rgba(self._A, self._alpha) > > >> File > "C:\app\Python2.6\lib\site-packages\matplotlib\cm.py", line 193, in > to_rgba > > >> x = self.norm(x) > > >> File > "C:\app\Python2.6\lib\site-packages\matplotlib\colors.py", line 820, > in __call__ > > >> result = (val-vmin) / (vmax-vmin) > > >> File > "C:\app\Python2.6\lib\site-packages\numpy\ma\core.py", line 3673, in > __div__ > > >> return divide(self, other) > > >> File > "C:\app\Python2.6\lib\site-packages\numpy\ma\core.py", line 1077, in > __call__ > > >> m |= filled(domain(da, db), True) > > >> File > "C:\app\Python2.6\lib\site-packages\numpy\ma\core.py", line 772, in > __call__ > > >> return umath.absolute(a) * self.tolerance>= > umath.absolute(b) > > >> MemoryError > > >> > > >> > > >> Thanks for any help, > > >> -robert > > >> > > > > > > > > > These are previous discussions on the issue: > > > > > > > > <http://www.mail-archive.com/matplotlib-users@lists.sourceforge.net/msg14727.html> > > > > > <http://www.mail-archive.com/matplotlib-users@lists.sourceforge.net/msg19815.html> > > > > > <http://www.mail-archive.com/matplotlib-users@lists.sourceforge.net/msg19614.html> > > > > > > Christoph > > > > > > > The first 2 discussions lead to suggestions of more memory on a > 64-bit installation, but that > > doesn't seem like a great solution. I use other image processing > software (ImageJ and IDL) and > > neither has any trouble with my images. As I mentioned ImageJ > uses 1/10th the memory for the same > > display, or about 1 byte of memory for 1 byte of image. I think > matplotlib should aim for the same. > > I also think it should free up memory when the image is closed, > but maybe I'm not doing the right > > thing. Is there something else I should be doing to free up memory? > > > > Things are even worse with plot. > > > > I'll file a bug report as Benjamin suggests. > > Please file it in the "enhancement" category, not as a "bug". The > difficulty that mpl has with images is the result of a design decision > long ago; as Christoph notes, mpl works primarily with float64 rgba, > which is horribly memory inefficient, but is very general and works fine > so long as the image is not too large. uint8 is already used in some > circumstances. I don't know how hard it would be to simply switch to > uint8 for images, or to make that an option, perhaps for all color > specification. It may involve getting into arcana of the extension code > and the agg library, which is a daunting prospect for most of us, > certainly for me. I agree entirely that a major reduction in memory > usage would be good; patches, or a mpl branch to achieve that, are > welcome. > > Eric > > > As a possible stop-gap solution, maybe we could consider the compressed > arrays (carry)? > > I am not a fan of doing special cases, though, especially at the very > base/core portions of the library. > > Ben Root >
Forget about the patch; it is too crude and breaks things. My point was that it should be possible to have huge memory savings without touching agg internals. I'll take another look. Christoph ------------------------------------------------------------------------------ Special Offer-- Download ArcSight Logger for FREE (a $49 USD value)! Finally, a world-class log management solution at an even better price-free! Download using promo code Free_Logger_4_Dev2Dev. Offer expires February 28th, so secure your free ArcSight Logger TODAY! http://p.sf.net/sfu/arcsight-sfd2d _______________________________________________ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users
