Christoph, if you're looking at special casing uint8's, you might want to keep in mind that uint16 greyscale images are also quite common as camera outputs in experimental setups. I think that the solution to this should ideally minimise memory usage for any greyscale image, be it uint8, uint16, float32 or float64. i.e. avoiding conversion to RGBA for any single-plane 2D array type would be best IMHO,
Gary R. On Thu, Feb 3, 2011 at 5:38 PM, Robert Abiad <ab...@ssl.berkeley.edu> wrote: > > > On 2/2/2011 6:06 PM, Eric Firing wrote: >> On 02/02/2011 03:08 PM, Robert Abiad wrote: >>> On 2/2/2011 3:59 PM, Christoph Gohlke wrote: >>>> On 2/2/2011 3:33 PM, Robert Abiad wrote: >>>>> Hello All, >>>>> >>>>> I'm very new to python, so bear with me. >>>>> >>>>> I'd like to use python to do my image processing, but I'm running into >>>>> behavior that doesn't make >>>>> sense to me. I'm using Windows 7 Pro (64-bit) with 4 gigs of memory, >>>>> python 2.6.6, and the newest >>>>> versions of ipython, pyfits, matplotlib (1.0.1), numpy (1.5.1), scipy. >>>>> I'm loading in a fits file >>>>> that's 26 MB (~16 Mpixels). When I load my image in ImageJ, I can see >>>>> memory usage go up by 50MB, >>>>> but when I try displaying the image using imshow(), my memory usage goes >>>>> up by around 500MB, each >>>>> time. If I close the figure and replot it, imshow() crashes. I don't >>>>> know if I'm doing something >>>>> wrong, or if it's a new or known bug. I tried the same thing on Linux >>>>> and got the same result. >>>>> Here's a transcript. >>>>> >>>>> Welcome to pylab, a matplotlib-based Python environment. >>>>> For more information, type 'help(pylab)'. >>>>> >>>>> In [1]: import pyfits >>>>> >>>>> In [2]: from Tkinter import * >>>>> >>>>> In [3]: import tkFileDialog >>>>> >>>>> In [4]: image=pyfits.getdata(tkFileDialog.askopenfilename()) >>>>> >>>>> In [5]: imshow(image) >>>>> Out[5]:<matplotlib.image.AxesImage object at 0x03BCA170> >>>>> >>>>> In [6]: close() >>>>> >>>>> In [7]: imshow(image,origin='lower') >>>>> Out[7]:<matplotlib.image.AxesImage object at 0x0440E170> >>>>> >>>>> In [8]: close() >>>>> >>>>> In [9]: imshow(image[100:3600,100:3600],origin='lower') >>>>> Out[9]:<matplotlib.image.AxesImage object at 0x045D9FB0> >>>>> >>>>> In [10]: Exception in Tkinter callback >>>>> Traceback (most recent call last): >>>>> File "C:\app\Python2.6\lib\lib-tk\Tkinter.py", line 1410, in >>>>> __call__ >>>>> return self.func(*args) >>>>> File "C:\app\Python2.6\lib\lib-tk\Tkinter.py", line 495, in callit >>>>> func(*args) >>>>> File >>>>> "C:\app\Python2.6\lib\site-packages\matplotlib\backends\backend_tkagg.py", >>>>> line 263, in >>>>> idle_draw >>>>> self.draw() >>>>> File >>>>> "C:\app\Python2.6\lib\site-packages\matplotlib\backends\backend_tkagg.py", >>>>> line 248, in draw >>>>> FigureCanvasAgg.draw(self) >>>>> File >>>>> "C:\app\Python2.6\lib\site-packages\matplotlib\backends\backend_agg.py", >>>>> line 394, in draw >>>>> self.figure.draw(self.renderer) >>>>> File "C:\app\Python2.6\lib\site-packages\matplotlib\artist.py", >>>>> line 55, in draw_wrapper >>>>> draw(artist, renderer, *args, **kwargs) >>>>> File "C:\app\Python2.6\lib\site-packages\matplotlib\figure.py", >>>>> line 798, in draw >>>>> func(*args) >>>>> File "C:\app\Python2.6\lib\site-packages\matplotlib\artist.py", >>>>> line 55, in draw_wrapper >>>>> draw(artist, renderer, *args, **kwargs) >>>>> File "C:\app\Python2.6\lib\site-packages\matplotlib\axes.py", line >>>>> 1946, in draw >>>>> a.draw(renderer) >>>>> File "C:\app\Python2.6\lib\site-packages\matplotlib\artist.py", >>>>> line 55, in draw_wrapper >>>>> draw(artist, renderer, *args, **kwargs) >>>>> File "C:\app\Python2.6\lib\site-packages\matplotlib\image.py", >>>>> line 354, in draw >>>>> im = self.make_image(renderer.get_image_magnification()) >>>>> File "C:\app\Python2.6\lib\site-packages\matplotlib\image.py", >>>>> line 569, in make_image >>>>> transformed_viewLim) >>>>> File "C:\app\Python2.6\lib\site-packages\matplotlib\image.py", >>>>> line 201, in _get_unsampled_image >>>>> x = self.to_rgba(self._A, self._alpha) >>>>> File "C:\app\Python2.6\lib\site-packages\matplotlib\cm.py", line >>>>> 193, in to_rgba >>>>> x = self.norm(x) >>>>> File "C:\app\Python2.6\lib\site-packages\matplotlib\colors.py", >>>>> line 820, in __call__ >>>>> result = (val-vmin) / (vmax-vmin) >>>>> File "C:\app\Python2.6\lib\site-packages\numpy\ma\core.py", line >>>>> 3673, in __div__ >>>>> return divide(self, other) >>>>> File "C:\app\Python2.6\lib\site-packages\numpy\ma\core.py", line >>>>> 1077, in __call__ >>>>> m |= filled(domain(da, db), True) >>>>> File "C:\app\Python2.6\lib\site-packages\numpy\ma\core.py", line >>>>> 772, in __call__ >>>>> return umath.absolute(a) * self.tolerance>= umath.absolute(b) >>>>> MemoryError >>>>> >>>>> >>>>> Thanks for any help, >>>>> -robert >>>>> >>>> >>>> These are previous discussions on the issue: >>>> >>>> <http://www.mail-archive.com/matplotlib-users@lists.sourceforge.net/msg14727.html> >>>> <http://www.mail-archive.com/matplotlib-users@lists.sourceforge.net/msg19815.html> >>>> <http://www.mail-archive.com/matplotlib-users@lists.sourceforge.net/msg19614.html> >>>> >>>> Christoph >>>> >>> The first 2 discussions lead to suggestions of more memory on a 64-bit >>> installation, but that >>> doesn't seem like a great solution. I use other image processing software >>> (ImageJ and IDL) and >>> neither has any trouble with my images. As I mentioned ImageJ uses 1/10th >>> the memory for the same >>> display, or about 1 byte of memory for 1 byte of image. I think matplotlib >>> should aim for the same. >>> I also think it should free up memory when the image is closed, but >>> maybe I'm not doing the right >>> thing. Is there something else I should be doing to free up memory? >>> >>> Things are even worse with plot. >>> >>> I'll file a bug report as Benjamin suggests. >> Please file it in the "enhancement" category, not as a "bug". The >> difficulty that mpl has with images is the result of a design decision >> long ago; as Christoph notes, mpl works primarily with float64 rgba, >> which is horribly memory inefficient, but is very general and works fine >> so long as the image is not too large. uint8 is already used in some >> circumstances. I don't know how hard it would be to simply switch to >> uint8 for images, or to make that an option, perhaps for all color >> specification. It may involve getting into arcana of the extension code >> and the agg library, which is a daunting prospect for most of us, >> certainly for me. I agree entirely that a major reduction in memory >> usage would be good; patches, or a mpl branch to achieve that, are welcome. >> >> Eric > I'll put it in as an enhancement, but I'm still unsure if there is a bug in > there as well. Is there something I should be doing to clear memory after the > first figure is closed other than close()? I don't understand why memory > usage > grows each time I replot, but I'm pretty sure it isn't desireable behavior. > As > I mentioned, this effect is worse with plot. > > So is this a bug or improper usage? > > As for how to implement a fix for memory usage, I'll let you folks figure that > out. But it seems that if I want a grayscale image, I could reduce memory > usage > by 4 if matplotlib could turn rgba into intensity. > > -robert > > ------------------------------------------------------------------------------ > Special Offer-- Download ArcSight Logger for FREE (a $49 USD value)! > Finally, a world-class log management solution at an even better price-free! > Download using promo code Free_Logger_4_Dev2Dev. Offer expires > February 28th, so secure your free ArcSight Logger TODAY! > http://p.sf.net/sfu/arcsight-sfd2d > _______________________________________________ > Matplotlib-users mailing list > Matplotlib-users@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/matplotlib-users > ------------------------------------------------------------------------------ Special Offer-- Download ArcSight Logger for FREE (a $49 USD value)! Finally, a world-class log management solution at an even better price-free! Download using promo code Free_Logger_4_Dev2Dev. Offer expires February 28th, so secure your free ArcSight Logger TODAY! http://p.sf.net/sfu/arcsight-sfd2d _______________________________________________ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users
