As a workaround you could do your own normalization and color mapping
and pass a uint8 RGB image to imshow. That avoids matplotlib's norm
function. The following example saves almost 500 MB for plotting a 16 MB
uint8 greyscale image, as compared to passing img directly to imshow:
import numpy
from matplotlib import pyplot
img = numpy.random.randint(0, 255, (4096, 4096)).astype('uint8')
lut = pyplot.cm.gray(numpy.arange(255), bytes=True)
rgb = lut.take(img, axis=0)
del img
pyplot.imshow(rgb)
pyplot.show()
Christoph
On 2/2/2011 11:00 PM, gary ruben wrote:
> Christoph, if you're looking at special casing uint8's, you might want
> to keep in mind that uint16 greyscale images are also quite common as
> camera outputs in experimental setups. I think that the solution to
> this should ideally minimise memory usage for any greyscale image, be
> it uint8, uint16, float32 or float64. i.e. avoiding conversion to RGBA
> for any single-plane 2D array type would be best IMHO,
>
> Gary R.
>
> On Thu, Feb 3, 2011 at 5:38 PM, Robert Abiad<ab...@ssl.berkeley.edu> wrote:
>>
>>
>> On 2/2/2011 6:06 PM, Eric Firing wrote:
>>> On 02/02/2011 03:08 PM, Robert Abiad wrote:
>>>> On 2/2/2011 3:59 PM, Christoph Gohlke wrote:
>>>>> On 2/2/2011 3:33 PM, Robert Abiad wrote:
>>>>>> Hello All,
>>>>>>
>>>>>> I'm very new to python, so bear with me.
>>>>>>
>>>>>> I'd like to use python to do my image processing, but I'm running into
>>>>>> behavior that doesn't make
>>>>>> sense to me. I'm using Windows 7 Pro (64-bit) with 4 gigs of memory,
>>>>>> python 2.6.6, and the newest
>>>>>> versions of ipython, pyfits, matplotlib (1.0.1), numpy (1.5.1), scipy.
>>>>>> I'm loading in a fits file
>>>>>> that's 26 MB (~16 Mpixels). When I load my image in ImageJ, I can see
>>>>>> memory usage go up by 50MB,
>>>>>> but when I try displaying the image using imshow(), my memory usage goes
>>>>>> up by around 500MB, each
>>>>>> time. If I close the figure and replot it, imshow() crashes. I don't
>>>>>> know if I'm doing something
>>>>>> wrong, or if it's a new or known bug. I tried the same thing on Linux
>>>>>> and got the same result.
>>>>>> Here's a transcript.
>>>>>>
>>>>>> Welcome to pylab, a matplotlib-based Python environment.
>>>>>> For more information, type 'help(pylab)'.
>>>>>>
>>>>>> In [1]: import pyfits
>>>>>>
>>>>>> In [2]: from Tkinter import *
>>>>>>
>>>>>> In [3]: import tkFileDialog
>>>>>>
>>>>>> In [4]: image=pyfits.getdata(tkFileDialog.askopenfilename())
>>>>>>
>>>>>> In [5]: imshow(image)
>>>>>> Out[5]:<matplotlib.image.AxesImage object at 0x03BCA170>
>>>>>>
>>>>>> In [6]: close()
>>>>>>
>>>>>> In [7]: imshow(image,origin='lower')
>>>>>> Out[7]:<matplotlib.image.AxesImage object at 0x0440E170>
>>>>>>
>>>>>> In [8]: close()
>>>>>>
>>>>>> In [9]: imshow(image[100:3600,100:3600],origin='lower')
>>>>>> Out[9]:<matplotlib.image.AxesImage object at 0x045D9FB0>
>>>>>>
>>>>>> In [10]: Exception in Tkinter callback
>>>>>> Traceback (most recent call last):
>>>>>> File "C:\app\Python2.6\lib\lib-tk\Tkinter.py", line 1410, in
>>>>>> __call__
>>>>>> return self.func(*args)
>>>>>> File "C:\app\Python2.6\lib\lib-tk\Tkinter.py", line 495, in
>>>>>> callit
>>>>>> func(*args)
>>>>>> File
>>>>>> "C:\app\Python2.6\lib\site-packages\matplotlib\backends\backend_tkagg.py",
>>>>>> line 263, in
>>>>>> idle_draw
>>>>>> self.draw()
>>>>>> File
>>>>>> "C:\app\Python2.6\lib\site-packages\matplotlib\backends\backend_tkagg.py",
>>>>>> line 248, in draw
>>>>>> FigureCanvasAgg.draw(self)
>>>>>> File
>>>>>> "C:\app\Python2.6\lib\site-packages\matplotlib\backends\backend_agg.py",
>>>>>> line 394, in draw
>>>>>> self.figure.draw(self.renderer)
>>>>>> File "C:\app\Python2.6\lib\site-packages\matplotlib\artist.py",
>>>>>> line 55, in draw_wrapper
>>>>>> draw(artist, renderer, *args, **kwargs)
>>>>>> File "C:\app\Python2.6\lib\site-packages\matplotlib\figure.py",
>>>>>> line 798, in draw
>>>>>> func(*args)
>>>>>> File "C:\app\Python2.6\lib\site-packages\matplotlib\artist.py",
>>>>>> line 55, in draw_wrapper
>>>>>> draw(artist, renderer, *args, **kwargs)
>>>>>> File "C:\app\Python2.6\lib\site-packages\matplotlib\axes.py",
>>>>>> line 1946, in draw
>>>>>> a.draw(renderer)
>>>>>> File "C:\app\Python2.6\lib\site-packages\matplotlib\artist.py",
>>>>>> line 55, in draw_wrapper
>>>>>> draw(artist, renderer, *args, **kwargs)
>>>>>> File "C:\app\Python2.6\lib\site-packages\matplotlib\image.py",
>>>>>> line 354, in draw
>>>>>> im = self.make_image(renderer.get_image_magnification())
>>>>>> File "C:\app\Python2.6\lib\site-packages\matplotlib\image.py",
>>>>>> line 569, in make_image
>>>>>> transformed_viewLim)
>>>>>> File "C:\app\Python2.6\lib\site-packages\matplotlib\image.py",
>>>>>> line 201, in _get_unsampled_image
>>>>>> x = self.to_rgba(self._A, self._alpha)
>>>>>> File "C:\app\Python2.6\lib\site-packages\matplotlib\cm.py", line
>>>>>> 193, in to_rgba
>>>>>> x = self.norm(x)
>>>>>> File "C:\app\Python2.6\lib\site-packages\matplotlib\colors.py",
>>>>>> line 820, in __call__
>>>>>> result = (val-vmin) / (vmax-vmin)
>>>>>> File "C:\app\Python2.6\lib\site-packages\numpy\ma\core.py", line
>>>>>> 3673, in __div__
>>>>>> return divide(self, other)
>>>>>> File "C:\app\Python2.6\lib\site-packages\numpy\ma\core.py", line
>>>>>> 1077, in __call__
>>>>>> m |= filled(domain(da, db), True)
>>>>>> File "C:\app\Python2.6\lib\site-packages\numpy\ma\core.py", line
>>>>>> 772, in __call__
>>>>>> return umath.absolute(a) * self.tolerance>= umath.absolute(b)
>>>>>> MemoryError
>>>>>>
>>>>>>
>>>>>> Thanks for any help,
>>>>>> -robert
>>>>>>
>>>>>
>>>>> These are previous discussions on the issue:
>>>>>
>>>>> <http://www.mail-archive.com/matplotlib-users@lists.sourceforge.net/msg14727.html>
>>>>> <http://www.mail-archive.com/matplotlib-users@lists.sourceforge.net/msg19815.html>
>>>>> <http://www.mail-archive.com/matplotlib-users@lists.sourceforge.net/msg19614.html>
>>>>>
>>>>> Christoph
>>>>>
>>>> The first 2 discussions lead to suggestions of more memory on a 64-bit
>>>> installation, but that
>>>> doesn't seem like a great solution. I use other image processing software
>>>> (ImageJ and IDL) and
>>>> neither has any trouble with my images. As I mentioned ImageJ uses 1/10th
>>>> the memory for the same
>>>> display, or about 1 byte of memory for 1 byte of image. I think
>>>> matplotlib should aim for the same.
>>>> I also think it should free up memory when the image is closed, but
>>>> maybe I'm not doing the right
>>>> thing. Is there something else I should be doing to free up memory?
>>>>
>>>> Things are even worse with plot.
>>>>
>>>> I'll file a bug report as Benjamin suggests.
>>> Please file it in the "enhancement" category, not as a "bug". The
>>> difficulty that mpl has with images is the result of a design decision
>>> long ago; as Christoph notes, mpl works primarily with float64 rgba,
>>> which is horribly memory inefficient, but is very general and works fine
>>> so long as the image is not too large. uint8 is already used in some
>>> circumstances. I don't know how hard it would be to simply switch to
>>> uint8 for images, or to make that an option, perhaps for all color
>>> specification. It may involve getting into arcana of the extension code
>>> and the agg library, which is a daunting prospect for most of us,
>>> certainly for me. I agree entirely that a major reduction in memory
>>> usage would be good; patches, or a mpl branch to achieve that, are welcome.
>>>
>>> Eric
>> I'll put it in as an enhancement, but I'm still unsure if there is a bug in
>> there as well. Is there something I should be doing to clear memory after
>> the
>> first figure is closed other than close()? I don't understand why memory
>> usage
>> grows each time I replot, but I'm pretty sure it isn't desireable behavior.
>> As
>> I mentioned, this effect is worse with plot.
>>
>> So is this a bug or improper usage?
>>
>> As for how to implement a fix for memory usage, I'll let you folks figure
>> that
>> out. But it seems that if I want a grayscale image, I could reduce memory
>> usage
>> by 4 if matplotlib could turn rgba into intensity.
>>
>> -robert
>>
>> ------------------------------------------------------------------------------
>> Special Offer-- Download ArcSight Logger for FREE (a $49 USD value)!
>> Finally, a world-class log management solution at an even better price-free!
>> Download using promo code Free_Logger_4_Dev2Dev. Offer expires
>> February 28th, so secure your free ArcSight Logger TODAY!
>> http://p.sf.net/sfu/arcsight-sfd2d
>> _______________________________________________
>> Matplotlib-users mailing list
>> Matplotlib-users@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/matplotlib-users
>>
>
> ------------------------------------------------------------------------------
> Special Offer-- Download ArcSight Logger for FREE (a $49 USD value)!
> Finally, a world-class log management solution at an even better price-free!
> Download using promo code Free_Logger_4_Dev2Dev. Offer expires
> February 28th, so secure your free ArcSight Logger TODAY!
> http://p.sf.net/sfu/arcsight-sfd2d
> _______________________________________________
> Matplotlib-users mailing list
> Matplotlib-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/matplotlib-users
>
>
------------------------------------------------------------------------------
Special Offer-- Download ArcSight Logger for FREE (a $49 USD value)!
Finally, a world-class log management solution at an even better price-free!
Download using promo code Free_Logger_4_Dev2Dev. Offer expires
February 28th, so secure your free ArcSight Logger TODAY!
http://p.sf.net/sfu/arcsight-sfd2d
_______________________________________________
Matplotlib-users mailing list
Matplotlib-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/matplotlib-users
