On 2/2/2011 6:06 PM, Eric Firing wrote:
On 02/02/2011 03:08 PM, Robert Abiad wrote:On 2/2/2011 3:59 PM, Christoph Gohlke wrote:On 2/2/2011 3:33 PM, Robert Abiad wrote:Hello All, I'm very new to python, so bear with me. I'd like to use python to do my image processing, but I'm running into behavior that doesn't make sense to me. I'm using Windows 7 Pro (64-bit) with 4 gigs of memory, python 2.6.6, and the newest versions of ipython, pyfits, matplotlib (1.0.1), numpy (1.5.1), scipy. I'm loading in a fits file that's 26 MB (~16 Mpixels). When I load my image in ImageJ, I can see memory usage go up by 50MB, but when I try displaying the image using imshow(), my memory usage goes up by around 500MB, each time. If I close the figure and replot it, imshow() crashes. I don't know if I'm doing something wrong, or if it's a new or known bug. I tried the same thing on Linux and got the same result. Here's a transcript. Welcome to pylab, a matplotlib-based Python environment. For more information, type 'help(pylab)'. In [1]: import pyfits In [2]: from Tkinter import * In [3]: import tkFileDialog In [4]: image=pyfits.getdata(tkFileDialog.askopenfilename()) In [5]: imshow(image) Out[5]:<matplotlib.image.AxesImage object at 0x03BCA170> In [6]: close() In [7]: imshow(image,origin='lower') Out[7]:<matplotlib.image.AxesImage object at 0x0440E170> In [8]: close() In [9]: imshow(image[100:3600,100:3600],origin='lower') Out[9]:<matplotlib.image.AxesImage object at 0x045D9FB0> In [10]: Exception in Tkinter callback Traceback (most recent call last): File "C:\app\Python2.6\lib\lib-tk\Tkinter.py", line 1410, in __call__ return self.func(*args) File "C:\app\Python2.6\lib\lib-tk\Tkinter.py", line 495, in callit func(*args) File "C:\app\Python2.6\lib\site-packages\matplotlib\backends\backend_tkagg.py", line 263, in idle_draw self.draw() File "C:\app\Python2.6\lib\site-packages\matplotlib\backends\backend_tkagg.py", line 248, in draw FigureCanvasAgg.draw(self) File "C:\app\Python2.6\lib\site-packages\matplotlib\backends\backend_agg.py", line 394, in draw self.figure.draw(self.renderer) File "C:\app\Python2.6\lib\site-packages\matplotlib\artist.py", line 55, in draw_wrapper draw(artist, renderer, *args, **kwargs) File "C:\app\Python2.6\lib\site-packages\matplotlib\figure.py", line 798, in draw func(*args) File "C:\app\Python2.6\lib\site-packages\matplotlib\artist.py", line 55, in draw_wrapper draw(artist, renderer, *args, **kwargs) File "C:\app\Python2.6\lib\site-packages\matplotlib\axes.py", line 1946, in draw a.draw(renderer) File "C:\app\Python2.6\lib\site-packages\matplotlib\artist.py", line 55, in draw_wrapper draw(artist, renderer, *args, **kwargs) File "C:\app\Python2.6\lib\site-packages\matplotlib\image.py", line 354, in draw im = self.make_image(renderer.get_image_magnification()) File "C:\app\Python2.6\lib\site-packages\matplotlib\image.py", line 569, in make_image transformed_viewLim) File "C:\app\Python2.6\lib\site-packages\matplotlib\image.py", line 201, in _get_unsampled_image x = self.to_rgba(self._A, self._alpha) File "C:\app\Python2.6\lib\site-packages\matplotlib\cm.py", line 193, in to_rgba x = self.norm(x) File "C:\app\Python2.6\lib\site-packages\matplotlib\colors.py", line 820, in __call__ result = (val-vmin) / (vmax-vmin) File "C:\app\Python2.6\lib\site-packages\numpy\ma\core.py", line 3673, in __div__ return divide(self, other) File "C:\app\Python2.6\lib\site-packages\numpy\ma\core.py", line 1077, in __call__ m |= filled(domain(da, db), True) File "C:\app\Python2.6\lib\site-packages\numpy\ma\core.py", line 772, in __call__ return umath.absolute(a) * self.tolerance>= umath.absolute(b) MemoryError Thanks for any help, -robertThese are previous discussions on the issue: <http://www.mail-archive.com/matplotlib-users@lists.sourceforge.net/msg14727.html> <http://www.mail-archive.com/matplotlib-users@lists.sourceforge.net/msg19815.html> <http://www.mail-archive.com/matplotlib-users@lists.sourceforge.net/msg19614.html> ChristophThe first 2 discussions lead to suggestions of more memory on a 64-bit installation, but that doesn't seem like a great solution. I use other image processing software (ImageJ and IDL) and neither has any trouble with my images. As I mentioned ImageJ uses 1/10th the memory for the same display, or about 1 byte of memory for 1 byte of image. I think matplotlib should aim for the same. I also think it should free up memory when the image is closed, but maybe I'm not doing the right thing. Is there something else I should be doing to free up memory? Things are even worse with plot. I'll file a bug report as Benjamin suggests.Please file it in the "enhancement" category, not as a "bug". The difficulty that mpl has with images is the result of a design decision long ago; as Christoph notes, mpl works primarily with float64 rgba, which is horribly memory inefficient, but is very general and works fine so long as the image is not too large. uint8 is already used in some circumstances. I don't know how hard it would be to simply switch to uint8 for images, or to make that an option, perhaps for all color specification. It may involve getting into arcana of the extension code and the agg library, which is a daunting prospect for most of us, certainly for me. I agree entirely that a major reduction in memory usage would be good; patches, or a mpl branch to achieve that, are welcome. Eric
How about special-casing uint8 arrays, assuming that these don't need to be normalized? This gives the user the option to bypass matplotlib's internal conversions and saves around 450 MB for plotting one 4096x4096 uint8 image. A preliminary, mostly untested, patch is attached. Similar changes could be done for other signed and unsigned integer data.
Christoph
Index: cm.py
===================================================================
--- cm.py (revision 8944)
+++ cm.py (working copy)
@@ -210,7 +210,8 @@
except AttributeError:
pass
x = ma.asarray(x)
- x = self.norm(x)
+ if x.dtype != np.uint8:
+ x = self.norm(x)
x = self.cmap(x, alpha=alpha, bytes=bytes)
return x
Index: image.py
===================================================================
--- image.py (revision 8944)
+++ image.py (working copy)
@@ -193,8 +193,15 @@
self._oldyslice = yslice
if self._imcache is None:
- if self._A.dtype == np.uint8 and len(self._A.shape) == 3:
- im = _image.frombyte(self._A[yslice,xslice,:], 0)
+ if self._A.dtype == np.uint8:
+ if len(self._A.shape) == 3:
+ im = _image.frombyte(self._A[yslice,xslice,:], 0)
+ else:
+ if self._rgbacache is None:
+ x = self.to_rgba(self._A, self._alpha, True)
+ else:
+ x = self._rgbacache
+ im = _image.frombyte(x[yslice,xslice,:], 0)
im.is_grayscale = False
else:
if self._rgbacache is None:
------------------------------------------------------------------------------ Special Offer-- Download ArcSight Logger for FREE (a $49 USD value)! Finally, a world-class log management solution at an even better price-free! Download using promo code Free_Logger_4_Dev2Dev. Offer expires February 28th, so secure your free ArcSight Logger TODAY! http://p.sf.net/sfu/arcsight-sfd2d
_______________________________________________ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users
