Re: [Matplotlib-users] Trouble with imshow
On 02/02/2011 08:38 PM, Robert Abiad wrote: [...] I'll put it in as an enhancement, but I'm still unsure if there is a bug in there as well. Is there something I should be doing to clear memory after the first figure is closed other than close()? I don't understand why memory usage grows each time I replot, but I'm pretty sure it isn't desireable behavior. As I mentioned, this effect is worse with plot. So is this a bug or improper usage? I'm not quite sure, but I don't think there is a specifically matplotlib memory leak bug at work here. Are you using ipython, and if so, have you turned off the caching? In its default mode, ipython keeps lots of references, thereby keeping memory in use. Also, memory management and reporting can be a bit tricky and misleading. Nevertheless, the attached script may be illustrating the problem. Try running it from the command line as-is (maybe shorten the loop--it doesn't take 100 iterations to show the pattern) and then commenting out the line as indicated in the comment. It seems that if anything is done that adds ever so slightly to memory use while the figure is displayed, then when the figure is closed, its memory is not reused. I'm puzzled. Eric As for how to implement a fix for memory usage, I'll let you folks figure that out. But it seems that if I want a grayscale image, I could reduce memory usage by 4 if matplotlib could turn rgba into intensity. -robert import time import numpy as np import matplotlib.pyplot as plt from matplotlib.cbook import report_memory plt.ion() stuff = [] for i in range(100): z = np.random.rand(2000, 2000) plt.imshow(z) plt.draw() time.sleep(0.5) plt.close('all') # Comment the following line, and the memory use is stable. stuff.append(z[0]) print report_memory() -- Special Offer-- Download ArcSight Logger for FREE (a $49 USD value)! Finally, a world-class log management solution at an even better price-free! Download using promo code Free_Logger_4_Dev2Dev. Offer expires February 28th, so secure your free ArcSight Logger TODAY! http://p.sf.net/sfu/arcsight-sfd2d___ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users
Re: [Matplotlib-users] Trouble with imshow
As a workaround you could do your own normalization and color mapping and pass a uint8 RGB image to imshow. That avoids matplotlib's norm function. The following example saves almost 500 MB for plotting a 16 MB uint8 greyscale image, as compared to passing img directly to imshow: import numpy from matplotlib import pyplot img = numpy.random.randint(0, 255, (4096, 4096)).astype('uint8') lut = pyplot.cm.gray(numpy.arange(255), bytes=True) rgb = lut.take(img, axis=0) del img pyplot.imshow(rgb) pyplot.show() Christoph On 2/2/2011 11:00 PM, gary ruben wrote: Christoph, if you're looking at special casing uint8's, you might want to keep in mind that uint16 greyscale images are also quite common as camera outputs in experimental setups. I think that the solution to this should ideally minimise memory usage for any greyscale image, be it uint8, uint16, float32 or float64. i.e. avoiding conversion to RGBA for any single-plane 2D array type would be best IMHO, Gary R. On Thu, Feb 3, 2011 at 5:38 PM, Robert Abiadab...@ssl.berkeley.edu wrote: On 2/2/2011 6:06 PM, Eric Firing wrote: On 02/02/2011 03:08 PM, Robert Abiad wrote: On 2/2/2011 3:59 PM, Christoph Gohlke wrote: On 2/2/2011 3:33 PM, Robert Abiad wrote: Hello All, I'm very new to python, so bear with me. I'd like to use python to do my image processing, but I'm running into behavior that doesn't make sense to me. I'm using Windows 7 Pro (64-bit) with 4 gigs of memory, python 2.6.6, and the newest versions of ipython, pyfits, matplotlib (1.0.1), numpy (1.5.1), scipy. I'm loading in a fits file that's 26 MB (~16 Mpixels). When I load my image in ImageJ, I can see memory usage go up by 50MB, but when I try displaying the image using imshow(), my memory usage goes up by around 500MB, each time. If I close the figure and replot it, imshow() crashes. I don't know if I'm doing something wrong, or if it's a new or known bug. I tried the same thing on Linux and got the same result. Here's a transcript. Welcome to pylab, a matplotlib-based Python environment. For more information, type 'help(pylab)'. In [1]: import pyfits In [2]: from Tkinter import * In [3]: import tkFileDialog In [4]: image=pyfits.getdata(tkFileDialog.askopenfilename()) In [5]: imshow(image) Out[5]:matplotlib.image.AxesImage object at 0x03BCA170 In [6]: close() In [7]: imshow(image,origin='lower') Out[7]:matplotlib.image.AxesImage object at 0x0440E170 In [8]: close() In [9]: imshow(image[100:3600,100:3600],origin='lower') Out[9]:matplotlib.image.AxesImage object at 0x045D9FB0 In [10]: Exception in Tkinter callback Traceback (most recent call last): File C:\app\Python2.6\lib\lib-tk\Tkinter.py, line 1410, in __call__ return self.func(*args) File C:\app\Python2.6\lib\lib-tk\Tkinter.py, line 495, in callit func(*args) File C:\app\Python2.6\lib\site-packages\matplotlib\backends\backend_tkagg.py, line 263, in idle_draw self.draw() File C:\app\Python2.6\lib\site-packages\matplotlib\backends\backend_tkagg.py, line 248, in draw FigureCanvasAgg.draw(self) File C:\app\Python2.6\lib\site-packages\matplotlib\backends\backend_agg.py, line 394, in draw self.figure.draw(self.renderer) File C:\app\Python2.6\lib\site-packages\matplotlib\artist.py, line 55, in draw_wrapper draw(artist, renderer, *args, **kwargs) File C:\app\Python2.6\lib\site-packages\matplotlib\figure.py, line 798, in draw func(*args) File C:\app\Python2.6\lib\site-packages\matplotlib\artist.py, line 55, in draw_wrapper draw(artist, renderer, *args, **kwargs) File C:\app\Python2.6\lib\site-packages\matplotlib\axes.py, line 1946, in draw a.draw(renderer) File C:\app\Python2.6\lib\site-packages\matplotlib\artist.py, line 55, in draw_wrapper draw(artist, renderer, *args, **kwargs) File C:\app\Python2.6\lib\site-packages\matplotlib\image.py, line 354, in draw im = self.make_image(renderer.get_image_magnification()) File C:\app\Python2.6\lib\site-packages\matplotlib\image.py, line 569, in make_image transformed_viewLim) File C:\app\Python2.6\lib\site-packages\matplotlib\image.py, line 201, in _get_unsampled_image x = self.to_rgba(self._A, self._alpha) File C:\app\Python2.6\lib\site-packages\matplotlib\cm.py, line 193, in to_rgba x = self.norm(x) File C:\app\Python2.6\lib\site-packages\matplotlib\colors.py, line 820, in __call__ result = (val-vmin) / (vmax-vmin) File C:\app\Python2.6\lib\site-packages\numpy\ma\core.py, line 3673, in __div__ return divide(self, other) File C:\app\Python2.6\lib\site-packages\numpy\ma\core.py, line 1077, in __call__ m |= filled(domain(da, db), True)
Re: [Matplotlib-users] 3D Data to 2d Plots
2011/2/3 Mike Alger mal...@ryerson.ca Sorry I was out of touch for a while I have been busy with other things, You would have to do some sort of a bin solution with the method I suggested. So m/z values would not have to be exact but you would group ranges of them together. To be honest based on the plots you showed in your initial question I am surprised you don’t already have the data in a 2d array already. An example of the data in the format you intend to start with would have really helped explain the situation. Matplotlib has a plot module that bins things automatically for you based on the data http://matplotlib.sourceforge.net/examples/api/histogram_demo.html or you can use numpy.historgram function directly. If you are doing something like that already to compute your m/z values, just make sure use the same sequence for your bins (see examples at http://docs.scipy.org/doc/numpy/reference/generated/numpy.histogram.html) and presto you have the makings of a perfectly aligned 2d array that can be plotted with pcolor or with 3d surface Again I hope this helps ok, thanks. i thought there was a accurate solution, since i figured that the amount of data points is so low that even lowering the resolution wouldn’t help. but then i remembered that somebody told me that binning would most likely be the only solution and now your mail. i’ll certainly manage to do that, then; thanks again! -- Special Offer-- Download ArcSight Logger for FREE (a $49 USD value)! Finally, a world-class log management solution at an even better price-free! Download using promo code Free_Logger_4_Dev2Dev. Offer expires February 28th, so secure your free ArcSight Logger TODAY! http://p.sf.net/sfu/arcsight-sfd2d___ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users
[Matplotlib-users] description: unknown (?!?!?)
In many places in the mpl docs there are tables of supported kwarg properties containing at least one (usually many) entries where the description given for the property is simply unknown. What's up with that??? How can the description of a property be unknown??? Any clarification for what this unknown notation means would be appreciated. Thanks, G -- Special Offer-- Download ArcSight Logger for FREE (a $49 USD value)! Finally, a world-class log management solution at an even better price-free! Download using promo code Free_Logger_4_Dev2Dev. Offer expires February 28th, so secure your free ArcSight Logger TODAY! http://p.sf.net/sfu/arcsight-sfd2d___ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users
[Matplotlib-users] Help with imshow() extent.
Hello I am using matplotlib 1.0.0 version. and would like to use the imshow() to display my 2D image [512 X 1024] so when I use the imshow() command as imshow(array,origin='lower',interpolation='bilinear') xlables - [0,100,200,400,500] ylabels - [0,200,400,600,800,1000] which is correct as by default the imshow will set the grid number as the labels. Now, physical size of this grid is [52 X 152] so to get that values, I use the following command imshow(array,origin='lower',interpolation='bilinear',extend=[0.0,52.0,0.0,152.0]) and the image that I obtain is not correct. By not correct I mean that the important features in the image are not at the positions where I expect them to be while transforming from [512 X 1024] grid to [52 X 152] physical grid. I presume this could be due to the fact that the extent just divides the axis in uniform grid ... however my grid is not uniform. But I do know the pixel width (dx and dy) at each position. I have tried to use pcolor but somehow the pcolor() is too slow for such a large grid. Can anyone help me in getting the X, and Y values correct on the axes , using imshow() and complete grid information, when I transform from [512 X 1024] grid to [52 X 152] physical grid. Cheers Bhargav Vaidya. -- Special Offer-- Download ArcSight Logger for FREE (a $49 USD value)! Finally, a world-class log management solution at an even better price-free! Download using promo code Free_Logger_4_Dev2Dev. Offer expires February 28th, so secure your free ArcSight Logger TODAY! http://p.sf.net/sfu/arcsight-sfd2d ___ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users
[Matplotlib-users] latex ' (prime)
Observe the following image: import pylab pylab.plot([0,1],[1,2],label=$\sigma'_0$) pylab.legend() pylab.savefig('test.png') Notice that the \prime introduced by the single quote in the legend is not raised above the \sigma, like it would be in TeX (i.e., in TeX, $\sigma'_0$ is equivalent to $\sigma^\prime_0$, IIRC). Is this a design decision, or is it easy to fix? This is with matplotlib 1.0.0. Thanks, Jason -- Special Offer-- Download ArcSight Logger for FREE (a $49 USD value)! Finally, a world-class log management solution at an even better price-free! Download using promo code Free_Logger_4_Dev2Dev. Offer expires February 28th, so secure your free ArcSight Logger TODAY! http://p.sf.net/sfu/arcsight-sfd2d ___ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users
[Matplotlib-users] Is it possible to plot axes with arrows ?
Hello, I use matplolib by the mathematical system Sage in order to plot a function. The Sage code calls matplotlib and uses its options : The Sage command is plot (sin, x, -5, 5) I add labels par axes_labels or remove axes by : plot (sin(x), x, -5, 5, axes_label = ['x', 'y']) plot (sin(x), x, -5, 5, axes=false) French users (and maybe others) uses arrows and not lines for axes. I'm looking for a plot (sin(x), x, -5, 5, axes=arrows) Is there a pretty way to get these arrows. The result of this code isn't so fine. length, width and color don't match. plot (sin(x), x, -5, 5, axes=false) + arrow ((-5,0),(5,0)) + arrow ((0,-1),(0,1)) What options do you propose ? I don't find relevant answers in the archive. F. from France. -- Special Offer-- Download ArcSight Logger for FREE (a $49 USD value)! Finally, a world-class log management solution at an even better price-free! Download using promo code Free_Logger_4_Dev2Dev. Offer expires February 28th, so secure your free ArcSight Logger TODAY! http://p.sf.net/sfu/arcsight-sfd2d ___ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users
[Matplotlib-users] Line2D: drawstyle + pick_event
'pick_event' fires when I pick on either the marker or the line segment of a Line2D object. But if I change the drawstyle to use steps (steps-pre, steps-post, steps-mid), picking on the line segment is broken. It still seems to think the line is linear. This leads to very strange behavior where picking on a line gives no feedback (e.g. a tooltip), but picking on a blank part of the graph does. Do I need to write my own custom picker function or is this an oversight in the code? Thanks, Jim -- Special Offer-- Download ArcSight Logger for FREE (a $49 USD value)! Finally, a world-class log management solution at an even better price-free! Download using promo code Free_Logger_4_Dev2Dev. Offer expires February 28th, so secure your free ArcSight Logger TODAY! http://p.sf.net/sfu/arcsight-sfd2d___ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users
Re: [Matplotlib-users] Trouble with imshow
On 02/02/2011 10:17 PM, Eric Firing wrote: On 02/02/2011 08:38 PM, Robert Abiad wrote: [...] I'll put it in as an enhancement, but I'm still unsure if there is a bug in there as well. Is there something I should be doing to clear memory after the first figure is closed other than close()? I don't understand why memory usage grows each time I replot, but I'm pretty sure it isn't desireable behavior. As I mentioned, this effect is worse with plot. So is this a bug or improper usage? I'm not quite sure, but I don't think there is a specifically matplotlib memory leak bug at work here. Are you using ipython, and if so, have you turned off the caching? In its default mode, ipython keeps lots of references, thereby keeping memory in use. Also, memory management and reporting can be a bit tricky and misleading. Nevertheless, the attached script may be illustrating the problem. Try running it from the command line as-is (maybe shorten the loop--it doesn't take 100 iterations to show the pattern) and then commenting out the line as indicated in the comment. It seems that if anything is done that adds ever so slightly to memory use while the figure is displayed, then when the figure is closed, its memory is not reused. I'm puzzled. I wasn't thinking straight--there is no mystery and no memory leak. Ignore my example script referred to above. It was saving rows of the z array, not single elements as I had intended, so of course memory use was growing substantially. Eric Eric As for how to implement a fix for memory usage, I'll let you folks figure that out. But it seems that if I want a grayscale image, I could reduce memory usage by 4 if matplotlib could turn rgba into intensity. -robert -- Special Offer-- Download ArcSight Logger for FREE (a $49 USD value)! Finally, a world-class log management solution at an even better price-free! Download using promo code Free_Logger_4_Dev2Dev. Offer expires February 28th, so secure your free ArcSight Logger TODAY! http://p.sf.net/sfu/arcsight-sfd2d ___ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users -- Special Offer-- Download ArcSight Logger for FREE (a $49 USD value)! Finally, a world-class log management solution at an even better price-free! Download using promo code Free_Logger_4_Dev2Dev. Offer expires February 28th, so secure your free ArcSight Logger TODAY! http://p.sf.net/sfu/arcsight-sfd2d ___ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users
Re: [Matplotlib-users] Trouble with imshow
On 2/3/2011 10:06 AM, Eric Firing wrote: On 02/02/2011 10:17 PM, Eric Firing wrote: On 02/02/2011 08:38 PM, Robert Abiad wrote: [...] I'll put it in as an enhancement, but I'm still unsure if there is a bug in there as well. Is there something I should be doing to clear memory after the first figure is closed other than close()? I don't understand why memory usage grows each time I replot, but I'm pretty sure it isn't desireable behavior. As I mentioned, this effect is worse with plot. So is this a bug or improper usage? I'm not quite sure, but I don't think there is a specifically matplotlib memory leak bug at work here. Are you using ipython, and if so, have you turned off the caching? In its default mode, ipython keeps lots of references, thereby keeping memory in use. Also, memory management and reporting can be a bit tricky and misleading. Nevertheless, the attached script may be illustrating the problem. Try running it from the command line as-is (maybe shorten the loop--it doesn't take 100 iterations to show the pattern) and then commenting out the line as indicated in the comment. It seems that if anything is done that adds ever so slightly to memory use while the figure is displayed, then when the figure is closed, its memory is not reused. I'm puzzled. I wasn't thinking straight--there is no mystery and no memory leak. Ignore my example script referred to above. It was saving rows of the z array, not single elements as I had intended, so of course memory use was growing substantially. Eric You may not see a memory leak, but I still can't get my memory back without killing python. I turned off the ipython caching and even ran without iPython on both Windows and Ubuntu, but when I use imshow(), followed by close('all') and another imshow(), I run out of memory. I can see from the OS that the memory does not come back after close() and that it grows after the second imshow(). Any other ideas? Looks like a bug to me otherwise. -robert -- The modern datacenter depends on network connectivity to access resources and provide services. The best practices for maximizing a physical server's connectivity to a physical network are well understood - see how these rules translate into the virtual world? http://p.sf.net/sfu/oracle-sfdevnlfb ___ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users
Re: [Matplotlib-users] Trouble with imshow
On 02/03/2011 11:30 AM, Robert Abiad wrote: On 2/3/2011 10:06 AM, Eric Firing wrote: On 02/02/2011 10:17 PM, Eric Firing wrote: On 02/02/2011 08:38 PM, Robert Abiad wrote: [...] I'll put it in as an enhancement, but I'm still unsure if there is a bug in there as well. Is there something I should be doing to clear memory after the first figure is closed other than close()? I don't understand why memory usage grows each time I replot, but I'm pretty sure it isn't desireable behavior. As I mentioned, this effect is worse with plot. So is this a bug or improper usage? I'm not quite sure, but I don't think there is a specifically matplotlib memory leak bug at work here. Are you using ipython, and if so, have you turned off the caching? In its default mode, ipython keeps lots of references, thereby keeping memory in use. Also, memory management and reporting can be a bit tricky and misleading. Nevertheless, the attached script may be illustrating the problem. Try running it from the command line as-is (maybe shorten the loop--it doesn't take 100 iterations to show the pattern) and then commenting out the line as indicated in the comment. It seems that if anything is done that adds ever so slightly to memory use while the figure is displayed, then when the figure is closed, its memory is not reused. I'm puzzled. I wasn't thinking straight--there is no mystery and no memory leak. Ignore my example script referred to above. It was saving rows of the z array, not single elements as I had intended, so of course memory use was growing substantially. Eric You may not see a memory leak, but I still can't get my memory back without killing python. I turned off the ipython caching and even ran without iPython on both Windows and Ubuntu, but when I use imshow(), followed by close('all') and another imshow(), I run out of memory. I can see from the OS that the memory does not come back after close() and that it grows after the second imshow(). Any other ideas? Looks like a bug to me otherwise. Except that I tried the same things and did not get quite the same result. Let's track this down. Please try the attached script, and see if the memory usage grows substantially, or just oscillates a bit. Eric import time import numpy as np import matplotlib.pyplot as plt from matplotlib.cbook import report_memory plt.ion() mem = [] for i in range(20): z = np.random.rand(2000, 2000) plt.imshow(z) #plt.draw() #time.sleep(0.5) plt.savefig(test.png) plt.close('all') m = report_memory() mem.append(m) print m -- The modern datacenter depends on network connectivity to access resources and provide services. The best practices for maximizing a physical server's connectivity to a physical network are well understood - see how these rules translate into the virtual world? http://p.sf.net/sfu/oracle-sfdevnlfb___ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users
Re: [Matplotlib-users] Trouble with imshow
On 2/3/2011 2:15 PM, Eric Firing wrote: On 02/03/2011 11:30 AM, Robert Abiad wrote: On 2/3/2011 10:06 AM, Eric Firing wrote: On 02/02/2011 10:17 PM, Eric Firing wrote: On 02/02/2011 08:38 PM, Robert Abiad wrote: [...] I'll put it in as an enhancement, but I'm still unsure if there is a bug in there as well. Is there something I should be doing to clear memory after the first figure is closed other than close()? I don't understand why memory usage grows each time I replot, but I'm pretty sure it isn't desireable behavior. As I mentioned, this effect is worse with plot. So is this a bug or improper usage? I'm not quite sure, but I don't think there is a specifically matplotlib memory leak bug at work here. Are you using ipython, and if so, have you turned off the caching? In its default mode, ipython keeps lots of references, thereby keeping memory in use. Also, memory management and reporting can be a bit tricky and misleading. Nevertheless, the attached script may be illustrating the problem. Try running it from the command line as-is (maybe shorten the loop--it doesn't take 100 iterations to show the pattern) and then commenting out the line as indicated in the comment. It seems that if anything is done that adds ever so slightly to memory use while the figure is displayed, then when the figure is closed, its memory is not reused. I'm puzzled. I wasn't thinking straight--there is no mystery and no memory leak. Ignore my example script referred to above. It was saving rows of the z array, not single elements as I had intended, so of course memory use was growing substantially. Eric You may not see a memory leak, but I still can't get my memory back without killing python. I turned off the ipython caching and even ran without iPython on both Windows and Ubuntu, but when I use imshow(), followed by close('all') and another imshow(), I run out of memory. I can see from the OS that the memory does not come back after close() and that it grows after the second imshow(). Any other ideas? Looks like a bug to me otherwise. Except that I tried the same things and did not get quite the same result. Let's track this down. Please try the attached script, and see if the memory usage grows substantially, or just oscillates a bit. Eric One thing I noticed is that if I add a def __del__(self): print 'del' to image._AxesImageBase, it never gets called. _AxesImageBase keeps float64 and uint8 rgba images in a cache, which is never freed. Christoph -- The modern datacenter depends on network connectivity to access resources and provide services. The best practices for maximizing a physical server's connectivity to a physical network are well understood - see how these rules translate into the virtual world? http://p.sf.net/sfu/oracle-sfdevnlfb ___ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users
Re: [Matplotlib-users] Trouble with imshow
On 02/03/2011 12:28 PM, Christoph Gohlke wrote: On 2/3/2011 2:15 PM, Eric Firing wrote: On 02/03/2011 11:30 AM, Robert Abiad wrote: On 2/3/2011 10:06 AM, Eric Firing wrote: On 02/02/2011 10:17 PM, Eric Firing wrote: On 02/02/2011 08:38 PM, Robert Abiad wrote: [...] I'll put it in as an enhancement, but I'm still unsure if there is a bug in there as well. Is there something I should be doing to clear memory after the first figure is closed other than close()? I don't understand why memory usage grows each time I replot, but I'm pretty sure it isn't desireable behavior. As I mentioned, this effect is worse with plot. So is this a bug or improper usage? I'm not quite sure, but I don't think there is a specifically matplotlib memory leak bug at work here. Are you using ipython, and if so, have you turned off the caching? In its default mode, ipython keeps lots of references, thereby keeping memory in use. Also, memory management and reporting can be a bit tricky and misleading. Nevertheless, the attached script may be illustrating the problem. Try running it from the command line as-is (maybe shorten the loop--it doesn't take 100 iterations to show the pattern) and then commenting out the line as indicated in the comment. It seems that if anything is done that adds ever so slightly to memory use while the figure is displayed, then when the figure is closed, its memory is not reused. I'm puzzled. I wasn't thinking straight--there is no mystery and no memory leak. Ignore my example script referred to above. It was saving rows of the z array, not single elements as I had intended, so of course memory use was growing substantially. Eric You may not see a memory leak, but I still can't get my memory back without killing python. I turned off the ipython caching and even ran without iPython on both Windows and Ubuntu, but when I use imshow(), followed by close('all') and another imshow(), I run out of memory. I can see from the OS that the memory does not come back after close() and that it grows after the second imshow(). Any other ideas? Looks like a bug to me otherwise. Except that I tried the same things and did not get quite the same result. Let's track this down. Please try the attached script, and see if the memory usage grows substantially, or just oscillates a bit. Eric One thing I noticed is that if I add a def __del__(self): print 'del' to image._AxesImageBase, it never gets called. _AxesImageBase keeps float64 and uint8 rgba images in a cache, which is never freed. Adding a __del__ method defeats (or blocks) the garbage collection. Since self._imcache is an instance attribute, when the instance is no longer referenced, it should get garbage-collected, provided there is no __del__ method. Eric Christoph -- The modern datacenter depends on network connectivity to access resources and provide services. The best practices for maximizing a physical server's connectivity to a physical network are well understood - see how these rules translate into the virtual world? http://p.sf.net/sfu/oracle-sfdevnlfb ___ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users -- The modern datacenter depends on network connectivity to access resources and provide services. The best practices for maximizing a physical server's connectivity to a physical network are well understood - see how these rules translate into the virtual world? http://p.sf.net/sfu/oracle-sfdevnlfb ___ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users
Re: [Matplotlib-users] Trouble with imshow
On 2/3/2011 2:44 PM, Eric Firing wrote: On 02/03/2011 12:28 PM, Christoph Gohlke wrote: On 2/3/2011 2:15 PM, Eric Firing wrote: On 02/03/2011 11:30 AM, Robert Abiad wrote: On 2/3/2011 10:06 AM, Eric Firing wrote: On 02/02/2011 10:17 PM, Eric Firing wrote: On 02/02/2011 08:38 PM, Robert Abiad wrote: [...] I'll put it in as an enhancement, but I'm still unsure if there is a bug in there as well. Is there something I should be doing to clear memory after the first figure is closed other than close()? I don't understand why memory usage grows each time I replot, but I'm pretty sure it isn't desireable behavior. As I mentioned, this effect is worse with plot. So is this a bug or improper usage? I'm not quite sure, but I don't think there is a specifically matplotlib memory leak bug at work here. Are you using ipython, and if so, have you turned off the caching? In its default mode, ipython keeps lots of references, thereby keeping memory in use. Also, memory management and reporting can be a bit tricky and misleading. Nevertheless, the attached script may be illustrating the problem. Try running it from the command line as-is (maybe shorten the loop--it doesn't take 100 iterations to show the pattern) and then commenting out the line as indicated in the comment. It seems that if anything is done that adds ever so slightly to memory use while the figure is displayed, then when the figure is closed, its memory is not reused. I'm puzzled. I wasn't thinking straight--there is no mystery and no memory leak. Ignore my example script referred to above. It was saving rows of the z array, not single elements as I had intended, so of course memory use was growing substantially. Eric You may not see a memory leak, but I still can't get my memory back without killing python. I turned off the ipython caching and even ran without iPython on both Windows and Ubuntu, but when I use imshow(), followed by close('all') and another imshow(), I run out of memory. I can see from the OS that the memory does not come back after close() and that it grows after the second imshow(). Any other ideas? Looks like a bug to me otherwise. Except that I tried the same things and did not get quite the same result. Let's track this down. Please try the attached script, and see if the memory usage grows substantially, or just oscillates a bit. Eric One thing I noticed is that if I add a def __del__(self): print 'del' to image._AxesImageBase, it never gets called. _AxesImageBase keeps float64 and uint8 rgba images in a cache, which is never freed. Adding a __del__ method defeats (or blocks) the garbage collection. Sorry, never heard of that. I thought __del__() is called when the reference count reaches 0. Christoph Since self._imcache is an instance attribute, when the instance is no longer referenced, it should get garbage-collected, provided there is no __del__ method. Eric Christoph -- The modern datacenter depends on network connectivity to access resources and provide services. The best practices for maximizing a physical server's connectivity to a physical network are well understood - see how these rules translate into the virtual world? http://p.sf.net/sfu/oracle-sfdevnlfb ___ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users -- The modern datacenter depends on network connectivity to access resources and provide services. The best practices for maximizing a physical server's connectivity to a physical network are well understood - see how these rules translate into the virtual world? http://p.sf.net/sfu/oracle-sfdevnlfb ___ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users -- The modern datacenter depends on network connectivity to access resources and provide services. The best practices for maximizing a physical server's connectivity to a physical network are well understood - see how these rules translate into the virtual world? http://p.sf.net/sfu/oracle-sfdevnlfb ___ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users
Re: [Matplotlib-users] Trouble with imshow
On 02/03/2011 01:02 PM, Christoph Gohlke wrote: On 2/3/2011 2:44 PM, Eric Firing wrote: On 02/03/2011 12:28 PM, Christoph Gohlke wrote: On 2/3/2011 2:15 PM, Eric Firing wrote: On 02/03/2011 11:30 AM, Robert Abiad wrote: On 2/3/2011 10:06 AM, Eric Firing wrote: On 02/02/2011 10:17 PM, Eric Firing wrote: On 02/02/2011 08:38 PM, Robert Abiad wrote: [...] I'll put it in as an enhancement, but I'm still unsure if there is a bug in there as well. Is there something I should be doing to clear memory after the first figure is closed other than close()? I don't understand why memory usage grows each time I replot, but I'm pretty sure it isn't desireable behavior. As I mentioned, this effect is worse with plot. So is this a bug or improper usage? I'm not quite sure, but I don't think there is a specifically matplotlib memory leak bug at work here. Are you using ipython, and if so, have you turned off the caching? In its default mode, ipython keeps lots of references, thereby keeping memory in use. Also, memory management and reporting can be a bit tricky and misleading. Nevertheless, the attached script may be illustrating the problem. Try running it from the command line as-is (maybe shorten the loop--it doesn't take 100 iterations to show the pattern) and then commenting out the line as indicated in the comment. It seems that if anything is done that adds ever so slightly to memory use while the figure is displayed, then when the figure is closed, its memory is not reused. I'm puzzled. I wasn't thinking straight--there is no mystery and no memory leak. Ignore my example script referred to above. It was saving rows of the z array, not single elements as I had intended, so of course memory use was growing substantially. Eric You may not see a memory leak, but I still can't get my memory back without killing python. I turned off the ipython caching and even ran without iPython on both Windows and Ubuntu, but when I use imshow(), followed by close('all') and another imshow(), I run out of memory. I can see from the OS that the memory does not come back after close() and that it grows after the second imshow(). Any other ideas? Looks like a bug to me otherwise. Except that I tried the same things and did not get quite the same result. Let's track this down. Please try the attached script, and see if the memory usage grows substantially, or just oscillates a bit. Eric One thing I noticed is that if I add a def __del__(self): print 'del' to image._AxesImageBase, it never gets called. _AxesImageBase keeps float64 and uint8 rgba images in a cache, which is never freed. Adding a __del__ method defeats (or blocks) the garbage collection. Sorry, never heard of that. I thought __del__() is called when the reference count reaches 0. It is, but if there are circular reference chains (cycles--and mpl is full of them) then the garbage collector has to identify them and remove them. If it encounters a __del__ it stops and leaves that cycle alone. Eric Christoph Since self._imcache is an instance attribute, when the instance is no longer referenced, it should get garbage-collected, provided there is no __del__ method. Eric Christoph -- The modern datacenter depends on network connectivity to access resources and provide services. The best practices for maximizing a physical server's connectivity to a physical network are well understood - see how these rules translate into the virtual world? http://p.sf.net/sfu/oracle-sfdevnlfb ___ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users
Re: [Matplotlib-users] Trouble with imshow
On 2/3/2011 3:13 PM, Eric Firing wrote: On 02/03/2011 01:02 PM, Christoph Gohlke wrote: On 2/3/2011 2:44 PM, Eric Firing wrote: On 02/03/2011 12:28 PM, Christoph Gohlke wrote: On 2/3/2011 2:15 PM, Eric Firing wrote: On 02/03/2011 11:30 AM, Robert Abiad wrote: On 2/3/2011 10:06 AM, Eric Firing wrote: On 02/02/2011 10:17 PM, Eric Firing wrote: On 02/02/2011 08:38 PM, Robert Abiad wrote: [...] I'll put it in as an enhancement, but I'm still unsure if there is a bug in there as well. Is there something I should be doing to clear memory after the first figure is closed other than close()? I don't understand why memory usage grows each time I replot, but I'm pretty sure it isn't desireable behavior. As I mentioned, this effect is worse with plot. So is this a bug or improper usage? I'm not quite sure, but I don't think there is a specifically matplotlib memory leak bug at work here. Are you using ipython, and if so, have you turned off the caching? In its default mode, ipython keeps lots of references, thereby keeping memory in use. Also, memory management and reporting can be a bit tricky and misleading. Nevertheless, the attached script may be illustrating the problem. Try running it from the command line as-is (maybe shorten the loop--it doesn't take 100 iterations to show the pattern) and then commenting out the line as indicated in the comment. It seems that if anything is done that adds ever so slightly to memory use while the figure is displayed, then when the figure is closed, its memory is not reused. I'm puzzled. I wasn't thinking straight--there is no mystery and no memory leak. Ignore my example script referred to above. It was saving rows of the z array, not single elements as I had intended, so of course memory use was growing substantially. Eric You may not see a memory leak, but I still can't get my memory back without killing python. I turned off the ipython caching and even ran without iPython on both Windows and Ubuntu, but when I use imshow(), followed by close('all') and another imshow(), I run out of memory. I can see from the OS that the memory does not come back after close() and that it grows after the second imshow(). Any other ideas? Looks like a bug to me otherwise. Except that I tried the same things and did not get quite the same result. Let's track this down. Please try the attached script, and see if the memory usage grows substantially, or just oscillates a bit. Eric One thing I noticed is that if I add a def __del__(self): print 'del' to image._AxesImageBase, it never gets called. _AxesImageBase keeps float64 and uint8 rgba images in a cache, which is never freed. Adding a __del__ method defeats (or blocks) the garbage collection. Sorry, never heard of that. I thought __del__() is called when the reference count reaches 0. It is, but if there are circular reference chains (cycles--and mpl is full of them) then the garbage collector has to identify them and remove them. If it encounters a __del__ it stops and leaves that cycle alone. My understanding is that if there is a circular reference then the refcount will not be zero anyway. In this case _AxesImageBase instances and their image caches will never be deleted by the gc (__del__ method present or not) unless the circle is broken. When the interpreter quits things are deleted by other means. I don't know matplotlib code good enough to fix this and will instead work on reducing the memory overhead needed to plot an image. In the meantime it could help to explicitly delete the image cache when a plot closes or to avoid caching altogether. Christoph Eric Christoph Since self._imcache is an instance attribute, when the instance is no longer referenced, it should get garbage-collected, provided there is no __del__ method. Eric Christoph -- The modern datacenter depends on network connectivity to access resources and provide services. The best practices for maximizing a physical server's connectivity to a physical network are well understood - see how these rules translate into the virtual world? http://p.sf.net/sfu/oracle-sfdevnlfb ___ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users -- The modern datacenter depends on network connectivity to access resources and provide services. The best practices for maximizing a physical server's connectivity to a physical network are well understood - see how these rules translate into the virtual world? http://p.sf.net/sfu/oracle-sfdevnlfb ___ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net
Re: [Matplotlib-users] Trouble with imshow
On Thu, Feb 3, 2011 at 6:37 PM, Christoph Gohlke cgoh...@uci.edu wrote: On 2/3/2011 3:13 PM, Eric Firing wrote: On 02/03/2011 01:02 PM, Christoph Gohlke wrote: On 2/3/2011 2:44 PM, Eric Firing wrote: On 02/03/2011 12:28 PM, Christoph Gohlke wrote: On 2/3/2011 2:15 PM, Eric Firing wrote: On 02/03/2011 11:30 AM, Robert Abiad wrote: On 2/3/2011 10:06 AM, Eric Firing wrote: On 02/02/2011 10:17 PM, Eric Firing wrote: On 02/02/2011 08:38 PM, Robert Abiad wrote: [...] I'll put it in as an enhancement, but I'm still unsure if there is a bug in there as well. Is there something I should be doing to clear memory after the first figure is closed other than close()? I don't understand why memory usage grows each time I replot, but I'm pretty sure it isn't desireable behavior. As I mentioned, this effect is worse with plot. So is this a bug or improper usage? I'm not quite sure, but I don't think there is a specifically matplotlib memory leak bug at work here. Are you using ipython, and if so, have you turned off the caching? In its default mode, ipython keeps lots of references, thereby keeping memory in use. Also, memory management and reporting can be a bit tricky and misleading. Nevertheless, the attached script may be illustrating the problem. Try running it from the command line as-is (maybe shorten the loop--it doesn't take 100 iterations to show the pattern) and then commenting out the line as indicated in the comment. It seems that if anything is done that adds ever so slightly to memory use while the figure is displayed, then when the figure is closed, its memory is not reused. I'm puzzled. I wasn't thinking straight--there is no mystery and no memory leak. Ignore my example script referred to above. It was saving rows of the z array, not single elements as I had intended, so of course memory use was growing substantially. Eric You may not see a memory leak, but I still can't get my memory back without killing python. I turned off the ipython caching and even ran without iPython on both Windows and Ubuntu, but when I use imshow(), followed by close('all') and another imshow(), I run out of memory. I can see from the OS that the memory does not come back after close() and that it grows after the second imshow(). Any other ideas? Looks like a bug to me otherwise. Except that I tried the same things and did not get quite the same result. Let's track this down. Please try the attached script, and see if the memory usage grows substantially, or just oscillates a bit. Eric One thing I noticed is that if I add a def __del__(self): print 'del' to image._AxesImageBase, it never gets called. _AxesImageBase keeps float64 and uint8 rgba images in a cache, which is never freed. Adding a __del__ method defeats (or blocks) the garbage collection. Sorry, never heard of that. I thought __del__() is called when the reference count reaches 0. It is, but if there are circular reference chains (cycles--and mpl is full of them) then the garbage collector has to identify them and remove them. If it encounters a __del__ it stops and leaves that cycle alone. My understanding is that if there is a circular reference then the refcount will not be zero anyway. In this case _AxesImageBase instances and their image caches will never be deleted by the gc (__del__ method present or not) unless the circle is broken. When the interpreter quits things are deleted by other means. I don't know matplotlib code good enough to fix this and will instead work on reducing the memory overhead needed to plot an image. In the meantime it could help to explicitly delete the image cache when a plot closes or to avoid caching altogether. Christoph Just to clarify the circular reference paradox. The garbage collector will pick it up because it is using weak refs instead of regular references. So, when a circular reference occurs, but is not hard ref-ed to anything, then it is detached and the gc picks it up. I am not very familiar on this concept in particular, but I do remember it being explained this way. Anyone who is more knowledgeable about this, I would welcome further comment or corrections to what I remember. Also, not to sound too annoying, but has anyone considered the idea of using compressed arrays for holding those rgba values? Ben Root -- The modern datacenter depends on network connectivity to access resources and provide services. The best practices for maximizing a physical server's connectivity to a physical network are well understood - see how these rules translate into the virtual world? http://p.sf.net/sfu/oracle-sfdevnlfb___ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net
Re: [Matplotlib-users] Is it possible to plot axes with arrows ?
From: Paul Ivanov pivanov...@gmail.com To: Francois Maltey fmal...@nerim.fr Cc: cc: Subject: Re: [Matplotlib-users] Is it possible to plot axes with arrows ? Reply-To: In-Reply-To: 4d496f84.7070...@nerim.fr X-PGP-Key: http://pirsquared.org/PaulIvanov0F3E28F7.asc Francois Maltey, on 2011-02-02 15:51, wrote: Hello, I use matplolib by the mathematical system Sage in order to plot a function. The Sage code calls matplotlib and uses its options : The Sage command is plot (sin, x, -5, 5) I add labels par axes_labels or remove axes by : plot (sin(x), x, -5, 5, axes_label = ['x', 'y']) plot (sin(x), x, -5, 5, axes=false) French users (and maybe others) uses arrows and not lines for axes. I'm looking for a plot (sin(x), x, -5, 5, axes=arrows) Is there a pretty way to get these arrows. The result of this code isn't so fine. length, width and color don't match. plot (sin(x), x, -5, 5, axes=false) + arrow ((-5,0),(5,0)) + arrow ((0,-1),(0,1)) What options do you propose ? I don't find relevant answers in the archive. Hi Francois, I'm not sure I understand - but do you want the arrows at the end of the axes spines? I don't think there's a direct way to adjust the spines to become arrows at the moment, but we can remedy that by making annotations in axes coordinates. The important thing to know here is that in axes coordinates, which are always between 0-1, spine endpoints are at these locations: (0,0), (0,1), (1,0), and (1,1). Here's the code, and attached is the resulting image import matplotlib.pyplot as plt ax = plt.subplot(1,1,1) al = 7 # arrow length in points arrowprops=dict(clip_on=False, # plotting outside axes on purpose frac=1., # make end arrowhead the whole size of arrow headwidth=al, # in points facecolor='k') kwargs = dict( xycoords='axes fraction', textcoords='offset points', arrowprops= arrowprops, ) ax.annotate(,(1,0),xytext=(-al,0), **kwargs) # bottom spine arrow ax.annotate(,(0,1),xytext=(0,-al), **kwargs) # left spin arrow # hide the top and right spines [sp.set_visible(False) for sp in ax.spines['top'],ax.spines['right']] #hide the right and top tick marks ax.yaxis.tick_left() ax.xaxis.tick_bottom() x = np.linspace(-5,5,50) ax.plot(x, np.sin(x)) # adjust the view a little bit ax.set_xlim(-5,5) ax.set_ylim(-1.1,1.1) plt.draw() I'm not familiar with how SAGE exposes matplotlib functionality, though, since the syntax you used differs from how matplotlib is utilized. best, -- Paul Ivanov 314 address only used for lists, off-list direct email at: http://pirsquared.org | GPG/PGP key id: 0x0F3E28F7 attachment: spine-arrows.png signature.asc Description: Digital signature -- The modern datacenter depends on network connectivity to access resources and provide services. The best practices for maximizing a physical server's connectivity to a physical network are well understood - see how these rules translate into the virtual world? http://p.sf.net/sfu/oracle-sfdevnlfb___ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users
Re: [Matplotlib-users] Trouble with imshow
On 02/03/2011 03:04 PM, Benjamin Root wrote: Also, not to sound too annoying, but has anyone considered the idea of using compressed arrays for holding those rgba values? I don't see how that really helps; as far as I know, a full rgba array has to be passed into agg. What *does* help is using uint8 from start to finish. It might also be possible to use some smart downsampling before generating the rgba array, but the uint8 route seems to me the first thing to attack. Eric Ben Root -- The modern datacenter depends on network connectivity to access resources and provide services. The best practices for maximizing a physical server's connectivity to a physical network are well understood - see how these rules translate into the virtual world? http://p.sf.net/sfu/oracle-sfdevnlfb ___ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users
Re: [Matplotlib-users] Trouble with imshow
On 2/3/2011 6:50 PM, Eric Firing wrote: On 02/03/2011 03:04 PM, Benjamin Root wrote: Also, not to sound too annoying, but has anyone considered the idea of using compressed arrays for holding those rgba values? I don't see how that really helps; as far as I know, a full rgba array has to be passed into agg. What *does* help is using uint8 from start to finish. It might also be possible to use some smart downsampling before generating the rgba array, but the uint8 route seems to me the first thing to attack. Eric Ben Root Please review the attached patch. It avoids generating and storing float64 rgba arrays and uses uint8 rgba instead. That's a huge memory saving and also faster. I can't see any side effects as _image.fromarray() converts the float64 input to uint8 anyway. So far other attempts to optimize memory usage were thwarted by matplotlib's internal use of masked arrays. As mentioned before, users can provide their own normalized rgba arrays to avoid all this processing. Christoph Index: image.py === --- image.py(revision 8944) +++ image.py(working copy) @@ -198,11 +198,11 @@ im.is_grayscale = False else: if self._rgbacache is None: -x = self.to_rgba(self._A, self._alpha) +x = self.to_rgba(self._A, self._alpha, True) self._rgbacache = x else: x = self._rgbacache -im = _image.fromarray(x[yslice,xslice], 0) +im = _image.frombyte(x[yslice,xslice,:], 0) if len(self._A.shape) == 2: im.is_grayscale = self.cmap.is_gray() else: -- The modern datacenter depends on network connectivity to access resources and provide services. The best practices for maximizing a physical server's connectivity to a physical network are well understood - see how these rules translate into the virtual world? http://p.sf.net/sfu/oracle-sfdevnlfb___ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users
Re: [Matplotlib-users] Is it possible to plot axes with arrows ?
On 2/2/11 8:51 AM, Francois Maltey wrote: Hello, I use matplolib by the mathematical system Sage in order to plot a function. The Sage code calls matplotlib and uses its options : The Sage command is plot (sin, x, -5, 5) I add labels par axes_labels or remove axes by : plot (sin(x), x, -5, 5, axes_label = ['x', 'y']) plot (sin(x), x, -5, 5, axes=false) French users (and maybe others) uses arrows and not lines for axes. I'm looking for a plot (sin(x), x, -5, 5, axes=arrows) Is there a pretty way to get these arrows. The result of this code isn't so fine. length, width and color don't match. plot (sin(x), x, -5, 5, axes=false) + arrow ((-5,0),(5,0)) + arrow ((0,-1),(0,1)) What options do you propose ? I've made a Sage ticket for this: http://trac.sagemath.org/sage_trac/ticket/10740 As a clumsy workaround, you could use the .matplotlib() method for Sage graphics objects to get the matplotlib figure object for the graphics object. Then you could: 1. figure out which spine was being used as the drawn axes 2. use that spine's transform to place an arrow at the end of the spine (using the example code that was just posted in another message to draw the arrow). 3. Draw the figure Thanks, Jason -- The modern datacenter depends on network connectivity to access resources and provide services. The best practices for maximizing a physical server's connectivity to a physical network are well understood - see how these rules translate into the virtual world? http://p.sf.net/sfu/oracle-sfdevnlfb ___ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users