On 2/4/2011 11:54 AM, Eric Firing wrote: > On 02/03/2011 05:35 PM, Christoph Gohlke wrote: >> >> >> On 2/3/2011 6:50 PM, Eric Firing wrote: >>> On 02/03/2011 03:04 PM, Benjamin Root wrote: >>> >>>> Also, not to sound too annoying, but has anyone considered the idea of >>>> using compressed arrays for holding those rgba values? >>> >>> I don't see how that really helps; as far as I know, a full rgba array >>> has to be passed into agg. What *does* help is using uint8 from start >>> to finish. It might also be possible to use some smart downsampling >>> before generating the rgba array, but the uint8 route seems to me the >>> first thing to attack. >>> >>> Eric >>> >>>> >>>> Ben Root >>> >> >> Please review the attached patch. It avoids generating and storing >> float64 rgba arrays and uses uint8 rgba instead. That's a huge memory >> saving and also faster. I can't see any side effects as >> _image.fromarray() converts the float64 input to uint8 anyway. > > Christoph, > > Thank you! I haven't found anything wrong with that delightfully simple > patch, so I have committed it to the trunk. Back in 2007 I added the > ability of colormapping to generate uint8 directly, precisely to enable > this sort of optimization. Why it was not already being used in imshow, > I don't know--maybe I was going to do it, got sidetracked, and never > finished. > > I suspect it won't be as simple as for the plain image, but there may be > opportunities for optimizing with uint8 in other image-like operations. > >> >> So far other attempts to optimize memory usage were thwarted by >> matplotlib's internal use of masked arrays. As mentioned before, users >> can provide their own normalized rgba arrays to avoid all this processing. >> > > Did you see other potential low-hanging fruit that might be harvested > with some changes to the code associated with masked arrays? > > Eric >
The norm function currently converts the data to double precision floating point and also creates temporary arrays that can be avoided. For float32 and low precision integer images this seems overkill and one could use float32. It might be possible to replace the norm function with numpy.digitize if that works with masked arrays. Last, the _image.frombyte function does a copy of 'strided arrays' (only relevant when zooming/panning large images). I try to provide a patch for each. Christoph ------------------------------------------------------------------------------ The modern datacenter depends on network connectivity to access resources and provide services. The best practices for maximizing a physical server's connectivity to a physical network are well understood - see how these rules translate into the virtual world? http://p.sf.net/sfu/oracle-sfdevnlfb _______________________________________________ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users
