Re: [Matplotlib-users] QT draw issue in 1.1.0 and PyQt4v2 missing?
Benjamin Root-2 wrote:
I think that might have been a little unclear. You should only need to
select 'PyQt4' or 'PySide'. If PyQt4 is selected, then (I think) the v2
is
automatically tested for internally.
hmm, please explain where you think it is used internally, as the code seem
to expect that this param can be set when there is no env variable 'QT_API'
set, in backends\qt4_compat.py:
# Available APIs.
QT_API_PYQT = 'PyQt4' # API is not set here; Python 2.x default is V 1
QT_API_PYQTv2 = 'PyQt4v2' # forced to Version 2 API
QT_API_PYSIDE = 'PySide'# only supports Version 2 API
ETS = dict(pyqt=QT_API_PYQTv2, pyside=QT_API_PYSIDE)
# If the ETS QT_API environment variable is set, use it. Note that
# ETS requires the version 2 of PyQt4, which is not the platform
# default for Python 2.x.
QT_API_ENV = os.environ.get('QT_API')
if QT_API_ENV is not None:
try:
QT_API = ETS[QT_API_ENV]
except KeyError:
raise RuntimeError(
'Unrecognized environment variable %r, valid values are: %r or %r'
%
(QT_API_ENV, 'pyqt', 'pyside'))
else:
# No ETS environment, so use rcParams.
QT_API = rcParams['backend.qt4']
but trying to set it, via code or matplotlibrc results in:
Unrecognized backend.qt4 string PyQt4v2: valid strings are
['PySide', 'PyQt4']
%s\n\t%s' % (val, cnt, line, fname, msg))
--
View this message in context:
http://old.nabble.com/QT-draw-issue-in-1.1.0-and-PyQt4v2-missing--tp32676093p32680414.html
Sent from the matplotlib - users mailing list archive at Nabble.com.
--
All the data continuously generated in your IT infrastructure contains a
definitive record of customers, application performance, security
threats, fraudulent activity and more. Splunk takes this data and makes
sense of it. Business sense. IT sense. Common sense.
http://p.sf.net/sfu/splunk-d2d-oct___
Matplotlib-users mailing list
Matplotlib-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/matplotlib-users
Re: [Matplotlib-users] QT draw issue in 1.1.0 and PyQt4v2 missing?
efiring wrote: https://github.com/matplotlib/matplotlib/pull/539 I think this pull request fixes it. Eric Great, thanks for confirming and fixing! Whilst waiting for this to make it into a release, I've hacked this to achieve the same effect (so I don't have to patch + distribute the mpl code): def do_draw_hack(self): fig = self.figure fig.canvas.draw() fig.draw(fig.canvas.get_renderer()) fig.canvas.update() -- View this message in context: http://old.nabble.com/QT-draw-issue-in-1.1.0-and-PyQt4v2-missing--tp32676093p32680445.html Sent from the matplotlib - users mailing list archive at Nabble.com. -- All the data continuously generated in your IT infrastructure contains a definitive record of customers, application performance, security threats, fraudulent activity and more. Splunk takes this data and makes sense of it. Business sense. IT sense. Common sense. http://p.sf.net/sfu/splunk-d2d-oct___ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users
Re: [Matplotlib-users] QT draw issue in 1.1.0 and PyQt4v2 missing?
On 10/18/2011 09:44 PM, RuiDC wrote:
Benjamin Root-2 wrote:
I think that might have been a little unclear. You should only need
to select 'PyQt4' or 'PySide'. If PyQt4 is selected, then (I think)
the v2 is automatically tested for internally.
hmm, please explain where you think it is used internally, as the code
seem to expect that this param can be set when there is no env variable
'QT_API' set, in backends\qt4_compat.py:
After quite a bit of thrashing around, we settled on the present system.
The rcParam deliberately does *not* set the API version if PyQt4 is
used. Whatever your Qt4 comes up with when the backend is imported is
what is used. That way there is no conflict with applications that
import pyqt4, and perhaps set the API, before importing mpl. If you want
to use the v2 api, you can import pyqt4 and set the API yourself before
importing mpl. Or, if you want the API to be forced to v2 by mpl when
the backend is imported, you can do what ETS does, which is to set the
QT_API environment variable to pyqt.
This moderately ugly mess was necessitated by the need to be compatible
with ipython, ETS, and existing user code, while dealing with two
versions of pyqt4 and one of pyside, and with different default versions
of pyqt4 for python 2 and python 3 (which we will be supporting in the
not-too-distant future, I believe.)
Eric
# Available APIs.
QT_API_PYQT = 'PyQt4' # API is not set here; Python 2.x default is V 1
QT_API_PYQTv2 = 'PyQt4v2' # forced to Version 2 API
QT_API_PYSIDE = 'PySide'# only supports Version 2 API
ETS = dict(pyqt=QT_API_PYQTv2, pyside=QT_API_PYSIDE)
# If the ETS QT_API environment variable is set, use it. Note that
# ETS requires the version 2 of PyQt4, which is not the platform
# default for Python 2.x.
QT_API_ENV = os.environ.get('QT_API')
if QT_API_ENV is not None:
try:
QT_API = ETS[QT_API_ENV]
except KeyError:
raise RuntimeError(
'Unrecognized environment variable %r, valid values are: %r or %r'
%
(QT_API_ENV, 'pyqt', 'pyside'))
else:
# No ETS environment, so use rcParams.
QT_API = rcParams['backend.qt4']
but trying to set it, via code or matplotlibrc results in: Unrecognized
backend.qt4 string PyQt4v2: valid strings are ['PySide', 'PyQt4']
%s\n\t%s' % (val, cnt, line, fname, msg))
View this message in context: Re: QT draw issue in 1.1.0 and PyQt4v2
missing?
http://old.nabble.com/QT-draw-issue-in-1.1.0-and-PyQt4v2-missing--tp32676093p32680414.html
Sent from the matplotlib - users mailing list archive
http://old.nabble.com/matplotlib---users-f2906.html at Nabble.com.
--
All the data continuously generated in your IT infrastructure contains a
definitive record of customers, application performance, security
threats, fraudulent activity and more. Splunk takes this data and makes
sense of it. Business sense. IT sense. Common sense.
http://p.sf.net/sfu/splunk-d2d-oct
___
Matplotlib-users mailing list
Matplotlib-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/matplotlib-users
--
All the data continuously generated in your IT infrastructure contains a
definitive record of customers, application performance, security
threats, fraudulent activity and more. Splunk takes this data and makes
sense of it. Business sense. IT sense. Common sense.
http://p.sf.net/sfu/splunk-d2d-oct
___
Matplotlib-users mailing list
Matplotlib-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/matplotlib-users
Re: [Matplotlib-users] QT draw issue in 1.1.0 and PyQt4v2 missing?
Thanks for the comprehensive explanation. So it would seem it's really only the http://matplotlib.sourceforge.net/users/whats_new.html page that is misleading on this. Many thanks! -- View this message in context: http://old.nabble.com/QT-draw-issue-in-1.1.0-and-PyQt4v2-missing--tp32676093p32680797.html Sent from the matplotlib - users mailing list archive at Nabble.com. -- All the data continuously generated in your IT infrastructure contains a definitive record of customers, application performance, security threats, fraudulent activity and more. Splunk takes this data and makes sense of it. Business sense. IT sense. Common sense. http://p.sf.net/sfu/splunk-d2d-oct ___ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users
Re: [Matplotlib-users] memory error - netcdf plots
On 10/18/11 8:55 PM, questions anon wrote: Thanks Jeff, that certainly speeds it up! But when I take them out of the loop and place them elsewhere they are no longer added to the map. Is there someway I can call them in the loop but still get it to run quickly? Thanks Just the Basemap instance creation and the transformation of coordinates to projection space should be hoisted out of the loop map = Basemap(projection='merc',llcrnrlat=-40,urcrnrlat=-33, llcrnrlon=139.0,urcrnrlon=151.0,lat_ts=0,resolution='i') x,y=map(*N.meshgrid(LON,LAT)) you can leave the other statements in. If you still have memory issues, bring the readshapefile call out, and draw the shapes whose coordinates are stored in the instance variable map.DSE_REGIONS manually in the loop. -Jeff On Fri, Oct 14, 2011 at 10:54 PM, Jeff Whitaker jsw...@fastmail.fm mailto:jsw...@fastmail.fm wrote: On 10/12/11 8:20 PM, questions anon wrote: Hi All, I keep receiving a memory error when processing many netcdf files. I assumed it had something to do with how I loop things and maybe needed to close things off properly but I recently received an error that made me think it might be because of matplotlib. In the code below I am looping through a bunch of netcdf files (each file is hourly data for one month) and within each netcdf file I am outputting a *png file every three hours. This works for one netcdf file (therefore one month) but when it begins to process the next netcdf file I receive a memory error (see below). Since I have tidied some of my code up it seems to process partly into the second file but then I still receive the memory error. I have tried a few suggestions such as: -Combining the dataset using MFDataset (using NETCDF4) is not an option because the files do not have unlimited dimension. - gc.collect() but that just results in a /GEOS_ERROR: bad allocation error/. -only open LAT and LON once (which worked) System Details: Python 2.7.2 |EPD 7.1-2 (32-bit)| (default, Jul 3 2011, 15:13:59) [MSC v.1500 32 bit (Intel)] on win32 Any feedback will be greatly appreciated as I seem to keep ending up with memory errors when working with netcdf files this even happens if I am using a much better computer. *Most recent error: * Traceback (most recent call last): File C:\plot_netcdf_merc_multiplot_across_multifolders_TSFC.py, line 78, in module plt.savefig((os.path.join(outputfolder, 'TSFC'+date_string+'UTC.png'))) File C:\Python27\lib\site-packages\matplotlib\pyplot.py, line 363, in savefig return fig.savefig(*args, **kwargs) File C:\Python27\lib\site-packages\matplotlib\figure.py, line 1084, in savefig self.canvas.print_figure(*args, **kwargs) File C:\Python27\lib\site-packages\matplotlib\backends\backend_wxagg.py, line 100, in print_figure FigureCanvasAgg.print_figure(self, filename, *args, **kwargs) File C:\Python27\lib\site-packages\matplotlib\backend_bases.py, line 1923, in print_figure **kwargs) File C:\Python27\lib\site-packages\matplotlib\backends\backend_agg.py, line 438, in print_png FigureCanvasAgg.draw(self) File C:\Python27\lib\site-packages\matplotlib\backends\backend_agg.py, line 393, in draw self.renderer = self.get_renderer() File C:\Python27\lib\site-packages\matplotlib\backends\backend_agg.py, line 404, in get_renderer self.renderer = RendererAgg(w, h, self.figure.dpi) File C:\Python27\lib\site-packages\matplotlib\backends\backend_agg.py, line 59, in __init__ self._renderer = _RendererAgg(int(width), int(height), dpi, debug=False) RuntimeError: Could not allocate memory for image *Error when I added gc.collect()* GEOS_ERROR: bad allocation *Old error (before adding gc.collect() )* /Traceback (most recent call last): File d:/plot_netcdf_merc_multiplot_across_multifolders__memoryerror.py, line 44, in module TSFC=ncfile.variables['T_SFC'][1::3] File netCDF4.pyx, line 2473, in netCDF4.Variable.__getitem__ (netCDF4.c:23094) MemoryError/ from netCDF4 import Dataset import numpy as N import matplotlib.pyplot as plt from mpl_toolkits.basemap import Basemap from netcdftime import utime from datetime import datetime import os import gc shapefile1=E:/ griddeddatasamples/GIS/DSE_REGIONS MainFolder=rE:/griddeddatasamples/GriddedData/InputsforValidation/T_SFC/ OutputFolder=rE:/griddeddatasamples/GriddedData/OutputsforValidation fileforlatlon=Dataset(E:/griddeddatasamples/GriddedData/InputsforValidation/T_SFC/TSFC_1974_01/IDZ00026_VIC_ADFD_T_SFC.nc, 'r+', 'NETCDF4')
Re: [Matplotlib-users] strange behavior of images when they have an alpha channel [resend]
You are right that Agg is doing the resizing here. Agg expects premultiplied alpha. See [1] for information about what that means. [1] http://en.wikipedia.org/wiki/Alpha_compositing After Agg interpolates the pixel values, to prevent oversaturation it truncates all values to be less than alpha (which makes sense if everything is assumed to be premultiplied alpha). Arguably, the bug here is that nearest neighbor (which doesn't have to do any blending) doesn't perform the truncation step -- then both would look wrong. It happens in this code snippet in span_image_filter_rgba: (base_mask is 255) if(fg[order_type::A] base_mask) fg[order_type::A] = base_mask; if(fg[order_type::R] fg[order_type::A]) fg[order_type::R] = fg[order_type::A]; if(fg[order_type::G] fg[order_type::A]) fg[order_type::G] = fg[order_type::A]; if(fg[order_type::B] fg[order_type::A]) fg[order_type::B] = fg[order_type::A]; So, the solution to make a partially transparent image is to not do: pix[:,:,3] = 127 but instead, do pix *= 0.5 Of course, the real fix here is to support alpha blending properly in the image class, then the user wouldn't have to deal with such details. A bug should probably be filed in the matplotlib issue tracker for this. Mike On 10/19/2011 12:23 PM, Daniel Hyams wrote: [Sorry, I keep getting tripped up with HTML mailresent in ascii, and resaved one of the attachment png's to make it smaller.] Example script attached (PIL required). Basically, if I impose a specific value into an image's alpha channel and use any interpolation scheme other than 'nearest', there appears gray all where the figure didn't have any color to begin with. I've also attached a screenshot of the output of the script on my machine. Hopefully I'm doing something wrongly? I chased the problem and managed to hack in a solution that fixes the problem, but it's extremely inefficient...basically, in matplotlib's image.py, routine BboxImage.make_image, you can create two images thereone with no alpha channel (call it imRGB) and one with (call it imRGBA). Go through all of the routine, doing exactly the same things to both of the images *except* for the interpolation, which is set to 'nearest' for imRGBA. Then, rip the colors out of imRGB, the alpha channel off of imRGBA, and put them togethergo through all of the routine again with this composited image, and it works. I know...I told you it was bad ;) The problem seems to be in the resize call in that routine...resize, which calls into C code, does not appear to handle things correctly when the alpha is anything other than 255's across the board. It might be a problem in the agg routines, but hopefully it is just maybe a misuse of the agg routines. The behavior seems to be backend independent as far as I could test (I tried with wxagg and tk backends). I am using mpl 1.0.0 on Windows if it matters. -- Daniel Hyams dhy...@gmail.com -- All the data continuously generated in your IT infrastructure contains a definitive record of customers, application performance, security threats, fraudulent activity and more. Splunk takes this data and makes sense of it. Business sense. IT sense. Common sense. http://p.sf.net/sfu/splunk-d2d-oct ___ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users -- The demand for IT networking professionals continues to grow, and the demand for specialized networking skills is growing even more rapidly. Take a complimentary Learning@Ciosco Self-Assessment and learn about Cisco certifications, training, and career opportunities. http://p.sf.net/sfu/cisco-dev2dev___ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users
Re: [Matplotlib-users] strange behavior of images when they have an alpha channel [resend]
Ah, thanks so much Michael! That explanation helps a great deal; I was always considering things in straight alpha format, not even knowing that there was alternative. I'll play with this tonight; I don't see any problem getting the thing working, though, now that I know what agg expects to see... And yes, alpha support in the image class would be very helpful ;) On Wed, Oct 19, 2011 at 2:16 PM, Michael Droettboom md...@stsci.edu wrote: You are right that Agg is doing the resizing here. Agg expects premultiplied alpha. See [1] for information about what that means. [1] http://en.wikipedia.org/wiki/Alpha_compositing After Agg interpolates the pixel values, to prevent oversaturation it truncates all values to be less than alpha (which makes sense if everything is assumed to be premultiplied alpha). Arguably, the bug here is that nearest neighbor (which doesn't have to do any blending) doesn't perform the truncation step -- then both would look wrong. It happens in this code snippet in span_image_filter_rgba: (base_mask is 255) if(fg[order_type::A] base_mask) fg[order_type::A] = base_mask; if(fg[order_type::R] fg[order_type::A]) fg[order_type::R] = fg[order_type::A]; if(fg[order_type::G] fg[order_type::A]) fg[order_type::G] = fg[order_type::A]; if(fg[order_type::B] fg[order_type::A]) fg[order_type::B] = fg[order_type::A]; So, the solution to make a partially transparent image is to not do: pix[:,:,3] = 127 but instead, do pix *= 0.5 Of course, the real fix here is to support alpha blending properly in the image class, then the user wouldn't have to deal with such details. A bug should probably be filed in the matplotlib issue tracker for this. Mike On 10/19/2011 12:23 PM, Daniel Hyams wrote: [Sorry, I keep getting tripped up with HTML mailresent in ascii, and resaved one of the attachment png's to make it smaller.] Example script attached (PIL required). Basically, if I impose a specific value into an image's alpha channel and use any interpolation scheme other than 'nearest', there appears gray all where the figure didn't have any color to begin with. I've also attached a screenshot of the output of the script on my machine. Hopefully I'm doing something wrongly? I chased the problem and managed to hack in a solution that fixes the problem, but it's extremely inefficient...basically, in matplotlib's image.py, routine BboxImage.make_image, you can create two images thereone with no alpha channel (call it imRGB) and one with (call it imRGBA). Go through all of the routine, doing exactly the same things to both of the images *except* for the interpolation, which is set to 'nearest' for imRGBA. Then, rip the colors out of imRGB, the alpha channel off of imRGBA, and put them togethergo through all of the routine again with this composited image, and it works. I know...I told you it was bad ;) The problem seems to be in the resize call in that routine...resize, which calls into C code, does not appear to handle things correctly when the alpha is anything other than 255's across the board. It might be a problem in the agg routines, but hopefully it is just maybe a misuse of the agg routines. The behavior seems to be backend independent as far as I could test (I tried with wxagg and tk backends). I am using mpl 1.0.0 on Windows if it matters. -- Daniel Hyams dhy...@gmail.com -- All the data continuously generated in your IT infrastructure contains a definitive record of customers, application performance, security threats, fraudulent activity and more. Splunk takes this data and makes sense of it. Business sense. IT sense. Common sense. http://p.sf.net/sfu/splunk-d2d-oct ___ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users -- The demand for IT networking professionals continues to grow, and the demand for specialized networking skills is growing even more rapidly. Take a complimentary Learning@Ciosco Self-Assessment and learn about Cisco certifications, training, and career opportunities. http://p.sf.net/sfu/cisco-dev2dev ___ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users -- Daniel Hyams dhy...@gmail.com -- The demand for IT networking professionals continues to grow, and the demand for specialized networking skills is growing even more rapidly. Take a complimentary Learning@Ciosco Self-Assessment and learn about Cisco
Re: [Matplotlib-users] Adding Arrows to polar plots
On Fri, Oct 14, 2011 at 3:42 AM, Nils Wagner nwag...@iam.uni-stuttgart.dewrote: Hi all, what is the native data coordinate system for Arrows in a polar plot ? How do I add arrows to a polar plot ? An example would be appreciated. fig = figure(figsize=(12,12)) ax = fig.add_subplot(111, polar=True) Nils I don't know if it is an existing feature or not. If it isn't, then I think a feature request should be filed. If it is, then a request should be filed anyway for an example to be added to the gallery. Ben Root -- The demand for IT networking professionals continues to grow, and the demand for specialized networking skills is growing even more rapidly. Take a complimentary Learning@Ciosco Self-Assessment and learn about Cisco certifications, training, and career opportunities. http://p.sf.net/sfu/cisco-dev2dev___ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users
Re: [Matplotlib-users] memory error - netcdf plots
thank you, I am not quite sure how to 'draw' the shapefile but making those changes and removing the shapefile has sped the processing up considerably! Thank you for your help On Wed, Oct 19, 2011 at 11:42 PM, Jeff Whitaker jsw...@fastmail.fm wrote: On 10/18/11 8:55 PM, questions anon wrote: Thanks Jeff, that certainly speeds it up! But when I take them out of the loop and place them elsewhere they are no longer added to the map. Is there someway I can call them in the loop but still get it to run quickly? Thanks Just the Basemap instance creation and the transformation of coordinates to projection space should be hoisted out of the loop map = Basemap(projection='merc',llcrnrlat=-40,urcrnrlat=-33, llcrnrlon=139.0,urcrnrlon=151.0,lat_ts=0,resolution='i') x,y=map(*N.meshgrid(LON,LAT)) you can leave the other statements in. If you still have memory issues, bring the readshapefile call out, and draw the shapes whose coordinates are stored in the instance variable map.DSE_REGIONS manually in the loop. -Jeff On Fri, Oct 14, 2011 at 10:54 PM, Jeff Whitaker jsw...@fastmail.fmwrote: On 10/12/11 8:20 PM, questions anon wrote: Hi All, I keep receiving a memory error when processing many netcdf files. I assumed it had something to do with how I loop things and maybe needed to close things off properly but I recently received an error that made me think it might be because of matplotlib. In the code below I am looping through a bunch of netcdf files (each file is hourly data for one month) and within each netcdf file I am outputting a *png file every three hours. This works for one netcdf file (therefore one month) but when it begins to process the next netcdf file I receive a memory error (see below). Since I have tidied some of my code up it seems to process partly into the second file but then I still receive the memory error. I have tried a few suggestions such as: -Combining the dataset using MFDataset (using NETCDF4) is not an option because the files do not have unlimited dimension. - gc.collect() but that just results in a *GEOS_ERROR: bad allocation error*. -only open LAT and LON once (which worked) System Details: Python 2.7.2 |EPD 7.1-2 (32-bit)| (default, Jul 3 2011, 15:13:59) [MSC v.1500 32 bit (Intel)] on win32 Any feedback will be greatly appreciated as I seem to keep ending up with memory errors when working with netcdf files this even happens if I am using a much better computer. *Most recent error: * Traceback (most recent call last): File C:\plot_netcdf_merc_multiplot_across_multifolders_TSFC.py, line 78, in module plt.savefig((os.path.join(outputfolder, 'TSFC'+date_string+'UTC.png'))) File C:\Python27\lib\site-packages\matplotlib\pyplot.py, line 363, in savefig return fig.savefig(*args, **kwargs) File C:\Python27\lib\site-packages\matplotlib\figure.py, line 1084, in savefig self.canvas.print_figure(*args, **kwargs) File C:\Python27\lib\site-packages\matplotlib\backends\backend_wxagg.py, line 100, in print_figure FigureCanvasAgg.print_figure(self, filename, *args, **kwargs) File C:\Python27\lib\site-packages\matplotlib\backend_bases.py, line 1923, in print_figure **kwargs) File C:\Python27\lib\site-packages\matplotlib\backends\backend_agg.py, line 438, in print_png FigureCanvasAgg.draw(self) File C:\Python27\lib\site-packages\matplotlib\backends\backend_agg.py, line 393, in draw self.renderer = self.get_renderer() File C:\Python27\lib\site-packages\matplotlib\backends\backend_agg.py, line 404, in get_renderer self.renderer = RendererAgg(w, h, self.figure.dpi) File C:\Python27\lib\site-packages\matplotlib\backends\backend_agg.py, line 59, in __init__ self._renderer = _RendererAgg(int(width), int(height), dpi, debug=False) RuntimeError: Could not allocate memory for image *Error when I added gc.collect()* GEOS_ERROR: bad allocation *Old error (before adding gc.collect() )* *Traceback (most recent call last): File d:/plot_netcdf_merc_multiplot_across_multifolders__memoryerror.py, line 44, in module TSFC=ncfile.variables['T_SFC'][1::3] File netCDF4.pyx, line 2473, in netCDF4.Variable.__getitem__ (netCDF4.c:23094) MemoryError* from netCDF4 import Dataset import numpy as N import matplotlib.pyplot as plt from mpl_toolkits.basemap import Basemap from netcdftime import utime from datetime import datetime import os import gc shapefile1=E:/ griddeddatasamples/GIS/DSE_REGIONS MainFolder=rE:/griddeddatasamples/GriddedData/InputsforValidation/T_SFC/ OutputFolder=rE:/griddeddatasamples/GriddedData/OutputsforValidation fileforlatlon=Dataset(E:/griddeddatasamples/GriddedData/InputsforValidation/T_SFC/TSFC_1974_01/IDZ00026_VIC_ADFD_T_SFC.nc, 'r+', 'NETCDF4') LAT=fileforlatlon.variables['latitude'][:] LON=fileforlatlon.variables['longitude'][:] for (path, dirs, files) in os.walk(MainFolder):
Re: [Matplotlib-users] memory error - netcdf plots
On 10/19/11 4:37 PM, questions anon wrote: thank you, I am not quite sure how to 'draw' the shapefile from matplotlib.collections import LineCollection ax = plt.gca() # get current axes instance # 'DSE_REGIONS' instance variable created by readshapefile method call. lines = LineCollection(map.DSE_REGIONS) ax.add_collection(lines) -Jeff but making those changes and removing the shapefile has sped the processing up considerably! Thank you for your help On Wed, Oct 19, 2011 at 11:42 PM, Jeff Whitaker jsw...@fastmail.fm mailto:jsw...@fastmail.fm wrote: On 10/18/11 8:55 PM, questions anon wrote: Thanks Jeff, that certainly speeds it up! But when I take them out of the loop and place them elsewhere they are no longer added to the map. Is there someway I can call them in the loop but still get it to run quickly? Thanks Just the Basemap instance creation and the transformation of coordinates to projection space should be hoisted out of the loop map = Basemap(projection='merc',llcrnrlat=-40,urcrnrlat=-33, llcrnrlon=139.0,urcrnrlon=151.0,lat_ts=0,resolution='i') x,y=map(*N.meshgrid(LON,LAT)) you can leave the other statements in. If you still have memory issues, bring the readshapefile call out, and draw the shapes whose coordinates are stored in the instance variable map.DSE_REGIONS manually in the loop. -Jeff On Fri, Oct 14, 2011 at 10:54 PM, Jeff Whitaker jsw...@fastmail.fm mailto:jsw...@fastmail.fm wrote: On 10/12/11 8:20 PM, questions anon wrote: Hi All, I keep receiving a memory error when processing many netcdf files. I assumed it had something to do with how I loop things and maybe needed to close things off properly but I recently received an error that made me think it might be because of matplotlib. In the code below I am looping through a bunch of netcdf files (each file is hourly data for one month) and within each netcdf file I am outputting a *png file every three hours. This works for one netcdf file (therefore one month) but when it begins to process the next netcdf file I receive a memory error (see below). Since I have tidied some of my code up it seems to process partly into the second file but then I still receive the memory error. I have tried a few suggestions such as: -Combining the dataset using MFDataset (using NETCDF4) is not an option because the files do not have unlimited dimension. - gc.collect() but that just results in a /GEOS_ERROR: bad allocation error/. -only open LAT and LON once (which worked) System Details: Python 2.7.2 |EPD 7.1-2 (32-bit)| (default, Jul 3 2011, 15:13:59) [MSC v.1500 32 bit (Intel)] on win32 Any feedback will be greatly appreciated as I seem to keep ending up with memory errors when working with netcdf files this even happens if I am using a much better computer. *Most recent error: * Traceback (most recent call last): File C:\plot_netcdf_merc_multiplot_across_multifolders_TSFC.py, line 78, in module plt.savefig((os.path.join(outputfolder, 'TSFC'+date_string+'UTC.png'))) File C:\Python27\lib\site-packages\matplotlib\pyplot.py, line 363, in savefig return fig.savefig(*args, **kwargs) File C:\Python27\lib\site-packages\matplotlib\figure.py, line 1084, in savefig self.canvas.print_figure(*args, **kwargs) File C:\Python27\lib\site-packages\matplotlib\backends\backend_wxagg.py, line 100, in print_figure FigureCanvasAgg.print_figure(self, filename, *args, **kwargs) File C:\Python27\lib\site-packages\matplotlib\backend_bases.py, line 1923, in print_figure **kwargs) File C:\Python27\lib\site-packages\matplotlib\backends\backend_agg.py, line 438, in print_png FigureCanvasAgg.draw(self) File C:\Python27\lib\site-packages\matplotlib\backends\backend_agg.py, line 393, in draw self.renderer = self.get_renderer() File C:\Python27\lib\site-packages\matplotlib\backends\backend_agg.py, line 404, in get_renderer self.renderer = RendererAgg(w, h, self.figure.dpi) File C:\Python27\lib\site-packages\matplotlib\backends\backend_agg.py, line 59, in __init__ self._renderer = _RendererAgg(int(width), int(height), dpi, debug=False) RuntimeError: Could not allocate memory for image *Error when I added gc.collect()* GEOS_ERROR: bad allocation *Old error
[Matplotlib-users] matplotlib-grayscale
Hi, I appreciate all the effort and, lastly, the large functionality offered by matplotlib. But I found no way around formulating things a bit provocative on http://friedrichromstedt.github.com/matplotlib-grayscale/index.html. The project is about matplotlib-grayscale, a matplotlib patch that aims at grayscale toggle-switch behaviour as well as colour filtering / conversion. The project has a proof of concept (see under the URL above), but I disregard this since it has major design flaws, although *it works*. But as most things in matplotlib, *which work*, I would rather prefer a well thought-through solution. No offense, it's just the way it goes with matplotlib. I have some examples in the roadmap. I'm telling upfront that I don't want to get engaged with mpl, I just want to get this my project getting an ex-project. So I want to do it quickly and properly, let it go to the main repository, and forget about it except bug-fixes, the usual maintenance. Time frame, is, including good testing, say 3 monthes. I want to get it started quickly and just want to let you all let know about this project. I'll do it irrespective of if there's clubbing, it's up to the maintainers to accept or reject it in the end. I'm asking for ideas about the general design, which do not go too far with mpl, but use the existing framework to its best. I've put some of my ideas on the web page noted above. I did not dive into it again yet after the few monthes of letting it put aside. I just dug out my old commit messages and have put them together on the web page together with the superficial things I remembered. But I think the detail knowledge can be recovered easily. Only constructive feedback please. Especially a documenter/tester with a fresh eye on the things I know now would be great. Same applies for testing. The testing done for the pre-github proof-of-concept by Benjamin Root must be noted here, it made flaws and side-tracks apparent I either didn't know about or did overlook. Documenting and testing of the technical things is up to me, only the interface documentation and testing boredom. Last time I kept the project secret, I'm not gonna do commit that mistake again. Without matplotlib, lastly, also this patch would simply not exist, neither in my mind nor in reality. cu, Friedrich P.S.: I keep the right of dropping the project at any time. I'll do my diploma next monthes, so time might be short. But I think it might serve its purpose. -- I want to be honest and open to you from the beginning who I am and what I'm doing. So here's a small collection of informative links: http://github.com/friedrichromstedt/ (github code sharing); http://friedrichr.blogspot.com/2011/10/home-dashboard.html (All-Purpose Blog); http://picasaweb.google.com/friedrichromstedt/ (photos which are public); http://vimeo.com/friedrichr/ (videos which are public). There's a legacy web page http://www.friedrichromstedt.org/ which is to be replaced. There's a legacy YouTube channel http://www.youtube.com/user/friedrichromstedt/ which is unmaintained. And there's a news feed on Twitter: @f_romstedt. -- The demand for IT networking professionals continues to grow, and the demand for specialized networking skills is growing even more rapidly. Take a complimentary Learning@Ciosco Self-Assessment and learn about Cisco certifications, training, and career opportunities. http://p.sf.net/sfu/cisco-dev2dev ___ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users
Re: [Matplotlib-users] Figure legend for 1.1.0 doesn't properly handle errorbars?
JJ, The workaround works. Thanks. -Sterling On Oct 17, 2011, at 6:58PM, Jae-Joon Lee wrote: Meanwhile, you may do from matplotlib.legend import Legend l = Legend(fig, h, l, loc='lower right') fig.legends.append(l) This should be equivalent to fig.legend(h,l,loc='lower right'). Or, if you don't need axes legend, you may do legend(h,l,loc='lower right', bbox_to_anchor=[0,0,1,1], bbox_transform=fig.transFigure) Regards, -JJ On Tue, Oct 18, 2011 at 6:59 AM, Sterling Smith smit...@fusion.gat.com wrote: Let me first say that I appreciate the work that the developers have put into matplotlib. You're doing a great job. I have filed a bug report at https://github.com/matplotlib/matplotlib/issues/533 wherein I post the following Consider: from pylab import * x = arange(0,1,.01) y = x**2 fig = figure(2) ax = fig.add_subplot(111) errorbar(x,y,yerr=x/10.,label='$x^2$') errorbar(x,y**3,yerr=x/10.,label='$x^6$') legend(loc='upper center') h,l = ax.get_legend_handles_labels() fig.legend(h,l,loc='lower right') I am getting the right legend for the axes based legend, but the figure based legend seems to be using the different parts of the errorbar for subsequent handles, instead of using them as a group. From what I can tell, this has appeared since the upgrade to version 1.1.0. I am running on Linux, python 2.7, gtkAgg backend. Thanks, Sterling -- All the data continuously generated in your IT infrastructure contains a definitive record of customers, application performance, security threats, fraudulent activity and more. Splunk takes this data and makes sense of it. Business sense. IT sense. Common sense. http://p.sf.net/sfu/splunk-d2d-oct ___ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users -- The demand for IT networking professionals continues to grow, and the demand for specialized networking skills is growing even more rapidly. Take a complimentary Learning@Ciosco Self-Assessment and learn about Cisco certifications, training, and career opportunities. http://p.sf.net/sfu/cisco-dev2dev ___ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users
