antonv wrote: > I have a bit of experience programming and I am pretty sure I get my parts of > the code pretty well optimized. I made sure that in the loop I have only the > stuff needed and I'm loading all the stuff before. > > The biggest bottleneck is happening because I'm unpacking grib files to csv > files using Degrib in command line. That operation is usually around half an
Instead of going to csv files--which are *very* inefficient to write, store, and then read in again--why not convert directly to netcdf, and then read your data in from netcdf as needed for plotting? I suspect this will speed things up quite a bit. Numpy support for netcdf is very good. Of course, direct numpy-enabled access to the grib files might be even better, eliminating the translation phase entirely. Have you looked into http://www.pyngl.ucar.edu/Nio.shtml? Eric > hour using no more than 50% of the processor but it maxes out the memory > usage and it definitely is hard drive intensive as it ends up writing over 4 > GB of data. I have noticed also that on a lower spec AMD desktop this runs > faster than on my P4 Intel Laptop, my guess being that the laptop hdd is > 5400 rpm and the desktop is 7200 rpm. > > Next step is to take all those csv files and make images from them. For this > one I haven't dug too deep to see what is happening but it seems to be the > other way, using the cpu a lot more while keeping the memory usage high too. > > Thanks, > Anton ------------------------------------------------------------------------------ This SF.net email is sponsored by: High Quality Requirements in a Collaborative Environment. Download a free trial of Rational Requirements Composer Now! http://p.sf.net/sfu/www-ibm-com _______________________________________________ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users
