Thank you Pierre,
I will test the other options. I did not know the
number limitation in case of plt.xcorr.
Thanks a lot
with best regards,
Sudheer
***************************************************************
Sudheer Joseph
Indian National Centre for Ocean Information Services
Ministry of Earth Sciences, Govt. of India
POST BOX NO: 21, IDA Jeedeemetla P.O.
Via Pragathi Nagar,Kukatpally, Hyderabad; Pin:5000 55
Tel:+91-40-23886047(O),Fax:+91-40-23895011(O),
Tel:+91-40-23044600(R),Tel:+91-40-9440832534(Mobile)
E-mail:sjo.in...@gmail.com;sudheer.jos...@yahoo.com
Web- http://oppamthadathil.tripod.com
***************************************************************
________________________________
From: Pierre Haessig <pierre.haes...@crans.org>
To: matplotlib-users@lists.sourceforge.net
Sent: Thursday, 21 February 2013 9:52 PM
Subject: Re: [Matplotlib-users] cross correlation
Hi Sudheer,
Le 21/02/2013 02:22, Sudheer Joseph a écrit :
Thank you very much Smith and Paul,
> I was away from office due to a
>medical situation. So could not respond and thank you regarding the help. I
>have got the results now and the tips from both of you were extremely useful.
>I am facing an issue with the code when I call plt.xcorr, in a loop. it
>builds up usage of memory by python and reaches to the RAM what ever available
>( in my 4 GB laptop it reaches almost full and in my 24 GB desktop it reaches
>the available. I suspected the plot not being closed during each iteration so
>have given a plt.close('all') in the loop. after which it is taking a good
>time to run the code which was otherwise faster until ram usage reaches its
>maximum.
>Is there a way to get out of this situation?. I am attaching the code here and
>also the link to the data I am using. If possible kindly help.
>
Thanks for sharing the code. By a quick look at gen_xcorr_wnd.py,
you are generating a quite high number (about len(lons)*len(lats))
of xcorr series over 365 lags. Here are two thoughts about why I
would not recommend using xcorr from matplotlib for this job :
1) There is an overhead in creating a plot object which is
unnecessary since you're only interested in correlation values
2) internally, plt.xcorr uses numpy.correlate
(https://github.com/matplotlib/matplotlib/blob/master/lib/matplotlib/axes.py#L4319
and https://github.com/numpy/numpy/blob/master/numpy/core/numeric.py#L731)
which is quite fast but unfortunately cannot be well tuned in terms of the
output length (only three modes : 'valid', 'same' or 'full'. Matplotlib uses
'full' )
All this to say that when you're interested in 365 correlation
values, the internal computations takes place on (N+M-1) points
(where N, M are the length of the input vectors, i.e. 2189 if I'm
right) and so about 90 % of the output is thrown away.
This being said, there is a tiny issue : I don't know a good module
which has the (x)correlation function. statsmodel has acf (aka
correlation) but I don't remember if there is crosscorrelation. For
acf has two computation modes : one based on fft, one based on
numpy.correlate which suffer from the same problem as matplotlib's
xcorr (
https://github.com/statsmodels/statsmodels/blob/master/statsmodels/tsa/stattools.py#L347)
best,
Pierre
------------------------------------------------------------------------------
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics
Download AppDynamics Lite for free today:
http://p.sf.net/sfu/appdyn_d2d_feb
_______________________________________________
Matplotlib-users mailing list
Matplotlib-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/matplotlib-users------------------------------------------------------------------------------
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics
Download AppDynamics Lite for free today:
http://p.sf.net/sfu/appdyn_d2d_feb
_______________________________________________
Matplotlib-users mailing list
Matplotlib-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/matplotlib-users
