Infinite impulse response (IIR) filters are when you apply a linear recursive filter that takes into account past outputs, up to order 'p', p being finite. If p is not finite, then you have essentially an infinite memory system, that is, your PACF would be significant even at extremely high lags, out to infinity. This is more like a nonlinear process (loosely speaking).
When you inspect the autocorrelation function of the raw signal, you don't see convergence. It is literally flat, and definitely above the required level for significance, and shows no sign of decay after the first 25 lags (i've observed it out to 1000 lags, and this is a signal that contains 45,000 points so I can do that...although matlab starts choking at this point). When you detrend it (linearly) you find that it reduces the ACF peaks, but they are still flat and still extend to infinity. Furthermore, if you look at the power spectrum (of either the original or the linearly detrended series), although the filter removed a significant amount of power at higher frequencies (>400Hz), the power is not 'zero' in these higher frequencies after filtering. This corresponds to the fact that both the autocorrelation function and the power spectrum are derived from the autocovariance function, so they would reflect the same thing in two different ways. My point is that the signal still has a significant amount of power which does not seem to diminish to zero; in fact, it exhibits a small peak near 480 Hz (max. ampl. ~10^2, where Power is |A|^2). The power in the low frequencies (<400Hz) is around O(10^4), while in the filtered region, the power is O(10^2). An IIR filter would show a finite PACF, not one with significant peaks at 198 lags, and so on out to infinity... On the other extreme, we can rule out the possibility that this is due to a first-order smoothing operation. Comparing the signal to a simple MA(1) process (infinite AR) I find that the profile of the PACF is different. That is, the original signal does not show a 'decaying' envelope (recall, an MA(1) process has an infinite AR representation, BUT the coefficients of past influence have successively less and less effect on the present (-phi^2 - phi^3 - phi^4 - ...). This type of decay is not observed in the signal i'm looking at. I see an inverted parabolic 'decay' (if you can call it that), with a constant 'regenerated' significant peak (that varies in significance) every 100 ms or so O(0.01) which are just large enough to be above the 95% confidence limits. If there was only one or two of these, one could think it was spurious, but if its effect is repeated every 100 ms or so, one is less inclined to believe this. It's difficult to describe without pictures unfortunately. P _____________________________________ Pradyumna Sribharga Upadrashta, PhD Student Scientific Computation, UofMN >-----Original Message----- >From: [EMAIL PROTECTED] >[mailto:[EMAIL PROTECTED] On Behalf Of Jason Turner >Sent: Saturday, September 27, 2003 1:55 AM >To: [EMAIL PROTECTED] >Cc: Pradyumna S Upadrashta >Subject: Re: [edstat] stationarity, time-series analysis, >power spectra, etc > > >On Sat, 2003-09-27 at 15:00, Pradyumna S Upadrashta wrote: >> If one applies a linear filter, with a lowpass of 400Hz, is it >> possible for it to induce spurious autocorrelation (that persists >> essentially what seems like 'forever') in a time-series? > >Was the filter a real-time filter (analog or digital), or was >it a frequency-domain filter applied after the fact? > >The effect you describe is how most linear filters work, in >practice; the real-time ones are called "recursive". Another >name for them is "infinite impulse response" filters (that >should give you a hint). > >... >> However, upon inspection of the power spectrum, I notice a roll-off >> behavior ~400Hz (orig sampling rate was 1017Hz) and the ACF and PACF >> display very odd behavior as mentioned. > >That's only 100 Hz you've got until you hit the Nyquist >frequency (which I'm sure you know). Data near the Nyquist >frequency, and near 0 Hz, are both a bit biased if you're >using an FFT anyway (are you using the fftw libraries? >they're quite nice for this). > >> Our data acquisition person >> insists that a standard low-pass filter was used, with a DC >offset. If >> this is the case, then i'm unable to understand why a >time-series that >> should be pure white noise, isn't... > >See above. > >> The caveat of not being able to 'know for sure' when something is >> stationary a priori, is that one can't rely on the power spectrum >> since it assumes a stationary signal structure for its >estimation! (or >> am I wrong about this?) > >You are. Read Bloomfield's introduction, with the notion of >"detrending" the raw data. > >> I suppose one could look for stationary segments of the signal, and >> then attempt to compare spectra across multiple 'stationary looking' >> segments; if the signal really is stationary, i'd expect no changes >> right? I have 45,000 data points in a single time-series. > >See detrending. > >Cheers > >Jason >-- >Indigo Industrial Controls Ltd. http://www.indigoindustrial.co.nz >+64-(0)21-343-545 > > > > >. >. ================================================================= >Instructions for joining and leaving this list, remarks about >the problem of INAPPROPRIATE MESSAGES, and archives are available at: >. http://jse.stat.ncsu.edu/ . >================================================================= > . . ================================================================= Instructions for joining and leaving this list, remarks about the problem of INAPPROPRIATE MESSAGES, and archives are available at: . http://jse.stat.ncsu.edu/ . =================================================================
