Re: [R] autocorrelation function plot in lattice

[Richard M. Heiberger]

Yes.  You may use the acf.pacf.plot, tsacfplots and related functions
in the HH package.

>From ?HH::tsacfplots

 tsacfplots(co2)
 acf.pacf.plot(co2)

If you want just the acf, and not the pacf also, you can use

update(acf.pacf.plot(co2)[1], layout=c(1,1), main="ACF: co2")

On Sat, Aug 20, 2016 at 8:28 AM, Naresh Gurbuxani
 wrote:
> Using lattice package, is it possible to plot autocorrelation functions 
> similar to acf in stats?
>
> Thanks,
> Naresh
>
> [[alternative HTML version deleted]]
>
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] autocorrelation function plot in lattice

[Jeff Newmiller]

Undoubtedly.  Consider nlme::plot.ACF as one possibility. Roll your own is also 
feasible. 
-- 
Sent from my phone. Please excuse my brevity.

On August 20, 2016 5:28:04 AM PDT, Naresh Gurbuxani 
 wrote:
>Using lattice package, is it possible to plot autocorrelation functions
>similar to acf in stats?
>
>Thanks,
>Naresh
>
>   [[alternative HTML version deleted]]
>
>__
>R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
>https://stat.ethz.ch/mailman/listinfo/r-help
>PLEASE do read the posting guide
>http://www.R-project.org/posting-guide.html
>and provide commented, minimal, self-contained, reproducible code.

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] Autocorrelation and normal distribution of gaps for ping requests in an unstable network

[Ramon Hofer]

Hi all

I have a powerline network connection which I'm investigating.
The test network contains some nodes to which I ping from one host.
The source host is always the same and I split the data to get files
for each connection.
A lot of ping requests get lost and I'm trying to plot an
autocorrelation of the data.

Here's an example log:
http://people.ee.ethz.ch/~hoferr/download/data-20130603-192.168.72.33.csv

I tried to plot the autocorrelation graph:
 acf(A$pingRTT.ms.)
which didn't work because of missing ping values. I found a post at
stackoverflow [1] where they suggest to use coredata which didn't work
for me. They also suggest to use na.action = na.omit or na.action =
na.pass. The second option works for me.

With these two commands I can draw an autocorrelation graph.
 A - read.csv('data-20130603-192.168.72.33.csv')
 acf(A$pingRTT.ms., na.action = na.pass)

But they also warn that:
acf works on regularly spaced data so acf first expands the time
series to a regularly spaced one inserting NAs as needed to make it
regularly spaced.
This seems to me as if it introduces new periods of time where there's
no ping value and thus no connection which means the autocorrelation
graph I get is nonsense.
Is my fear for no reason or is there a way to get a meaningful plot?


I'd also like to plot a histogram with normal curve like the example
from statmethods [2].
In their example they have the data directly available.
In my case I need to prepare my data to get a list of gaps. E.g.

 TimestampStart,GapLength
 2013-06-03_15:20:25.374096766,16.2s
 2013-06-03_15:22:13.944293504,37.5s
 ...

My plan is to program a loop like

 A$Timestamp - strptime(as.character(A$Timestamp), %Y-%m-%d_%H:%M:%S)
 B - matrix(nrow = 0, ncol = 2)
 colnames(B) - c(TimestampStart,GapLength[s])
 j - 1
 gap.start - A$Timestamp[0]
 for(i in 2:length(A$Timestamp)) 
 { #For all rows
  if(is.na(A$pingRTT.ms.[i]))
  { #Currently no connection
   if(!is.na(A$pingRTT.ms.[i-1]))
   { #Connection lost now
gap.start - i
   } 
   else if(!is.na(A$pingRTT.ms.[i+1])) 
   { # Connection restores next time
gap.end - i+1
B -
 rbind( B,
  c(
   A$Timestamp[gap.start],
   A$Timestamp[gap.end]-A$Timestamp[gap.start]
  ) 
 ) 
   }
  } 
 }
 x - B$GapLength
 h-hist(x, xlab=Gap Length [s?], 

There's a problem with the rbind function which I'm using wrong.
Is this the right approach and could you please give me a hint on how
to add the line?
Or is there a better way to achieve this?


Best
Ramon


[1]
http://stackoverflow.com/questions/7309411/how-to-calculate-autocorrelation-in-r-zoo-object

[2]
http://www.statmethods.net/graphs/images/histogram3.jpg

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] autocorrelation of a series

[efthimiosm]

Hello all,

I have a general question about time series and I wonder if someone 
could help me. I have time series data of this form:


x=c(rnorm(500,0,1),rnorm(500,5,1),rnorm(500,10,1),rnorm(500,3,1),rnorm(500,8,1),rnorm(500,4,1),rnorm(500,1,1),rnorm(500,7,1))
time=1:4000
plot(time,x)

Each rnorm generates a different cluster. I would like to do a 
statistical test of mean difference between the first, the second and 
the seventh cluster:


HO: mean(rnorm(500,0,1)) = mean(rnorm(500,5,1)) = mean(rnorm(500,1,1))

My questions are: (a) Can I simply do this by an ANOVA model on these 
values or by change-point analysis (using multiple.mean.norm of the 
package changepoint)? (b) If I want to check for the autocorrelation 
of my series (by acf), which data should I use: the residuals of the 
ANOVA model or the actual data? The actual data will give me high 
autocorrelation due to the trend, is that correct?


Thank you for your help,
Mike

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] Autocorrelation values? How to extract?

[Anna Zakrisson]

 Hi,

I am attempting to correct my models p-values due to temporal 
autocorrelations. It is not possible to model the correlation, so I have to 
make do with the p-value correction. I am feeling a bit thick hereI 
cannot get the autocorrelation values. What is the argument? 
My aim is to multiply the dependent variable autocorrelation with the 
independent variable autocorrelation and then multiply by (N-j)/N where N is 
the sample size and j is the lag...calculate z-value...adjust my 
p-value...Sincerely
Anna Zakrisson Braeunlich
PhD Student

Department of Systems Ecology
Stockholm University
Svante Arrheniusv. 21A
SE-106 91 Stockholm

E-mail: a...@ecology.su.se
Tel work: +46 (0)8 161103
Mobile: +46-(0)700-525015
Web site: http://www.ecology.su.se/staff/personal.asp?id=163

º`•. . • `•. .• `•. . º`•. . • `•. .• 
`•. .º

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Autocorrelation values? How to extract?

[dnz.marcio]

Hi Anna,

I think you use acf() function to calculate the variable autocorrelation.
I'd do:

ac - acf(y, lag.max = 100)$acf

Here, you use $acf, then you can extract the values only.

Best regards,
Márcio Diniz
PhD Student

IME -  Mathematical and Statistics Institute
USP -  University - São Paulo

--
View this message in context: 
http://r.789695.n4.nabble.com/Autocorrelation-values-How-to-extract-tp4278381p4278592.html
Sent from the R help mailing list archive at Nabble.com.

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] autocorrelation problem with cointegration

[upani1982]

Dear All,

I am looking for a cointegration relationship between Spot and Future Price
of commodites. The problem i am facing follows:

1. After estimating by Engle-Grranger Method, i found that the residuals are
stationary at their level I (o), which is required to fulfill the
cointegration test. But the autocorrelation problem arises, as DW statistics
is signficantly low 0.50-0.88 for various commodities. My question is shall
i go ahead with the results or not. 

2. When i use Johansens Method i found at least one cointegrtion relation.
But i am confused with lag selection criteria. I use VAR to select the
lagselection criteria. But there is autocorrelation problem with the lags it
is providing for AIC. Whether i should take first difference of the price
level to estimate the VAR, then how to use the same lag selection criteria,
when i am using price series in levels to estimate the cointegration by
johansen method. 

Looking forward for your help

With sincere regards,
Upananda

--
View this message in context: 
http://r.789695.n4.nabble.com/autocorrelation-problem-with-cointegration-tp3851336p3851336.html
Sent from the R help mailing list archive at Nabble.com.

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] Autocorrelation using acf

[Vincy Pyne]

Dear R list

As suggested by Prof Brian Ripley, I have tried to read acf literature. The 
main problem is I am not the statistician and hence have some problem in 
understanding the concepts immediately. I came across one literature 
(http://www.stat.nus.edu.sg/~staxyc/REG32.pdf) on auto-correlation giving the 
methodology. As per that literature, the auto-correlation is arrived at as per 
following.

y = 
c(15.91,9.80,17.16,16.68,15.53,22.66,31.01,8.62,45.82,10.97,45.46,28.69,36.75,37.75,
 41.18,42.67,46.05, 43.70,53.08,47.56)

t = c(1:20) # defining time variable.

Fitting y = a + bt + e, I get the estimates of a and b as a = 9.12 and b = 
2.07. So using these estimates I obtain

y_fit = 
c(11.19,13.26,15.33,17.40,19.47,21.54,23.61,25.68,27.75,29.82,31.89,33.96, 
36.03,38.10, 40.17,42.24,44.31,46.38,48.45,50.52)  # these are fitted values.


e_t = (y - y_fit)   # dif between the observed y and fitted value of 
corresponding y

 e_t
 [1]   4.72  -3.46   1.83  -0.72  -3.94   1.12   7.40
 [8] -17.06  18.07 -18.85  13.57  -5.27   0.72  -0.35
[15]  
 1.01   0.43   1.74  -2.68   4.63  -2.96

# We define 

e_t1 = 
c(-3.46,1.83,-0.72,-3.94,1.12,7.40,-17.06,18.07,-18.85,13.57,-5.27,0.72,-0.35,1.01,
 0.43,1.74,-2.68,4.63,-2.96)   # 1 st element of e_t deleted

e_t2 = 
c(4.72,-3.46,1.83,-0.72,-3.94,1.12,7.40,-17.06,18.07,-18.85,13.57,-5.27,0.72,-0.35,
 1.01, 0.43,1.74,-2.68,4.63)     # Original series with last element deleted


cor(e_t1, e_t2)

 cor(e_t1, e_t2)
[1] -0.8732316


However, if I use 

acf(y, 1)

Autocorrelations of series ‘y’, by lag

    0     1 
1.000 0.343 

I am simply not able to figure out how acf is used? 

Thanking you in advance.

Regards

Vincy

--- On Wed, 8/24/11, Prof Brian Ripley rip...@stats.ox.ac.uk wrote:

From: Prof Brian Ripley rip...@stats.ox.ac.uk
Subject: Re: [R] Autocorrelation using library(tseries)
To: Vincy Pyne vincy_p...@yahoo.ca
Cc: r-help@r-project.org
Received:
 Wednesday, August 24, 2011, 9:08 AM

Your understanding is wrong.  For a start, there is no function acf() in 
package tseries: it is in stats.

And the autocorrelation at lag one is not the correlation omitting the first 
and last values: it uses the mean and variance estimated from the whole series 
and divisor n.

Have you looked at the reference given on ?acf ?  As the help says

     (This contains the exact definitions used.)

Neither the R help pages nor R-help are intended as tutorials in statistics.

On Wed, 24 Aug 2011, Vincy Pyne wrote:

 Dear R list
 
 I am trying to understand the auto-correlation concept. Auto-correlation is 
 the self-correlation of random variable X with a certain time lag of say t.
 
 The article 
 http://www.mit.tut.fi/MIT-3010/luentokalvot/lk10-11/MDA_lecture16_11.pdf; 
 (Page no. 9 and 10) gives the methodology as under.

But that is not the definitive reference, and no, it doesn't (and what it does 
give is not the conventional definition in the time series literature).

 Suppose you have a time series observations as say
 
 X = c(44,41,46,49,49,50,40,44,49,41)
 
 # For autocorrelation with time lag of 1 we define
 
 A = c(41,46,49,49,50,40,44,49,41)?? # first element of X not considered
 B = c(44,41,46,49,49,50,40,44,49) # Last element of X not considered
 
 cor(A,B)
 [1] -0.02581234
 
 However, if I try the acf command using library tseries I get
 
 acf(X, 1)
 
 Autocorrelations of series ???X???, by
 lag
 
  0?? 1
 ??1.000 -0.019
 
 So
 by usual correlation command (where same random variable X is converted into 
two series with a time lag of 1), I obtain auto-correlation as -0.02581234 and 
by acf command I get auto-correlation = -0.019 (for time lag of 1).
 
 I am not able to figure out where I am going wrong or is it my understanding 
 of auto-correlation procedure is wrong?
 
 Will be grateful if someone guides .
 
 Vincy
 
 
 
     [[alternative HTML version deleted]]
 
 

-- Brian D. Ripley,                  rip...@stats.ox.ac.uk
Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
University of Oxford,         
    Tel:  +44 1865 272861 (self)
1 South Parks Road,                     +44 1865 272866 (PA)
Oxford OX1 3TG, UK                Fax:  +44 1865 272595

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Autocorrelation using acf

[Raphael Saldanha]

Hi Vincy,

Take a look on the material bellow, maybe they can help you:

http://www.statoek.wiso.uni-goettingen.de/veranstaltungen/zeitreihen/sommer03/ts_r_intro.pdf

http://www.maths.bris.ac.uk/~mazlc/TSA/r-ts.pdf

http://www.stat.pitt.edu/stoffer/tsa2/R_time_series_quick_fix.htm

On Thu, Aug 25, 2011 at 7:18 AM, Vincy Pyne vincy_p...@yahoo.ca wrote:

 Dear R list

 As suggested by Prof Brian Ripley, I have tried to read acf literature. The
 main problem is I am not the statistician and hence have some problem in
 understanding the concepts immediately. I came across one literature (
 http://www.stat.nus.edu.sg/~staxyc/REG32.pdf) on auto-correlation giving
 the methodology. As per that literature, the auto-correlation is arrived at
 as per following.

 y =
 c(15.91,9.80,17.16,16.68,15.53,22.66,31.01,8.62,45.82,10.97,45.46,28.69,36.75,37.75,
 41.18,42.67,46.05, 43.70,53.08,47.56)

 t = c(1:20) # defining time variable.

 Fitting y = a + bt + e, I get the estimates of a and b as a = 9.12 and b =
 2.07. So using these estimates I obtain

 y_fit =
 c(11.19,13.26,15.33,17.40,19.47,21.54,23.61,25.68,27.75,29.82,31.89,33.96,
 36.03,38.10, 40.17,42.24,44.31,46.38,48.45,50.52)  # these are fitted
 values.


 e_t = (y - y_fit)   # dif between the observed y and fitted value of
 corresponding y

  e_t
  [1]   4.72  -3.46   1.83  -0.72  -3.94   1.12   7.40
  [8] -17.06  18.07 -18.85  13.57  -5.27   0.72  -0.35
 [15]
  1.01   0.43   1.74  -2.68   4.63  -2.96

 # We define

 e_t1 =
 c(-3.46,1.83,-0.72,-3.94,1.12,7.40,-17.06,18.07,-18.85,13.57,-5.27,0.72,-0.35,1.01,
 0.43,1.74,-2.68,4.63,-2.96)   # 1 st element of e_t deleted

 e_t2 =
 c(4.72,-3.46,1.83,-0.72,-3.94,1.12,7.40,-17.06,18.07,-18.85,13.57,-5.27,0.72,-0.35,
 1.01, 0.43,1.74,-2.68,4.63) # Original series with last element deleted


 cor(e_t1, e_t2)

  cor(e_t1, e_t2)
 [1] -0.8732316


 However, if I use

 acf(y, 1)

 Autocorrelations of series ‘y’, by lag

 0 1
 1.000 0.343

 I am simply not able to figure out how acf is used?

 Thanking you in advance.

 Regards

 Vincy

 --- On Wed, 8/24/11, Prof Brian Ripley rip...@stats.ox.ac.uk wrote:

 From: Prof Brian Ripley rip...@stats.ox.ac.uk
 Subject: Re: [R] Autocorrelation using library(tseries)
 To: Vincy Pyne vincy_p...@yahoo.ca
 Cc: r-help@r-project.org
 Received:
  Wednesday, August 24, 2011, 9:08 AM

 Your understanding is wrong.  For a start, there is no function acf() in
 package tseries: it is in stats.

 And the autocorrelation at lag one is not the correlation omitting the
 first and last values: it uses the mean and variance estimated from the
 whole series and divisor n.

 Have you looked at the reference given on ?acf ?  As the help says

  (This contains the exact definitions used.)

 Neither the R help pages nor R-help are intended as tutorials in
 statistics.

 On Wed, 24 Aug 2011, Vincy Pyne wrote:

  Dear R list
 
  I am trying to understand the auto-correlation concept. Auto-correlation
 is the self-correlation of random variable X with a certain time lag of say
 t.
 
  The article 
 http://www.mit.tut.fi/MIT-3010/luentokalvot/lk10-11/MDA_lecture16_11.pdf;
 (Page no. 9 and 10) gives the methodology as under.

 But that is not the definitive reference, and no, it doesn't (and what it
 does give is not the conventional definition in the time series literature).

  Suppose you have a time series observations as say
 
  X = c(44,41,46,49,49,50,40,44,49,41)
 
  # For autocorrelation with time lag of 1 we define
 
  A = c(41,46,49,49,50,40,44,49,41)?? # first element of X not considered
  B = c(44,41,46,49,49,50,40,44,49) # Last element of X not considered
 
  cor(A,B)
  [1] -0.02581234
 
  However, if I try the acf command using library tseries I get
 
  acf(X, 1)
 
  Autocorrelations of series ???X???, by
  lag
 
   0?? 1
  ??1.000 -0.019
 
  So
  by usual correlation command (where same random variable X is converted
 into two series with a time lag of 1), I obtain auto-correlation as
 -0.02581234 and by acf command I get auto-correlation = -0.019 (for time lag
 of 1).
 
  I am not able to figure out where I am going wrong or is it my
 understanding of auto-correlation procedure is wrong?
 
  Will be grateful if someone guides .
 
  Vincy
 
 
 
  [[alternative HTML version deleted]]
 
 

 -- Brian D. Ripley,  rip...@stats.ox.ac.uk
 Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
 University of Oxford,
 Tel:  +44 1865 272861 (self)
 1 South Parks Road, +44 1865 272866 (PA)
 Oxford OX1 3TG, UKFax:  +44 1865 272595

[[alternative HTML version deleted]]


 __
 R-help@r-project.org mailing list
 https://stat.ethz.ch/mailman/listinfo/r-help
 PLEASE do read the posting guide
 http://www.R-project.org/posting-guide.html
 and provide commented, minimal, self-contained, reproducible code.




-- 
Atenciosamente,

Raphael Saldanha
saldanha.plan

[R] Autocorrelation using library(tseries)

[Vincy Pyne]

Dear R list

I am trying to understand the auto-correlation concept. Auto-correlation is the 
self-correlation of random variable X with a certain time lag of say t.

The article 
http://www.mit.tut.fi/MIT-3010/luentokalvot/lk10-11/MDA_lecture16_11.pdf; 
(Page no. 9 and 10) gives the methodology as under. 

Suppose you have a time series observations as say

X = c(44,41,46,49,49,50,40,44,49,41) 

# For autocorrelation with time lag of 1 we define 

A = c(41,46,49,49,50,40,44,49,41)  # first element of X not considered
B = c(44,41,46,49,49,50,40,44,49) # Last element of X not considered

 cor(A,B)
[1] -0.02581234

However, if I try the acf command using library tseries I get

acf(X, 1)

Autocorrelations of series ‘X’, by
 lag

     0      1 
 1.000 -0.019 

So by usual correlation command (where same random variable X is converted into 
two series with a time lag of 1), I obtain auto-correlation as -0.02581234 and 
by acf command I get auto-correlation = -0.019 (for time lag of 1).

I am not able to figure out where I am going wrong or is it my understanding of 
auto-correlation procedure is wrong?

Will be grateful if someone guides .

Vincy



[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Autocorrelation using library(tseries)

[Prof Brian Ripley]

Your understanding is wrong.  For a start, there is no function acf() 
in package tseries: it is in stats.


And the autocorrelation at lag one is not the correlation omitting the 
first and last values: it uses the mean and variance estimated from 
the whole series and divisor n.


Have you looked at the reference given on ?acf ?  As the help says

 (This contains the exact definitions used.)

Neither the R help pages nor R-help are intended as tutorials in 
statistics.


On Wed, 24 Aug 2011, Vincy Pyne wrote:


Dear R list

I am trying to understand the auto-correlation concept. 
Auto-correlation is the self-correlation of random variable X with a 
certain time lag of say t.


The article 
http://www.mit.tut.fi/MIT-3010/luentokalvot/lk10-11/MDA_lecture16_11.pdf; 
(Page no. 9 and 10) gives the methodology as under.


But that is not the definitive reference, and no, it doesn't (and what 
it does give is not the conventional definition in the time series 
literature).



Suppose you have a time series observations as say

X = c(44,41,46,49,49,50,40,44,49,41)

# For autocorrelation with time lag of 1 we define

A = c(41,46,49,49,50,40,44,49,41)?? # first element of X not considered
B = c(44,41,46,49,49,50,40,44,49) # Last element of X not considered


cor(A,B)

[1] -0.02581234

However, if I try the acf command using library tseries I get

acf(X, 1)

Autocorrelations of series ???X???, by
lag

 0?? 1
??1.000 -0.019

So by usual correlation command (where same random variable X is 
converted into two series with a time lag of 1), I obtain 
auto-correlation as -0.02581234 and by acf command I get 
auto-correlation = -0.019 (for time lag of 1).


I am not able to figure out where I am going wrong or is it my 
understanding of auto-correlation procedure is wrong?


Will be grateful if someone guides .

Vincy



[[alternative HTML version deleted]]




--
Brian D. Ripley,  rip...@stats.ox.ac.uk
Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
University of Oxford, Tel:  +44 1865 272861 (self)
1 South Parks Road, +44 1865 272866 (PA)
Oxford OX1 3TG, UKFax:  +44 1865 272595

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] Autocorrelation in R

[Iuri Gavronski]

Hi,

I am trying to learn time series, and I am attending a colleague's
course on Econometrics. However, he uses e-views, and I use R. I am
trying to reproduce his examples in R, but I am having problems
specifying a AR(1) model. Would anyone help me with my code?

Thanks in advance!

Reproducible code follows:

download.file(https://sites.google.com/a/proxima.adm.br/main/ex_32.csv
--no-check-certificate, ex_32.csv, method=wget)

ex32=read.csv(ex_32.csv)

lm_ex32=lm(gc ~ yd, data=ex32)

summary(lm_ex32)

# Durbin-Watson (slide 26)
library(lmtest)

dwtest(gc ~ yd, data=ex32)
# or
dwtest(lm_ex32)

# Breusch-Godfrey
bgtest(lm_ex32, order=2)

# AR(1)

# In e-views, the specification was:
# GC = YD AR(1)
# and the output was:

# Dependent Variable: GC
# Method: Least Squares
# Sample: 1970Q2 1995Q2
# Included observations: 101
# Convergence achieved after 6 interations
# =
# Variable    Coefficient   Std.Error t-Statistic Prob.
# C           -56.99706     19.84692  -2.871835   0.0050
# YD          0.937035      0.006520  143.7170    0.
# AR(1)       0.752407      0.066565  11.30338    0.
# =
# R-squared 0.999691 Mean dependent var 2345.867
# Adjusted R-squared 0.999685 S.D. dependent var 1284.675
# S.E. of regression 22.81029 Akaike info criterion 9.121554
# Sum squared resid 50990.32 Schwarz criterion 9.199231
# Log likelihood -457.6385 F-statistic 158548.1
# Durbin-Watson stat 2.350440 Prob(F-statistic) 0.00

# following code based on
http://www.stat.pitt.edu/stoffer/tsa2/R_time_series_quick_fix.htm
# And now for some regression with autocorrelated errors.

# I've tried to follow the example in Pinheiro  Bates (2004), p.
239-244, with no success.

gc_ts = ts(ex32[66:166,gc])
yd_ts = ts(ex32[66:166,yd])

library(nlme)
trend = time(gc_ts)

fit_lm = lm(gc_ts ~ trend + yd_ts)
acf(resid(fit_lm))
pacf(resid(fit_lm))



gls_ex32_ar1 = gls(gc_ts ~ trend + yd_ts, correlation = corAR1(form=
~yd_ts),method=ML)
summary(gls_ex32_ar1)


_
Dr. Iuri Gavronski
Assistant Professor
Programa de Pós-Graduação em Administração
Universidade do Vale do Rio dos Sinos – UNISINOS
Av. Unisinos, 950 – São Leopoldo – RS – Brasil
Sala (Room) 5A 406 D
93022-000
www.unisinos.br

TEL +55-51-3591-1122 ext. 1589
FAX +55-51-3590-8447
Email: igavron...@unisinos.br

CV Lattes: http://lattes.cnpq.br/8843390959025944

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Autocorrelation in R

[Achim Zeileis]

On Wed, 8 Jun 2011, Iuri Gavronski wrote:


Hi,

I am trying to learn time series, and I am attending a colleague's
course on Econometrics. However, he uses e-views, and I use R. I am
trying to reproduce his examples in R, but I am having problems
specifying a AR(1) model. Would anyone help me with my code?

Thanks in advance!

Reproducible code follows:

download.file(https://sites.google.com/a/proxima.adm.br/main/ex_32.csv
--no-check-certificate, ex_32.csv, method=wget)

ex32=read.csv(ex_32.csv)

lm_ex32=lm(gc ~ yd, data=ex32)

summary(lm_ex32)

# Durbin-Watson (slide 26)
library(lmtest)

dwtest(gc ~ yd, data=ex32)
# or
dwtest(lm_ex32)

# Breusch-Godfrey
bgtest(lm_ex32, order=2)

# AR(1)

# In e-views, the specification was:
# GC = YD AR(1)
# and the output was:

# Dependent Variable: GC
# Method: Least Squares
# Sample: 1970Q2 1995Q2
# Included observations: 101
# Convergence achieved after 6 interations
# =
# Variable    Coefficient   Std.Error t-Statistic Prob.
# C           -56.99706     19.84692  -2.871835   0.0050
# YD          0.937035      0.006520  143.7170    0.
# AR(1)       0.752407      0.066565  11.30338    0.
# =
# R-squared 0.999691 Mean dependent var 2345.867
# Adjusted R-squared 0.999685 S.D. dependent var 1284.675
# S.E. of regression 22.81029 Akaike info criterion 9.121554
# Sum squared resid 50990.32 Schwarz criterion 9.199231
# Log likelihood -457.6385 F-statistic 158548.1
# Durbin-Watson stat 2.350440 Prob(F-statistic) 0.00


I'm not sure what exactly E-Views does here, but an ARIMAX(1,0,0) model 
estimated by least squares seems to come rather close.


## create a time series object of your data
ex32ts - ts(ex32[,-1], start = c(1954, 1), freq = 4)

## select relevant subset
ex32ts1 - window(ex32ts, start = c(1970, 2))

## fit ARIMAX(1,0,0) model
m - arima(ex32ts1[,gc], order = c(1, 0, 0),
  xreg = ex32ts1[,yd], method = CSS)

## print output, coefficient tests, etc.
m
coeftest(m)
logLik(m)

It seems to be slightly different but that can well be due to different 
fitting algorithms...


hth,
Z


# following code based on
http://www.stat.pitt.edu/stoffer/tsa2/R_time_series_quick_fix.htm
# And now for some regression with autocorrelated errors.

# I've tried to follow the example in Pinheiro  Bates (2004), p.
239-244, with no success.

gc_ts = ts(ex32[66:166,gc])
yd_ts = ts(ex32[66:166,yd])

library(nlme)
trend = time(gc_ts)

fit_lm = lm(gc_ts ~ trend + yd_ts)
acf(resid(fit_lm))
pacf(resid(fit_lm))



gls_ex32_ar1 = gls(gc_ts ~ trend + yd_ts, correlation = corAR1(form=
~yd_ts),method=ML)
summary(gls_ex32_ar1)


_
Dr. Iuri Gavronski
Assistant Professor
Programa de Pós-Graduação em Administração
Universidade do Vale do Rio dos Sinos ? UNISINOS
Av. Unisinos, 950 ? São Leopoldo ? RS ? Brasil
Sala (Room) 5A 406 D
93022-000
www.unisinos.br

TEL +55-51-3591-1122 ext. 1589
FAX +55-51-3590-8447
Email: igavron...@unisinos.br

CV Lattes: http://lattes.cnpq.br/8843390959025944

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Autocorrelation in linear models

[Ben Bolker]

Arni Magnusson arnima at hafro.is writes:

 
 I have been reading about autocorrelation in linear models over the last 
 couple of days, and I have to say the more I read, the more confused I 
 get. Beyond confusion lies enlightenment, so I'm tempted to ask R-Help for 
 guidance.
 
 Most authors are mainly worried about autocorrelation in the residuals, 
 but some authors are also worried about autocorrelation within Y and 
 within X vectors before any model is fitted. Would you test for 
 autocorrelation both in the data and in the residuals?

   My immediate reaction is that autocorrelation in the raw data
(marginal autocorrelation) is not relevant. (There are exceptions,
of course -- in many ecological systems the marginal autocorrelation
tells us something about the processes driving the system, so we
may want to quantify/estimate it -- but I wouldn't generally think
that *testing* it (e.g. trying to reject a null hypothesis of
ACF=0) makes sense.)

 
 If we limit our worries to the residuals, it looks like we have a variety 
 of tests for lag=1:
 
stats::cor.test(residuals(fm)[-n], residuals(fm)[-1])
stats::Box.test(residuals(fm))
lmtest::dwtest(fm, alternative=two.sided)
lmtest::bgtest(fm, type=F)

  Note that (I think) all of these tests are based on
lag-1 autocorrelation only (I see you mention this
later).  Have you looked at nlme:::ACF ?  It is possible
to get non-significant autocorrelation at lag 1 with sig.
autocorrelation at higher lags.

 
 In my model, a simple lm(y~x1+x2) with n=20 annual measurements, I have 
 significant _positive_ autocorrelation within Y and within both X vectors, 
 but _negative_ autocorrelation in the residuals. 

  That's plausible. Again, I think the residual autocorrelation
is what you should worry about.

 The residual 
 autocorrelation is not quite significant, with the p-values
 
0.070
0.064
0.125
0.077
 
 from the tests above. I seem to remember some authors saying that the 
 Durbin-Watson test has less power than some alternative tests, as 
 reflected here. The difference in p-values is substantial,

  ?? I wouldn't necessarily say so -- I would guess you could get this
range of p-values from a single test statistic if you had
multiple simulated data sets from the same underlying model
and parameters ...  Have you tried running such simulations?

 so choosing 
 which test to use could in many cases make a big difference for the 
 subsequent analysis and conclusions. Most of them (cor.test, Box.test, 
 bgtest) can also test lags1. Which test would you recommend? I imagine 
 the basic cor.test is somehow inappropriate for this; the other tests 
 wouldn't have been invented otherwise, right?

  I don't know the details (it's been a while since I did time
series analysis, and it wasn't in this particular vein.)

 The car::dwt(fm) has p-values fluctuating by a factor of 2, unless I run a 
 very long simulation, which results in a p-value similar to 
 lmtest::dwtest, at least in my case.
 
 Finally, one question regarding remedies. If there was significant 
 _positive_ autocorrelation in the residuals, some authors suggest 
 remedying this by deflating the df (fewer effective df in the data) and 
 redo the t-tests of the regression coefficients, rejecting fewer null 
 hypotheses. Does that mean if the residuals are _negatively_ correlated 
 then I should inflate the df (more effective df in the data) and reject 
 more null hypotheses?

   My personal taste is that these df adjustments are bit cheesy.
Most of the time I would prefer to fit a model that incorporated
autocorrelation (i.e. nlme::gls(y~x1+x2,correlation=corAR1()) [or pick
another choice of time-series model from ?corClasses].

  More generally, this whole approach falls into the category of
test for presence of XX; if XX is not statistically significant
then ignore it, which is worrisome (if your test for XX is very
powerful then you will be concerned about dealing with XX even when
its effect on your results would be trivial; if your test for XX
is weak or you have very little data then you won't detect
XX even when it is present).  I would say that if you're really
concerned about autocorrelation you should just automatically use
a modeling approach (see above) that incorporates it.

 
 That's four question marks. I'd greatly appreciate guidance on any of 
 them.
 
 Thanks in advance,
 

  cheers
Ben

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] Autocorrelation in non-linear regression model

[pruro]

Hey all!

I am working on my master thesis and I am desperate with my model. 
It looks as following:

Y(t) = β1*X1(t) + β2*X2(t) + δ*(β1*((1+c)/(δ+c))+β2)*IE(t) -
β2*α*((1+c)/(δ+c))*(δ+g)* IE(t-1)

note: c and g is a constant value

The problem I encounter is that between IE(t) and IE(t-1) there is strong
linear correlation (autocorrelation). How can I solve this problem? Of
utterly importance is to have finally a significant coefficient δ and α
which is than used for a consecutive model. However, I get either no
significant values for δ and α, or for one of the two some unrealistic
values.

Is there an option to combine both in using some non-linear time lagged
model, time series or plugged in autoregression? 

A following up question would be how to place penalties for this model. I
would like to restrict values for δ and α between 0 and 0.5 and add
penalties when they come closer to the boundaries.

I really need some help. Because I am stuck with it for the last two weeks
and don't know how to go about it.

Thanks for the support

Cheers,

Bob


--
View this message in context: 
http://r.789695.n4.nabble.com/Autocorrelation-in-non-linear-regression-model-tp3385647p3385647.html
Sent from the R help mailing list archive at Nabble.com.

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] Autocorrelation in linear models

[Arni Magnusson]

I have been reading about autocorrelation in linear models over the last 
couple of days, and I have to say the more I read, the more confused I 
get. Beyond confusion lies enlightenment, so I'm tempted to ask R-Help for 
guidance.


Most authors are mainly worried about autocorrelation in the residuals, 
but some authors are also worried about autocorrelation within Y and 
within X vectors before any model is fitted. Would you test for 
autocorrelation both in the data and in the residuals?


If we limit our worries to the residuals, it looks like we have a variety 
of tests for lag=1:


  stats::cor.test(residuals(fm)[-n], residuals(fm)[-1])
  stats::Box.test(residuals(fm))
  lmtest::dwtest(fm, alternative=two.sided)
  lmtest::bgtest(fm, type=F)

In my model, a simple lm(y~x1+x2) with n=20 annual measurements, I have 
significant _positive_ autocorrelation within Y and within both X vectors, 
but _negative_ autocorrelation in the residuals. The residual 
autocorrelation is not quite significant, with the p-values


  0.070
  0.064
  0.125
  0.077

from the tests above. I seem to remember some authors saying that the 
Durbin-Watson test has less power than some alternative tests, as 
reflected here. The difference in p-values is substantial, so choosing 
which test to use could in many cases make a big difference for the 
subsequent analysis and conclusions. Most of them (cor.test, Box.test, 
bgtest) can also test lags1. Which test would you recommend? I imagine 
the basic cor.test is somehow inappropriate for this; the other tests 
wouldn't have been invented otherwise, right?


The car::dwt(fm) has p-values fluctuating by a factor of 2, unless I run a 
very long simulation, which results in a p-value similar to 
lmtest::dwtest, at least in my case.


Finally, one question regarding remedies. If there was significant 
_positive_ autocorrelation in the residuals, some authors suggest 
remedying this by deflating the df (fewer effective df in the data) and 
redo the t-tests of the regression coefficients, rejecting fewer null 
hypotheses. Does that mean if the residuals are _negatively_ correlated 
then I should inflate the df (more effective df in the data) and reject 
more null hypotheses?


That's four question marks. I'd greatly appreciate guidance on any of 
them.


Thanks in advance,

Arni

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] Autocorrelation Rho Greater Than One

[Hock Ann Lim]

Dear R Users,

Kindly advice me what's wrong in my programming.

I'm using the Cochrane-Orcutt two stage procedure with Prais Wisten 
transformation, below is my R programming :

Y-c(60.8,62.5,64.6,66.1,67.7,69.1,71.7,73.5,76.2,77.3,78.8,80.2,82.6,84.3,83.3,84.1,86.4,87.6,89.1,89.3,89.1,
,
+ 
89.3,90.4,90.3,90.7,92.0,94.9,95.2,96.5,95.0,96.2,97.4,100.0,99.7,99.0,98.7,99.4,100.5,105.2,108.0,112.0,113.5,

+ 115.7,117.7,119.0,120.2)
 
X-c(48.9,50.6,52.9,55.0,56.8,58.8,61.2,62.5,64.7,65.0,66.3,69.0,71.2,73.4,72.3,74.8,77.1,78.5,79.3,79.3,79.2,
,
+ 
80.8,80.1,83.0,85.2,87.1,89.7,90.1,91.5,92.4,94.4,95.9,100.0,100.4,101.3,101.5,104.5,106.5,109.5,112.8,116.1,

+ 119.1,124.0,128.7,132.7,135.7)
 model-lm(Y~X)
 e-resid(model)
 mylag-function(e,d=1) { 
+   n-length(e) 
+   c(rep(NA,d),e)[1:n] 
+ }
 n-length(e)
 e1-mylag(e)
 modele-lm(e~e1-1)
 rho-coef(modele)
 rho
   e1 
0.8875926 
 n-length(e)
 xstar-c(X[1]*(1-rho^2)^0.5,X[2:n]-rho*X[1:(n-1)])
 ystar-c(Y[1]*(1-rho^2)^0.5,Y[2:n]-rho*Y[1:(n-1)])
 modelb-lm(ystar~xstar)
 bstar-coef(modelb)
 a-(bstar[[1]][[1]])/(1-rho)
 a[1:n]-a
 b-bstar[[2]][[1]]
 u-Y-(a+X*b)
 u-u
 myu-function(u,d=1) { 
+   n-length(u) 
+   c(rep(NA,d),u)[1:n] 
+ }
 u1-myu(u)
 modelu-lm(u~u1-1)
 Rho-coef(modelu)
 Rho
  u1 
1.029970 

 The answer should be less than one but I got 1.029970. Any correction in the 
programming part? Any preventive action on this matter?

Thank you.

Regards,
Lim Hock Ann



  
[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] autocorrelation in count data

[dave fournier]

You can fit this model with AD Model Builder's random effects module.
there is an example fitting a Poisson and negative binomial to the 
venerable

polio data set with ar(1) random effects at

   
http://admb-project.org/examples/count-data/negative-binomial-serially-correlated-counts


A big advantage of ADMB is flexible modeling of both the mean and 
overdispersion.


__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] autocorrelation in count data

[sahin]

hello,

I try to model traffic accidents with the following model:

glm.nb(y~j+w+m+sf+b+ft,data=fr[]). the problem is that there exist  
autocorrelation  in the data. one possibility is to model traffic  
accidents with inar(1)-models. has anyone an idea how to change this  
model in order to abtain an integer valued time series model?


thanks
nazli

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] autocorrelation in count data

[Kjetil Halvorsen]

see
http://onlinelibrary.wiley.com/doi/10./j.1467-9892.2010.00684.x/abstract

kjetil

On Fri, Nov 19, 2010 at 6:02 PM,  sa...@hsu-hh.de wrote:
 hello,

 I try to model traffic accidents with the following model:

 glm.nb(y~j+w+m+sf+b+ft,data=fr[]). the problem is that there exist
 autocorrelation  in the data. one possibility is to model traffic accidents
 with inar(1)-models. has anyone an idea how to change this model in order to
 abtain an integer valued time series model?

 thanks
 nazli

 __
 R-help@r-project.org mailing list
 https://stat.ethz.ch/mailman/listinfo/r-help
 PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
 and provide commented, minimal, self-contained, reproducible code.


__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] Autocorrelation with merMCMC object

[berthouly cecile]

Hello,

*I have some issues with furnction that I've used before and that are not
working anymore. I want to check the autocorrelation of an object merMCMC
but the function autocorr did not acccept merMCMC object.Is there any ither
funtion that I could use?*

*mm0.REML-lmer(TD~1+SEXE+(1|PROVINCE),data=tab,REML=TRUE)*
* samp3-mcmcsamp(mm0.REML,n=6,saveb=TRUE)*
* autocorr.diag(samp3)*
Error in UseMethod(autocorr.diag) :
 no applicable method for 'autocorr.diag' applied to an object of class
merMCMC

* When I want to use the  sndow function I've got strange message...*
*I'm using R 2.11.1 version *

* samp4-window(samp3,start=10002,end=6,thin=1)*
Error in window.default(samp3, 1002, 6, thin = 1) :
  'start' cannot be after 'end'
In addition: Warning message:
In window.default(samp3, 1002, 6, thin = 1) : 'end' value not changed

*I'm sure I'm missing something, but I can not see it*

*Thank you*
*Cecile*

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] Autocorrelation in R for NLS

[ruchita gupta]

Hello

Can someone please let me know how to test for Autocorrelation in R ( eg.
like durbin-watson statistic or any other test) after performing Non linear
least squares  and what can be the best solution for it.

Thanks
Regards
Ruchita

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] Autocorrelation and t-tests

[B Garcia Carreras]

Hi,

I have two sets of data for a given set of (non-lattice) locations. I would
like to know whether the two are significantly different. This would be
simple enough if it wasn't for the fact that the data is spatially
autocorrelated. I have come across several possible solutions (including
Cliff  Ord which however appears to be for gridded data), or using gls.
However, they don't quite fit the bill (I think). Ideally it would simply be
a modified t-test which somehow 'takes care' of the spatial autocorrelation,
and it would be perfect (perhaps asking too much!) if it were a
function/package in R. I could also take the differences between the two
sets and test whether they are significantly different to 0 or not. I get
the impression that this issue must have come up again and again, so I am
hoping someone might know of an appropriate solution! Thank you very much in
advance for any help!

Bernardo

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Autocorrelation and t-tests

[milton ruser]

Hi Bernardo,

I suggest you give a look at:
Dale MRT  Fortin MJ, 2009. Spatial Autocorrelation and Statistical Tests:
Some Solutions.
Journal of Agricultural, Biological and Environmental Statistics,
14(2):188-206.

Cheers

milton

On Tue, Aug 25, 2009 at 1:08 PM, B Garcia Carreras 
bernardo.garcia-carrera...@imperial.ac.uk wrote:

 Hi,

 I have two sets of data for a given set of (non-lattice) locations. I would
 like to know whether the two are significantly different. This would be
 simple enough if it wasn't for the fact that the data is spatially
 autocorrelated. I have come across several possible solutions (including
 Cliff  Ord which however appears to be for gridded data), or using gls.
 However, they don't quite fit the bill (I think). Ideally it would simply
 be
 a modified t-test which somehow 'takes care' of the spatial
 autocorrelation,
 and it would be perfect (perhaps asking too much!) if it were a
 function/package in R. I could also take the differences between the two
 sets and test whether they are significantly different to 0 or not. I get
 the impression that this issue must have come up again and again, so I am
 hoping someone might know of an appropriate solution! Thank you very much
 in
 advance for any help!

 Bernardo

[[alternative HTML version deleted]]

 __
 R-help@r-project.org mailing list
 https://stat.ethz.ch/mailman/listinfo/r-help
 PLEASE do read the posting guide
 http://www.R-project.org/posting-guide.htmlhttp://www.r-project.org/posting-guide.html
 and provide commented, minimal, self-contained, reproducible code.


[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] autocorrelation

[Michael Denslow]

 
 Hi
 Is any multiple regression-like test with correction for
 autocorrelation ?

If I understand your question, yes. Take a look at the spdep package for 
starters. Also you may find the following references helpful. 

Dormann et al. 2007. Methods to account for spatial autocorrelation in the 
analysis of species distributional data: a review. Ecography 30:609-628

Also the book by Bivand et al. 2008. (Applied Spatial Data Analysis with R. 
from Springer) is very good.

Hope this helps,

Michael Denslow

I.W. Carpenter Jr. Herbarium [BOON]
Appalachian State University
Boone, North Carolina U.S.A.

-- AND --

Communications Manager
Southeastern Regional Network of Expertise and Collections
sernec.org

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] autocorrelation

[jan nowak]

Hi
Is any multiple regression-like test with correction for autocorrelation ?

Wojciech

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] autocorrelation in gams

[Gavin Simpson]

On Thu, 2008-08-14 at 16:12 +0100, Abigail McQuatters-Gollop wrote:
 Hi,
 
 I am looking at the effects of two explanatory variables on chlorophyll.
 The data are an annual time-series (so are autocorrelated) and the
 relationships are non-linear. I want to account for autocorrelation in
 my model. 
 
  
 
 The model I am trying to use is this:
 
  
 
 Library(mgcv)
 
  
 
 gam1 -gam(Chl~s(wintersecchi)+s(SST),family=gaussian,
 na.action=na.omit, correlation=corAR1(form =~ Year)) 
 

There is no correlation argument in mgcv::gam you need gamm(). gam() has
a ... argument which I suspect is silently mopping up your correlation
argument so that no error/warning is raised.

Note that gamm() uses lme from the nlme package (in the Gaussian case)
and works that function very hard (see Wood 2006 GAM book). In my
experience with this function separating trend from the correlation is
quite difficult when also estimating the degree of smoothness and one
has to work hard with the options.

As an alternative you might also take a look at the paper by Ferguson et
al (2007):

http://www3.interscience.wiley.com/journal/119392062/abstract?CRETRY=1SRETRY=0

Which has R code using the sm package to do a very similar thing.

HTH

G
 
 
 the result I get is this: 
 
  
 
 Family: gaussian 
 
 Link function: identity 
 
  
 
 Formula:
 
 CPRChl ~ s(wintersecchi) + s(SST)
 
  
 
 Parametric coefficients:
 
 Estimate Std. Error t value Pr(|t|)
 
 (Intercept)  3.570000.05061   70.54   2e-16 ***
 
 ---
 
 Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 
 
  
 
 Approximate significance of smooth terms:
 
   edf Est.rank F p-value   
 
 s(wintersecchi) 2.4455 4.672 0.00887 **
 
 s(SST)  2.4085 4.301 0.01237 * 
 
 ---
 
 Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 
 
  
 
 R-sq.(adj) =  0.676   Deviance explained = 75.4%
 
 GCV score = 0.074563   Scale est. = 0.053781  n = 21
 
  
 
 The result look good - significant, with a lot of deviance explained,
 but I am not convinced the model is actually accounting for the
 autocorrelation (based on the formula in the results). How can I tell? 
 
  
 
 Many thanks,
 
  
 
 
 
 
 
 Dr Abigail McQuatters-Gollop
 
 Sir Alister Hardy Foundation for Ocean Science (SAHFOS)
 
 The Laboratory
 
 Citadel Hill
 
 Plymouth UK PL1 2PB
 
 tel: +44 1752 633233
 
  
 
 
   [[alternative HTML version deleted]]
 
 __
 R-help@r-project.org mailing list
 https://stat.ethz.ch/mailman/listinfo/r-help
 PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
 and provide commented, minimal, self-contained, reproducible code.

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] autocorrelation in gams

[Kingsford Jones]

Keeping Gavin's advice in mind, you may also want to look at ?acf (and
see section 14.1 of MASS) and help(ACF, package=nlme) (see section 5.3
of MEMSS).  These are useful functions for exploring the 1d empirical
autocorrelation structure of model residuals.

hth,
Kingsford Jones

On Fri, Aug 15, 2008 at 1:18 AM, Gavin Simpson [EMAIL PROTECTED] wrote:
 On Thu, 2008-08-14 at 16:12 +0100, Abigail McQuatters-Gollop wrote:
 Hi,

 I am looking at the effects of two explanatory variables on chlorophyll.
 The data are an annual time-series (so are autocorrelated) and the
 relationships are non-linear. I want to account for autocorrelation in
 my model.



 The model I am trying to use is this:



 Library(mgcv)



 gam1 -gam(Chl~s(wintersecchi)+s(SST),family=gaussian,
 na.action=na.omit, correlation=corAR1(form =~ Year))


 There is no correlation argument in mgcv::gam you need gamm(). gam() has
 a ... argument which I suspect is silently mopping up your correlation
 argument so that no error/warning is raised.

 Note that gamm() uses lme from the nlme package (in the Gaussian case)
 and works that function very hard (see Wood 2006 GAM book). In my
 experience with this function separating trend from the correlation is
 quite difficult when also estimating the degree of smoothness and one
 has to work hard with the options.

 As an alternative you might also take a look at the paper by Ferguson et
 al (2007):

 http://www3.interscience.wiley.com/journal/119392062/abstract?CRETRY=1SRETRY=0

 Which has R code using the sm package to do a very similar thing.

 HTH

 G


 the result I get is this:



 Family: gaussian

 Link function: identity



 Formula:

 CPRChl ~ s(wintersecchi) + s(SST)



 Parametric coefficients:

 Estimate Std. Error t value Pr(|t|)

 (Intercept)  3.570000.05061   70.54   2e-16 ***

 ---

 Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1



 Approximate significance of smooth terms:

   edf Est.rank F p-value

 s(wintersecchi) 2.4455 4.672 0.00887 **

 s(SST)  2.4085 4.301 0.01237 *

 ---

 Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1



 R-sq.(adj) =  0.676   Deviance explained = 75.4%

 GCV score = 0.074563   Scale est. = 0.053781  n = 21



 The result look good - significant, with a lot of deviance explained,
 but I am not convinced the model is actually accounting for the
 autocorrelation (based on the formula in the results). How can I tell?



 Many thanks,







 Dr Abigail McQuatters-Gollop

 Sir Alister Hardy Foundation for Ocean Science (SAHFOS)

 The Laboratory

 Citadel Hill

 Plymouth UK PL1 2PB

 tel: +44 1752 633233




   [[alternative HTML version deleted]]

 __
 R-help@r-project.org mailing list
 https://stat.ethz.ch/mailman/listinfo/r-help
 PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
 and provide commented, minimal, self-contained, reproducible code.

 __
 R-help@r-project.org mailing list
 https://stat.ethz.ch/mailman/listinfo/r-help
 PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
 and provide commented, minimal, self-contained, reproducible code.


__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] autocorrelation in gams

[Abigail McQuatters-Gollop]

Hi,

I am looking at the effects of two explanatory variables on chlorophyll.
The data are an annual time-series (so are autocorrelated) and the
relationships are non-linear. I want to account for autocorrelation in
my model. 

 

The model I am trying to use is this:

 

Library(mgcv)

 

gam1 -gam(Chl~s(wintersecchi)+s(SST),family=gaussian,
na.action=na.omit, correlation=corAR1(form =~ Year)) 

 

the result I get is this: 

 

Family: gaussian 

Link function: identity 

 

Formula:

CPRChl ~ s(wintersecchi) + s(SST)

 

Parametric coefficients:

Estimate Std. Error t value Pr(|t|)

(Intercept)  3.570000.05061   70.54   2e-16 ***

---

Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 

 

Approximate significance of smooth terms:

  edf Est.rank F p-value   

s(wintersecchi) 2.4455 4.672 0.00887 **

s(SST)  2.4085 4.301 0.01237 * 

---

Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 

 

R-sq.(adj) =  0.676   Deviance explained = 75.4%

GCV score = 0.074563   Scale est. = 0.053781  n = 21

 

The result look good - significant, with a lot of deviance explained,
but I am not convinced the model is actually accounting for the
autocorrelation (based on the formula in the results). How can I tell? 

 

Many thanks,

 

 

 

Dr Abigail McQuatters-Gollop

Sir Alister Hardy Foundation for Ocean Science (SAHFOS)

The Laboratory

Citadel Hill

Plymouth UK PL1 2PB

tel: +44 1752 633233

 


[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] autocorrelation error: cannot allocate vector of size 220979 Kb

[J S]

Thanks. Here are some information about my computer and file:



Operating system: Windows 2000



RAM: 1.99 GB



After I run the program:

 gc()

  used (Mb) gc trigger  (Mb) max used  (Mb)

Ncells  468065 12.5 818163  21.9   818163  21.9

Vcells 1160828  8.9   46162021 352.2 54869490 418.7



 object.size(a)

[1] 2144392



 str(a)

'data.frame':   16925 obs. of  14 variables:

 $ Site  : Factor w/ 6 levels HD,LEA,MCD,..: 6 6 6 6 6 6 6 6 6 6
...

 $ Plot  : num  1 1 1 1 1 1 1 1 1 1 ...

 $ Veg   : Factor w/ 2 levels Forest,Grass: 2 2 2 2 2 2 2 2 2 2 ...

 $ Landuse   : Factor w/ 2 levels Rural,Urban: 2 2 2 2 2 2 2 2 2 2 ...

 $ Date  : Factor w/ 2484 levels 01-Apr-01,01-Apr-03,..: 1078 1162
1244 1326 1407 1488 1569 1650 1731 1812 ...

 $ Soil.temp : num  26.1 25.9 26.0 25.7 25.5 ...

 $ combin: Factor w/ 6 levels HDForestUrban,..: 6 6 6 6 6 6 6 6 6 6
...

 $ year_scale: num  -1.4 -1.4 -1.4 -1.4 -1.4 ...

 $ day   : num  14 15 16 17 18 19 20 21 22 23 ...

 $ M : num  8 8 8 8 8 8 8 8 8 8 ...

 $ year  : Factor w/ 8 levels 2000,2001,..: 4 4 4 4 4 4 4 4 4 4 ...

 $ combin2   : Factor w/ 11 levels HDForestUrban1,..: 10 10 10 10 10 10 10
10 10 10 ...

 $ time  : num  225 226 227 228 229 230 231 232 233 234 ...

 $ time_scale: num  32.8 33.8 34.8 35.8 36.8 ...




On Sat, May 17, 2008 at 5:04 PM, jim holtman [EMAIL PROTECTED] wrote:

 More information is needed.  What is your operating system?  How much RAM
 do you have?  Are there other objects in memory that you could delete to
 recover some space?  What does 'str' and 'object.size' say for the data you
 are analyzing?  What does 'gc()' report  -  you may want to do this
 before/after sections of code to see how memory might be growing.

   On Fri, May 16, 2008 at 1:56 PM, J S [EMAIL PROTECTED] wrote:

  Dear R community,



 I used a linear mixed model (named lm11) to model daily soil temperature
 depending upon vegetation cover and air temperature. I have almost 17,000
 observations for six years.



 I can not account for autocorrelation in my model, since I receive the
 error
 message after applying the function:



 update(lm11, corr=corAR1())



 Error: cannot allocate vector of size 220979 Kb



 Do you have any suggestions?



 Thanks, Julia

[[alternative HTML version deleted]]

 __
 R-help@r-project.org mailing list
 https://stat.ethz.ch/mailman/listinfo/r-help
 PLEASE do read the posting guide
 http://www.R-project.org/posting-guide.htmlhttp://www.r-project.org/posting-guide.html
 and provide commented, minimal, self-contained, reproducible code.




 --
 Jim Holtman
 Cincinnati, OH
 +1 513 646 9390

 What is the problem you are trying to solve?

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] autocorrelation in nlme: Error: cannot allocate vector of size 220979 Kb

[J S]

Dear R community,



Below you may find the details of my model (lm11). I receive the error
message Error: cannot allocate vector of size 220979 Kb after
applying the autocorrelation function update(lm11, corr=corAR1()).



lm11-lme(Soil.temp ~ Veg*M+Veg*year,

   data=a,

   random = list(Site=pdDiag(~Veg),

   Plot=pdDiag(~Veg))

Dataset:

a-data frame of daily measurements of soil temperature (Soil.temp)
over six years

Site (6 sites),

Plot(2 plots per site),

Veg(2 vegetation types: 2 sites as grassland, 4 sites as forest)

M-month (categorical predictor)

year (continues)



Thanks,

Julia

P.S. I a sorry if this message showed up a few times, since I had
trouble posting it.

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] autocorrelation error: cannot allocate vector of size 220979 Kb

[jim holtman]

More information is needed.  What is your operating system?  How much RAM do
you have?  Are there other objects in memory that you could delete to
recover some space?  What does 'str' and 'object.size' say for the data you
are analyzing?  What does 'gc()' report  -  you may want to do this
before/after sections of code to see how memory might be growing.

On Fri, May 16, 2008 at 1:56 PM, J S [EMAIL PROTECTED] wrote:

 Dear R community,



 I used a linear mixed model (named lm11) to model daily soil temperature
 depending upon vegetation cover and air temperature. I have almost 17,000
 observations for six years.



 I can not account for autocorrelation in my model, since I receive the
 error
 message after applying the function:



 update(lm11, corr=corAR1())



 Error: cannot allocate vector of size 220979 Kb



 Do you have any suggestions?



 Thanks, Julia

[[alternative HTML version deleted]]

 __
 R-help@r-project.org mailing list
 https://stat.ethz.ch/mailman/listinfo/r-help
 PLEASE do read the posting guide
 http://www.R-project.org/posting-guide.htmlhttp://www.r-project.org/posting-guide.html
 and provide commented, minimal, self-contained, reproducible code.




-- 
Jim Holtman
Cincinnati, OH
+1 513 646 9390

What is the problem you are trying to solve?

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] autocorrelation error: cannot allocate vector of size 220979 Kb

[J S]

Dear R community,



I used a linear mixed model (named lm11) to model daily soil temperature
depending upon vegetation cover and air temperature. I have almost 17,000
observations for six years.



I can not account for autocorrelation in my model, since I receive the error
message after applying the function:



update(lm11, corr=corAR1())



Error: cannot allocate vector of size 220979 Kb



Do you have any suggestions?



Thanks, Julia

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] autocorrelation in nlme; Error: cannot allocate vector of size

[J S]

Dear R community,



I used a linear mixed model (named lm11) to model daily soil temperature
depending upon vegetation cover and air temperature. I have almost 17,000
observations for six years.



I can not account for autocorrelation in my model, since I receive the error
message after applying the function:



update(lm11, corr=corAR1())



Error: cannot allocate vector of size 220979 Kb



Do you have any suggestions?



Thanks, Julia

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] autocorrelation in nlme; Error: cannot allocate vector of size

[ONKELINX, Thierry]

Dear Julia,

You'll need to give more details on your model and the structure of your
dataset. The problem will probably be in the specification of the random
effects. But without the detail we can't check that.

Thierry




ir. Thierry Onkelinx
Instituut voor natuur- en bosonderzoek / Research Institute for Nature
and Forest
Cel biometrie, methodologie en kwaliteitszorg / Section biometrics,
methodology and quality assurance
Gaverstraat 4
9500 Geraardsbergen
Belgium 
tel. + 32 54/436 185
[EMAIL PROTECTED] 
www.inbo.be 

To call in the statistician after the experiment is done may be no more
than asking him to perform a post-mortem examination: he may be able to
say what the experiment died of.
~ Sir Ronald Aylmer Fisher

The plural of anecdote is not data.
~ Roger Brinner

The combination of some data and an aching desire for an answer does not
ensure that a reasonable answer can be extracted from a given body of
data.
~ John Tukey

-Oorspronkelijk bericht-
Van: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]
Namens J S
Verzonden: vrijdag 16 mei 2008 23:06
Aan: r-help@r-project.org
Onderwerp: [R] autocorrelation in nlme; Error: cannot allocate vector of
size

Dear R community,



I used a linear mixed model (named lm11) to model daily soil temperature
depending upon vegetation cover and air temperature. I have almost
17,000
observations for six years.



I can not account for autocorrelation in my model, since I receive the
error
message after applying the function:



update(lm11, corr=corAR1())



Error: cannot allocate vector of size 220979 Kb



Do you have any suggestions?



Thanks, Julia

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] autocorrelation in nlme; Error: cannot allocate vector of size

[J S]

Dear R community,



Here are details of my model, which gives me trouble modeling
autocorrelation.



lm11-lme(Soil.temp ~ Veg*M+Veg*year,

   data=a,

   random = list(Site=pdDiag(~Veg),

   Plot=pdDiag(~Veg))



dataset:



a-data frame of daily measurements of soil temperature (Soil.temp) over six
years

Site (6 sites),

Plot(2 plots per site),

Veg(2 vegetation types: 2 sites as grassland, 4 sites as forest)

M-month (categorical predictor)

year (continues)



Thanks,

Julia


 Van: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]
 Namens J S
 Verzonden: vrijdag 16 mei 2008 23:06
 Aan: r-help@r-project.org
 Onderwerp: [R] autocorrelation in nlme; Error: cannot allocate vector of
 size

 Dear R community,



 I used a linear mixed model (named lm11) to model daily soil temperature
 depending upon vegetation cover and air temperature. I have almost
 17,000
 observations for six years.



 I can not account for autocorrelation in my model, since I receive the
 error
 message after applying the function:



 update(lm11, corr=corAR1())



 Error: cannot allocate vector of size 220979 Kb



 Do you have any suggestions?



 Thanks, Julia

[[alternative HTML version deleted]]

 __
 R-help@r-project.org mailing list
 https://stat.ethz.ch/mailman/listinfo/r-help
 PLEASE do read the posting guide
 http://www.R-project.org/posting-guide.htmlhttp://www.r-project.org/posting-guide.html
 and provide commented, minimal, self-contained, reproducible code.


[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] autocorrelation by group in mixed model

[Irene Mantzouni]

Hi all!
 
(How) is it possible to fit a mixed model with group specific auto-correlation 
structure ? For instance, not all my groups display auto-correlation so I would 
like to use a corMatrix (corAR1) only for the auto-correlated ones. If I 
construct manually a
the corMatrix, is it possible to use it  as input somehow?
 
thank you!
 
Irene

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] Autocorrelation Matrix

[Jan Feyereisl]

Hi,
I am trying to calculate the autocorrelation matrix for an input matrix with 
the size n*m where n=7 (the dimensionality of my input feature vectors) and m 
being the time. Thus one could think of the input data as a 7-dimensional 
time-series. Does anyone know of any way to calculate the autocorrelation 
matrix for such an input? I tried various functions, but none give me a matrix 
that returns this information.
 
Thank you very much in advance,
Jan
_
Celeb spotting – Play CelebMashup and win cool prizes

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.