Re: [R] Need help with time series
Thanks for providing direction. By redefining time using as.yearmon(), I got
acf() working.
Sent from my iPhone
On Jan 14, 2025, at 6:51 PM, CALUM POLWART wrote:
acf wants a time series, so tries to make one:
as.ts(myts)
Time Series:
Start = 19357
End = 20027
Frequency = 1
[1] 24957 NA NA NA NA NA NA NA NA NA
[11] NA NA NA NA NA NA NA NA NA NA
[21] NA NA NA NA NA NA NA NA NA NA
[31] NA 10577 NA NA NA NA NA NA NA NA
[41] NA NA NA NA NA NA NA NA NA NA
[51] NA NA NA NA NA NA NA NA NA NA
[61] NA NA -18516 NA NA NA NA NA NA NA
[71] NA NA NA NA NA NA NA NA NA NA
[81] NA NA NA NA NA NA NA NA NA NA
[91] 2940 NA NA NA NA NA NA NA NA NA
[101] NA NA NA NA NA NA NA NA NA NA
[111] NA NA NA NA NA NA NA NA NA NA
[121] NA -1458 NA NA NA NA NA NA NA NA
[131] NA NA
I'm not familiar enough with TS to know how you make them 31d month units...
On Tue, 14 Jan 2025, 23:08 Naresh Gurbuxani,
mailto:naresh_gurbux...@hotmail.com>> wrote:
For below data, I find strange results in basic time series analysis. Why does
acf() function find missing values? When crossprod(xmat) is invertible, why
does arima() find system exactly singular?
Thanks,
Naresh
x <- c(24957, 10577, -18516, 2940, -1458, 32704, -26697, -46902, 48413, -11937,
2043, 26431, -55336, -16838, 89651, 25363,-50388, -41012, -28242, -18213,
58759, -15290, -7413, 124098)
myts <- zoo::zoo(x, seq.Date(as.Date("2022-12-31"),
as.Date("2024-11-30"), by = "1 month"))
> myts
2022-12-31 2023-01-31 2023-03-03 2023-03-31 2023-05-01 2023-05-31 2023-07-01
24957 10577 -18516 2940 -1458 32704 -26697
2023-07-31 2023-08-31 2023-10-01 2023-10-31 2023-12-01 2023-12-31 2024-01-31
-46902 48413 -11937 2043 26431 -55336 -16838
2024-03-02 2024-03-31 2024-05-01 2024-05-31 2024-07-01 2024-07-31 2024-08-31
89651 25363 -50388 -41012 -28242 -18213 58759
2024-10-01 2024-10-31
-15290 -7413
> coredata(myts) |> class()
[1] "numeric"
> acf(myts, lag.max = 2)
Error in na.fail.default(as.ts(x)) : missing values in object
> arima(myts, order = c(1, 0, 0))
Error in solve.default(res$hessian * n.used, A) :
Lapack routine dgesv: system is exactly singular: U[1,1] = 0
> xmat <- na.omit(cbind(x = myts, xlag1 = lag(myts, k = -1)))
> crossprod(xmat) |> solve()
xxlag1
x 3.488838e-11 1.949958e-12
xlag1 1.949958e-12 3.421268e-11
[[alternative HTML version deleted]]
__
R-help@r-project.org mailing list -- To
UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide https://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
[[alternative HTML version deleted]]
__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide https://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Re: [R] Need help with time series
acf wants a time series, so tries to make one:
as.ts(myts)
Time Series:
Start = 19357
End = 20027
Frequency = 1
[1] 24957 NA NA NA NA NA NA NA NA NA
[11] NA NA NA NA NA NA NA NA NA NA
[21] NA NA NA NA NA NA NA NA NA NA
[31] NA 10577 NA NA NA NA NA NA NA NA
[41] NA NA NA NA NA NA NA NA NA NA
[51] NA NA NA NA NA NA NA NA NA NA
[61] NA NA -18516 NA NA NA NA NA NA NA
[71] NA NA NA NA NA NA NA NA NA NA
[81] NA NA NA NA NA NA NA NA NA NA
[91] 2940 NA NA NA NA NA NA NA NA NA
[101] NA NA NA NA NA NA NA NA NA NA
[111] NA NA NA NA NA NA NA NA NA NA
[121] NA -1458 NA NA NA NA NA NA NA NA
[131] NA NA
I'm not familiar enough with TS to know how you make them 31d month
units...
On Tue, 14 Jan 2025, 23:08 Naresh Gurbuxani,
wrote:
> For below data, I find strange results in basic time series analysis. Why
> does acf() function find missing values? When crossprod(xmat) is
> invertible, why does arima() find system exactly singular?
>
> Thanks,
> Naresh
>
> x <- c(24957, 10577, -18516, 2940, -1458, 32704, -26697, -46902, 48413,
> -11937, 2043, 26431, -55336, -16838, 89651, 25363,-50388, -41012,
> -28242, -18213, 58759, -15290, -7413, 124098)
>
> myts <- zoo::zoo(x, seq.Date(as.Date("2022-12-31"),
> as.Date("2024-11-30"), by = "1 month"))
>
> > myts
>
> 2022-12-31 2023-01-31 2023-03-03 2023-03-31 2023-05-01 2023-05-31
> 2023-07-01
>
> 24957 10577 -18516 2940 -1458 32704
> -26697
>
> 2023-07-31 2023-08-31 2023-10-01 2023-10-31 2023-12-01 2023-12-31
> 2024-01-31
>
> -46902 48413 -11937 2043 26431 -55336
> -16838
>
> 2024-03-02 2024-03-31 2024-05-01 2024-05-31 2024-07-01 2024-07-31
> 2024-08-31
>
> 89651 25363 -50388 -41012 -28242 -18213
> 58759
>
> 2024-10-01 2024-10-31
>
> -15290 -7413
>
> > coredata(myts) |> class()
>
> [1] "numeric"
>
> > acf(myts, lag.max = 2)
>
> Error in na.fail.default(as.ts(x)) : missing values in object
>
> > arima(myts, order = c(1, 0, 0))
>
> Error in solve.default(res$hessian * n.used, A) :
>
> Lapack routine dgesv: system is exactly singular: U[1,1] = 0
>
> > xmat <- na.omit(cbind(x = myts, xlag1 = lag(myts, k = -1)))
>
> > crossprod(xmat) |> solve()
>
> xxlag1
>
> x 3.488838e-11 1.949958e-12
>
> xlag1 1.949958e-12 3.421268e-11
>
>
>
> [[alternative HTML version deleted]]
>
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
> https://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>
[[alternative HTML version deleted]]
__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide https://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Re: [R] Need help with time series data
Hi,
Just to add:
As you have more columns of data, it would be better to do in this way:
Time1<-as.POSIXct("2013-01-01 00:00:00")
tseq<-seq(Time1,length.out=50,by="secs")
dat2<-data.frame(TIME=tseq,matrix(NA,nrow=50,ncol=3)) #change nrow and ncol
names(dat2)[-1]<-paste("Data",1:3,sep="")
dat3<-read.table(text="
ID Date TIME Data1 Data2 Data3
1 2013-01-01 00:00:00 34 53 66
2 2013-01-01 00:00:01 333 4 5
3 2013-01-01 00:00:05 333 4 5
4 2013-01-01 00:00:20 34 63 66
5 2013-01-01 00:00:25 433 5 8
6 2013-01-01 00:00:26 533 8 5
",sep="",stringsAsFactors=FALSE,header=TRUE)
dat4<-data.frame(TIME=as.POSIXct(paste(dat3$Date,dat3$TIME)),dat3[,4:6])
dat2[,-1][dat2[,1]%in%dat4[,1],]<-dat4[,-1]
head(dat2)
# TIME Data1 Data2 Data3
#1 2013-01-01 00:00:00 34 53 66
#2 2013-01-01 00:00:01 333 4 5
#3 2013-01-01 00:00:02 NA NA NA
#4 2013-01-01 00:00:03 NA NA NA
#5 2013-01-01 00:00:04 NA NA NA
#6 2013-01-01 00:00:05 333 4 5
A.K.
- Original Message -
From: Simonas Kecorius
To: r-help@r-project.org
Cc:
Sent: Saturday, January 5, 2013 8:54 AM
Subject: [R] Need help with time series data
Dear R users,
Could you share your knowledge on following problem:
Suppose we have dataframe:
ID TIME Data1 Data2 Data3
... Data700
1. 2013-01-01 00:00:00 34 53 66
55
2. 2013-01-01 00:00:01 333 4 5
50
3. 2013-01-01 00:00:02 and so on
4. 2013-01-01 00:00:03 and so on
5. 2013-01-01 03:00:45 44 33 4
. 66
6. 2013-01-01 03:00:46 46 35 7
. 6
Notice that between ID 4 and ID 5 there is a gap, that instrument was
turned off. It started automatically in 2013-01-01 03:00:45 and continues
till the next turn off.
What I need R to do, is to find these gaps, and insert missing time values
with empty Data lines to have no gaps in time and empty (or NA) values in
data place:
ID TIME Data1 Data2 Data3
... Data700
1. 2013-01-01 00:00:00 34 53 66
55
2. 2013-01-01 00:00:01 333 4 5
50
3. 2013-01-01 00:00:02 and so on
4. 2013-01-01 00:00:03 and so on
5. 2013-01-01
00:00:04
<- time with empty (or NA) data values inserted
6. 2013-01-01
00:00:05
<- time with empty (or NA) data values inserted
7. 2013-01-01
00:00:06
<- time with empty (or NA) data values inserted
and so on, until the time when instrument works again:
2013-01-01 03:00:45 44 33 4
. 66
2013-01-01 03:00:46 46 35 7
. 6
All your suggestions will be appreciated!
Simonas Kecorius
**
[[alternative HTML version deleted]]
__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Re: [R] Need help with time series data
HI,
You could try this:
Time1<-as.POSIXct("2013-01-01 00:00:00")
tseq<-seq(Time1,length.out=50,by="secs")
dat2<-data.frame(TIME=tseq,Data1=NA,Data2=NA,Data3=NA)
dat3<-read.table(text="
ID Date TIME Data1 Data2 Data3
1 2013-01-01 00:00:00 34 53 66
2 2013-01-01 00:00:01 333 4 5
3 2013-01-01 00:00:05 333 4 5
4 2013-01-01 00:00:20 34 63 66
5 2013-01-01 00:00:25 433 5 8
6 2013-01-01 00:00:26 533 8 5
",sep="",stringsAsFactors=FALSE,header=TRUE)
dat4<-data.frame(TIME=as.POSIXct(paste(dat3$Date,dat3$TIME)),dat3[,4:6])
dat2[,-1][dat2[,1]%in%dat4[,1],]<-dat4[,-1]
head(dat2)
# TIME Data1 Data2 Data3
#1 2013-01-01 00:00:00 34 53 66
#2 2013-01-01 00:00:01 333 4 5
#3 2013-01-01 00:00:02 NA NA NA
#4 2013-01-01 00:00:03 NA NA NA
#5 2013-01-01 00:00:04 NA NA NA
#6 2013-01-01 00:00:05 333 4 5
A.K.
- Original Message -
From: Simonas Kecorius
To: r-help@r-project.org
Cc:
Sent: Saturday, January 5, 2013 8:54 AM
Subject: [R] Need help with time series data
Dear R users,
Could you share your knowledge on following problem:
Suppose we have dataframe:
ID TIME Data1 Data2 Data3
... Data700
1. 2013-01-01 00:00:00 34 53 66
55
2. 2013-01-01 00:00:01 333 4 5
50
3. 2013-01-01 00:00:02 and so on
4. 2013-01-01 00:00:03 and so on
5. 2013-01-01 03:00:45 44 33 4
. 66
6. 2013-01-01 03:00:46 46 35 7
. 6
Notice that between ID 4 and ID 5 there is a gap, that instrument was
turned off. It started automatically in 2013-01-01 03:00:45 and continues
till the next turn off.
What I need R to do, is to find these gaps, and insert missing time values
with empty Data lines to have no gaps in time and empty (or NA) values in
data place:
ID TIME Data1 Data2 Data3
... Data700
1. 2013-01-01 00:00:00 34 53 66
55
2. 2013-01-01 00:00:01 333 4 5
50
3. 2013-01-01 00:00:02 and so on
4. 2013-01-01 00:00:03 and so on
5. 2013-01-01
00:00:04
<- time with empty (or NA) data values inserted
6. 2013-01-01
00:00:05
<- time with empty (or NA) data values inserted
7. 2013-01-01
00:00:06
<- time with empty (or NA) data values inserted
and so on, until the time when instrument works again:
2013-01-01 03:00:45 44 33 4
. 66
2013-01-01 03:00:46 46 35 7
. 6
All your suggestions will be appreciated!
Simonas Kecorius
**
[[alternative HTML version deleted]]
__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Re: [R] Need help with time series data
On Sat, 5 Jan 2013, Simonas Kecorius wrote:
Dear R users,
Could you share your knowledge on following problem:
Suppose we have dataframe:
ID TIME Data1 Data2Data3
... Data700
1.2013-01-01 00:00:00 3453 66
55
2.2013-01-01 00:00:01 333 4 5
50
3.2013-01-01 00:00:02 and so on
4.2013-01-01 00:00:03 and so on
5.2013-01-01 03:00:45 4433 4
. 66
6.2013-01-01 03:00:46 4635 7
. 6
Notice that between ID 4 and ID 5 there is a gap, that instrument was
turned off. It started automatically in 2013-01-01 03:00:45 and continues
till the next turn off.
What I need R to do, is to find these gaps, and insert missing time values
with empty Data lines to have no gaps in time and empty (or NA) values in
data place:
ID TIME Data1 Data2Data3
... Data700
1.2013-01-01 00:00:00 3453 66
55
2.2013-01-01 00:00:01 333 4 5
50
3.2013-01-01 00:00:02 and so on
4.2013-01-01 00:00:03 and so on
5.2013-01-01
00:00:04
<- time with empty (or NA) data values inserted
6.2013-01-01
00:00:05
<- time with empty (or NA) data values inserted
7.2013-01-01
00:00:06
<- time with empty (or NA) data values inserted
and so on, until the time when instrument works again:
2013-01-01 03:00:45 4433 4
. 66
2013-01-01 03:00:46 4635 7
. 6
You can use zoo time series with POSIXct time stamps (or chron or
timeDate) and then extend to a regular grid. For example:
## regular time grid (by second)
t <- seq(as.POSIXct("2000-01-01 00:00"), by = "1 sec", length = 5)
## zoo series with only part of the time stamps (3rd stamp missing)
z <- zoo(1:4, t[-3])
## extend to full time grid by merging with an empty zoo series
z2 <- merge(zoo( , t), z)
This yields:
2000-01-01 00:00:00 2000-01-01 00:00:01 2000-01-01 00:00:02
1 2 NA
2000-01-01 00:00:03 2000-01-01 00:00:04
3 4
You can also use different strategies for filling these NAs.
See help("merge.zoo", package = "zoo") for another example and the
vignettes in vignette(package = "zoo") for more details.
All your suggestions will be appreciated!
Simonas Kecorius
**
[[alternative HTML version deleted]]
__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
