Re: [R-SIG-Finance] Remove first two weeks of data in half hourly resolution
This is brilliant. Thanks very much! On Thu, Mar 31, 2016 at 12:24 PM, Brian G. Peterson wrote: > Peter, > > You haven't published a reproducible example, and I'm not going to take > the time to write a complete example from scratch. > > We use xts subsetting for this type of thing, so I suggest using xts for > your time series (this is always good advice for time series in R). > > Here's a partial example to get you started. > > ### > > #load some data from the PerformanceAnalytics package > data(edhec) > > #check the range > range(index(edhec)) > > #add 14 days from the start > first(index(edhec))+14 > > #now assume that you have an object 'z' with intraday data > range(z) > > #check the range of Dates by forcing the index to Date type > range(as.Date(index(z))) > > #add 114 days, as before > first(as.Date(index(z)))+14 > > # now subset by cutting off the first 14 calendar days > # from the start of the series > zs <- z[paste0(first(as.Date(index(z)))+14,'/')] > > #check the range > range(as.Date(index(zs))) > > ## > > Regards, > > Brian > > -- > Brian G. Peterson > http://braverock.com/brian/ > Ph: 773-459-4973 > IM: bgpbraverock > > > On Thu, 2016-03-31 at 11:50 +0100, Peter Neumaier wrote: > > Hi all, > > > > I am doing some analysis on monthly futures contracts from 2011-2016. > > Each monthly contract goes for eight weeks, in half hourly resolution. > > > > I'd like to remove first two weeks of eight weeks history for each > monthly > > contract. > > My approach was to work out the start and end date and cut the 1st two > weeks > > data off, but problem is that the half hourly resolution is sometimes > > incomplete( > > i.e. a trading day goes from 7:00am - 4:00pm but sometimes starts at > > 7:30am). > > > > Any suggestion on how to resolve this? Below a sample trading day in half > > hourly: > > > > NGFH6.Open NGFH6.High NGFH6.Low NGFH6.Close > > NGFH6.Volume NGFH6.WAP NGFH6.hasGaps NGFH6.Count > > 2016-01-06 07:30:00 0.3395 0.33950.3375 0.3380 > > 45 0.33811 0 5 > > 2016-01-06 08:00:00 0.3400 0.34000.3387 0.3395 > > 140 0.33928 0 12 > > 2016-01-06 08:30:00 0.3395 0.33950.3379 0.3379 > > 70 0.33884 0 5 > > 2016-01-06 09:00:00 0.3379 0.33790.3379 0.3379 > > 0 0.33790 0 0 > > 2016-01-06 09:30:00 0.3379 0.33790.3379 0.3379 > > 0 0.33790 0 0 > > 2016-01-06 10:00:00 0.3375 0.33800.3373 0.3373 > > 230 0.33738 0 14 > > 2016-01-06 10:30:00 0.3376 0.33790.3376 0.3379 > > 20 0.33775 0 2 > > 2016-01-06 11:00:00 0.3370 0.33700.3370 0.3370 > > 105 0.33700 0 5 > > 2016-01-06 11:30:00 0.3366 0.33660.3365 0.3365 > > 65 0.33658 0 4 > > 2016-01-06 12:00:00 0.3370 0.33700.3370 0.3370 > > 10 0.33700 0 1 > > 2016-01-06 12:30:00 0.3372 0.33720.3361 0.3361 > > 125 0.33686 0 9 > > 2016-01-06 13:00:00 0.3360 0.33600.3357 0.3360 > > 225 0.33585 0 17 > > 2016-01-06 13:30:00 0.3357 0.33570.3355 0.3355 > > 50 0.33560 0 5 > > 2016-01-06 14:00:00 0.3350 0.33590.3350 0.3359 > > 25 0.33554 0 2 > > 2016-01-06 14:30:00 0.3359 0.33590.3359 0.3359 > > 0 0.33590 0 0 > > 2016-01-06 15:00:00 0.3352 0.33520.3348 0.3352 > > 150 0.33492 0 15 > > 2016-01-06 15:30:00 0.3352 0.33520.3334 0.3341 > > 280 0.33364 0 24 > > 2016-01-06 16:00:00 0.3341 0.33750.3341 0.3370 > > 145 0.33543 0 17 > > 2016-01-06 16:30:00 0.3380 0.33850.3380 0.3385 > > 25 0.33830 0 3 > > > > Many Thanks > > Peter > > > > [[alternative HTML version deleted]] > > > > ___ > > R-SIG-Finance@r-project.org mailing list > > https://stat.ethz.ch/mailman/listinfo/r-sig-finance > > -- Subscriber-posting only. If you want to post, subscribe first. > > -- Also note that this is not the r-help list where general R questions > should go. > > [[alternative HTML version deleted]] ___ R-SIG-Finance@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-sig-finance -- Subscriber-posting only. If you want to post, subscribe first. -- Also note that this is not the r-help list where general R questions should go.
Re: [R-SIG-Finance] Remove first two weeks of data in half hourly resolution
Brian's example is good if you want to remove the first 14 calendar days. If you want to remove the first 14 dates that appear in your data, it's even simpler #create sample data x <- .xts(1:1000, .POSIXct(1:1000*60*30)) # remove 1st 14 days by using a negative with first() first(x, "-14 days") Garrett On Thu, Mar 31, 2016 at 6:24 AM, Brian G. Peterson wrote: > Peter, > > You haven't published a reproducible example, and I'm not going to take > the time to write a complete example from scratch. > > We use xts subsetting for this type of thing, so I suggest using xts for > your time series (this is always good advice for time series in R). > > Here's a partial example to get you started. > > ### > > #load some data from the PerformanceAnalytics package > data(edhec) > > #check the range > range(index(edhec)) > > #add 14 days from the start > first(index(edhec))+14 > > #now assume that you have an object 'z' with intraday data > range(z) > > #check the range of Dates by forcing the index to Date type > range(as.Date(index(z))) > > #add 114 days, as before > first(as.Date(index(z)))+14 > > # now subset by cutting off the first 14 calendar days > # from the start of the series > zs <- z[paste0(first(as.Date(index(z)))+14,'/')] > > #check the range > range(as.Date(index(zs))) > > ## > > Regards, > > Brian > > -- > Brian G. Peterson > http://braverock.com/brian/ > Ph: 773-459-4973 > IM: bgpbraverock > > > On Thu, 2016-03-31 at 11:50 +0100, Peter Neumaier wrote: >> Hi all, >> >> I am doing some analysis on monthly futures contracts from 2011-2016. >> Each monthly contract goes for eight weeks, in half hourly resolution. >> >> I'd like to remove first two weeks of eight weeks history for each monthly >> contract. >> My approach was to work out the start and end date and cut the 1st two weeks >> data off, but problem is that the half hourly resolution is sometimes >> incomplete( >> i.e. a trading day goes from 7:00am - 4:00pm but sometimes starts at >> 7:30am). >> >> Any suggestion on how to resolve this? Below a sample trading day in half >> hourly: >> >> NGFH6.Open NGFH6.High NGFH6.Low NGFH6.Close >> NGFH6.Volume NGFH6.WAP NGFH6.hasGaps NGFH6.Count >> 2016-01-06 07:30:00 0.3395 0.33950.3375 0.3380 >> 45 0.33811 0 5 >> 2016-01-06 08:00:00 0.3400 0.34000.3387 0.3395 >> 140 0.33928 0 12 >> 2016-01-06 08:30:00 0.3395 0.33950.3379 0.3379 >> 70 0.33884 0 5 >> 2016-01-06 09:00:00 0.3379 0.33790.3379 0.3379 >> 0 0.33790 0 0 >> 2016-01-06 09:30:00 0.3379 0.33790.3379 0.3379 >> 0 0.33790 0 0 >> 2016-01-06 10:00:00 0.3375 0.33800.3373 0.3373 >> 230 0.33738 0 14 >> 2016-01-06 10:30:00 0.3376 0.33790.3376 0.3379 >> 20 0.33775 0 2 >> 2016-01-06 11:00:00 0.3370 0.33700.3370 0.3370 >> 105 0.33700 0 5 >> 2016-01-06 11:30:00 0.3366 0.33660.3365 0.3365 >> 65 0.33658 0 4 >> 2016-01-06 12:00:00 0.3370 0.33700.3370 0.3370 >> 10 0.33700 0 1 >> 2016-01-06 12:30:00 0.3372 0.33720.3361 0.3361 >> 125 0.33686 0 9 >> 2016-01-06 13:00:00 0.3360 0.33600.3357 0.3360 >> 225 0.33585 0 17 >> 2016-01-06 13:30:00 0.3357 0.33570.3355 0.3355 >> 50 0.33560 0 5 >> 2016-01-06 14:00:00 0.3350 0.33590.3350 0.3359 >> 25 0.33554 0 2 >> 2016-01-06 14:30:00 0.3359 0.33590.3359 0.3359 >> 0 0.33590 0 0 >> 2016-01-06 15:00:00 0.3352 0.33520.3348 0.3352 >> 150 0.33492 0 15 >> 2016-01-06 15:30:00 0.3352 0.33520.3334 0.3341 >> 280 0.33364 0 24 >> 2016-01-06 16:00:00 0.3341 0.33750.3341 0.3370 >> 145 0.33543 0 17 >> 2016-01-06 16:30:00 0.3380 0.33850.3380 0.3385 >> 25 0.33830 0 3 >> >> Many Thanks >> Peter >> >> [[alternative HTML version deleted]] >> >> ___ >> R-SIG-Finance@r-project.org mailing list >> https://stat.ethz.ch/mailman/listinfo/r-sig-finance >> -- Subscriber-posting only. If you want to post, subscribe first. >> -- Also note that this is not the r-help list where general R questions >> should go. > > ___ > R-SIG-Finance@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-sig-finance > -- Subscriber-posting only. If you want to post, subscribe first. > -- Also note that this is not the r-help list where gener
Re: [R-SIG-Finance] Remove first two weeks of data in half hourly resolution
Peter, You haven't published a reproducible example, and I'm not going to take the time to write a complete example from scratch. We use xts subsetting for this type of thing, so I suggest using xts for your time series (this is always good advice for time series in R). Here's a partial example to get you started. ### #load some data from the PerformanceAnalytics package data(edhec) #check the range range(index(edhec)) #add 14 days from the start first(index(edhec))+14 #now assume that you have an object 'z' with intraday data range(z) #check the range of Dates by forcing the index to Date type range(as.Date(index(z))) #add 114 days, as before first(as.Date(index(z)))+14 # now subset by cutting off the first 14 calendar days # from the start of the series zs <- z[paste0(first(as.Date(index(z)))+14,'/')] #check the range range(as.Date(index(zs))) ## Regards, Brian -- Brian G. Peterson http://braverock.com/brian/ Ph: 773-459-4973 IM: bgpbraverock On Thu, 2016-03-31 at 11:50 +0100, Peter Neumaier wrote: > Hi all, > > I am doing some analysis on monthly futures contracts from 2011-2016. > Each monthly contract goes for eight weeks, in half hourly resolution. > > I'd like to remove first two weeks of eight weeks history for each monthly > contract. > My approach was to work out the start and end date and cut the 1st two weeks > data off, but problem is that the half hourly resolution is sometimes > incomplete( > i.e. a trading day goes from 7:00am - 4:00pm but sometimes starts at > 7:30am). > > Any suggestion on how to resolve this? Below a sample trading day in half > hourly: > > NGFH6.Open NGFH6.High NGFH6.Low NGFH6.Close > NGFH6.Volume NGFH6.WAP NGFH6.hasGaps NGFH6.Count > 2016-01-06 07:30:00 0.3395 0.33950.3375 0.3380 > 45 0.33811 0 5 > 2016-01-06 08:00:00 0.3400 0.34000.3387 0.3395 > 140 0.33928 0 12 > 2016-01-06 08:30:00 0.3395 0.33950.3379 0.3379 > 70 0.33884 0 5 > 2016-01-06 09:00:00 0.3379 0.33790.3379 0.3379 > 0 0.33790 0 0 > 2016-01-06 09:30:00 0.3379 0.33790.3379 0.3379 > 0 0.33790 0 0 > 2016-01-06 10:00:00 0.3375 0.33800.3373 0.3373 > 230 0.33738 0 14 > 2016-01-06 10:30:00 0.3376 0.33790.3376 0.3379 > 20 0.33775 0 2 > 2016-01-06 11:00:00 0.3370 0.33700.3370 0.3370 > 105 0.33700 0 5 > 2016-01-06 11:30:00 0.3366 0.33660.3365 0.3365 > 65 0.33658 0 4 > 2016-01-06 12:00:00 0.3370 0.33700.3370 0.3370 > 10 0.33700 0 1 > 2016-01-06 12:30:00 0.3372 0.33720.3361 0.3361 > 125 0.33686 0 9 > 2016-01-06 13:00:00 0.3360 0.33600.3357 0.3360 > 225 0.33585 0 17 > 2016-01-06 13:30:00 0.3357 0.33570.3355 0.3355 > 50 0.33560 0 5 > 2016-01-06 14:00:00 0.3350 0.33590.3350 0.3359 > 25 0.33554 0 2 > 2016-01-06 14:30:00 0.3359 0.33590.3359 0.3359 > 0 0.33590 0 0 > 2016-01-06 15:00:00 0.3352 0.33520.3348 0.3352 > 150 0.33492 0 15 > 2016-01-06 15:30:00 0.3352 0.33520.3334 0.3341 > 280 0.33364 0 24 > 2016-01-06 16:00:00 0.3341 0.33750.3341 0.3370 > 145 0.33543 0 17 > 2016-01-06 16:30:00 0.3380 0.33850.3380 0.3385 > 25 0.33830 0 3 > > Many Thanks > Peter > > [[alternative HTML version deleted]] > > ___ > R-SIG-Finance@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-sig-finance > -- Subscriber-posting only. If you want to post, subscribe first. > -- Also note that this is not the r-help list where general R questions > should go. ___ R-SIG-Finance@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-sig-finance -- Subscriber-posting only. If you want to post, subscribe first. -- Also note that this is not the r-help list where general R questions should go.
[R-SIG-Finance] Remove first two weeks of data in half hourly resolution
Hi all, I am doing some analysis on monthly futures contracts from 2011-2016. Each monthly contract goes for eight weeks, in half hourly resolution. I'd like to remove first two weeks of eight weeks history for each monthly contract. My approach was to work out the start and end date and cut the 1st two weeks data off, but problem is that the half hourly resolution is sometimes incomplete( i.e. a trading day goes from 7:00am - 4:00pm but sometimes starts at 7:30am). Any suggestion on how to resolve this? Below a sample trading day in half hourly: NGFH6.Open NGFH6.High NGFH6.Low NGFH6.Close NGFH6.Volume NGFH6.WAP NGFH6.hasGaps NGFH6.Count 2016-01-06 07:30:00 0.3395 0.33950.3375 0.3380 45 0.33811 0 5 2016-01-06 08:00:00 0.3400 0.34000.3387 0.3395 140 0.33928 0 12 2016-01-06 08:30:00 0.3395 0.33950.3379 0.3379 70 0.33884 0 5 2016-01-06 09:00:00 0.3379 0.33790.3379 0.3379 0 0.33790 0 0 2016-01-06 09:30:00 0.3379 0.33790.3379 0.3379 0 0.33790 0 0 2016-01-06 10:00:00 0.3375 0.33800.3373 0.3373 230 0.33738 0 14 2016-01-06 10:30:00 0.3376 0.33790.3376 0.3379 20 0.33775 0 2 2016-01-06 11:00:00 0.3370 0.33700.3370 0.3370 105 0.33700 0 5 2016-01-06 11:30:00 0.3366 0.33660.3365 0.3365 65 0.33658 0 4 2016-01-06 12:00:00 0.3370 0.33700.3370 0.3370 10 0.33700 0 1 2016-01-06 12:30:00 0.3372 0.33720.3361 0.3361 125 0.33686 0 9 2016-01-06 13:00:00 0.3360 0.33600.3357 0.3360 225 0.33585 0 17 2016-01-06 13:30:00 0.3357 0.33570.3355 0.3355 50 0.33560 0 5 2016-01-06 14:00:00 0.3350 0.33590.3350 0.3359 25 0.33554 0 2 2016-01-06 14:30:00 0.3359 0.33590.3359 0.3359 0 0.33590 0 0 2016-01-06 15:00:00 0.3352 0.33520.3348 0.3352 150 0.33492 0 15 2016-01-06 15:30:00 0.3352 0.33520.3334 0.3341 280 0.33364 0 24 2016-01-06 16:00:00 0.3341 0.33750.3341 0.3370 145 0.33543 0 17 2016-01-06 16:30:00 0.3380 0.33850.3380 0.3385 25 0.33830 0 3 Many Thanks Peter [[alternative HTML version deleted]] ___ R-SIG-Finance@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-sig-finance -- Subscriber-posting only. If you want to post, subscribe first. -- Also note that this is not the r-help list where general R questions should go.
