I wouldn't use the IB historical data route. I've had a lot of trouble with it (on stocks) in the past. Better to keep our own data and keep it as a known good reference.
The trade-off you want to make is more-difficult data reading (as opposed to just reading from a file) at the gain of not having to record anything. Recording indexes is pretty simple, but adding in all the glue around it does require more work (I found the GUI to be one of the trickier parts). You would still need a lot of this glue, regardless. I don't think recording our own files is too much additional code compared to the IB historical data method. On Wed, Jan 9, 2013 at 4:17 PM, Marcus Williford <[email protected]> wrote: > I see, there is a flaw with my plan in backtesting and optimization. My > design proposal was to use IB to get historical data (so no data file > needed, and this would have made it almost automatic for the user), but this > has a limitation of only going back 1-year in history. So, to backtest, It > would ALSO need to support this historical index file you pointed out. This > does add some complexity, as you again need to know if you have valid date, > gaps, then what to do if you do. I guess I don't see another way, unless we > use some other online data source dependency that can be automatic, and goes > back longer in history for backtesting and optimization automatically. Is > there such a free db of open/close for popular indexes, I'm kinda weak in > the where do get data part of this. > > Summary: If the user can deal with only backtesting and optmization over > the last year (minus and period time required for indicators to settle), > then it would be pretty easy to support. > Is 1-year enough valid data to satisfy most backtesters? Is it worth a demo > branch to try this setup? > > I think I agree with Judson on using index as the historical trend line, so > it would be ES index (not the real securities). > > From a trading perspective, I think if a macro indicator was known, it might > help influence entry/exit points with respect to short/long bias. How, I > don't know, but I'm sure someone would attempt to make this part of a valid > strategy modification (even for short-lived ones). > > Marcus > > > > On Wed, Jan 9, 2013 at 3:55 PM, Eugene Kononov <[email protected]> > wrote: >> >> >> On Wed, Jan 9, 2013 at 5:02 PM, Marcus Williford <[email protected]> >> wrote: >>> >>> Looking at the code, and this feature request. I think we can use IB API >>> to get the historical data, since we all pay for it! >>> >>> My suggestion is to create a new type of object, which lazy loads a 1yr >>> security historical data for any security. Let's call it SecurityHistory >>> (think of this like a macro marketbook, with no book data). >>> Next, we need to have a tier for HistoricalIndicators, which can get >>> access to SecurityHistory. These HIstoricalIndicators can have state, for >>> EMA calculations, etc. Then, any Strategy can create as many >>> HistoricalIndicators it wants. I don't think this would break the existing >>> object oriented model too much ;). >>> >> >> This is certainly doable. My only objection is that it would complicate >> things quite a bit. Specifically, for the purposes of backtesting and >> optimization (as well as trading and forward-testing), you would now need 2 >> historical data files (one containing 1-sec market depth samples, and the >> other one containing historical prices with some lower resolution). On top >> of that, from what I hear from Judson, the market depth historical data file >> would contain the future prices (as it's now), but the second one would >> contain the underlying index (such as SPY, if I am trading the ES). >> >> I think there may be a simpler solution, and I am willing to look into it, >> but I'd like to see the evidence that the multi-week and multi-months trends >> affect the intraday strategies. The average holding period of my strategies >> is about 30-40 minutes. I really don't anticipate that these types of >> strategies would be affected by taking long-term trends into consideration. >> More importantly, all my strategies are actually anti-trend (or >> mean-reversion, using a different terminology). However, there are of course >> so many ways to trade (and to auto-trade), that it would be silly for me to >> say that long term trends don't matter. If someone has such a strategy, it's >> still possible to use it with JBT as it is, by simply specifying filters >> (such as EMAs and SMAs) as indicators of long length. As Judson noted, the >> discontinuity associated with contracts rollovers would be a problem, but >> that can be hacked for the "proof of concept" purposes. If I see such as >> proof of good performance, I'd be motivated to enhance JBT so that it can >> support the time frames larger than a day. >> >> >> -- >> You received this message because you are subscribed to the Google Groups >> "JBookTrader" group. >> To post to this group, send email to [email protected]. >> To unsubscribe from this group, send email to >> [email protected]. >> For more options, visit this group at >> http://groups.google.com/group/jbooktrader?hl=en. > > > -- > You received this message because you are subscribed to the Google Groups > "JBookTrader" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]. > For more options, visit this group at > http://groups.google.com/group/jbooktrader?hl=en. -- You received this message because you are subscribed to the Google Groups "JBookTrader" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/jbooktrader?hl=en.
