I think maybe a phased approach might work. And, for sure I'd recommend a separate branch for this experiment.
Here is my current idea, base on our thread. I think it addresses most of the requirements: 1. Create a directory where index long term data files will live, no UI needed if we start with a static file name. Yes, this is ugly long term, but this is just for phase1. It gets us off the ground, and means we can postpone the ability to name the file yourself until other parts are proven. 2. Create a SecurityHistory object, or every security found in this directory, which is loaded from this file location. It will parse this file, and make available the data to other classes. 3. A HistoricalIndicator object is then created, this is created by the user in code, EMA, RSI, etc.... I'll supply some exmples. It also needs some isValid(date) indication, so downstream strategies will know if it is valid for a given date. It will know if it is valid by making sure that the SecurityHistory object that it needs exists, and has enough data, with no gaps, etc. 4. Setup a demo Sample strategy, and adds this HistoricalIndicator as an indicator, and it can use params too!. So, it looks to the strategy just like another Indicator object. Optimization should work fine. The only error case is when isValid() returns false, in which case the Strategy should not trade, and should probably log this. 5. Come up with a way to populate and maintain our data file(s), my first suggestion was through IB data feed (auto-fill the entire year, then each day), maybe user will supply the file, maybe IB supplies it "if and only if" the user didn't. 6. The file format could have enough information for the IB system to query quotes. Phase 2: 1. Allow the UI to add a security (which creates the underlying file with the right info), also configure how the file is to be maintained, though IB or some other. 2. Allow the UI to show the status of the data, if it is valid, date ranges it covers, etc. 3. During backtesting and Optimization, interrogate the HistoricalIndicator dependency for valid date ranges, warning the user if Optimization or Backtesting is going to be invalid due to lack of this historical indicator data. I "think" that covers most of the issues. Advantages: - We could grow this to have the right checks it needs, and to help the user maintain the index data. - Users can write their own Indicator logic for historical stuff, so they can decide what is important, we just give them access to high/low/open/close - Strategies can have generic use of this, just like today's indicators. The system just needs to be aware of what Historical Indicators it depends on, and for what dates it is valid. This is probably how I'd code a beta version of this. If there are other feature priorities, they should be considered too. Is this one of the highest requested features? Marcus On Wed, Jan 9, 2013 at 5:43 PM, Judson Wilson <[email protected]>wrote: > > Ok, so you prefer to have us calculate high/low/close for the day from > the > > market book data? Or, should we still use IB to get today's > high/low/close > > (in the event we are recording)? > > > > What if our jbook is briefly turned off while a high occurs. Shouldn't > we > > trust IB more than ourselves due to this possibility? > > There are several different ways to get this data from IB in > real-time. I would have to look at my raw-recorder data again to see > what comes up. I feel like there is some specific open/high/low/close > data for the entire day that gets broadcast, but I am not sure. If > you mess up a day then we should record something that says the data > is bad. Then we can hand edit the file later (should be easy to find > the values). > > > Regarding the GUI, I think I can do this with no GUI impact. This would > > simply be another type of indicator that a Strategy can get a reference > too. > > This would keep things really simple for the initial impl, and reduce the > > risk of any impact to the existing trading code. > > In the long term we would need a visual display that the index's > stream is alive. At least, I would like that. > As Eugene says, we would probably need some modifications to the > backtesting window. > > > > There is a much simpler way out, though. We can add another column to the > > existing recording format, to record whatever we define as a "long term > > trend". For example: > > I admire the simplicity of the idea, but I want more data. More > columns could work. It would probably be a significant amount of > additional time for the optimizer. Also, I'm not sure how you would > read the data between strategy restarts - except for the obvious way > (read the end of the file) which itself is a little tricky for really > long files. > > > > > > > On Wed, Jan 9, 2013 at 5:23 PM, Eugene Kononov <[email protected]> > wrote: > > > >> Regarding the GUI, I think I can do this with no GUI impact. This would > >> simply be another type of indicator that a Strategy can get a reference > too. > >> This would keep things really simple for the initial impl, and reduce > the > >> risk of any impact to the existing trading code. > > > > > > > > Not sure how you'd do this without affecting the GUI. Wouldn't you need > to > > add the corresponding controls to the backtesting and optimization > dialogs > > to specify the *second* historical data file? The GUI changes aside, > there > > is a lot more to do in the back end to somehow synchronize these data > files. > > > > There is a much simpler way out, though. We can add another column to the > > existing recording format, to record whatever we define as a "long term > > trend". For example: > > > > # This historical data file was created by JBookTrader > > # 1. date in the MMddyy format > > # 2. time in the HHmmss format > > # 3. book balance > > # 4. price > > # 5. volume > > # 6. long-term trend (in relative strength units) > > > > timeZone=America/New_York > > > > 010713,080904,4.0,1455.875,10,95 > > 010713,080905,4.61,1455.875,5,95 > > 010713,080906,4.63,1455.875,1,95 > > 010713,080907,4.64,1455.875,0,95 > > 010713,080908,4.59,1455.875,1,95 > > 010713,080909,4.63,1455.875,0,95 > > > > That way, very few things would need to change, and all components of the > > system would be intacts. The downside is, of course, you've got to > collect > > this for a year or so, before it becomes usable. > > > > > > > > > > -- > > You received this message because you are subscribed to the Google Groups > > "JBookTrader" group. > > To post to this group, send email to [email protected]. > > To unsubscribe from this group, send email to > > [email protected]. > > For more options, visit this group at > > http://groups.google.com/group/jbooktrader?hl=en. > > -- > You received this message because you are subscribed to the Google Groups > "JBookTrader" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]. > For more options, visit this group at > http://groups.google.com/group/jbooktrader?hl=en. > > -- You received this message because you are subscribed to the Google Groups "JBookTrader" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/jbooktrader?hl=en.
