Lots of text in this message, but I want to make sure we are in agreement :
I am trying to figure out what to do with "gaps" in the data. Generally speaking it looks like I will get occasional seconds where there was no update to the book in the CME data, so there is no trigger to the parser to write a snapshot to the file. Right now I only write the values to the output file when there is a new update to the book and it's calendar second differs from the previous calendar second (and day, month, hour, and minute). Digging through JBT to figure out how these gaps are handled elsewhere in JBT, I am reading the code as such: 1)At normal JBT run time, the StrategyRunner creates an instance of the SnapshotHandler once every second. The depth book is written to a snapshot, and fed to both the strategy and the backtest file writer. In this case it looks like the data is always written to the file, even if the data stream stops (assuming the book is valid when the data stream stops), because the book has data and the 1 second tick keeps happening? Am I interpreting this correctly, and is this the behavior we want? Slightly off topic: What "should" happen if the depth data updates stop for 1 second, 10 seconds, 1000 seconds? Is there some code handling this problem that I am not seeing? It looks like the last valid data value could continue to be added to the backtest file for as long as the time is within the trading session. 2) The back tester takes in the snapshots as they are in the file, sets the time according to these snapshots. If the time between successive snapshots is too long, the indicators reset. This is pretty straightforward. So I can see two ways to handle the CME data: I can fill in the gaps with repeat data, which will keep the time steps uniform for the backtester. This probably makes things like EMA indicators more predictable, and for small amounts of time these periods of no-update are likely "real" - although I haven't analyzed the update timestamps to see just how frequent they really are. This method also seems to match the behavior of the JBT recordings. I could also omit the data, just as things are happening now wiht my parser. This prevents the problem of "inventing" data where there may not have been before. On the flip side, behavior for things like EMAs and strategies that are counting ticks will be different than in the real world run-time, from what I read in the JBT code. I am leaning towards the "repeat" method with some kind of consecutive repeat limit, maybe 20 seconds before I gap? For reference, during trading hours I see a noticeable number of gaps of 1 to 2 seconds, I have seen gaps as large as 5 seconds. I haven't inspected PM/AH trading hours yet. On Sun, Nov 8, 2009 at 11:42 PM, Eugene Kononov <[email protected]>wrote: > > > The values look good. There appears to be a time delay on the order of 1 > to > > 10 seconds. > > At the time when that data file was created, JBT used the PC clock > instead of the NTP time, so this would explain the 1 to 10 second > delay. > > > I looked at your Excel graphs, they are somewhat difficult to > interpret. Might be easier if you plotted just the difference between > the parsed and the recorded book balance. There also still appears to > be alignment issues, probably because of the drifting clock. But > overall, it looks like you are on a good track. > > > > --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "JBookTrader" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/jbooktrader?hl=en -~----------~----~----~----~------~----~------~--~---
