Hi all, I have been reading through the archived posts on the raw data format. The general conclusion seems to have been that raw data (i.e. the full order book) is nice but impractical because there are ~2 million rows per day per contract and so backtesting would take much too long.
This may be true if the data is stored as ASCII-CSV in an ordinary disk file, but has anyone looked into column-oriented DBMS such as MonetDB? It is specifically designed for that type of problem. http://en.wikipedia.org/wiki/Column-oriented_DBMS http://en.wikipedia.org/wiki/MonetDB http://monetdb.cwi.nl/ http://monetdb.cwi.nl/projects/monetdb/SQL/Benchmark/TPCH/index.html I haven't tried it. I suspect it might be possible to do JBT-type stuff with full order books (raw data) on a reasonable modern system with MonetDB or one of the similar column-oriented databases. Any thoughts? Best, Paul --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "JBookTrader" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/jbooktrader?hl=en -~----------~----~----~----~------~----~------~--~---
