Thanks for the insight. Less confusion now. It appears that minute or other data can be accumulated just the same as EOD data. With IQ feed they claim 500 intraday symbols with 8 months backfill. So to get an 8 month intraday (1 minute) data base to back test with a stategy might be to reduce my universe of stocks to maybe 1000 stocks and then split that into 2 watchlists of 500 tickers each so it can be downloaded and then updated. Does this sound like a reasonable approach to using intra day data?
Wow. Lots of data: 6.5 hours/day x 60 min bars/hr =390bars /day/ticker 390 bars/day/ ticker x 1000 tickers = 390,000 bars/day 390,000 bars/day x 10 bytes/bar( estimate?) = 3,900,000 bytes/day = 4mb /day 4mb /day x 250 day/yr = 1 gig/yr of data with 1000 tickers So 8 months of 1 minute data is a big download and would make for very long back tests. It would be nice to have the one minute data but maybe 30 or 60 minute bars would be a more reasonable time selection for back testing. Is 8 months reasonable for 30 minute intraday bar back tests? I use 8 years for EOD back testing which will show performance through full bear and bull market cycles. What types intra day data / time frame strategies are some of you using? Tod Pellaton --- In [email protected], "brian_z111" <[EMAIL PROTECTED]> wrote: > > Hello Tod, > > > Looking at IQ feed or starting an account at IB. Both of these data > > suppliers have limits on simultaneous data you can have and > >backfill data > > days? Not sure what this means. > > Basically - every bar == a set amount of bytes download adn memory to > store. > 1 EOD bar == 60 * 1 minute bars (it has OHLCV just the same) > therefore 60 times the download. > > The providers allow so many symbols * so many bars == a set download > (if you use a smaller timeframe then you can't go back so far before > exhausting the limit). > > The details vay but this is the theme. > > If you want more symbols then you pay more. > > > > I guess I just do not have a good understanding of how RT data is > >gathered > > and used as compared to EOD data > > Basically every transaction is recorded by the exchanges = 10000 > bought at $30 == a tick - in the leading stocks the ticks fly - the > ticks are 'compressed' to create, say a one minute bar (AB can handle > tick data and you can view any timeframe you want up to the daily bar > which is then compressed to weekly or monthly. > > I have eSignal RT - I don't use below one minute so I set that as my > database timeframe. > > If I look at a 24 hour chart of a leading stock it will trade slowly > after hours and premarket. > > If I look at the opening bar, based on the open time of the exchange > where the stock trades and the same for the closing one minute bar > (measured by time) i.e. say tge 4pm or 1600 hour one minute bar then > the close of this bar will be the close of the daily bar. > > The daily bar == the one minute bars from market open > hours 'compressed' into 1 bar. > > RE backtesting: > > With eSignal I have a 200 symbol limit so I can take the max history > for the timeframe I want - say 6 months for 1 minute - and it is > saved locally by AB. > > Then I can do another 200 symbols etc. > > I am not a big user of back data compared to others and this is > enough for me but everyone is different - some do go through a lot > more symbols and depth. > > I bought a CD with ASX tick data history on it and manually imported > it into AB to use for backtesxting - it was a PITA to import - very > slow and I had to manually clean out data errors - not recommended. > > I use eSignal because I live outside the US and they are the best > choice for multiple international markets. > > IB also have international market data but I have held off opening an > account there for personal reasons - their data isn't quite as good > as eSignal. > > eSignal is just a matter of following the help manual - the plugin > works like a dream as per the notes. > > Plenty of US citzs seem happy with their RT data from the other main > supported providers (IQ or QP). > > Pity you don't live in Aus because locals can ring the eSignal Aus > desk and get the 30 day trial for free (except exchange fees)- AFAIK > if you continue on with eSignal US the 30 day trial is back charged > once you sign up. > > I didn't answer all of your questions - others might help with the > rest. > > brian_z > > --- In [email protected], "Tod Pellaton" <tod@> wrote: > > > > I am looking for help in clarifying how to set up and use Real time > data. > > Currently using worden EOD data and can program very simple systems > and back > > tests, etc. > > > > > > > > I guess I just do not have a good understanding of how RT data is > gathered > > and used as compared to EOD data. I have looked through the > documentation > > but am confused how it all works. It looks easy enough to set up a > data > > supplier but confusing how to best utilize RT data. > > > > > > > > Looking at IQ feed or starting an account at IB. Both of these data > > suppliers have limits on simultaneous data you can have and > backfill data > > days? Not sure what this means. With EOD data I just download data > for all > > US stocks every day. > > > > > > > > Is it realistic to download 1 minute data for all stocks everyday > to build a > > database for back testing? Are these data suppliers set up to > provide us > > with this much data every day? Should a smaller universe of stocks > be > > specified to work with to reduce the amount of data? Is it > possible to > > obtain a few years of data for a universe of stocks for back > testing systems > > with real time data? How does AB manage all this data gathering? Is > it > > realistic to think I will need to build a RT database for a > universe of > > stocks in AB over time? Is there a special group of AFL commands > in AB I > > should learn about to work with RT data? What methods are currently > being > > used to gather and mange RT Data? > > > > > > > > Any clarification or insight on this topic would be appreciated. > > > > > > > > > > > > Tod Pellaton > > >
