I currently do this with 500 symbols using IQFeed. I have tried everything possible with downloading data from IB - I was never able to get it running to my satisfaction. I tried this using Amibroker as well as a variety of other utilities and software.. the bottleneck is IB - i wished I stopped trying and moved to to proper data source earlier.
Once you have IQFeed or similar, then the way to do it is to run an exploration and check the "Wait for Backfill" option. This will wait until the current backfill is complete before moving to the next symbol. When backfilling for the first time, try to do this over a weekend. Trying to backfill such a large number of symbols during market hours will be a nightmare - AB will both backfill and subscribe the symbol for realtime streaming, so after a number of symbols, the realtime streaming on the symbols already backfilled will cause your current backfill to slow down. Once you have your initial set, I would suggest scheduling a save + restart + an exploration every night to keep your database updated. On 500 symbols this process takes me around 25 minutes. hope this helps, regards, eToke --- In [email protected], "papadawg1" <papada...@...> wrote: > > Anyone have suggestions as to how to most efficiently build a 1-minute > historical DB for ~1000 stocks or so...I've begun trying leveraging IB/TWS, > but, with the throttling, and inability to skip over previously filled > symbols automatically (at least as far as I can tell), seems rather > difficult. I could probably take the time to build up a 30 day DB, but not > sure how to keep it current without spending an inordinate amount of time. > > Any thoughts? Anyone have suggestions on an optimal data source? > > Thanks a bunch, > Jayson >
