I went through the same frustration with IB and now use IQFeed also. Two comments: I have created 2 IQFeed databases - one for real time during market hours that has just enough 1 min bars to properly calculate my indicators. This short database loads and calculates quickly when the markets are open.
My second database has many more bars. This one I use for backtesting and I only refresh it after market hours. This combo gives me speed during the day and good backtesting afterhours. ReefBreak --- In [email protected], "etoketrader" <et...@...> wrote: > > I currently do this with 500 symbols using IQFeed. > > I have tried everything possible with downloading data from IB - I was never > able to get it running to my satisfaction. I tried this using Amibroker as > well as a variety of other utilities and software.. the bottleneck is IB - i > wished I stopped trying and moved to to proper data source earlier. > > Once you have IQFeed or similar, then the way to do it is to run an > exploration and check the "Wait for Backfill" option. This will wait until > the current backfill is complete before moving to the next symbol. > > When backfilling for the first time, try to do this over a weekend. Trying to > backfill such a large number of symbols during market hours will be a > nightmare - AB will both backfill and subscribe the symbol for realtime > streaming, so after a number of symbols, the realtime streaming on the > symbols already backfilled will cause your current backfill to slow down. > > Once you have your initial set, I would suggest scheduling a save + restart + > an exploration every night to keep your database updated. On 500 symbols this > process takes me around 25 minutes. > > hope this helps, > regards, > eToke > > > --- In [email protected], "papadawg1" <papadawg1@> wrote: > > > > Anyone have suggestions as to how to most efficiently build a 1-minute > > historical DB for ~1000 stocks or so...I've begun trying leveraging IB/TWS, > > but, with the throttling, and inability to skip over previously filled > > symbols automatically (at least as far as I can tell), seems rather > > difficult. I could probably take the time to build up a 30 day DB, but not > > sure how to keep it current without spending an inordinate amount of time. > > > > Any thoughts? Anyone have suggestions on an optimal data source? > > > > Thanks a bunch, > > Jayson > > >
