Hello, I may add that if you are focused on keeping 8000 symbols historical database, it would be good idea to use / buy QCollector because it is specifically designed to download history for large lists and makes it easier to control the download process. The only thing that needs to be understood is that 8000 symbols database collected via QCollector won't be real-time.
Best regards, Tomasz Janeczko amibroker.com ----- Original Message ----- From: Tomasz Janeczko To: [email protected] Sent: Friday, June 20, 2008 8:20 AM Subject: Re: [amibroker] Metastock and Amibroker Hello, >Wouldn't that be 16 seconds or 16 * 500 tickers? That would be true only if you were needing "last tick" for those 500 tickers. But if you need anything else than last tick, you need to request backfill. Backfill is by nature much slower due to technological differences (the backfill server is separate and each backfill query pulls data from much bigger database that by nature is slower) and is handled on "query-response". Streaming on the other hand is a multicast protocol (similar to TV "broadcast") that does not have such overhead as backfill. How IQFeed decides which 500 tickers? It is not IQFeed who decides, but rather end-user. If you put 500 tickers into Real-time quote list this would be it. If you don't AMiBroker will use 500 most recently used (accessed) tickers. As to backfilling - you may get holes because you are not using "Wait for backfill" checkbox in AA and IQFeed is not able to backfill at Automatic Analysis scan speed. You need to use "wait for backfill" checkbox in automatic analysis window. That's the only thing you need to do. And no right-clicking on "force backfill" is not necessary. It is only needed when you specifically want to erase everything and start from the scratch. You should really read this part of manual: http://www.amibroker.com/guide/h_rtsource.html Realistically using any data source and any program, you can not perform real-time scan for more symbols that given data source allows. If your account allows 500 symbols, you can scan 500 symbols in REAL TIME If your account allows 1300 symbols, then you can scan 1300 symbols in REAL TIME. Expecting anything more than your account limit is unrealistic, because backfill is much slower process and does not allow real-time delivery of 2000+ symbols as you are wishing. Your expectations are too high. The only source that allows accounts with 2000 streaming symbols is eSignal. You are free to use any software you wish, but I can safely tell you that you won't be able to stream 8000 symbols in real time using consumer-level internet-based data vendors. Best regards, Tomasz Janeczko amibroker.com ----- Original Message ----- From: Louis Préfontaine To: [email protected] Sent: Friday, June 20, 2008 4:48 AM Subject: Re: [amibroker] Metastock and Amibroker Hi Tomasz, There is something I don't understand. If IQFeed can only do real-time for 500 tickers at a time, why would it take 8000 seconds for 8000 tickers? Wouldn't that be 16 seconds or 16 * 500 tickers? I can live with 2000 tickers instead of 8000, but I'd need to be able to scan them for 15-minute bars. I mean: how can I know in advance which ticker would do a breakout if that is my strategy? Wouldn't that be possible to scan them all this way? And how do IQFeed decides which 500 tickers it downloads RT? This is so confusing... The real thing that makes me think about QCollector was all the backfilling problems, the holes and things like that that appeared in AB. Sometimes, the data would go from let's say March 24 and the next day is April 4 and there is no way to fill the holes. I talked with support at IQFeed and they suspect a problem with Amibroker, where the software would overwrite the data instead of adding to it, but of course if that was the case I don't understand why there would be holes... The only way I could deal with those holes was to right-click "backfilling" but it is a long thing to do with over 5000 tickers (a REALLY long thing to do considering one would have to do it often) but when it backfills it erases the first bars, so instead of seeing my database getting deeper with more and more data I get the maximum one year of intraday data offered by IQFeed. Kind of mixed. All I want is something reliable that works. I thought QCollector could help me collect the data and make it grow. My goal is to have many years of intraday data, and since I didn't find any way to obtain that I wanted to extend my own database. What we need to know now is if that backfilling/holes problem (which many people talked about) is a problem by IQFeed or by Amibroker. That seems crucial to me. Louis 2008/6/19 Tomasz Janeczko <[EMAIL PROTECTED]>: Hello, Before you buy QCollector, you need to understand that both AmiBroker and QCollector use the same *IQFEED API* and the same servers. That means that you won't get ANY better backfills that you get with AmiBroker because it is the IQFEED API that handles backfill. The client application (AB or QC) just sends REQUEST and WAITS for data to arrive. No magic here. Then data arrive as text via socket. The speed is totally limited by the API and IQFeed server response time. It can be easily seen when using IQFeed own History demo that just downloads historical data and displays in the list box. The time since request to first response is well above 1 second and transfer speed is *exactly* the same as you would see in AmiBroker. You will see the same using QCollector. As to your question - there is no problem you can have 4 folders, 2000 securities in each and AmiBroker can access all folders simultaneously, but you need to understand that no matter what you do, you won't get 8000 symbols updated in real time from IQFeed. Streaming via IQFeed (regular account) is for 500 symbols. It will take more than 8000 seconds (2+ hours) to backfill 8000 symbols. And that does not depend on program you are using for IQFeed. Generally speaking internet-based streaming RT sources are not designed to track 8000 symbols at once. You would need dedicated pipe or satellite dish for that many. Best regards, Tomasz Janeczko amibroker.com ----- Original Message ----- From: Louis Préfontaine To: [email protected] Sent: Thursday, June 19, 2008 8:00 PM Subject: [amibroker] Metastock and Amibroker Hi, I am considering buying Qcollector for IQFeed, but it saves the data in Metastock files. Metastock can only have 2000 securities per file, so I was wondering if there would be any problem for a large 1-minute databse with 8000 securities in Amibroker. Would that be easy to manage? Thanks. Louis
