Hi Kurasake, Now that you have added the QuoteTracker info to your problem, I'll confirm that I too have seen the same problem. I see it using QT with a data feed from Ameritrade (two computers and two different AMTD data servers). The differences I've noticed from your experience are that I can find the bad data (lately a low or open of 0 on a one minute bar) when I check the raw data / quotes list in either AB or QT (in QT to see the raw data for an instrument, highlight the instrument and press <CTRL>D).
These outlier data points do not show up on a chart in QT. If you see a bad data point on a QT chart you can zoom in on it and then press <Shift><CTRL>H to erase the high spike data and <Shift><CTRL>L to erase the low spike data. Sometimes multiple executions of this are required to get rid of the bad data. To my knowledge, when the QT chart doesn't display the bad data, there is no way to edit the data in QT other than to scroll thru the raw data to find the bad stuff and delete it. In AB I can see the bad data when I open the quote editor and then sort the column for low or high. In AB you may edit the data or delete it. However if you catch a bad data point on the day it occurs, and correct it in QT, it usually will automatically correct in AB. If it is more than a couple day bars in the past or perhaps if you have saved your database in AB, then you must manually correct the AB database. Perhaps there is a way to write a filter which will delete any intraday quote with a 0 entry or an entry X % away from the H or L -- I haven't had time to develop such. Recently (past several weeks), I have seen increasing instances of the outlier condition, especially 0 at the open and several times in the first minute, but sometimes intraday as well. I have talked to Ameritrade about this and in the case of the spike data, they say it is the result of delayed reporting from dark pools and they can't do anything about it (eSignal on the other hand does report it on a T&S window but doesn't pass it to a chart or to AB). For some years, the SPY, or other heavily traded stocks, often will have intraday erratic spikes as well, especially on FOMC or other heavy volume days or after the close in the aftermarket. No explanation from AMTD yet to my inquiry about the 0 values. As recently as 1/29 there were many, many bad data spikes on many, many stocks as well as the 0 low value. I finally wrote an exploration to find stocks with outlier values so I could correct them. In the case of AB on my laptop I finally dumped the database for both QT and AB and reimported the quotes. A usage tip: if you do not delete the bad data from QT, then it will be introduced into AB the next time you run an exploration, so step one is to clean up your QT data, then clean up the AB data (by edit or rebuild). A good habit is to backup your AB data every so often (at least within the 20 days of storage QT provides) so that if you do get corrupted big time, it is easier to reinstall a "clean" historic intraday database and then update it from clean QT data. Just one more reason I wish there were a plug in for Ameritrade, rather than having to import via QT. (I am not knocking QT -- I have used it for longer than the 10+ years I've used AB and Jerry and Mike are second only to Tomasz for product support). Peace and Justice --- Patrick ----- Original Message ----- From: kurasake To: [email protected] Sent: Wednesday, February 03, 2010 11:09 AM Subject: [amibroker] Re: Bad data with IQFeed? I wanted to add that I have QuoteTracker running on the same machine using the same IQFeed and the data in QT is fine so it's definitely an AB issue. The problem seems to come up only after I run an "Explore" in "Automatic Analysis" over a large number of stock symbols. The code in the .afl file I'm running can be as simple as: Filter=1; AddColumn(O,"Open"); AddColumn(H,"High"); AddColumn(L,"Low"); AddColumn(C,"Close"); AddColumn(V,"Volume",1.0); which is run on >400 symbols and "Wait for backfill" checked and "n last days" equal to 1. Any help, clues, hints on how I can fix this would be appreciated. Thanks ------------------------------------ **** IMPORTANT PLEASE READ **** This group is for the discussion between users only. This is *NOT* technical support channel. TO GET TECHNICAL SUPPORT send an e-mail directly to SUPPORT {at} amibroker.com TO SUBMIT SUGGESTIONS please use FEEDBACK CENTER at http://www.amibroker.com/feedback/ (submissions sent via other channels won't be considered) For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG: http://www.amibroker.com/devlog/ Yahoo! Groups Links
