Hi Keith, Thank you for your response.
Your solution might be what I have to end up going. Those this makes symbol maintenance a little hectic since I important symbols from different screens and the number of symbols I have monitoring in QuoteTracker varies so I'd have to keep count of QT symbols, the symbols that I import, split it up into chunks etc. I just wrote a response to Tomasz where I think I outline that I think everything can be solved if the IQFeed subscription slots were simply released after each use. Anyway, maybe I'm being unreasonable, but I don't think the points I'm making are outrageous or out of bounds. And I think if this "fix" were implemented it would make AB that much better/robust (since you would be able to run an Exploration on an unlimited number of symbols and there'd be no problems with data corruption if a user "miscounted" the number of symbols etc) and make life better for everyone involved. I don't know... Thanks again for your response. Cheers! --- In [email protected], Keith McCombs <kmcco...@...> wrote: > > Kurasake -- > I don't use IQ feed, but IB instead. IB has a limit on 100tickers at a > time. > I can shut down the AB/IB connection. Then run an explore on the entire > data base. Then run my program with Filter = InWatchList(WLnum), where > there are less than 100 stocks in WLnum. It takes quite a while to > backfill all in the watchlist. But once that is done, I can shut down > AB/IB connection and do another explore including the newly downloaded > stocks. > > You might try the same with IQfeed. > -- Keith > > On 5/18/2010 15:39, kurasake wrote: > > > > > > > > Hi Tomasz > > > > Thank you for your response. I sincerely appreciate the time you take > > to monitor these forums and answer questions. > > > > What I'm paying IQFeed for is real time data for 500 _Simultaneous_ > > symbol access. I am allowed to view as many symbols as I want so long > > as I don't view more than 500 at the same time. (I've confirmed this > > with IQFeed support). So, if I were to add 500 symbols to the Realtime > > Quote window, I wouldn't expect anything more. But when I run an > > Exploration, I'm not asking for more than 500 _simultaneous_ symbol > > updates at a time. > > > > As I understand it, an Exploration and Automatic Scan in AB is not > > _continually_ requesting data for 500 symbols simultaneously even > > after it's done with the exploration. Rather, (again, as I understand > > it), an Exploration and Automatic Scan goes through the list of > > symbols sequentially, requests data from IQFeed one at a time, and > > then goes on to the next symbol. > > > > So, once an Exploration is done backfilling a symbol, it's no longer > > requesting or getting data for that symbol. That means the whole > > _simultaneous_ and _realtime_ part that I'm paying for is being wasted > > since I'm _not_ actually updating data for 500 symbols with an > > Exploration. In effect, an Exploration takes a "snapshot" of the > > symbol's data and that's it. As far as AB is concerned, until another > > Exploration is run or the user selects another symbol or the Realtime > > Quote window is used to monitor symbols etc it really has no need for > > IQFeed except for the currently selected symbol. > > > > Clearly the IQFeed client provides the means for users to "look" at as > > many symbols as they want as long as it's not 500 symbols viewed > > _simultaneously_. And as IQFeed support personnel confirms, I'm well > > within my rights to view as many symbols as I want so long as it's not > > more than 500 at the same time. So, whether or not I exceed my 500 > > symbol subscription limit depends entirely on the stock investment > > programs to release each slot/ resource when it's done using it. > > > > If I open up the IQFeed Client, it will display how many symbols are > > currently being tracked/monitored by IQFeed. And if I go into > > QuoteTracker and remove symbols from the list of symbols I want to > > monitor, the IQFeed Client will reflect that change and show that I've > > freed up those slots and that I'm now able to replace those symbols > > with new ones. However in AB, once I select a symbol, it holds on to > > an IQFeed "slot" even if I _delete_ the symbol from the AB database. > > So in this case, AB is still holding on to the IQFeed resource even > > though it can not possibly have any use for that resource. ( Or it may > > not "technically" be AB that's holding on to the resource but it > > clearly hasn't told IQFeed to free up that resource, so the end result > > is the same.) > > > > Using another example, let's say I run an exploration on 500 symbols > > in AB. After the Exploration runs, that means all 500 "slots" that I'm > > contracted for are filled up but are _NOT_ actually being used for > > simultaneous realtime updates. All 500 slots are in effect, used but > > dormant. Now, lets say in this state, I decide to run QuoteTracker on > > the same machine using the same IQFeed data source. Since AB is still > > holding on to the 500 slots, QuoteTracker would be unable to get any > > data, _EVEN THOUGH_ AB is not actually actively getting real time data > > for any symbol (except the currently selected). > > > > I've confirmed with IQFeed support that I'm allowed to connect as many > > programs to the IQFeed client as I want. This makes each "slot" in > > IQFeed is a limited and _shared_ resource. If an Exploration in AB ran > > through 500 symbols but didn't release those IQFeed slots, this would > > be analogous to a program using up all the memory and disk drive space > > so other programs are unable to run even though it's not really using > > any of it . So, if AB no longer uses a "slot", why not release it > > after each use? I think this is reasonable? > > > > I can appreciate that AB is too fast for IQFeed to handle. I think > > it's fantastic that you've made AB so fast. But with the backfill > > option in an Exploration turned on, I would think it would be better > > if there were a level of flow-control/handshaking between AB and > > IQFeed so that they "play nicely" with each other. > > > > If one of the main functions of any stock analysis/monitoring program > > is to provide accurate data to the user then I would think that AB > > should wait for IQFeed to "catch up" instead of continuing on and > > leaving IQFeed "behind" since we know that if AB keeps going for a > > large number of symbols, there's going to be corrupt data appearing in > > AB. The two programs must work nicely together in order to produce the > > desired result. The role of IQFeed is to service requests for data and > > to provide that data. The role of AB is to make requests for data from > > IQFeed and display and analyze that data accurately. In this case, > > IQFeed is clearly the bottleneck but as you mention, the data provider > > is always going to be the bottleneck, even if I switched to a faster > > provider. And clearly corrupt data has a major effect on things like > > buy/sell triggers and is an undesired problem. IQFeed, however slow, > > is still the data provider and AB is reliant on data from the d ata > > provider. Everything should work fine if there's proper > > communication/flow control between the two. > > > > This would be somewhat analogous to a computer with a near infinitely > > fast CPU working with a very, very slow disk drive and a very, very > > slow system bus and very, very slow memory etc. No matter how fast the > > CPU is, it must still wait for all the other components in order for > > it to produce the correct result. It's unfortunate that everything > > else isn't fast enough for the CPU but the CPU can not just go off and > > keep going on it's own just because everything else is too slow. I > > think this is a reasonable point as well? > > > > I think it would be fair if... : > > > > - AB released each IQFeed slot/resource as soon as it's done using it, > > since each IQFeed slot is a shared resource. > > - an Exploration waited for IQFeed (or any data provider) to "catch > > up" so that data corruption doesn't occur. Or perhaps have a switch > > that let's the user choose between having AB wait for the data > > provider or to just keep going. > > - or perhaps as another option, an ability for the user to release > > unused IQFeed resources via AFL/OLE etc... Though I would think that > > to ensure that the correct data is always displayed, AB should wait > > for the data provider to catch up. > > > > Otherwise, the user would need to change stock trading behavior to > > suit the limitations of the program. Constant symbol maintenance and > > tracking in AB would be a nightmare. This is especially true if for > > example I'm using up a lot of IQFeed slots in QuoteTracker and the > > available "free" slots left for Amibroker is far less then 500. What > > if I'm already tracking 499 symbols in QuoteTracker? Should this > > condition cause AB to start displaying corrupt data on Explorations on > > 10, 20, 50 symbols? > > > > Clearly these are limitations that don't need to exist (ie: there's > > nothing technically preventing AB from working well with IQFeed). Nor > > would I think that this is the type of behavior you would want in your > > program. > > > > I've talked over the 500 symbol limit issue with IQFeed Support and > > confirmed that I'm not asking for anything out of bounds . I'm not > > sure if my explanation were clear (I hope they were) but I think what > > I'm asking for is very reasonable and expected behavior for a stock > > monitoring/investing/trading program. > > > > Thank you for your time. > > > > With respect and best regards. > > > > --- In [email protected] <mailto:amibroker%40yahoogroups.com>, > > Tomasz Janeczko <groups@> wrote: > > > > > > Hello, > > > > > > Simply speaking: you paid for 500 symbols and you receive 500 symbols. > > > It is that simple. > > > If you buy Fiat do not expect it to be Bugatti Veyron. > > > > > > a) you are not allowed to exceed your subscription limits with IQFeed. > > > IQFeed simply won't allow you to get data for more symbols at the > > same time > > > (subscription to any more symbols will simply fail and you won't get > > data) > > > > > > b) if you exceed the subscription limit AmiBroker will unsubscribe > > > oldest symbol > > > (the one that was accessed first) and subscribe new and will request > > > backfill, > > > BUT .... because the backfill from IQFeed is SLOW, IQFeed is literally > > > unable to > > > keep up with exploration or any automatic scan. > > > IQFeed (and many other RT sources) takes many seconds to backfill > > ONE symbol > > > (as IQFeed was designed to display CHARTS, not to perform whole market > > > scans) > > > and is designed with "manual", i.e. *SLOW* operation in mind (so you > > > manually select the chart > > > and watch it) - the way how QuoteTracker works - MANUALLY (slowly) > > > adding/removing symbols > > > instead of running automated scans on huge portfolios. > > > AmiBroker on the other hand is able to scan thousands of symbols in one > > > second, therefore > > > IQFeed is not able to deliver data fast enough (via backfill) if you > > exceed > > > subscription limits. Becuase of the time IQFeed needs to do the > > backfill > > > (if you exceed subscription counts), > > > you may end up with data holes (if you exceed the subscription limit). > > > > > > Why do you expect receiving more data that you paid for ? > > > > > > Best regards, > > > Tomasz Janeczko > > > amibroker.com > > > > > > On 2010-05-14 22:12, kurasake wrote: > > > > I'm running AB 5.30pro using IQFeed as my RT data feed on a Core > > i7 w/4gig ram. > > > > > > > > In a different post, I'd been describing a problem I've been > > having when performing an exploration on a large number of symbols. > > The problem was that if I scanned too many symbols, I would start to > > get bad data appearing in AB. > > > > > > > > After some feedback, it appears that this problem occurs when I've > > scanned more symbols then what my contract with IQFeed allows. In > > other words, my contract with IQFeed allows me to monitor 500 symbols > > simultaneously at a time and if I exceed that 500 symbol limit, I > > would start getting bad data in AB. I haven't confirmed this > > "scientifically" yet but the problem does seem to happen after I've > > scanned somewhere in the neighborhood over 500 symbols. > > > > > > > > The question I have though is if I've only "looked" at a symbol in > > AB and I'm not actively monitoring it, like say in the Realtime Quote > > window, should AB still be holding on to an IQFeed > > "slot"/resource/token? I would think that if I'm not actively > > collecting data on a symbol and I've only "looked" at it for a brief > > moment, that AB should release this slot back to IQFeed. Or is AB > > still collecting data on that symbol even if it's not in the Realtime > > Quote window? > > > > > > > > If I open up the IQFeed Connection Manager, I can see how many > > symbols IQFeed is "tracking" for me. Every time I click on a symbol in > > AB and it backfills that symbol with data, the Number of Symbols > > counter in the IQFeed Connection Manager goes up. The counter however, > > never appers to go down unless I either break the connection from AB > > to the IQFeed client or I exit from AB. > > > > > > > > This puts a rather strong restriction on how I use AB as I can > > only scan/explore/look at 500 symbols in a single AB session and > > IQFeed thinks I'm using up my entire allotment when I'm not. > > > > > > > > Incidentally, if I run QuoteTracker on the same machine and have > > it connect to the same IQFeed client, it will "release" an IQFeed slot > > for every symbol I remove from the active QT portfolio. So it should > > be possible to do this in AB as well. > > > > > > > > If I perform a Reconnect or Shutdown from AB by right clicking in > > the status bar where it displays the client connection status, the > > IQFeed symbol count drops back down to what it was before I started > > AB. Does anyone know if there's any way to do this "reset"/release > > using COM objects or other programming means? Maybe I can write a vb > > script or something that keeps calling this obj method? Or is this a > > feature request that I need to place with AB? > > > > > > > > Many thanks. > > > > > > > > > > > > > > > > ------------------------------------ > > > > > > > > **** IMPORTANT PLEASE READ **** > > > > This group is for the discussion between users only. > > > > This is *NOT* technical support channel. > > > > > > > > TO GET TECHNICAL SUPPORT send an e-mail directly to > > > > SUPPORT {at} amibroker.com > > > > > > > > TO SUBMIT SUGGESTIONS please use FEEDBACK CENTER at > > > > http://www.amibroker.com/feedback/ > > > > (submissions sent via other channels won't be considered) > > > > > > > > For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG: > > > > http://www.amibroker.com/devlog/ > > > > > > > > Yahoo! Groups Links > > > > > > > > > > > > > > > > > > > > > > > > > > > >
