Hello, "But when I run an Exploration, I'm not asking for more than 500 _simultaneous_ symbol updates at a time. "
Yes you DO. This is so because of technical limitations of IQFeed API I described earlier. I wrote everything I wanted to write previously and you should re-read what I wrote. If you think what I wrote is not correct, go on take free AmiBroker Development Kit and take IQFeed API and write your own better plugin. If you think that IQFeed staff knows better - ask them to write better plugin. And make sure you tell me what is their response. I would have a good laugh. Best regards, Tomasz Janeczko amibroker.com On 2010-05-18 21:39, kurasake wrote: > > Hi Tomasz > > Thank you for your response. I sincerely appreciate the time you take to > monitor these forums and answer questions. > > What I'm paying IQFeed for is real time data for 500 _Simultaneous_ symbol > access. I am allowed to view as many symbols as I want so long as I don't > view more than 500 at the same time. (I've confirmed this with IQFeed > support). So, if I were to add 500 symbols to the Realtime Quote window, I > wouldn't expect anything more. But when I run an Exploration, I'm not asking > for more than 500 _simultaneous_ symbol updates at a time. > > As I understand it, an Exploration and Automatic Scan in AB is not > _continually_ requesting data for 500 symbols simultaneously even after it's > done with the exploration. Rather, (again, as I understand it), an > Exploration and Automatic Scan goes through the list of symbols > sequentially, requests data from IQFeed one at a time, and then goes on to > the next symbol. > > So, once an Exploration is done backfilling a symbol, it's no longer > requesting or getting data for that symbol. That means the whole > _simultaneous_ and _realtime_ part that I'm paying for is being wasted since > I'm _not_ actually updating data for 500 symbols with an Exploration. In > effect, an Exploration takes a "snapshot" of the symbol's data and that's it. > As far as AB is concerned, until another Exploration is run or the user > selects another symbol or the Realtime Quote window is used to monitor > symbols etc it really has no need for IQFeed except for the currently > selected symbol. > > Clearly the IQFeed client provides the means for users to "look" at as many > symbols as they want as long as it's not 500 symbols viewed _simultaneously_. > And as IQFeed support personnel confirms, I'm well within my rights to view > as many symbols as I want so long as it's not more than 500 at the same time. > So, whether or not I exceed my 500 symbol subscription limit depends > entirely on the stock investment programs to release each slot/ resource when > it's done using it. > > If I open up the IQFeed Client, it will display how many symbols are > currently being tracked/monitored by IQFeed. And if I go into QuoteTracker > and remove symbols from the list of symbols I want to monitor, the IQFeed > Client will reflect that change and show that I've freed up those slots and > that I'm now able to replace those symbols with new ones. However in AB, > once I select a symbol, it holds on to an IQFeed "slot" even if I _delete_ > the symbol from the AB database. So in this case, AB is still holding on to > the IQFeed resource even though it can not possibly have any use for that > resource. ( Or it may not "technically" be AB that's holding on to the > resource but it clearly hasn't told IQFeed to free up that resource, so the > end result is the same.) > > Using another example, let's say I run an exploration on 500 symbols in AB. > After the Exploration runs, that means all 500 "slots" that I'm contracted > for are filled up but are _NOT_ actually being used for simultaneous > realtime updates. All 500 slots are in effect, used but dormant. Now, lets > say in this state, I decide to run QuoteTracker on the same machine using the > same IQFeed data source. Since AB is still holding on to the 500 slots, > QuoteTracker would be unable to get any data, _EVEN THOUGH_ AB is not > actually actively getting real time data for any symbol (except the currently > selected). > > I've confirmed with IQFeed support that I'm allowed to connect as many > programs to the IQFeed client as I want. This makes each "slot" in IQFeed is > a limited and _shared_ resource. If an Exploration in AB ran through 500 > symbols but didn't release those IQFeed slots, this would be analogous to a > program using up all the memory and disk drive space so other programs are > unable to run even though it's not really using any of it . So, if AB no > longer uses a "slot", why not release it after each use? I think this is > reasonable? > > I can appreciate that AB is too fast for IQFeed to handle. I think it's > fantastic that you've made AB so fast. But with the backfill option in an > Exploration turned on, I would think it would be better if there were a > level of flow-control/handshaking between AB and IQFeed so that they "play > nicely" with each other. > > If one of the main functions of any stock analysis/monitoring program is to > provide accurate data to the user then I would think that AB should wait for > IQFeed to "catch up" instead of continuing on and leaving IQFeed "behind" > since we know that if AB keeps going for a large number of symbols, there's > going to be corrupt data appearing in AB. The two programs must work nicely > together in order to produce the desired result. The role of IQFeed is to > service requests for data and to provide that data. The role of AB is to > make requests for data from IQFeed and display and analyze that data > accurately. In this case, IQFeed is clearly the bottleneck but as you > mention, the data provider is always going to be the bottleneck, even if I > switched to a faster provider. And clearly corrupt data has a major effect > on things like buy/sell triggers and is an undesired problem. IQFeed, > however slow, is still the data provider and AB is reliant on data from the > data provider. Everything should work fine if there's proper > communication/flow control between the two. > > This would be somewhat analogous to a computer with a near infinitely fast > CPU working with a very, very slow disk drive and a very, very slow system > bus and very, very slow memory etc. No matter how fast the CPU is, it must > still wait for all the other components in order for it to produce the > correct result. It's unfortunate that everything else isn't fast enough for > the CPU but the CPU can not just go off and keep going on it's own just > because everything else is too slow. I think this is a reasonable point as > well? > > I think it would be fair if... : > > - AB released each IQFeed slot/resource as soon as it's done using it, > since each IQFeed slot is a shared resource. > - an Exploration waited for IQFeed (or any data provider) to "catch up" > so that data corruption doesn't occur. Or perhaps have a switch that let's > the user choose between having AB wait for the data provider or to just keep > going. > - or perhaps as another option, an ability for the user to release unused > IQFeed resources via AFL/OLE etc... Though I would think that to ensure that > the correct data is always displayed, AB should wait for the data provider to > catch up. > > Otherwise, the user would need to change stock trading behavior to suit the > limitations of the program. Constant symbol maintenance and tracking in AB > would be a nightmare. This is especially true if for example I'm using up a > lot of IQFeed slots in QuoteTracker and the available "free" slots left for > Amibroker is far less then 500. What if I'm already tracking 499 symbols in > QuoteTracker? Should this condition cause AB to start displaying corrupt > data on Explorations on 10, 20, 50 symbols? > > Clearly these are limitations that don't need to exist (ie: there's nothing > technically preventing AB from working well with IQFeed). Nor would I think > that this is the type of behavior you would want in your program. > > I've talked over the 500 symbol limit issue with IQFeed Support and confirmed > that I'm not asking for anything out of bounds . I'm not sure if my > explanation were clear (I hope they were) but I think what I'm asking for is > very reasonable and expected behavior for a stock > monitoring/investing/trading program. > > Thank you for your time. > > With respect and best regards. > > > > --- In [email protected], Tomasz Janeczko<gro...@...> wrote: > >> Hello, >> >> Simply speaking: you paid for 500 symbols and you receive 500 symbols. >> It is that simple. >> If you buy Fiat do not expect it to be Bugatti Veyron. >> >> a) you are not allowed to exceed your subscription limits with IQFeed. >> IQFeed simply won't allow you to get data for more symbols at the same time >> (subscription to any more symbols will simply fail and you won't get data) >> >> b) if you exceed the subscription limit AmiBroker will unsubscribe >> oldest symbol >> (the one that was accessed first) and subscribe new and will request >> backfill, >> BUT .... because the backfill from IQFeed is SLOW, IQFeed is literally >> unable to >> keep up with exploration or any automatic scan. >> IQFeed (and many other RT sources) takes many seconds to backfill ONE symbol >> (as IQFeed was designed to display CHARTS, not to perform whole market >> scans) >> and is designed with "manual", i.e. *SLOW* operation in mind (so you >> manually select the chart >> and watch it) - the way how QuoteTracker works - MANUALLY (slowly) >> adding/removing symbols >> instead of running automated scans on huge portfolios. >> AmiBroker on the other hand is able to scan thousands of symbols in one >> second, therefore >> IQFeed is not able to deliver data fast enough (via backfill) if you exceed >> subscription limits. Becuase of the time IQFeed needs to do the backfill >> (if you exceed subscription counts), >> you may end up with data holes (if you exceed the subscription limit). >> >> Why do you expect receiving more data that you paid for ? >> >> Best regards, >> Tomasz Janeczko >> amibroker.com >> >> On 2010-05-14 22:12, kurasake wrote: >> >>> I'm running AB 5.30pro using IQFeed as my RT data feed on a Core i7 w/4gig >>> ram. >>> >>> In a different post, I'd been describing a problem I've been having when >>> performing an exploration on a large number of symbols. The problem was >>> that if I scanned too many symbols, I would start to get bad data appearing >>> in AB. >>> >>> After some feedback, it appears that this problem occurs when I've scanned >>> more symbols then what my contract with IQFeed allows. In other words, my >>> contract with IQFeed allows me to monitor 500 symbols simultaneously at a >>> time and if I exceed that 500 symbol limit, I would start getting bad data >>> in AB. I haven't confirmed this "scientifically" yet but the problem does >>> seem to happen after I've scanned somewhere in the neighborhood over 500 >>> symbols. >>> >>> The question I have though is if I've only "looked" at a symbol in AB and >>> I'm not actively monitoring it, like say in the Realtime Quote window, >>> should AB still be holding on to an IQFeed "slot"/resource/token? I would >>> think that if I'm not actively collecting data on a symbol and I've only >>> "looked" at it for a brief moment, that AB should release this slot back to >>> IQFeed. Or is AB still collecting data on that symbol even if it's not in >>> the Realtime Quote window? >>> >>> If I open up the IQFeed Connection Manager, I can see how many symbols >>> IQFeed is "tracking" for me. Every time I click on a symbol in AB and it >>> backfills that symbol with data, the Number of Symbols counter in the >>> IQFeed Connection Manager goes up. The counter however, never appers to go >>> down unless I either break the connection from AB to the IQFeed client or I >>> exit from AB. >>> >>> This puts a rather strong restriction on how I use AB as I can only >>> scan/explore/look at 500 symbols in a single AB session and IQFeed thinks >>> I'm using up my entire allotment when I'm not. >>> >>> Incidentally, if I run QuoteTracker on the same machine and have it connect >>> to the same IQFeed client, it will "release" an IQFeed slot for every >>> symbol I remove from the active QT portfolio. So it should be possible to >>> do this in AB as well. >>> >>> If I perform a Reconnect or Shutdown from AB by right clicking in the >>> status bar where it displays the client connection status, the IQFeed >>> symbol count drops back down to what it was before I started AB. Does >>> anyone know if there's any way to do this "reset"/release using COM objects >>> or other programming means? Maybe I can write a vb script or something >>> that keeps calling this obj method? Or is this a feature request that I >>> need to place with AB? >>> >>> Many thanks. >>> >>> >>> >>> ------------------------------------ >>> >>> **** IMPORTANT PLEASE READ **** >>> This group is for the discussion between users only. >>> This is *NOT* technical support channel. >>> >>> TO GET TECHNICAL SUPPORT send an e-mail directly to >>> SUPPORT {at} amibroker.com >>> >>> TO SUBMIT SUGGESTIONS please use FEEDBACK CENTER at >>> http://www.amibroker.com/feedback/ >>> (submissions sent via other channels won't be considered) >>> >>> For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG: >>> http://www.amibroker.com/devlog/ >>> >>> Yahoo! Groups Links >>> >>> >>> >>> >>> >>> >> > > > > ------------------------------------ > > **** IMPORTANT PLEASE READ **** > This group is for the discussion between users only. > This is *NOT* technical support channel. > > TO GET TECHNICAL SUPPORT send an e-mail directly to > SUPPORT {at} amibroker.com > > TO SUBMIT SUGGESTIONS please use FEEDBACK CENTER at > http://www.amibroker.com/feedback/ > (submissions sent via other channels won't be considered) > > For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG: > http://www.amibroker.com/devlog/ > > Yahoo! Groups Links > > > > >
