Kurasake --
I don't use IQ feed, but IB instead. IB has a limit on 100tickers at a
time.
I can shut down the AB/IB connection. Then run an explore on the entire
data base. Then run my program with Filter = InWatchList(WLnum), where
there are less than 100 stocks in WLnum. It takes quite a while to
backfill all in the watchlist. But once that is done, I can shut down
AB/IB connection and do another explore including the newly downloaded
stocks.
You might try the same with IQfeed.
-- Keith
On 5/18/2010 15:39, kurasake wrote:
Hi Tomasz
Thank you for your response. I sincerely appreciate the time you take
to monitor these forums and answer questions.
What I'm paying IQFeed for is real time data for 500 _Simultaneous_
symbol access. I am allowed to view as many symbols as I want so long
as I don't view more than 500 at the same time. (I've confirmed this
with IQFeed support). So, if I were to add 500 symbols to the Realtime
Quote window, I wouldn't expect anything more. But when I run an
Exploration, I'm not asking for more than 500 _simultaneous_ symbol
updates at a time.
As I understand it, an Exploration and Automatic Scan in AB is not
_continually_ requesting data for 500 symbols simultaneously even
after it's done with the exploration. Rather, (again, as I understand
it), an Exploration and Automatic Scan goes through the list of
symbols sequentially, requests data from IQFeed one at a time, and
then goes on to the next symbol.
So, once an Exploration is done backfilling a symbol, it's no longer
requesting or getting data for that symbol. That means the whole
_simultaneous_ and _realtime_ part that I'm paying for is being wasted
since I'm _not_ actually updating data for 500 symbols with an
Exploration. In effect, an Exploration takes a "snapshot" of the
symbol's data and that's it. As far as AB is concerned, until another
Exploration is run or the user selects another symbol or the Realtime
Quote window is used to monitor symbols etc it really has no need for
IQFeed except for the currently selected symbol.
Clearly the IQFeed client provides the means for users to "look" at as
many symbols as they want as long as it's not 500 symbols viewed
_simultaneously_. And as IQFeed support personnel confirms, I'm well
within my rights to view as many symbols as I want so long as it's not
more than 500 at the same time. So, whether or not I exceed my 500
symbol subscription limit depends entirely on the stock investment
programs to release each slot/ resource when it's done using it.
If I open up the IQFeed Client, it will display how many symbols are
currently being tracked/monitored by IQFeed. And if I go into
QuoteTracker and remove symbols from the list of symbols I want to
monitor, the IQFeed Client will reflect that change and show that I've
freed up those slots and that I'm now able to replace those symbols
with new ones. However in AB, once I select a symbol, it holds on to
an IQFeed "slot" even if I _delete_ the symbol from the AB database.
So in this case, AB is still holding on to the IQFeed resource even
though it can not possibly have any use for that resource. ( Or it may
not "technically" be AB that's holding on to the resource but it
clearly hasn't told IQFeed to free up that resource, so the end result
is the same.)
Using another example, let's say I run an exploration on 500 symbols
in AB. After the Exploration runs, that means all 500 "slots" that I'm
contracted for are filled up but are _NOT_ actually being used for
simultaneous realtime updates. All 500 slots are in effect, used but
dormant. Now, lets say in this state, I decide to run QuoteTracker on
the same machine using the same IQFeed data source. Since AB is still
holding on to the 500 slots, QuoteTracker would be unable to get any
data, _EVEN THOUGH_ AB is not actually actively getting real time data
for any symbol (except the currently selected).
I've confirmed with IQFeed support that I'm allowed to connect as many
programs to the IQFeed client as I want. This makes each "slot" in
IQFeed is a limited and _shared_ resource. If an Exploration in AB ran
through 500 symbols but didn't release those IQFeed slots, this would
be analogous to a program using up all the memory and disk drive space
so other programs are unable to run even though it's not really using
any of it . So, if AB no longer uses a "slot", why not release it
after each use? I think this is reasonable?
I can appreciate that AB is too fast for IQFeed to handle. I think
it's fantastic that you've made AB so fast. But with the backfill
option in an Exploration turned on, I would think it would be better
if there were a level of flow-control/handshaking between AB and
IQFeed so that they "play nicely" with each other.
If one of the main functions of any stock analysis/monitoring program
is to provide accurate data to the user then I would think that AB
should wait for IQFeed to "catch up" instead of continuing on and
leaving IQFeed "behind" since we know that if AB keeps going for a
large number of symbols, there's going to be corrupt data appearing in
AB. The two programs must work nicely together in order to produce the
desired result. The role of IQFeed is to service requests for data and
to provide that data. The role of AB is to make requests for data from
IQFeed and display and analyze that data accurately. In this case,
IQFeed is clearly the bottleneck but as you mention, the data provider
is always going to be the bottleneck, even if I switched to a faster
provider. And clearly corrupt data has a major effect on things like
buy/sell triggers and is an undesired problem. IQFeed, however slow,
is still the data provider and AB is reliant on data from the d ata
provider. Everything should work fine if there's proper
communication/flow control between the two.
This would be somewhat analogous to a computer with a near infinitely
fast CPU working with a very, very slow disk drive and a very, very
slow system bus and very, very slow memory etc. No matter how fast the
CPU is, it must still wait for all the other components in order for
it to produce the correct result. It's unfortunate that everything
else isn't fast enough for the CPU but the CPU can not just go off and
keep going on it's own just because everything else is too slow. I
think this is a reasonable point as well?
I think it would be fair if... :
- AB released each IQFeed slot/resource as soon as it's done using it,
since each IQFeed slot is a shared resource.
- an Exploration waited for IQFeed (or any data provider) to "catch
up" so that data corruption doesn't occur. Or perhaps have a switch
that let's the user choose between having AB wait for the data
provider or to just keep going.
- or perhaps as another option, an ability for the user to release
unused IQFeed resources via AFL/OLE etc... Though I would think that
to ensure that the correct data is always displayed, AB should wait
for the data provider to catch up.
Otherwise, the user would need to change stock trading behavior to
suit the limitations of the program. Constant symbol maintenance and
tracking in AB would be a nightmare. This is especially true if for
example I'm using up a lot of IQFeed slots in QuoteTracker and the
available "free" slots left for Amibroker is far less then 500. What
if I'm already tracking 499 symbols in QuoteTracker? Should this
condition cause AB to start displaying corrupt data on Explorations on
10, 20, 50 symbols?
Clearly these are limitations that don't need to exist (ie: there's
nothing technically preventing AB from working well with IQFeed). Nor
would I think that this is the type of behavior you would want in your
program.
I've talked over the 500 symbol limit issue with IQFeed Support and
confirmed that I'm not asking for anything out of bounds . I'm not
sure if my explanation were clear (I hope they were) but I think what
I'm asking for is very reasonable and expected behavior for a stock
monitoring/investing/trading program.
Thank you for your time.
With respect and best regards.
--- In [email protected] <mailto:amibroker%40yahoogroups.com>,
Tomasz Janeczko <gro...@...> wrote:
>
> Hello,
>
> Simply speaking: you paid for 500 symbols and you receive 500 symbols.
> It is that simple.
> If you buy Fiat do not expect it to be Bugatti Veyron.
>
> a) you are not allowed to exceed your subscription limits with IQFeed.
> IQFeed simply won't allow you to get data for more symbols at the
same time
> (subscription to any more symbols will simply fail and you won't get
data)
>
> b) if you exceed the subscription limit AmiBroker will unsubscribe
> oldest symbol
> (the one that was accessed first) and subscribe new and will request
> backfill,
> BUT .... because the backfill from IQFeed is SLOW, IQFeed is literally
> unable to
> keep up with exploration or any automatic scan.
> IQFeed (and many other RT sources) takes many seconds to backfill
ONE symbol
> (as IQFeed was designed to display CHARTS, not to perform whole market
> scans)
> and is designed with "manual", i.e. *SLOW* operation in mind (so you
> manually select the chart
> and watch it) - the way how QuoteTracker works - MANUALLY (slowly)
> adding/removing symbols
> instead of running automated scans on huge portfolios.
> AmiBroker on the other hand is able to scan thousands of symbols in one
> second, therefore
> IQFeed is not able to deliver data fast enough (via backfill) if you
exceed
> subscription limits. Becuase of the time IQFeed needs to do the
backfill
> (if you exceed subscription counts),
> you may end up with data holes (if you exceed the subscription limit).
>
> Why do you expect receiving more data that you paid for ?
>
> Best regards,
> Tomasz Janeczko
> amibroker.com
>
> On 2010-05-14 22:12, kurasake wrote:
> > I'm running AB 5.30pro using IQFeed as my RT data feed on a Core
i7 w/4gig ram.
> >
> > In a different post, I'd been describing a problem I've been
having when performing an exploration on a large number of symbols.
The problem was that if I scanned too many symbols, I would start to
get bad data appearing in AB.
> >
> > After some feedback, it appears that this problem occurs when I've
scanned more symbols then what my contract with IQFeed allows. In
other words, my contract with IQFeed allows me to monitor 500 symbols
simultaneously at a time and if I exceed that 500 symbol limit, I
would start getting bad data in AB. I haven't confirmed this
"scientifically" yet but the problem does seem to happen after I've
scanned somewhere in the neighborhood over 500 symbols.
> >
> > The question I have though is if I've only "looked" at a symbol in
AB and I'm not actively monitoring it, like say in the Realtime Quote
window, should AB still be holding on to an IQFeed
"slot"/resource/token? I would think that if I'm not actively
collecting data on a symbol and I've only "looked" at it for a brief
moment, that AB should release this slot back to IQFeed. Or is AB
still collecting data on that symbol even if it's not in the Realtime
Quote window?
> >
> > If I open up the IQFeed Connection Manager, I can see how many
symbols IQFeed is "tracking" for me. Every time I click on a symbol in
AB and it backfills that symbol with data, the Number of Symbols
counter in the IQFeed Connection Manager goes up. The counter however,
never appers to go down unless I either break the connection from AB
to the IQFeed client or I exit from AB.
> >
> > This puts a rather strong restriction on how I use AB as I can
only scan/explore/look at 500 symbols in a single AB session and
IQFeed thinks I'm using up my entire allotment when I'm not.
> >
> > Incidentally, if I run QuoteTracker on the same machine and have
it connect to the same IQFeed client, it will "release" an IQFeed slot
for every symbol I remove from the active QT portfolio. So it should
be possible to do this in AB as well.
> >
> > If I perform a Reconnect or Shutdown from AB by right clicking in
the status bar where it displays the client connection status, the
IQFeed symbol count drops back down to what it was before I started
AB. Does anyone know if there's any way to do this "reset"/release
using COM objects or other programming means? Maybe I can write a vb
script or something that keeps calling this obj method? Or is this a
feature request that I need to place with AB?
> >
> > Many thanks.
> >
> >
> >
> > ------------------------------------
> >
> > **** IMPORTANT PLEASE READ ****
> > This group is for the discussion between users only.
> > This is *NOT* technical support channel.
> >
> > TO GET TECHNICAL SUPPORT send an e-mail directly to
> > SUPPORT {at} amibroker.com
> >
> > TO SUBMIT SUGGESTIONS please use FEEDBACK CENTER at
> > http://www.amibroker.com/feedback/
> > (submissions sent via other channels won't be considered)
> >
> > For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:
> > http://www.amibroker.com/devlog/
> >
> > Yahoo! Groups Links
> >
> >
> >
> >
> >
>
