To answer the *actual* question you posed, regarding R and IBrokers
(not Excel and Yahoo!?)
In the talks I have given at useR and R/Finance I have a slide/video
that addresses this issue. As Russ pointed out in his reply, the TWS
has some rather picky rules to keep in mind, so caveat emptor as
always.
snapshot=TRUE in the API is really useless - this is a design flaw of
the API (not IBrokers). It doesn't assure timeliness of the data,
even IF it managed to do things quickly - which it doesn't do either.
IBrokers maps to the API and leaves you the user with the burden of
dealing with the APIs warts.
Of course I also made the functions R like where possible (fixing the
API, if you will)...
The workaround is exactly what you are pursuing. The weakness at
present is in the error handling - at present the code I will paste
below chokes if it encounters a bad ticker. I'll work on a patch to
make that ugliness go away.
With that in mind ...
# some symbols to start with ..
> nms <-
> c("AAPL","YHOO","AMZN","CSCO","BBY","A","AAR","AA","PCM","PEP","SAP","ZMH","MSFT")
> reqMktData(tws, lapply(nms,twsSTK),
> eventWrapper=eWrapper.data(length(nms)),CALLBACK=snapShot)
BidSize BidPrice AskPrice AskSize Last LastSize Volume
AAPL 7 252.92 253.00 3 253.00 1 40937
YHOO 138 13.89 13.90 474 13.89 1 29920
AMZN 4 128.87 128.92 1 128.93 1 17413
CSCO 346 22.30 22.31 631 22.30 2 206266
BBY 6 32.95 32.96 44 32.95 1 11780
A 32 29.17 29.18 30 29.18 3 12172
AAR 7 21.81 21.85 1 21.85 1 87
AA 372 10.84 10.85 536 10.85 2 70867
PCM 4 10.87 10.95 2 10.85 12 41
PEP 10 65.42 65.43 1 65.44 1 6528
SAP 11 45.28 45.30 9 45.30 1 2197
ZMH 1 51.48 51.49 21 51.48 1 3458
MSFT 453 24.56 24.57 661 24.55 1 136815
> system.time( reqMktData(tws, lapply(nms,twsSTK),
> eventWrapper=eWrapper.data(length(nms)),
user system elapsed
0.259 0.004 0.889
Extending to 500 symbols or so is doable as well, though takes more
time to fetch. I am breaking it up into 100 symbol chunks, as I think
you might have some issues that Russ alluded to if you don't ... but
experimenting is easy with R.
> nms100 <- rep(nms[1:10],10)
> reqMktData(tws, lapply(nms100,twsSTK),
> eventWrapper=eWrapper.data(length(nms100)),CALLBACK=snapShot)
> system.time( reqMktData(tws, lapply(nms100,twsSTK),
> eventWrapper=eWrapper.data(length(nms100)),CALLBACK=snapShot) )
user system elapsed
10.931 0.054 11.038
> str(reqMktData(tws, lapply(nms100,twsSTK),
> eventWrapper=eWrapper.data(length(nms100)),CALLBACK=snapShot))
'data.frame': 100 obs. of 7 variables:
$ BidSize : num 1 299 1 400 20 36 5 215 4 35 ...
$ BidPrice: num 253.8 13.9 129.2 22.3 33 ...
$ AskPrice: num 253.8 13.9 129.3 22.4 33 ...
$ AskSize : num 2 192 1 937 24 34 4 434 4 7 ...
$ Last : num 253.8 13.9 129.2 22.4 33 ...
$ LastSize: num 2 1 1 1 1 1 2 1 1 1 ...
$ Volume : num 45476 34769 19535 235881 16038 ...
The "snapShot" code is only twsCALLBACK modified in 2 places...
> dput(snapShot)
function (twsCon, eWrapper, timestamp, file, playback = 1, ...)
{
if (missing(eWrapper))
eWrapper <- eWrapper()
names(eWrapper$.Data$data) <- eWrapper$.Data$symbols
con <- twsCon[[1]]
if (inherits(twsCon, "twsPlayback")) {
sys.time <- NULL
while (TRUE) {
if (!is.null(timestamp)) {
last.time <- sys.time
sys.time <- as.POSIXct(strptime(paste(readBin(con,
character(), 2), collapse = " "), timestamp))
if (!is.null(last.time)) {
Sys.sleep((sys.time - last.time) * playback)
}
curMsg <- .Internal(readBin(con, "character",
1L, NA_integer_, TRUE, FALSE))
if (length(curMsg) < 1)
next
processMsg(curMsg, con, eWrapper, format(sys.time,
timestamp), file, ...)
}
else {
curMsg <- readBin(con, character(), 1)
if (length(curMsg) < 1)
next
processMsg(curMsg, con, eWrapper, timestamp,
file, ...)
if (curMsg == .twsIncomingMSG$REAL_TIME_BARS)
Sys.sleep(5 * playback)
}
}
}
else {
while (TRUE) {
socketSelect(list(con), FALSE, NULL)
curMsg <- .Internal(readBin(con, "character", 1L,
NA_integer_, TRUE, FALSE))
if (!is.null(timestamp)) {
processMsg(curMsg, con, eWrapper, format(Sys.time(),
timestamp), file, ...)
}
else {
processMsg(curMsg, con, eWrapper, timestamp,
file, ...)
}
if (!any(sapply(eWrapper$.Data$data, is.na)))
return(do.call(rbind, lapply(eWrapper$.Data$data,
as.data.frame)))
}
}
}
The lines changed are:
names(eWrapper$.Data$data) <- eWrapper$.Data$symbols
and:
if (!any(sapply(eWrapper$.Data$data, is.na)))
return(do.call(rbind, lapply(eWrapper$.Data$data,
as.data.frame)))
HTH
Jeff
On Tue, Aug 17, 2010 at 11:20 PM, zerdna <az...@yahoo.com> wrote:
>
> Hi, i need to get a quotes for a large set of stocks at 5 min after the open.
> I am trying to use IBrokers, but frankly not making much progress. I
> understand that i need to use reqMktData. It seems that taking snapshot is
> what i need, but when i do something like
>
> df<-reqMktData(tws, lapply(nms, twsSTK), tickGenerics="", snapshot=T),
>
> where nms is my vector of tickers,
>
> i get a dataframe back after waiting for a while -- about 12 sec per ticker,
> which comes out to more than 1.5 hours for my list of 500 tickers. Also,
> most of the data is NA. Maybe it has to do with stocks on the list being
> relatively illiquid or maybe i am doing something wrong. Could someone
> comment, please?
>
> I looked into not using snapshot and try instead to use
> myWrapper <- eWrapper.MktData.CSV(1) and
> myWrapper$get.Data("data") . While i could do it for one stock by manually
> interrupting reqMktData and getting last quote, i cannot figure out what
> needs to be done for a list of stocks. Do i need to repeat reqMktData and
> somehow programmatically interrupt inside the loop? Do i modify twsCALLBACK
> to return after gathering full quote? Would really appreciate any pointers,
> references, or help of any kind.
>
> --
> View this message in context:
> http://r.789695.n4.nabble.com/how-to-get-quotes-with-IBrokers-tp2329328p2329328.html
> Sent from the Rmetrics mailing list archive at Nabble.com.
>
> _______________________________________________
> R-SIG-Finance@stat.math.ethz.ch mailing list
> https://stat.ethz.ch/mailman/listinfo/r-sig-finance
> -- Subscriber-posting only. If you want to post, subscribe first.
> -- Also note that this is not the r-help list where general R questions
> should go.
>
--
Jeffrey Ryan
jeff.a.r...@gmail.com
_______________________________________________
R-SIG-Finance@stat.math.ethz.ch mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-finance
-- Subscriber-posting only. If you want to post, subscribe first.
-- Also note that this is not the r-help list where general R questions should
go.
