Dear Eugene, right. This was the last event that was registered. Then till I stopped it 10 hours later nothing happened. Regarding the re-subscribe I am wondering what this means: is the instrument also reselected? I am wondering because of the following case: - out of a type of contract (e.g, ES) always the most liquid one is chosen. Now let's assume a position is held, while the problem occurs: if the resubscribe happens (and the very unfortunately it is exactly when the handover between instruments occurs, later the other duration of ES is sold, when the attempt is made to close the position. Is this a relevant scenario?
Cheers Klaus Am Mittwoch, 10. Oktober 2012 18:37:06 UTC-4 schrieb nonlinear: > > One important thing I noticed in your log is that the message: > > 10/01/12 00:16:16.630 IB API 1102: Connectivity between IB and Trader > Workstation has been restored - data maintained. > > was NOT followed up with this the message: > > IB API 2104: Market data farm connection is OK:usfuture > > So, my plan for implementing the "re-subscribe" functionality is to detect > this condition and resubscribe. > > > > > > On Tue, Oct 9, 2012 at 11:48 PM, Klaus <[email protected] <javascript:> > > wrote: > >> Dear Eugene, >> right. I did not check every line, but it was exactly the same price at >> the end and all checks in between gave the same data >> (It had run by this time for 10 hours). It was only afterwards that I >> found that exactly at the same time these events happened. >> >> Klaus >> >> >> >> >>> Between 23:59:05 and 00:16:16, there is some activity indicating IB >>> restart processes kicking in, which ultimately end at 00:16:16. The message >>> says "Connectivity between IB and Trader Workstation has been restored - >>> data maintained", which means that all market data subscriptions are still >>> valid. But you are saying that the price data never changed after 00:16:16. >>> Correct? >>> >>> >>> >>> On Tuesday, October 9, 2012 2:48:01 PM UTC-4, Klaus wrote: >>>> >>>> Dear Eugene, Dear Judson, >>>> >>>> first thanks to all for clarifying this. This was puzzling me and now I >>>> understand how it can technically >>>> happen. I was mistaken to believe it would be the same stream. >>>> >>>> Certainly, if you give me a proposed patch, I would be happy to analyze >>>> the relevant effects. >>>> >>>> Cheers >>>> Klaus >>>> >>>> >>>> Am Dienstag, 9. Oktober 2012 14:43:00 UTC-4 schrieb nonlinear: >>>>> >>>>> Correct, Judson. >>>>> >>>>> The book data is requested via this line in TraderAssistant.java: >>>>> socket.reqMktDepth(ticker, contract, 10); >>>>> The corresponding callback method in Trader.java: >>>>> updateMktDepth(); >>>>> This is where the price is determined as the midpoint between the >>>>> current best bid and current best ask. >>>>> >>>>> The volume data is requested via this line in TraderAssistant.java: >>>>> socket.reqMktData(ticker, contract, "", false); >>>>> The corresponding callback method in Trader.java: >>>>> tickSize() >>>>> Price data (such as the "last price") can also be handled here, but >>>>> JBT only uses the volume data here. >>>>> >>>>> The subscription to market depth and volume happens here: >>>>> if (!subscribedTickers.contains(**ticker)) { >>>>> subscribedTickers.add(ticker); >>>>> socket.reqContractDetails(**strategy.getContract().m_**conId, >>>>> strategy.getContract()); >>>>> eventReport.report(**JBookTrader.APP_NAME, "Requested >>>>> contract details for instrument " + instrument); >>>>> socket.reqMktDepth(ticker, contract, 10); >>>>> eventReport.report(**JBookTrader.APP_NAME, "Requested >>>>> book data for instrument " + instrument); >>>>> socket.reqMktData(ticker, contract, "", false); >>>>> eventReport.report(**JBookTrader.APP_NAME, "Requested >>>>> market data for instrument " + instrument); >>>>> } >>>>> >>>>> >>>>> What I suggested to Klaus is that we can detect the loss of market >>>>> data via the error() method, and resubscribe if this happens. Klaus, I'll >>>>> see if I can make that code change and I'll send it over to you to test. >>>>> Are you OK with it? >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> On Tue, Oct 9, 2012 at 1:58 PM, Judson Wilson <[email protected]>wrote: >>>>> >>>>>> I thought price was not coming from the same stream as volume, i >>>>>> thought it comes from the book stream (isn't it the midpoint of the >>>>>> best bid/ask on the book?) >>>>>> >>>>>> >>>>>> On Tue, Oct 9, 2012 at 8:59 AM, Klaus <[email protected]> wrote: >>>>>> > Dear nonlinear, >>>>>> > >>>>>> > what irritates me most is that it partially recovers: some data >>>>>> (for the >>>>>> > same instrument) is reconnected and correctly logged (volume = at >>>>>> least I >>>>>> > assume it is correct, it is at least reasonable), while price is not >>>>>> > correctly logged. To me this looks more like a SW issue, as both >>>>>> types of >>>>>> > data are coming from the same stream. >>>>>> > Regarding network connection, I assume there is probably few better >>>>>> > connections than AWS. It certainly is way ahead of private >>>>>> > connections I dealt with. It might also be interesting to note >>>>>> that I do >>>>>> > not use TWS, but IB Gateway. >>>>>> > >>>>>> > Cheers >>>>>> > Klaus >>>>>> > >>>>>> > >>>>>> > Am Montag, 8. Oktober 2012 10:44:27 UTC-4 schrieb nonlinear: >>>>>> >> >>>>>> >> Thanks for reporting the problem, Klaus. With respect to the >>>>>> midnight >>>>>> >> market data interruptions, this has always been the case, as IB >>>>>> servers >>>>>> >> restart at that time. The expected behavior is that once the >>>>>> restart is >>>>>> >> completed, the market data resumes and TWS and JBT would resume >>>>>> receiving >>>>>> >> the data stream without the need to restart. However, I noticed >>>>>> that >>>>>> >> occasionally this fails. My remedy is restart both TWS and JBT >>>>>> every day in >>>>>> >> the morning before regular trading starts. With respect to the >>>>>> interruptions >>>>>> >> during the other times of the day, I belive it all has to do with >>>>>> the >>>>>> >> quality of your network connection. The better quality, the less >>>>>> frequent is >>>>>> >> the loss. In my experience, both JBT and TWS recover from these >>>>>> >> interruptions (except for the occasional non-recoverable midnight >>>>>> resets). >>>>>> >> The typical pattern in the log is the message "market data lost", >>>>>> followed >>>>>> >> by in a minute or two by message "market data restored". Now, with >>>>>> respect >>>>>> >> to the stock data that you see in the TWS log, it's probably your >>>>>> standard >>>>>> >> subscription. I'd suggest visiting your "market subscriptions" page >>>>>> >> associated with your account on the IB we site. >>>>>> >> >>>>>> >> >>>>>> >> >>>>>> >> >>>>>> >> On Mon, Oct 8, 2012 at 9:45 AM, Klaus <[email protected]> >>>>>> wrote: >>>>>> >>> >>>>>> >>> Just had the same again. This time at 3:25 Monday morning. >>>>>> >>> Registered a temporary connection loss in the event report. Book >>>>>> and >>>>>> >>> price data did not change for any of the collected data from >>>>>> thereon, but >>>>>> >>> volume does. >>>>>> >>> >>>>>> >>> In addition, I found the following messages in the terminal >>>>>> window. >>>>>> >>> Perhaps somehow can make sense of all this? >>>>>> >>> (I include one message before and after for context) >>>>>> >>> --- >>>>>> >>> 03:08:11:547 JTS-CCPPing-269: Warning: not received heartbeat at >>>>>> 20904000 >>>>>> >>> elapsed:11000 >>>>>> >>> 03:25:30:728 JTS-PostAuthenticate-513: pre-logon message >>>>>> successfully >>>>>> >>> sent >>>>>> >>> 03:25:36:225 AWT-EventQueue-0: Warning: hot restart not equal to >>>>>> quick >>>>>> >>> restart >>>>>> >>> 03:25:36:226 AWT-EventQueue-0: Unsubscribe MD before routing table >>>>>> >>> rerequest >>>>>> >>> 03:25:36:227 AWT-EventQueue-0: Unsubscribe HMDS before updating >>>>>> routing >>>>>> >>> table >>>>>> >>> 03:25:36:227 AWT-EventQueue-0: Could not find route table entry >>>>>> for >>>>>> >>> path:exch=NEWS secType=STK currency=USD dataType=DayChart >>>>>> >>> 03:25:36:228 AWT-EventQueue-0: No data of type DayChart is >>>>>> available for >>>>>> >>> the exchange 'NEWS' and the security type 'Stock' >>>>>> >>> 03:25:36:228 AWT-EventQueue-0: Could not find route table entry >>>>>> for >>>>>> >>> path:exch=RTRSFND secType=STK currency=USD dataType=DayChart >>>>>> >>> 03:25:36:228 AWT-EventQueue-0: No data of type DayChart is >>>>>> available for >>>>>> >>> the exchange 'RTRSFND' and the security type 'Stock' >>>>>> >>> 03:25:36:228 AWT-EventQueue-0: Could not find route table entry >>>>>> for >>>>>> >>> path:exch=RTRSFND secType=STK currency=USD dataType=DayChart >>>>>> >>> 03:25:36:228 AWT-EventQueue-0: No data of type DayChart is >>>>>> available for >>>>>> >>> the exchange 'RTRSFND' and the security type 'Stock' >>>>>> >>> 03:25:36:229 AWT-EventQueue-0: Reconnected! >>>>>> >>> 03:25:37:151 JTS-ForwardRouting-HISTORICAL_**DATA-532: >>>>>> >>> handleCompatibleRouting HISTORICAL_DATA compmode=true ready=false >>>>>> >>> farm=ushmds >>>>>> >>> 03:25:37:939 JTS-ForwardRouting-MARKET_**DATA-536: >>>>>> handleCompatibleRouting >>>>>> >>> MARKET_DATA compmode=true ready=false farm=usfarm >>>>>> >>> 03:26:36:399 JTS-Fuse-sd-533: Reset transferrable socket:ushmds >>>>>> >>> gw1.ibllc.com:4000 >>>>>> >>> 03:26:37:187 JTS-Fuse-sd-537: Reset transferrable socket:usfarm >>>>>> >>> gw1.ibllc.com:4000 >>>>>> >>> 05:20:18:052 JTS-CCPPing-516: Warning: not received heartbeat at >>>>>> 28813000 >>>>>> >>> elapsed:11000 >>>>>> >>> ------ >>>>>> >>> >>>>>> >>> Data that is collected is only for the three instruments: EUR, >>>>>> NQ, ES >>>>>> >>> (thus I am puzzled also by the stock request above) >>>>>> >>> >>>>>> >>> Cheers >>>>>> >>> Klaus >>>>>> >>> >>>>>> >>> >>>>>> >>> Am Montag, 1. Oktober 2012 10:42:52 UTC-4 schrieb Klaus: >>>>>> >>>> >>>>>> >>>> Hi, >>>>>> >>>> >>>>>> >>>> for quite some time, I am using JBT to gather data. For this >>>>>> purpose, it >>>>>> >>>> is running on a AWS micro-instance. >>>>>> >>>> Once in a while I found it stuck (actually , but was not sure >>>>>> where it >>>>>> >>>> was coming from). Usually the happened also after a >>>>>> >>>> full week of data. >>>>>> >>>> >>>>>> >>>> However, today it happened again and I found some strange things >>>>>> (I >>>>>> >>>> start recording usually some time Sunday evening and let it run >>>>>> till >>>>>> >>>> Friday). Perhaps someone has an idea where it might come from. >>>>>> The fact is, >>>>>> >>>> it seems not data itself gets stuck only the price information >>>>>> is kept the >>>>>> >>>> same: the volume does continue to change. >>>>>> >>>> >>>>>> >>>> This is how the end of the recording looks like: then there is a >>>>>> 30sec >>>>>> >>>> gap where I restarted JBT. >>>>>> >>>> >>>>>> >>>> 100112,101425,-11.6,1.28335,14 >>>>>> >>>> >>>>>> >>>> 100112,101426,-11.6,1.28335,6 >>>>>> >>>> >>>>>> >>>> 100112,101427,-11.6,1.28335,0 >>>>>> >>>> >>>>>> >>>> 100112,101458,9.95,1.29335,0 >>>>>> >>>> >>>>>> >>>> 100112,101459,9.16,1.29335,2 >>>>>> >>>> >>>>>> >>>> 100112,101500,8.24,1.29335,1 >>>>>> >>>> >>>>>> >>>> The gap is not due to a change in the market, but because the >>>>>> data >>>>>> >>>> before was no longer correct (btw, this is the EUR) >>>>>> >>>> >>>>>> >>>> >>>>>> >>>> This is the part of the data where it seemed to happen: >>>>>> >>>> >>>>>> >>>> 100112,001556,-9.99,1.2833,0 >>>>>> >>>> >>>>>> >>>> 100112,001557,-4.27,1.28325,0 >>>>>> >>>> >>>>>> >>>> 100112,001558,-2.62,1.28325,0 >>>>>> >>>> >>>>>> >>>> 100112,001559,-11.47,1.28335,**16 >>>>>> >>>> >>>>>> >>>> 100112,001600,-9.75,1.2833,1 >>>>>> >>>> >>>>>> >>>> 100112,001601,-10.32,1.2833,0 >>>>>> >>>> >>>>>> >>>> 100112,001602,-10.26,1.2833,0 >>>>>> >>>> >>>>>> >>>> 100112,001603,-13.42,1.28335,1 >>>>>> >>>> >>>>>> >>>> 100112,001604,-13.61,1.28335,0 >>>>>> >>>> >>>>>> >>>> 100112,001605,-12.64,1.28335,0 >>>>>> >>>> >>>>>> >>>> 100112,001606,-12.04,1.28335,0 >>>>>> >>>> >>>>>> >>>> 100112,001607,-11.83,1.28335,0 >>>>>> >>>> >>>>>> >>>> 100112,001608,-11.71,1.28335,0 >>>>>> >>>> >>>>>> >>>> 100112,001609,-11.71,1.28335,0 >>>>>> >>>> >>>>>> >>>> 100112,001610,-11.71,1.28335,0 >>>>>> >>>> >>>>>> >>>> 100112,001611,-11.71,1.28335,0 >>>>>> >>>> >>>>>> >>>> 100112,001612,-11.93,1.28335,0 >>>>>> >>>> >>>>>> >>>> 100112,001613,-11.98,1.28335,0 >>>>>> >>>> >>>>>> >>>> 100112,001614,-11.04,1.28335,7 >>>>>> >>>> >>>>>> >>>> the price keeped changing until 001602, then was stuck - as far >>>>>> as I >>>>>> >>>> could see in the file all the way until I restarted. >>>>>> >>>> >>>>>> >>>> >>>>>> >>>> The following is an excerpt from the Eventreport file - it >>>>>> contains the >>>>>> >>>> end before the restart (the were no further events recorded) >>>>>> >>>> >>>>>> >>>> 09/30/12 23:59:05.196 IB API 2103: Market data farm connection is >>>>>> >>>> broken:usfuture >>>>>> >>>> >>>>>> >>>> 09/30/12 23:59:21.450 IB API 2104: Market data farm connection is >>>>>> >>>> OK:usfuture >>>>>> >>>> >>>>>> >>>> 10/01/12 00:14:04.896 IB API 317: Market depth data has been >>>>>> RESET. >>>>>> >>>> Please empty deep book contents before applying any new entries. >>>>>> (for id 3) >>>>>> >>>> >>>>>> >>>> 10/01/12 00:14:04.896 My JBookTrader Market data for book 3 has >>>>>> been >>>>>> >>>> reset. >>>>>> >>>> >>>>>> >>>> 10/01/12 00:14:04.896 IB API 317: Market depth data has been >>>>>> RESET. >>>>>> >>>> Please empty deep book contents before applying any new entries. >>>>>> (for id 2) >>>>>> >>>> >>>>>> >>>> 10/01/12 00:14:04.896 My JBookTrader Market data for book 2 has >>>>>> been >>>>>> >>>> reset. >>>>>> >>>> >>>>>> >>>> 10/01/12 00:14:04.896 IB API 317: Market depth data has been >>>>>> RESET. >>>>>> >>>> Please empty deep book contents before applying any new entries. >>>>>> (for id 1) >>>>>> >>>> >>>>>> >>>> 10/01/12 00:14:04.896 My JBookTrader Market data for book 1 has >>>>>> been >>>>>> >>>> reset. >>>>>> >>>> >>>>>> >>>> 10/01/12 00:14:04.896 IB API 2103: Market data farm connection is >>>>>> >>>> broken:usfuture >>>>>> >>>> >>>>>> >>>> 10/01/12 00:14:42.166 IB API 2104: Market data farm connection is >>>>>> >>>> OK:usfuture >>>>>> >>>> >>>>>> >>>> 10/01/12 00:15:48.860 IB API 1100: Connectivity between IB and >>>>>> Trader >>>>>> >>>> Workstation has been lost. >>>>>> >>>> >>>>>> >>>> 10/01/12 00:16:16.630 IB API 1102: Connectivity between IB and >>>>>> Trader >>>>>> >>>> Workstation has been restored - data maintained. >>>>>> >>>> >>>>>> >>>> >>>>>> >>>> So, it looks like connectivity was lost and restored, but from >>>>>> this >>>>>> >>>> point forward no price updates were recorded.. >>>>>> >>>> >>>>>> >>>> Any idea what is going on, would be helpful. >>>>>> >>>> >>>>>> >>>> Cheers >>>>>> >>>> Klaus >>>>>> >>>> >>>>>> >>>> >>>>>> >>> -- >>>>>> >>> You received this message because you are subscribed to the >>>>>> Google Groups >>>>>> >>> "JBookTrader" group. >>>>>> >>> To view this discussion on the web visit >>>>>> >>> https://groups.google.com/d/**msg/jbooktrader/-/O4d4rFkJ3F0J<https://groups.google.com/d/msg/jbooktrader/-/O4d4rFkJ3F0J> >>>>>> **. >>>>>> >>> >>>>>> >>> To post to this group, send email to [email protected]. >>>>>> >>> To unsubscribe from this group, send email to >>>>>> >>> jbooktrader...@googlegroups.**com. >>>>>> >>> >>>>>> >>> For more options, visit this group at >>>>>> >>> http://groups.google.com/**group/jbooktrader?hl=en<http://groups.google.com/group/jbooktrader?hl=en> >>>>>> . >>>>>> >> >>>>>> >> >>>>>> >> >>>>>> > -- >>>>>> > You received this message because you are subscribed to the Google >>>>>> Groups >>>>>> > "JBookTrader" group. >>>>>> > To view this discussion on the web visit >>>>>> > https://groups.google.com/d/**msg/jbooktrader/-/tEYk3_95L5EJ<https://groups.google.com/d/msg/jbooktrader/-/tEYk3_95L5EJ> >>>>>> **. >>>>>> > >>>>>> > To post to this group, send email to [email protected]. >>>>>> > To unsubscribe from this group, send email to >>>>>> > jbooktrader...@googlegroups.**com. >>>>>> > For more options, visit this group at >>>>>> > http://groups.google.com/**group/jbooktrader?hl=en<http://groups.google.com/group/jbooktrader?hl=en> >>>>>> . >>>>>> >>>>>> -- >>>>>> You received this message because you are subscribed to the Google >>>>>> Groups "JBookTrader" group. >>>>>> To post to this group, send email to [email protected]. >>>>>> To unsubscribe from this group, send email to >>>>>> jbooktrader...@googlegroups.**com. >>>>>> For more options, visit this group at http://groups.google.com/** >>>>>> group/jbooktrader?hl=en<http://groups.google.com/group/jbooktrader?hl=en> >>>>>> . >>>>>> >>>>>> >>>>> -- >> You received this message because you are subscribed to the Google Groups >> "JBookTrader" group. >> To view this discussion on the web visit >> https://groups.google.com/d/msg/jbooktrader/-/VQInXSoseSUJ. >> >> To post to this group, send email to [email protected]<javascript:> >> . >> To unsubscribe from this group, send email to >> [email protected] <javascript:>. >> For more options, visit this group at >> http://groups.google.com/group/jbooktrader?hl=en. >> > > -- You received this message because you are subscribed to the Google Groups "JBookTrader" group. To view this discussion on the web visit https://groups.google.com/d/msg/jbooktrader/-/qcF5KIX3zu8J. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/jbooktrader?hl=en.
