Dear nonlinear, what irritates me most is that it partially recovers: some data (for the same instrument) is reconnected and correctly logged (volume = at least I assume it is correct, it is at least reasonable), while price is not correctly logged. To me this looks more like a SW issue, as both types of data are coming from the same stream. Regarding network connection, I assume there is probably few better connections than AWS. It certainly is way ahead of private connections I dealt with. It might also be interesting to note that I do not use TWS, but IB Gateway.
Cheers Klaus Am Montag, 8. Oktober 2012 10:44:27 UTC-4 schrieb nonlinear: > > Thanks for reporting the problem, Klaus. With respect to the midnight > market data interruptions, this has always been the case, as IB servers > restart at that time. The expected behavior is that once the restart is > completed, the market data resumes and TWS and JBT would resume receiving > the data stream without the need to restart. However, I noticed that > occasionally this fails. My remedy is restart both TWS and JBT every day in > the morning before regular trading starts. With respect to the > interruptions during the other times of the day, I belive it all has to do > with the quality of your network connection. The better quality, the less > frequent is the loss. In my experience, both JBT and TWS recover from these > interruptions (except for the occasional non-recoverable midnight resets). > The typical pattern in the log is the message "market data lost", followed > by in a minute or two by message "market data restored". Now, with respect > to the stock data that you see in the TWS log, it's probably your standard > subscription. I'd suggest visiting your "market subscriptions" page > associated with your account on the IB we site. > > > > > On Mon, Oct 8, 2012 at 9:45 AM, Klaus <[email protected] > <javascript:>>wrote: > >> Just had the same again. This time at 3:25 Monday morning. >> Registered a temporary connection loss in the event report. Book and >> price data did not change for any of the collected data from thereon, but >> volume does. >> >> In addition, I found the following messages in the terminal window. >> Perhaps somehow can make sense of all this? >> (I include one message before and after for context) >> --- >> 03:08:11:547 JTS-CCPPing-269: Warning: not received heartbeat at 20904000 >> elapsed:11000 >> 03:25:30:728 JTS-PostAuthenticate-513: pre-logon message successfully sent >> 03:25:36:225 AWT-EventQueue-0: Warning: hot restart not equal to quick >> restart >> 03:25:36:226 AWT-EventQueue-0: Unsubscribe MD before routing table >> rerequest >> 03:25:36:227 AWT-EventQueue-0: Unsubscribe HMDS before updating routing >> table >> 03:25:36:227 AWT-EventQueue-0: Could not find route table entry for >> path:exch=NEWS secType=STK currency=USD dataType=DayChart >> 03:25:36:228 AWT-EventQueue-0: No data of type DayChart is available for >> the exchange 'NEWS' and the security type 'Stock' >> 03:25:36:228 AWT-EventQueue-0: Could not find route table entry for >> path:exch=RTRSFND secType=STK currency=USD dataType=DayChart >> 03:25:36:228 AWT-EventQueue-0: No data of type DayChart is available for >> the exchange 'RTRSFND' and the security type 'Stock' >> 03:25:36:228 AWT-EventQueue-0: Could not find route table entry for >> path:exch=RTRSFND secType=STK currency=USD dataType=DayChart >> 03:25:36:228 AWT-EventQueue-0: No data of type DayChart is available for >> the exchange 'RTRSFND' and the security type 'Stock' >> 03:25:36:229 AWT-EventQueue-0: Reconnected! >> 03:25:37:151 JTS-ForwardRouting-HISTORICAL_DATA-532: >> handleCompatibleRouting HISTORICAL_DATA compmode=true ready=false >> farm=ushmds >> 03:25:37:939 JTS-ForwardRouting-MARKET_DATA-536: handleCompatibleRouting >> MARKET_DATA compmode=true ready=false farm=usfarm >> 03:26:36:399 JTS-Fuse-sd-533: Reset transferrable socket:ushmds >> gw1.ibllc.com:4000 >> 03:26:37:187 JTS-Fuse-sd-537: Reset transferrable socket:usfarm >> gw1.ibllc.com:4000 >> 05:20:18:052 JTS-CCPPing-516: Warning: not received heartbeat at 28813000 >> elapsed:11000 >> ------ >> >> Data that is collected is only for the three instruments: EUR, NQ, ES >> (thus I am puzzled also by the stock request above) >> >> Cheers >> Klaus >> >> >> Am Montag, 1. Oktober 2012 10:42:52 UTC-4 schrieb Klaus: >> >>> Hi, >>> >>> for quite some time, I am using JBT to gather data. For this purpose, it >>> is running on a AWS micro-instance. >>> Once in a while I found it stuck (actually , but was not sure where it >>> was coming from). Usually the happened also after a >>> full week of data. >>> >>> However, today it happened again and I found some strange things (I >>> start recording usually some time Sunday evening and let it run till >>> Friday). Perhaps someone has an idea where it might come from. The fact is, >>> it seems not data itself gets stuck only the price information is kept the >>> same: the volume does continue to change. >>> >>> This is how the end of the recording looks like: then there is a 30sec >>> gap where I restarted JBT. >>> >>> 100112,101425,-11.6,1.28335,14 >>> >>> 100112,101426,-11.6,1.28335,6 >>> >>> 100112,101427,-11.6,1.28335,0 >>> >>> 100112,101458,9.95,1.29335,0 >>> >>> 100112,101459,9.16,1.29335,2 >>> >>> 100112,101500,8.24,1.29335,1 >>> >>> The gap is not due to a change in the market, but because the data >>> before was no longer correct (btw, this is the EUR) >>> >>> >>> This is the part of the data where it seemed to happen: >>> >>> 100112,001556,-9.99,1.2833,0 >>> >>> 100112,001557,-4.27,1.28325,0 >>> >>> 100112,001558,-2.62,1.28325,0 >>> >>> 100112,001559,-11.47,1.28335,**16 >>> >>> 100112,001600,-9.75,1.2833,1 >>> >>> 100112,001601,-10.32,1.2833,0 >>> >>> 100112,001602,-10.26,1.2833,0 >>> >>> 100112,001603,-13.42,1.28335,1 >>> >>> 100112,001604,-13.61,1.28335,0 >>> >>> 100112,001605,-12.64,1.28335,0 >>> >>> 100112,001606,-12.04,1.28335,0 >>> >>> 100112,001607,-11.83,1.28335,0 >>> >>> 100112,001608,-11.71,1.28335,0 >>> >>> 100112,001609,-11.71,1.28335,0 >>> >>> 100112,001610,-11.71,1.28335,0 >>> >>> 100112,001611,-11.71,1.28335,0 >>> >>> 100112,001612,-11.93,1.28335,0 >>> >>> 100112,001613,-11.98,1.28335,0 >>> >>> 100112,001614,-11.04,1.28335,7 >>> >>> the price keeped changing until 001602, then was stuck - as far as I >>> could see in the file all the way until I restarted. >>> >>> >>> The following is an excerpt from the Eventreport file - it contains the >>> end before the restart (the were no further events recorded) >>> >>> 09/30/12 23:59:05.196 IB API 2103: Market data farm connection is >>> broken:usfuture >>> >>> 09/30/12 23:59:21.450 IB API 2104: Market data farm connection is >>> OK:usfuture >>> >>> 10/01/12 00:14:04.896 IB API 317: Market depth data has been RESET. >>> Please empty deep book contents before applying any new entries. (for id 3) >>> >>> 10/01/12 00:14:04.896 My JBookTrader Market data for book 3 has been >>> reset. >>> >>> 10/01/12 00:14:04.896 IB API 317: Market depth data has been RESET. >>> Please empty deep book contents before applying any new entries. (for id 2) >>> >>> 10/01/12 00:14:04.896 My JBookTrader Market data for book 2 has been >>> reset. >>> >>> 10/01/12 00:14:04.896 IB API 317: Market depth data has been RESET. >>> Please empty deep book contents before applying any new entries. (for id 1) >>> >>> 10/01/12 00:14:04.896 My JBookTrader Market data for book 1 has been >>> reset. >>> >>> 10/01/12 00:14:04.896 IB API 2103: Market data farm connection is >>> broken:usfuture >>> >>> 10/01/12 00:14:42.166 IB API 2104: Market data farm connection is >>> OK:usfuture >>> >>> 10/01/12 00:15:48.860 IB API 1100: Connectivity between IB and Trader >>> Workstation has been lost. >>> >>> 10/01/12 00:16:16.630 IB API 1102: Connectivity between IB and Trader >>> Workstation has been restored - data maintained. >>> >>> So, it looks like connectivity was lost and restored, but from this >>> point forward no price updates were recorded.. >>> >>> Any idea what is going on, would be helpful. >>> >>> Cheers >>> Klaus >>> >>> >>> -- >> You received this message because you are subscribed to the Google Groups >> "JBookTrader" group. >> To view this discussion on the web visit >> https://groups.google.com/d/msg/jbooktrader/-/O4d4rFkJ3F0J. >> >> To post to this group, send email to [email protected]<javascript:> >> . >> To unsubscribe from this group, send email to >> [email protected] <javascript:>. >> For more options, visit this group at >> http://groups.google.com/group/jbooktrader?hl=en. >> > > > -- You received this message because you are subscribed to the Google Groups "JBookTrader" group. To view this discussion on the web visit https://groups.google.com/d/msg/jbooktrader/-/tEYk3_95L5EJ. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/jbooktrader?hl=en.
