I'm running JBT on an AWS Small instance and tried switching to a Micro instance over the weekend. I use IB Gateway instead of TWS (it doesn't shutdown itself and uses less memory). After starting IB Gateway on the micro instance it used all the memory. I started another IB Gateway instance (my JBT sends orders to two different IB accounts). The result was, that while Ubuntu was freeing the memory for the second instance, the first one was shutdown by the OS.
I then started only one IB Gateway and JBT and monitored the memory. Over the weekend without any market data processing the memory was always 99% full. I didn't feel comfortable running JBT in such a low memory environment so I switched back to a small instance. What I'm trying to say is switch to a small instance for a week and see if that solves your problems. On Monday, October 8, 2012 4:44:27 PM UTC+2, nonlinear wrote: > > Thanks for reporting the problem, Klaus. With respect to the midnight > market data interruptions, this has always been the case, as IB servers > restart at that time. The expected behavior is that once the restart is > completed, the market data resumes and TWS and JBT would resume receiving > the data stream without the need to restart. However, I noticed that > occasionally this fails. My remedy is restart both TWS and JBT every day in > the morning before regular trading starts. With respect to the > interruptions during the other times of the day, I belive it all has to do > with the quality of your network connection. The better quality, the less > frequent is the loss. In my experience, both JBT and TWS recover from these > interruptions (except for the occasional non-recoverable midnight resets). > The typical pattern in the log is the message "market data lost", followed > by in a minute or two by message "market data restored". Now, with respect > to the stock data that you see in the TWS log, it's probably your standard > subscription. I'd suggest visiting your "market subscriptions" page > associated with your account on the IB we site. > > > > > On Mon, Oct 8, 2012 at 9:45 AM, Klaus <[email protected] > <javascript:>>wrote: > >> Just had the same again. This time at 3:25 Monday morning. >> Registered a temporary connection loss in the event report. Book and >> price data did not change for any of the collected data from thereon, but >> volume does. >> >> In addition, I found the following messages in the terminal window. >> Perhaps somehow can make sense of all this? >> (I include one message before and after for context) >> --- >> 03:08:11:547 JTS-CCPPing-269: Warning: not received heartbeat at 20904000 >> elapsed:11000 >> 03:25:30:728 JTS-PostAuthenticate-513: pre-logon message successfully sent >> 03:25:36:225 AWT-EventQueue-0: Warning: hot restart not equal to quick >> restart >> 03:25:36:226 AWT-EventQueue-0: Unsubscribe MD before routing table >> rerequest >> 03:25:36:227 AWT-EventQueue-0: Unsubscribe HMDS before updating routing >> table >> 03:25:36:227 AWT-EventQueue-0: Could not find route table entry for >> path:exch=NEWS secType=STK currency=USD dataType=DayChart >> 03:25:36:228 AWT-EventQueue-0: No data of type DayChart is available for >> the exchange 'NEWS' and the security type 'Stock' >> 03:25:36:228 AWT-EventQueue-0: Could not find route table entry for >> path:exch=RTRSFND secType=STK currency=USD dataType=DayChart >> 03:25:36:228 AWT-EventQueue-0: No data of type DayChart is available for >> the exchange 'RTRSFND' and the security type 'Stock' >> 03:25:36:228 AWT-EventQueue-0: Could not find route table entry for >> path:exch=RTRSFND secType=STK currency=USD dataType=DayChart >> 03:25:36:228 AWT-EventQueue-0: No data of type DayChart is available for >> the exchange 'RTRSFND' and the security type 'Stock' >> 03:25:36:229 AWT-EventQueue-0: Reconnected! >> 03:25:37:151 JTS-ForwardRouting-HISTORICAL_DATA-532: >> handleCompatibleRouting HISTORICAL_DATA compmode=true ready=false >> farm=ushmds >> 03:25:37:939 JTS-ForwardRouting-MARKET_DATA-536: handleCompatibleRouting >> MARKET_DATA compmode=true ready=false farm=usfarm >> 03:26:36:399 JTS-Fuse-sd-533: Reset transferrable socket:ushmds >> gw1.ibllc.com:4000 >> 03:26:37:187 JTS-Fuse-sd-537: Reset transferrable socket:usfarm >> gw1.ibllc.com:4000 >> 05:20:18:052 JTS-CCPPing-516: Warning: not received heartbeat at 28813000 >> elapsed:11000 >> ------ >> >> Data that is collected is only for the three instruments: EUR, NQ, ES >> (thus I am puzzled also by the stock request above) >> >> Cheers >> Klaus >> >> >> Am Montag, 1. Oktober 2012 10:42:52 UTC-4 schrieb Klaus: >> >>> Hi, >>> >>> for quite some time, I am using JBT to gather data. For this purpose, it >>> is running on a AWS micro-instance. >>> Once in a while I found it stuck (actually , but was not sure where it >>> was coming from). Usually the happened also after a >>> full week of data. >>> >>> However, today it happened again and I found some strange things (I >>> start recording usually some time Sunday evening and let it run till >>> Friday). Perhaps someone has an idea where it might come from. The fact is, >>> it seems not data itself gets stuck only the price information is kept the >>> same: the volume does continue to change. >>> >>> This is how the end of the recording looks like: then there is a 30sec >>> gap where I restarted JBT. >>> >>> 100112,101425,-11.6,1.28335,14 >>> >>> 100112,101426,-11.6,1.28335,6 >>> >>> 100112,101427,-11.6,1.28335,0 >>> >>> 100112,101458,9.95,1.29335,0 >>> >>> 100112,101459,9.16,1.29335,2 >>> >>> 100112,101500,8.24,1.29335,1 >>> >>> The gap is not due to a change in the market, but because the data >>> before was no longer correct (btw, this is the EUR) >>> >>> >>> This is the part of the data where it seemed to happen: >>> >>> 100112,001556,-9.99,1.2833,0 >>> >>> 100112,001557,-4.27,1.28325,0 >>> >>> 100112,001558,-2.62,1.28325,0 >>> >>> 100112,001559,-11.47,1.28335,**16 >>> >>> 100112,001600,-9.75,1.2833,1 >>> >>> 100112,001601,-10.32,1.2833,0 >>> >>> 100112,001602,-10.26,1.2833,0 >>> >>> 100112,001603,-13.42,1.28335,1 >>> >>> 100112,001604,-13.61,1.28335,0 >>> >>> 100112,001605,-12.64,1.28335,0 >>> >>> 100112,001606,-12.04,1.28335,0 >>> >>> 100112,001607,-11.83,1.28335,0 >>> >>> 100112,001608,-11.71,1.28335,0 >>> >>> 100112,001609,-11.71,1.28335,0 >>> >>> 100112,001610,-11.71,1.28335,0 >>> >>> 100112,001611,-11.71,1.28335,0 >>> >>> 100112,001612,-11.93,1.28335,0 >>> >>> 100112,001613,-11.98,1.28335,0 >>> >>> 100112,001614,-11.04,1.28335,7 >>> >>> the price keeped changing until 001602, then was stuck - as far as I >>> could see in the file all the way until I restarted. >>> >>> >>> The following is an excerpt from the Eventreport file - it contains the >>> end before the restart (the were no further events recorded) >>> >>> 09/30/12 23:59:05.196 IB API 2103: Market data farm connection is >>> broken:usfuture >>> >>> 09/30/12 23:59:21.450 IB API 2104: Market data farm connection is >>> OK:usfuture >>> >>> 10/01/12 00:14:04.896 IB API 317: Market depth data has been RESET. >>> Please empty deep book contents before applying any new entries. (for id 3) >>> >>> 10/01/12 00:14:04.896 My JBookTrader Market data for book 3 has been >>> reset. >>> >>> 10/01/12 00:14:04.896 IB API 317: Market depth data has been RESET. >>> Please empty deep book contents before applying any new entries. (for id 2) >>> >>> 10/01/12 00:14:04.896 My JBookTrader Market data for book 2 has been >>> reset. >>> >>> 10/01/12 00:14:04.896 IB API 317: Market depth data has been RESET. >>> Please empty deep book contents before applying any new entries. (for id 1) >>> >>> 10/01/12 00:14:04.896 My JBookTrader Market data for book 1 has been >>> reset. >>> >>> 10/01/12 00:14:04.896 IB API 2103: Market data farm connection is >>> broken:usfuture >>> >>> 10/01/12 00:14:42.166 IB API 2104: Market data farm connection is >>> OK:usfuture >>> >>> 10/01/12 00:15:48.860 IB API 1100: Connectivity between IB and Trader >>> Workstation has been lost. >>> >>> 10/01/12 00:16:16.630 IB API 1102: Connectivity between IB and Trader >>> Workstation has been restored - data maintained. >>> >>> So, it looks like connectivity was lost and restored, but from this >>> point forward no price updates were recorded.. >>> >>> Any idea what is going on, would be helpful. >>> >>> Cheers >>> Klaus >>> >>> >>> -- >> You received this message because you are subscribed to the Google Groups >> "JBookTrader" group. >> To view this discussion on the web visit >> https://groups.google.com/d/msg/jbooktrader/-/O4d4rFkJ3F0J. >> >> To post to this group, send email to [email protected]<javascript:> >> . >> To unsubscribe from this group, send email to >> [email protected] <javascript:>. >> For more options, visit this group at >> http://groups.google.com/group/jbooktrader?hl=en. >> > > -- You received this message because you are subscribed to the Google Groups "JBookTrader" group. To view this discussion on the web visit https://groups.google.com/d/msg/jbooktrader/-/14gZE_CaNPIJ. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/jbooktrader?hl=en.
