Hi Dan, Do you have any test client that could reproduce this quickly? That may help us a lot for figuring out what is wrong. Thanks. :-)
On 11/29/06, Daniel Daley <[EMAIL PROTECTED]> wrote:
Now that the new FLVReader code is in place I am able to handle many more concurrent users but continue having one strange issue. After about 4600 connections have occurred the heap suddenly grows to the point that a full garbage collect hardly drops it (as seen in the log below). A short bit after this occurs the server will not longer stream but will still accept connections. My only thought as to what it might be is that I've had trouble with the job scheduling service hanging suddenly after a period of running. I had my own scheduled jobs during that time so I think it accelerated the process but I know that the stop command is scheduled and perhaps if the scheduler is locking it would cause this issue? I wouldn't think it would happen so quickly though if that was the case, as everything runs perfectly beforehand with the heap being garbage collected down to around 60 or 70MB. Any ideas? Thanks, --Dan-- 2006-11-28 18:45:39.973964500 [INFO] 16657086 pool-1-thread-6: ( streamSubscriberClose ) 2006-11-28 18:45:39 Client: 4518 streamSubscriberClose 2006-11-28 18:45:39.985626500 [INFO] 16657098 pool-1-thread-1: ( streamPlaylistVODItemSeek ) 2006-11-28 18:45:39 Client: 4549 streamPlaylistVODItemSeek 2006-11-28 18:45:40.234276500 [Full GC 499584K->486582K(511808K), 3.9512268 secs] 2006-11-28 18:45:44.386985500 [Full GC 499584K->490826K(511808K), 4.0028856 secs] 2006-11-28 18:45:48.544464500 [Full GC 499584K->487926K(511808K), 5.3713444 secs] 2006-11-28 18:45:54.095456500 [Full GC 499583K->493405K(511808K), 4.0693999 secs] 2006-11-28 18:45:58.261174500 [Full GC[Unloading class sun.reflect.GeneratedConstructorAccessor4] 2006-11-28 18:46:02.380410500 499583K->495486K(511808K), 4.1192612 secs] 2006-11-28 18:46:02.454677500 [Full GC 499583K->496817K(511808K), 4.0788536 secs] 2006-11-28 18:46:06.576321500 [Full GC[Unloading class sun.reflect.GeneratedMethodAccessor24] 2006-11-28 18:46:08.452441500 [Unloading class sun.reflect.GeneratedMethodAccessor2] 2006-11-28 18:46:08.452444500 [Unloading class sun.reflect.GeneratedMethodAccessor15] 2006-11-28 18:46:08.452485500 [Unloading class sun.reflect.GeneratedMethodAccessor14] 2006-11-28 18:46:08.452739500 [Unloading class sun.reflect.GeneratedMethodAccessor12] 2006-11-28 18:46:08.452801500 [Unloading class sun.reflect.GeneratedMethodAccessor20] 2006-11-28 18:46:08.452805500 [Unloading class sun.reflect.GeneratedMethodAccessor3] 2006-11-28 18:46:08.452820500 [Unloading class sun.reflect.GeneratedMethodAccessor6] 2006-11-28 18:46:08.452836500 [Unloading class sun.reflect.GeneratedMethodAccessor5] 2006-11-28 18:46:08.452850500 [Unloading class sun.reflect.GeneratedMethodAccessor22] 2006-11-28 18:46:08.452890500 [Unloading class sun.reflect.GeneratedMethodAccessor11] 2006-11-28 18:46:08.453125500 [Unloading class sun.reflect.GeneratedMethodAccessor25] 2006-11-28 18:46:08.453197500 [Unloading class sun.reflect.GeneratedMethodAccessor17] 2006-11-28 18:46:08.453232500 [Unloading class sun.reflect.GeneratedMethodAccessor13] 2006-11-28 18:46:08.453314500 [Unloading class sun.reflect.GeneratedMethodAccessor4] 2006-11-28 18:46:12.096960500 499573K->495985K(511808K), 5.5206938 secs] 2006-11-28 18:46:12.154356500 [Full GC 499584K->498842K(511808K), 4.2219567 secs] 2006-11-28 18:46:16.387928500 [Full GC 499584K->499102K(511808K), 4.1133314 secs] 2006-11-28 18:46:20.524919500 [Full GC[Unloading class sun.reflect.GeneratedMethodAccessor1] 2006-11-28 18:46:24.684359500 499584K->499222K(511808K), 4.1594666 secs] 2006-11-28 18:46:24.706426500 [Full GC 499584K->497661K(511808K), 5.6004981 secs] 2006-11-28 18:46:30.357607500 [Full GC 499583K->499583K(511808K), 4.1456096 secs] 2006-11-28 18:46:34.503350500 [Full GC 499583K->498283K(511808K), 5.5813694 secs] 2006-11-28 18:46:40.125999500 [Full GC 499579K->499579K(511808K), 4.1222203 secs] 2006-11-28 18:46:44.248342500 [Full GC 499579K->498764K(511808K), 5.5525750 secs] 2006-11-28 18:46:49.836342500 [Full GC 499583K->499583K(511808K), 4.2713130 secs] 2006-11-28 18:46:54.107789500 [Full GC 499583K->499000K(511808K), 5.4783996 secs] 2006-11-28 18:46:59.616781500 [Full GC 499571K->499571K(511808K), 4.3601245 secs] 2006-11-28 18:47:03.977038500 [Full GC 499571K->499231K(511808K), 5.6121831 secs] 2006-11-28 18:47:09.598690500 [Full GC 499584K->499584K(511808K), 4.1334214 secs] 2006-11-28 18:47:13.732243500 [Full GC 499584K->499344K(511808K), 5.4238636 secs] 2006-11-28 18:47:19.162038500 [Full GC 499583K->499583K(511808K), 4.1761207 secs] 2006-11-28 18:47:23.338292500 [Full GC 499583K->499435K(511808K), 4.0381495 secs] 2006-11-28 18:47:27.397388500 [Full GC 499578K->499578K(511808K), 4.1922121 secs] 2006-11-28 18:47:31.589735500 [Full GC 499578K->499504K(511808K), 5.5022513 secs] 2006-11-28 18:47:37.095978500 [Full GC 499583K->499583K(511808K), 4.1805690 secs] 2006-11-28 18:47:41.276684500 [Full GC 499583K->499503K(511808K), 4.0814699 secs] 2006-11-28 18:47:45.363748500 [Full GC 499583K->499583K(511808K), 4.1720694 secs] 2006-11-28 18:47:49.535943500 [Full GC 499583K->499544K(511808K), 4.1699713 secs] 2006-11-28 18:47:53.755950500 [Full GC 499583K->499583K(511808K), 4.1593166 secs] 2006-11-28 18:47:57.917455500 [Full GC 499583K->499552K(511808K), 5.5514382 secs] 2006-11-28 18:48:03.472669500 [Full GC 499584K->499584K(511808K), 4.1725040 secs] 2006-11-28 18:48:07.659044500 [Full GC 499584K->499584K(511808K), 4.1750055 secs] 2006-11-28 18:48:11.847801500 [Full GC 499584K->499584K(511808K), 4.1131328 secs] 2006-11-28 18:48:15.979586500 [Full GC 499584K->496903K(511808K), 5.6763932 secs] 2006-11-28 18:48:21.657594500 Exception in thread "Callback Thread" [ERROR] 16818770 pool-1-thread-7: ( org.red5.server.service.ServiceInvoker.error ) Error executing call: Service: null Method: deleteStream Num Params: 10: 1 2006-11-28 18:48:21.658716500 java.lang.OutOfMemoryError: Java heap space[ERROR] 16818779 pool-1-thread-7: ( org.red5.server.service.ServiceInvoker.error ) Service invocation error 2006-11-28 18:48:21.667577500 java.lang.reflect.InvocationTargetException 2006-11-28 18:48:21.667580500 at sun.reflect.GeneratedMethodAccessor21.invoke(Unknown Source) 2006-11-28 18:48:21.667582500 at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:25) 2006-11-28 18:48:21.667585500 at java.lang.reflect.Method.invoke (Method.java:585) 2006-11-28 18:48:21.667731500 at org.red5.server.service.ServiceInvoker.invoke(ServiceInvoker.java:161) 2006-11-28 18:48:21.667734500 at org.red5.server.net.rtmp.RTMPHandler.invokeCall(RTMPHandler.java:135) 2006-11-28 18:48:21.667736500 at org.red5.server.net.rtmp.RTMPHandler.onInvoke(RTMPHandler.java:276) 2006-11-28 18:48:21.667739500 at org.red5.server.net.rtmp.BaseRTMPHandler.messageReceived (BaseRTMPHandler.java:114) 2006-11-28 18:48:21.667741500 at org.red5.server.net.rtmp.RTMPMinaIoHandler.messageReceived (RTMPMinaIoHandler.java:78) 2006-11-28 18:48:21.667752500 at org.apache.mina.common.support.AbstractIoFilterChain$2.messageReceived (AbstractIoFilterChain.java:189) 2006-11-28 18:48:21.667756500 at org.apache.mina.common.support.AbstractIoFilterChain.callNextMessageRece ived(AbstractIoFilterChain.java:502) 2006-11-28 18:48:21.667759500 at org.apache.mina.common.support.AbstractIoFilterChain.access$1000 (AbstractIoFilterChain.java:52) 2006-11-28 18:48:21.667775500 at org.apache.mina.common.support.AbstractIoFilterChain$EntryImpl $1.messageReceived(AbstractIoFilterChain.java:777) 2006-11-28 18:48:21.667778500 at org.red5.io.filter.ExecutorFilter.processEvent(ExecutorFilter.java:231) 2006-11-28 18:48:21.667781500 at org.red5.io.filter.ExecutorFilter $ProcessEventsRunnable.run(ExecutorFilter.java:279) 2006-11-28 18:48:21.667784500 at java.util.concurrent.ThreadPoolExecutor$Worker.runTask (ThreadPoolExecutor.java:650) 2006-11-28 18:48:21.667792500 at java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:675) 2006-11-28 18:48:21.667795500 at java.lang.Thread.run(Thread.java:595) 2006-11-28 18:48:21.667797500 Caused by: java.lang.OutOfMemoryError: Java heap space 2006-11-28 18:48:21.668241500 2006-11-28 18:48:21.669689500 Exception in thread "btpool0-1" java.lang.OutOfMemoryError: Java heap space 2006-11-28 18:48:21.671628500 Exception in thread "FlowControlService" java.lang.OutOfMemoryError: Java heap space _______________________________________________ Red5 mailing list [email protected] http://osflash.org/mailman/listinfo/red5_osflash.org
-- I cannot tell why this heart languishes in silence. It is for small needs it never asks, or knows or remembers. -- Tagore Best Regards Steven Gong
_______________________________________________ Red5 mailing list [email protected] http://osflash.org/mailman/listinfo/red5_osflash.org
