Hi Dan,
Do you have any test client that could reproduce this quickly?
That may help us a lot for figuring out what is wrong. Thanks. :-)
On 11/29/06, Daniel Daley <[EMAIL PROTECTED]> wrote:
Now that the new FLVReader code is in place I am able to handle many
more concurrent users but continue having one strange issue. After
about 4600 connections have occurred the heap suddenly grows to the
point that a full garbage collect hardly drops it (as seen in the log
below).
A short bit after this occurs the server will not longer stream but
will still accept connections. My only thought as to what it might be
is that I've had trouble with the job scheduling service hanging
suddenly after a period of running. I had my own scheduled jobs
during that time so I think it accelerated the process but I know
that the stop command is scheduled and perhaps if the scheduler is
locking it would cause this issue? I wouldn't think it would happen
so quickly though if that was the case, as everything runs perfectly
beforehand with the heap being garbage collected down to around 60 or
70MB. Any ideas?
Thanks,
--Dan--
2006-11-28 18:45: 39.973964500 [INFO] 16657086 pool-1-thread-6:
( streamSubscriberClose ) 2006-11-28 18:45:39 Client: 4518
streamSubscriberClose
2006-11-28 18:45:39.985626500 [INFO] 16657098 pool-1-thread-1:
( streamPlaylistVODItemSeek ) 2006-11-28 18:45:39 Client: 4549
streamPlaylistVODItemSeek
2006-11-28 18:45:40.234276500 [Full GC 499584K->486582K(511808K),
3.9512268 secs]
2006-11-28 18:45:44.386985500 [Full GC 499584K->490826K(511808K),
4.0028856 secs]
2006-11-28 18:45: 48.544464500 [Full GC 499584K->487926K(511808K),
5.3713444 secs]
2006-11-28 18:45: 54.095456500 [Full GC 499583K->493405K(511808K),
4.0693999 secs]
2006-11-28 18:45:58.261174500 [Full GC[Unloading class
sun.reflect.GeneratedConstructorAccessor4]
2006-11-28 18:46:02.380410500 499583K->495486K(511808K), 4.1192612
secs]
2006-11-28 18:46:02.454677500 [Full GC 499583K->496817K(511808K),
4.0788536 secs]
2006-11-28 18:46: 06.576321500 [Full GC[Unloading class
sun.reflect.GeneratedMethodAccessor24]
2006-11-28 18:46: 08.452441500 [Unloading class
sun.reflect.GeneratedMethodAccessor2]
2006-11-28 18:46:08.452444500 [Unloading class
sun.reflect.GeneratedMethodAccessor15]
2006-11-28 18:46:08.452485500 [Unloading class
sun.reflect.GeneratedMethodAccessor14 ]
2006-11-28 18:46:08.452739500 [Unloading class
sun.reflect.GeneratedMethodAccessor12 ]
2006-11-28 18:46:08.452801500 [Unloading class
sun.reflect.GeneratedMethodAccessor20]
2006-11-28 18:46:08.452805500 [Unloading class
sun.reflect.GeneratedMethodAccessor3]
2006-11-28 18:46:08.452820500 [Unloading class
sun.reflect.GeneratedMethodAccessor6]
2006-11-28 18:46:08.452836500 [Unloading class
sun.reflect.GeneratedMethodAccessor5 ]
2006-11-28 18:46:08.452850500 [Unloading class
sun.reflect.GeneratedMethodAccessor22 ]
2006-11-28 18:46:08.452890500 [Unloading class
sun.reflect.GeneratedMethodAccessor11]
2006-11-28 18:46:08.453125500 [Unloading class
sun.reflect.GeneratedMethodAccessor25]
2006-11-28 18:46:08.453197500 [Unloading class
sun.reflect.GeneratedMethodAccessor17]
2006-11-28 18:46:08.453232500 [Unloading class
sun.reflect.GeneratedMethodAccessor13 ]
2006-11-28 18:46:08.453314500 [Unloading class
sun.reflect.GeneratedMethodAccessor4 ]
2006-11-28 18:46:12.096960500 499573K->495985K(511808K), 5.5206938
secs]
2006-11-28 18:46:12.154356500 [Full GC 499584K->498842K(511808K),
4.2219567 secs]
2006-11-28 18:46:16.387928500 [Full GC 499584K->499102K(511808K),
4.1133314 secs]
2006-11-28 18:46:20.524919500 [Full GC[Unloading class
sun.reflect.GeneratedMethodAccessor1]
2006-11-28 18:46: 24.684359500 499584K->499222K(511808K), 4.1594666
secs]
2006-11-28 18:46: 24.706426500 [Full GC 499584K->497661K(511808K),
5.6004981 secs]
2006-11-28 18:46:30.357607500 [Full GC 499583K->499583K(511808K),
4.1456096 secs]
2006-11-28 18:46:34.503350500 [Full GC 499583K->498283K(511808K),
5.5813694 secs]
2006-11-28 18:46:40.125999500 [Full GC 499579K->499579K(511808K),
4.1222203 secs]
2006-11-28 18:46: 44.248342500 [Full GC 499579K->498764K(511808K),
5.5525750 secs]
2006-11-28 18:46: 49.836342500 [Full GC 499583K->499583K(511808K),
4.2713130 secs]
2006-11-28 18:46:54.107789500 [Full GC 499583K->499000K(511808K),
5.4783996 secs]
2006-11-28 18:46:59.616781500 [Full GC 499571K->499571K(511808K),
4.3601245 secs]
2006-11-28 18:47:03.977038500 [Full GC 499571K->499231K(511808K),
5.6121831 secs]
2006-11-28 18:47: 09.598690500 [Full GC 499584K->499584K(511808K),
4.1334214 secs]
2006-11-28 18:47: 13.732243500 [Full GC 499584K->499344K(511808K),
5.4238636 secs]
2006-11-28 18:47:19.162038500 [Full GC 499583K->499583K(511808K),
4.1761207 secs]
2006-11-28 18:47:23.338292500 [Full GC 499583K->499435K(511808K),
4.0381495 secs]
2006-11-28 18:47:27.397388500 [Full GC 499578K->499578K(511808K),
4.1922121 secs]
2006-11-28 18:47: 31.589735500 [Full GC 499578K->499504K(511808K),
5.5022513 secs]
2006-11-28 18:47: 37.095978500 [Full GC 499583K->499583K(511808K),
4.1805690 secs]
2006-11-28 18:47:41.276684500 [Full GC 499583K->499503K(511808K),
4.0814699 secs]
2006-11-28 18:47:45.363748500 [Full GC 499583K->499583K(511808K),
4.1720694 secs]
2006-11-28 18:47:49.535943500 [Full GC 499583K->499544K(511808K),
4.1699713 secs]
2006-11-28 18:47: 53.755950500 [Full GC 499583K->499583K(511808K),
4.1593166 secs]
2006-11-28 18:47: 57.917455500 [Full GC 499583K->499552K(511808K),
5.5514382 secs]
2006-11-28 18:48:03.472669500 [Full GC 499584K->499584K(511808K),
4.1725040 secs]
2006-11-28 18:48:07.659044500 [Full GC 499584K->499584K(511808K),
4.1750055 secs]
2006-11-28 18:48:11.847801500 [Full GC 499584K->499584K(511808K),
4.1131328 secs]
2006-11-28 18:48: 15.979586500 [Full GC 499584K->496903K(511808K),
5.6763932 secs]
2006-11-28 18:48: 21.657594500 Exception in thread "Callback
Thread" [ERROR] 16818770 pool-1-thread-7:
( org.red5.server.service.ServiceInvoker.error ) Error executing
call: Service: null Method: deleteStream Num Params: 10: 1
2006-11-28 18:48:21.658716500 java.lang.OutOfMemoryError: Java heap
space[ERROR] 16818779 pool-1-thread-7:
( org.red5.server.service.ServiceInvoker.error ) Service invocation
error
2006-11-28 18:48:21.667577500
java.lang.reflect.InvocationTargetException
2006-11-28 18:48:21.667580500 at
sun.reflect.GeneratedMethodAccessor21.invoke(Unknown Source)
2006-11-28 18:48: 21.667582500 at
sun.reflect.DelegatingMethodAccessorImpl.invoke
(DelegatingMethodAccessorImpl.java:25)
2006-11-28 18:48:21.667585500 at java.lang.reflect.Method.invoke
(Method.java:585)
2006-11-28 18:48: 21.667731500 at
org.red5.server.service.ServiceInvoker.invoke (ServiceInvoker.java:
161)
2006-11-28 18:48:21.667734500 at
org.red5.server.net.rtmp.RTMPHandler.invokeCall(RTMPHandler.java:135)
2006-11-28 18:48: 21.667736500 at
org.red5.server.net.rtmp.RTMPHandler.onInvoke (RTMPHandler.java:276)
2006-11-28 18:48:21.667739500 at
org.red5.server.net.rtmp.BaseRTMPHandler.messageReceived
(BaseRTMPHandler.java:114)
2006-11-28 18:48: 21.667741500 at
org.red5.server.net.rtmp.RTMPMinaIoHandler.messageReceived
(RTMPMinaIoHandler.java:78)
2006-11-28 18:48:21.667752500 at
org.apache.mina.common.support.AbstractIoFilterChain
$2.messageReceived
(AbstractIoFilterChain.java:189)
2006-11-28 18:48:21.667756500 at
org.apache.mina.common.support.AbstractIoFilterChain.callNextMessageR
ece
ived(AbstractIoFilterChain.java:502)
2006-11-28 18:48:21.667759500 at
org.apache.mina.common.support.AbstractIoFilterChain.access$1000
(AbstractIoFilterChain.java:52)
2006-11-28 18:48:21.667775500 at
org.apache.mina.common.support.AbstractIoFilterChain$EntryImpl
$1.messageReceived( AbstractIoFilterChain.java:777)
2006-11-28 18:48:21.667778500 at
org.red5.io.filter.ExecutorFilter.processEvent(ExecutorFilter.java:
231)
2006-11-28 18:48:21.667781500 at org.red5.io.filter.ExecutorFilter
$ProcessEventsRunnable.run( ExecutorFilter.java:279)
2006-11-28 18:48: 21.667784500 at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask
(ThreadPoolExecutor.java:650)
2006-11-28 18:48:21.667792500 at
java.util.concurrent.ThreadPoolExecutor$Worker.run
(ThreadPoolExecutor.java :675)
2006-11-28 18:48:21.667795500 at java.lang.Thread.run
(Thread.java:595)
2006-11-28 18:48:21.667797500 Caused by: java.lang.OutOfMemoryError:
Java heap space
2006-11-28 18:48: 21.668241500
2006-11-28 18:48: 21.669689500 Exception in thread "btpool0-1"
java.lang.OutOfMemoryError: Java heap space
2006-11-28 18:48:21.671628500 Exception in thread
"FlowControlService" java.lang.OutOfMemoryError: Java heap space
_______________________________________________
Red5 mailing list
[email protected]
http://osflash.org/mailman/listinfo/red5_osflash.org
--
I cannot tell why this heart languishes in silence. It is for
small needs it never asks, or knows or remembers. -- Tagore
Best Regards
Steven Gong
_______________________________________________
Red5 mailing list
[email protected]
http://osflash.org/mailman/listinfo/red5_osflash.org
