Re: Audio lags / user got disconnected

[Peter Dähn]

ok... should be the right address... I keep this browser-tab open... ;-)

Am 08.01.2016 um 15:18 schrieb Maxim Solodovnik:

done: https://groups.google.com/forum/#!topic/red5interest/uYwN_F1HmDQ
please feel free to comment :)

On Fri, Jan 8, 2016 at 2:31 PM, Peter Dähn <da...@vcrp.de> wrote:

Hi Maxim,

by now OpenJDK 7. With Java 8 it could be worth to test G1GC. By now I'm
not sure, how to test it under have load wi8thout using my live system...
;-)

Greetings Peter


Am 07.01.2016 um 15:28 schrieb Maxim Solodovnik:

Actually this file is part of red5 distribution, I'll try to propose your
options as default ones.
BTW what java are you using?

WBR, Maxim
(from mobile, sorry for the typos)
On Jan 7, 2016 20:22, "Peter Dähn" <da...@vcrp.de> wrote:

Hi Maxim,

you are right... "Väterchen Frost" is coming today... ;-) Then have a
nice
party and a lot of gifts.. ;-)

Probably someone could check highperf settings? If it doesn't work, it
shouldn't be shipped in that way...

Am 07.01.2016 um 13:05 schrieb Maxim Solodovnik:

Hello Peter,

Happy New Year :) Today it is also Christmas in Russia, so Merry
Christmas
:)

We are not using highperf script on our demo server (we are using
red.sh)
I'm glad the issue is resolved for you :)

On Wed, Jan 6, 2016 at 4:15 PM, Peter Dähn <da...@vcrp.de> wrote:

Happy new year to all!

Hi Maxim,

the server here seems to work again. I ended up to use java-opts from
my
3.0.2 installation.

export JVM_OPTS="-Xrs -Xms1024M -Xmx1024M -Xss512K -XX:NewSize=256m
-XX:SurvivorRatio=16 -XX:MinHeapFreeRatio=20 -XX:MaxPermSize=128m
-XX:+ExplicitGCInvokesConcurrent -XX:+UseConcMarkSweepGC
-Djava.net.preferIPv4Stack=true -Xverify:none"

This works for me. I don't know, it seems, that we are the only ones
with
this problem...

Anyway I will observe this and probably use this threat again later...

BR Peter



Am 16.12.2015 um 11:54 schrieb Peter Dähn:

Hi Maxim,

if it works better go on... ;-) I would need to test it somehow if its
help me out...

By now we figure out, it could be a Problem with the options in
highperf.sh

With the installation of OM 3.0.7 we also made an update from Ubuntu
12.04 to 14.04. So we are using OpenJDK 7 by now.

The troublemaker seemed to be "-XX:+UseParNewGC". It seems, that
during
a
gc some user loose the connection and others are delayed somehow...

"-XX:+UseConcMarkSweepGC" works better here. This one produce less
"Rejected messages". We had a test yesterday evening and the system
was
usable. By now I try  "-XX:+UseG1GC" this should also work better.
Which
JDK do you use and which java options shall I try.

BR Peter


Am 15.12.2015 um 15:44 schrieb Maxim Solodovnik:

I can switch 3.1.0 to red5 1.0.7 maybe it will work better :)

please let me know if you are interested

On Tue, Dec 15, 2015 at 3:44 PM, Peter Dähn <da...@vcrp.de> wrote:

Hello Maxim,

thanks for that hint. I will try this one also. Currently I still try

to
figure out whats going on...

BR Peter

Am 15.12.2015 um 07:53 schrieb Maxim Solodovnik:

sorry for the late response

I believe "Rejected message" are not good as well as WARN from

DeadLockScheduler
it seems red5 was unable to process messages on good speed
you might try to tweek red5.properties to optimize it

On Fri, Dec 11, 2015 at 2:52 PM, Peter Dähn <da...@vcrp.de> wrote:

Hi Maxim,

thank you for your response.

I will try to figure out, what went wrong. Use a different (test)
system
might work, but at least I will have problems to produce a kind of
traffic.

Unfortunately I don't know how to reproduce these error... Might
be
traffic, might be configuration... it seems we are the only one
with
that
problem... :-( I think you already had a look at your logs...

First I would like to know, do I have to worry about the "INFO
o.r.server.net.rtmp.RTMPConnection - Rejected message:"-entries?
Rejected
doesn't sound that good for me..

Greetings Peter

Am 10.12.2015 um 16:52 schrieb Maxim Solodovnik:

I never saw errors like this :(

Can only suggest to

1) try 3.1.0 SNAPSHOT should work better with audio/video
2) try to update red5 to latest RC might work better
3) ask red5 mailing list
try all above :)

is the issue easily reproducible?

On Thu, Dec 10, 2015 at 8:25 PM, Peter Dähn <da...@vcrp.de>
wrote:

Hi there,

we encounter a problem using om 3.0.7 on ubuntu-server with 4
cpus

and

6
gb ram. Network is 1 gb-ethernet.

A teacher reported, that a session was stopped due to several
disconnects
and a bad audio quality. Audio had glitches and user got
disconnected
very
often. The situation got worst by using screen-recording.

I found alot of warnings and disconnects in the log that time.

Here an example:

2015-12-09 18:51:23,504 [NioProcessor-4] INFO
o.r.server.net.rtmp.RTMPConnection - Rejected message: Packet
[[header
data
type=8, channel=16, timer=1768266], [message timestamp=1768266]]
on
CG2CSHDX5LFVD
2015-12-09 18:51:23,736 [NioProcessor-4] INFO
o.r.server.net.rtmp.RTMPConnection - Rejected message: Packet
[[header
data
type=8, channel=34, timer=1388395], [message timestamp=1388395]]
on
0XS9VHWF75UMX
2015-12-09 18:51:23,750 [NioProcessor-4] INFO
o.r.server.net.rtmp.RTMPConnection - Rejected message: Packet
[[header
data
type=8, channel=16, timer=1768498], [message timestamp=1768498]]
on
CG2CSHDX5LFVD
2015-12-09 18:51:23,750 [NioProcessor-1] INFO
o.r.server.net.rtmp.RTMPConnection - Rejected message: Packet
[[header
data
type=8, channel=58, timer=504480], [message timestamp=504480]]
on
V3GNK2XVSN2VO
2015-12-09 18:51:23,765 [NioProcessor-13] INFO
o.r.server.net.rtmp.RTMPConnection - Rejected message: Packet
[[header
data
type=8, channel=4, timer=665318], [message timestamp=665318]] on
3Y2OVUT1W0KWG
2015-12-09 18:51:23,766 [NioProcessor-4] INFO
o.r.server.net.rtmp.RTMPConnection - Rejected message: Packet
[[header
data
type=8, channel=16, timer=1768544], [message timestamp=1768544]]
on
CG2CSHDX5LFVD
2015-12-09 18:51:23,776 [NioProcessor-9] INFO
o.r.server.net.rtmp.RTMPConnection - Rejected message: Packet
[[header
data
type=8, channel=34, timer=1510203], [message timestamp=1510203]]
on
O6M69P8EUCWJX
2015-12-09 18:51:23,787 [NioProcessor-15] INFO
o.r.server.net.rtmp.RTMPConnection - Rejected message: Packet
[[header
data
type=8, channel=58, timer=1448863], [message timestamp=1448863]]
on
TIUYYRLR80D32
2015-12-09 18:51:24,126 [DeadlockGuardScheduler-17] WARN
o.r.s.net.rtmp.ReceivedMessageTask - Interrupting unfinished
active
task
on
K1EFB4AJLRGNK
2015-12-09 18:51:26,109 [DeadlockGuardScheduler-21] WARN
o.r.s.net.rtmp.ReceivedMessageTask - Interrupting unfinished
active
task
on
K1EFB4AJLRGNK
2015-12-09 18:51:26,393 [RTMPConnectionScheduler-12] WARN
o.r.server.net.rtmp.RTMPConnection - Closing 1MGWAR97JFXFW, due
to
long
handshake. State: handshake
2015-12-09 18:51:27,661 [DeadlockGuardScheduler-31] WARN
o.r.s.net.rtmp.ReceivedMessageTask - Interrupting unfinished
active
task
on
K1EFB4AJLRGNK
2015-12-09 18:51:28,283 [http-nio-0.0.0.0-5080-exec-20] WARN
o.r.server.net.rtmpt.RTMPTServlet - Null connection for session
id:
1MGWAR97JFXFW
2015-12-09 18:51:29,428 [DeadlockGuardScheduler-23] WARN
o.r.s.net.rtmp.ReceivedMessageTask - Interrupting unfinished
active
task
on
K1EFB4AJLRGNK
2015-12-09 18:51:31,447 [DeadlockGuardScheduler-13] WARN
o.r.s.net.rtmp.ReceivedMessageTask - Interrupting unfinished
active
task
on
YZO09I3B96EYW
2015-12-09 18:51:31,896 [RTMPConnectionExecutor-134] WARN
o.r.s.net.rtmp.RTMPMinaConnection - Interrupted while waiting
for
write
lock. State: connected
2015-12-09 18:51:33,027 [RTMPConnectionExecutor-157] INFO
o.a.o.r.r.ScopeApplicationAdapter - W3C x-category:session
x-event:connect
c-ip:xxx.xxx.xxx.xxx c-client-id:HUDY62CJY8UI0
2015-12-09 18:51:34,321 [http-nio-0.0.0.0-5080-exec-35] ERROR
o.r.s.n.r.codec.RTMPProtocolDecoder - Last header null not new,
headerSize:
1, channelId 37
2015-12-09 18:51:34,436 [http-nio-0.0.0.0-5080-exec-35] WARN
o.r.s.n.r.codec.RTMPProtocolDecoder - Closing connection because
decoding
failed: RTMPTConnection from xxx.xxx.xxx.xxx (in: 3914 out:
3900)
session:
HUDY62CJY8UI0 state: connected
2015-12-09 18:51:34,724 [http-nio-0.0.0.0-5080-exec-35] INFO
o.a.o.r.r.ScopeApplicationAdapter - W3C x-category:session
x-event:disconnect c-ip:xxx.xxx.xxx.xxx
c-client-id:HUDY62CJY8UI0
2015-12-09 18:51:34,991 [http-nio-0.0.0.0-5080-exec-34] WARN
o.r.server.net.rtmpt.RTMPTServlet - Null connection for session
id:
HUDY62CJY8UI0
2015-12-09 18:51:41,518 [DeadlockGuardScheduler-30] WARN
o.r.s.net.rtmp.ReceivedMessageTask - Interrupting unfinished
active
task
on
5SYB0WBTBVCMZ
2015-12-09 18:51:41,819 [RTMPConnectionExecutor-154] WARN
o.r.s.net.rtmp.RTMPMinaConnection - Interrupted while waiting
for
write
lock. State: connected
2015-12-09 18:52:01,167 [DeadlockGuardScheduler-21] WARN
o.r.s.net.rtmp.ReceivedMessageTask - Interrupting unfinished
active
task
on
5SYB0WBTBVCMZ
2015-12-09 18:52:01,404 [RTMPConnectionExecutor-133] WARN
o.r.s.net.rtmp.RTMPMinaConnection - Interrupted while waiting
for
write
lock. State: connected
2015-12-09 18:52:10,091 [RTMPConnectionExecutor-138] INFO
o.a.o.r.r.ScopeApplicationAdapter - W3C x-category:session
x-event:connect
c-ip:xxx.xxx.xxx.xxx c-client-id:567
2015-12-09 18:52:29,221 [DeadlockGuardScheduler-17] WARN
o.r.s.net.rtmp.ReceivedMessageTask - Interrupting unfinished
active
task
on
QI81ELGF3OZSJ
2015-12-09 18:52:29,369 [DeadlockGuardScheduler-10] WARN
o.r.s.net.rtmp.ReceivedMessageTask - Interrupting unfinished
active
task
on
QI81ELGF3OZSJ
2015-12-09 18:52:29,483 [RTMPConnectionExecutor-145] WARN
o.r.s.net.rtmp.RTMPMinaConnection - Interrupted while waiting
for
write
lock. State: connected
2015-12-09 18:52:29,617 [RTMPConnectionExecutor-138] WARN
o.r.s.net.rtmp.RTMPMinaConnection - Interrupted while waiting
for
write
lock. State: connected
2015-12-09 18:52:44,665 [DeadlockGuardScheduler-24] WARN
o.r.s.net.rtmp.ReceivedMessageTask - Interrupting unfinished
active
task
on
GARSOFM7QNIBF
2015-12-09 18:52:45,080 [DeadlockGuardScheduler-10] WARN
o.r.s.net.rtmp.ReceivedMessageTask - Interrupting unfinished
active
task
on
YZO09I3B96EYW
2015-12-09 18:52:45,199 [RTMPConnectionExecutor-154] WARN
o.r.s.net.rtmp.RTMPMinaConnection - Interrupted while waiting
for
write
lock. State: connected
2015-12-09 18:52:48,382 [DeadlockGuardScheduler-28] WARN
o.r.s.net.rtmp.ReceivedMessageTask - Interrupting unfinished
active
task
on
XYPAYVIJODN57
2015-12-09 18:53:06,901 [DeadlockGuardScheduler-30] WARN
o.r.s.net.rtmp.ReceivedMessageTask - Interrupting unfinished
active
task
on
5SYB0WBTBVCMZ
2015-12-09 18:53:07,011 [RTMPConnectionExecutor-155] WARN
o.r.s.net.rtmp.RTMPMinaConnection - Interrupted while waiting
for
write
lock. State: connected
2015-12-09 18:53:12,984 [DeadlockGuardScheduler-18] WARN
o.r.s.net.rtmp.ReceivedMessageTask - Interrupting unfinished
active
task
on
5SYB0WBTBVCMZ
2015-12-09 18:53:13,052 [RTMPConnectionExecutor-159] WARN
o.r.s.net.rtmp.RTMPMinaConnection - Interrupted while waiting
for
write
lock. State: connected
2015-12-09 18:53:19,812 [DeadlockGuardScheduler-13] WARN
o.r.s.net.rtmp.ReceivedMessageTask - Interrupting unfinished
active
task
on
XYPAYVIJODN57
2015-12-09 18:53:19,816 [DeadlockGuardScheduler-32] WARN
o.r.s.net.rtmp.ReceivedMessageTask - Interrupting unfinished
active
task
on
XYPAYVIJODN57
2015-12-09 18:53:19,818 [RTMPConnectionExecutor-157] WARN
o.r.s.net.rtmp.RTMPMinaConnection - Interrupted while waiting
for
write
lock. State: connected
2015-12-09 18:53:22,358 [NioProcessor-5] INFO
o.a.o.r.r.ScopeApplicationAdapter - W3C x-category:session
x-event:disconnect c-ip:xxx.xxx.xxx.xxx c-client-id:559

Does anyone see something like this? Where do I have to start to
investigate, what happens?

BR Peter