On Thu, 11 Feb 2021 at 09:02, [email protected] <[email protected]> wrote:
> Yeah that is what I meant, maybe yo should check n case of flowType == > FLOW_STATE.*NOT_FLOWING*: > > (1) If the client is stream audio+video && the evt.getType == AUDIO => Do > *NOT* stopBroadcast > (2) If the client is stream audio-only && the evt.getType == AUDIO => Do > STOP KStream by calling stopBroadcast > > I think (2) would not be ideal still, cause basically you are killing audio > streams that potentially are just "quiet" but not dead. But at least we are > not killing off a video stream because audio is quiet currently. > > Maybe for (2) we can also increase the timeout which may help a bit. > Timeout is configurable `kurento.flowout.timeout` (5secs by default) > is there no other more reliable event we could use ? > Unfortunately I'm not aware of such :((( > > Thanks, > Seb > > Sebastian Wagner > Director Arrakeen Solutions, OM-Hosting.com > http://arrakeen-solutions.co.nz/ > https://om-hosting.com - Cloud & Server Hosting for HTML5 > Video-Conferencing OpenMeetings > < > https://www.youracclaim.com/badges/da4e8828-743d-4968-af6f-49033f10d60a/public_url > > > < > https://www.youracclaim.com/badges/b7e709c6-aa87-4b02-9faf-099038475e36/public_url > > > > > On Thu, 11 Feb 2021 at 14:49, Maxim Solodovnik <[email protected]> > wrote: > > > (sorry for multi-posting) > > > > Can you check if this code will work as expected in audio-only room? > > I'm would like to ensure we will not break this code :) > > > > On Thu, 11 Feb 2021 at 08:43, Maxim Solodovnik <[email protected]> > > wrote: > > > > > BTW there is cancel to cancel stopBroadcast() if flow is restored > > > the timeout seems to be insufficient ... :( > > > > > > On Thu, 11 Feb 2021 at 08:39, Maxim Solodovnik <[email protected]> > > > wrote: > > > > > >> > > >> > > >> On Thu, 11 Feb 2021 at 02:26, [email protected] < > > >> [email protected]> wrote: > > >> > > >>> According to my observations: if stream is NOT_FLOWING it will never > > flow > > >>> again :( > > >>> this is why the connection is dropped > > >>> > > >>> That might be true for MediaType VIDEO but NOT for MediaType *AUDIO* > > >>> > > >>> In the log file (I've added streamIds to the logs) you can see > > >>> NOT_FLOWING > > >>> but later again you can see FLOWING events. For the *same* streamId > > >>> > > >>> Example 1 > > >>> [34mINFO [0;39m 02-10 08:11:38.220 [36mo.a.o.c.r.KStream:160 > > >>> [ventExec-e2-t48] [0;39m - Media Flow STATE :: *NOT_FLOWING*, type > > >>> MediaFlowOutStateChange, evt *AUDIO*, sid > > >>> *d6c5db69-0d30-4ba6-b863-aa3c297e17ac*, uid > > >>> 9ad475f3-e994-4850-97a3-3e3717bf755a > > >>> [34mINFO [0;39m 02-10 08:11:47.022 [36mo.a.o.c.r.KStream:160 > > >>> [ventExec-e2-t73] [0;39m - Media Flow STATE :: *FLOWING*, type > > >>> MediaFlowOutStateChange, evt *AUDIO*, sid > > >>> *d6c5db69-0d30-4ba6-b863-aa3c297e17ac*, uid > > >>> 9ad475f3-e994-4850-97a3-3e3717bf755a > > >>> > > >>> Example 2 > > >>> [34mINFO [0;39m 02-10 08:11:52.066 [36mo.a.o.c.r.KStream:160 > > >>> [ventExec-e2-t65] [0;39m - Media Flow STATE :: *NOT_FLOWING*, type > > >>> MediaFlowOutStateChange, evt *AUDIO*, sid > > >>> *b8e9f1a6-d29f-482c-8faf-4ac9b9939527*, uid > > >>> eb6a7aa7-b913-4d51-a34f-ab1ac684f27d > > >>> [34mINFO [0;39m 02-10 08:12:03.501 [36mo.a.o.c.r.KStream:160 > > >>> [ventExec-e2-t51] [0;39m - Media Flow STATE :: *FLOWING*, type > > >>> MediaFlowOutStateChange, evt *AUDIO*, sid > > >>> *b8e9f1a6-d29f-482c-8faf-4ac9b9939527*, uid > > >>> eb6a7aa7-b913-4d51-a34f-ab1ac684f27d > > >>> > > >>> Example 3 > > >>> [34mINFO [0;39m 02-10 08:12:16.099 [36mo.a.o.c.r.KStream:160 > > >>> [ventExec-e2-t40] [0;39m - Media Flow STATE :: *NOT_FLOWING*, type > > >>> MediaFlowOutStateChange, evt *AUDIO*, sid > > >>> *46420fab-fcaa-493f-9283-1b2fbf9f3170*, uid > > >>> b996144c-9ab1-43d5-8500-da359446ad13 > > >>> [34mINFO [0;39m 02-10 08:12:19.305 [36mo.a.o.c.r.KStream:160 > > >>> [ventExec-e2-t40] [0;39m - Media Flow STATE :: *FLOWING*, type > > >>> MediaFlowOutStateChange, evt *AUDIO*, sid > > >>> *46420fab-fcaa-493f-9283-1b2fbf9f3170*, uid > > >>> b996144c-9ab1-43d5-8500-da359446ad13 > > >>> > > >>> I have copied the full log of all MediaFlow State events when doing > the > > >>> 140 > > >>> users test here: https://pastebin.com/ZEmtZVyH > > >>> > > >>> => You can find another 20 examples of FLOWING after NOT_FLOWING for > > type > > >>> AUDIO. You can _never_ find it for VIDEO. But type AUDIO can indeed > > start > > >>> flowing again! > > >>> > > >>> I understand that this might be a challenge to detect stop events if > we > > >>> can't use this one. But just to be clear, I'm talking about type > AUDIO. > > >>> There must be a way to filter them out to not kill the stream in > those > > >>> cases. > > >>> > > >> > > >> Great! > > >> Thanks for the clarifications! > > >> I'll add this check to the master ASAP! > > >> > > >> > > >>> > > >>> Thanks > > >>> Sebastian > > >>> > > >>> Sebastian Wagner > > >>> Director Arrakeen Solutions, OM-Hosting.com > > >>> http://arrakeen-solutions.co.nz/ > > >>> https://om-hosting.com - Cloud & Server Hosting for HTML5 > > >>> Video-Conferencing OpenMeetings > > >>> < > > >>> > > > https://www.youracclaim.com/badges/da4e8828-743d-4968-af6f-49033f10d60a/public_url > > >>> > > > >>> < > > >>> > > > https://www.youracclaim.com/badges/b7e709c6-aa87-4b02-9faf-099038475e36/public_url > > >>> > > > >>> > > >>> > > >>> On Wed, 10 Feb 2021 at 23:53, Maxim Solodovnik <[email protected] > > > > >>> wrote: > > >>> > > >>> > On Wed, 10 Feb 2021 at 16:22, [email protected] < > > >>> [email protected] > > >>> > > > > >>> > wrote: > > >>> > > > >>> > > I was able to fix the missing Video Pods issue. And I think it > was > > a > > >>> > > successful test with 140 users. > > >>> > > > > >>> > > See: > > >>> > > > > >>> > > > > >>> > > > >>> > > > https://cwiki.apache.org/confluence/display/OPENMEETINGS/OpenMeetings+140+users+test+10-02-2021+with+256crypt+and+FLOW_STATE+fix+for+not+terminating+AUDIO > > >>> > > > > >>> > > Fixes in the branch I was testing: > > >>> > > > > >>> > > - The previously described fix to reduce the N factor in > SCrypt > > >>> > > calculation > > >>> > > - The previous fix for adding some DB INdex on Address.email > > >>> column > > >>> > > > > >>> > > And also in this one: Fix for not calling "stopBroadcast" when > > >>> > > MediaFlowState.NOT_FLOWING is of type AUDIO. > > >>> > > => I discovered in previous tests that there are almost exactly > the > > >>> > amount > > >>> > > of video pods missing compared to the log event: > > >>> > > [34mINFO [0;39m 02-10 08:11:38.220 [36mo.a.o.c.r.KStream:160 > > >>> > > [ventExec-e2-t48] [0;39m - Media Flow STATE :: NOT_FLOWING, type > > >>> > > MediaFlowOutStateChange, evt *AUDIO*, sid > > >>> > > d6c5db69-0d30-4ba6-b863-aa3c297e17ac, uid > > >>> > > 9ad475f3-e994-4850-97a3-3e3717bf755a > > >>> > > > > >>> > > I think AUDIO can stop flowing! there is no need to kill the > stream > > >>> in > > >>> > this > > >>> > > scenario. > > >>> > > > > >>> > > To test and verify I updated my performance investigating branch > to > > >>> > modify > > >>> > > and add && MediaType.VIDEO == evt.getMediaType(): before > > >>> > > > > >>> > > > > >>> > > > >>> > > > https://github.com/apache/openmeetings/blob/feature/OPENMEETINGS-2567-investigate-performance-monitoring/openmeetings-core/src/main/java/org/apache/openmeetings/core/remote/KStream.java#L162 > > >>> > > > > >>> > > And I repeated the 140 user test and it's fine now! (well a lot > > >>> better!) > > >>> > > > > >>> > > > >>> > This code was added because clients thought everything is OK while > > >>> other > > >>> > people in the room were not seeing/hearing > > >>> > And such misunderstanding is much worse than just reloading :( > > >>> > I believe the change like this need to be tested extremely > carefully > > >>> > > > >>> > According to my observations: if stream is NOT_FLOWING it will > never > > >>> flow > > >>> > again :( > > >>> > this is why the connection is dropped > > >>> > > > >>> > > > >>> > > During this test there were almost 100 "Media Flow STATE :: > > >>> NOT_FLOWING, > > >>> > > type AUDIO" events BUT as you can see in the screenshots most > > video > > >>> pods > > >>> > > are fine: > > >>> > > > > >>> > > > > >>> > > > >>> > > > https://cwiki.apache.org/confluence/display/OPENMEETINGS/OpenMeetings+140+users+test+10-02-2021+with+256crypt+and+FLOW_STATE+fix+for+not+terminating+AUDIO#OpenMeetings140userstest10022021with256cryptandFLOW_STATEfixfornotterminatingAUDIO-ScreenshotsfromeachroomwithVideoPods > > >>> > > > > >>> > > There are other issues :) There is a low percentage of video pods > > >>> hanging > > >>> > > (see screenshots on some pods). But previously those pods were > > >>> completely > > >>> > > missing and the ratio of pods missing was 50-70%!! > > >>> > > => So I still call this a success :) Since it's a massive > > improvement > > >>> > from > > >>> > > the results before. > > >>> > > > > >>> > > I will raise a separated PR for discussing lowering the SCrypt > > value > > >>> (1) > > >>> > > and how to address the issue around MediaState.NOT_FLOWING and > type > > >>> AUDIO > > >>> > > to not close the broadCast in those cases (2). > > >>> > > > > >>> > > Unfortunate NOT_FLOWING for type AUDIO, I'm not sure but we may > not > > >>> be > > >>> > able > > >>> > > to use this. I can also remember from the past that gaps in Audio > > >>> streams > > >>> > > can happen in a normal streaming scenario. Cause streaming tries > to > > >>> save > > >>> > > bandwidth. So NOT_FLOWING for Audio may actually not mean > anything. > > >>> > > > > >>> > > Thanks, > > >>> > > Seb > > >>> > > > > >>> > > Sebastian Wagner > > >>> > > Director Arrakeen Solutions, OM-Hosting.com > > >>> > > http://arrakeen-solutions.co.nz/ > > >>> > > https://om-hosting.com - Cloud & Server Hosting for HTML5 > > >>> > > Video-Conferencing OpenMeetings > > >>> > > < > > >>> > > > > >>> > > > >>> > > > https://www.youracclaim.com/badges/da4e8828-743d-4968-af6f-49033f10d60a/public_url > > >>> > > > > > >>> > > < > > >>> > > > > >>> > > > >>> > > > https://www.youracclaim.com/badges/b7e709c6-aa87-4b02-9faf-099038475e36/public_url > > >>> > > > > > >>> > > > > >>> > > > >>> > > > >>> > -- > > >>> > Best regards, > > >>> > Maxim > > >>> > > > >>> > > >> > > >> > > >> -- > > >> Best regards, > > >> Maxim > > >> > > > > > > > > > -- > > > Best regards, > > > Maxim > > > > > > > > > -- > > Best regards, > > Maxim > > > -- Best regards, Maxim
