(sorry for multi-posting) Can you check if this code will work as expected in audio-only room? I'm would like to ensure we will not break this code :)
On Thu, 11 Feb 2021 at 08:43, Maxim Solodovnik <[email protected]> wrote: > BTW there is cancel to cancel stopBroadcast() if flow is restored > the timeout seems to be insufficient ... :( > > On Thu, 11 Feb 2021 at 08:39, Maxim Solodovnik <[email protected]> > wrote: > >> >> >> On Thu, 11 Feb 2021 at 02:26, [email protected] < >> [email protected]> wrote: >> >>> According to my observations: if stream is NOT_FLOWING it will never flow >>> again :( >>> this is why the connection is dropped >>> >>> That might be true for MediaType VIDEO but NOT for MediaType *AUDIO* >>> >>> In the log file (I've added streamIds to the logs) you can see >>> NOT_FLOWING >>> but later again you can see FLOWING events. For the *same* streamId >>> >>> Example 1 >>> [34mINFO [0;39m 02-10 08:11:38.220 [36mo.a.o.c.r.KStream:160 >>> [ventExec-e2-t48] [0;39m - Media Flow STATE :: *NOT_FLOWING*, type >>> MediaFlowOutStateChange, evt *AUDIO*, sid >>> *d6c5db69-0d30-4ba6-b863-aa3c297e17ac*, uid >>> 9ad475f3-e994-4850-97a3-3e3717bf755a >>> [34mINFO [0;39m 02-10 08:11:47.022 [36mo.a.o.c.r.KStream:160 >>> [ventExec-e2-t73] [0;39m - Media Flow STATE :: *FLOWING*, type >>> MediaFlowOutStateChange, evt *AUDIO*, sid >>> *d6c5db69-0d30-4ba6-b863-aa3c297e17ac*, uid >>> 9ad475f3-e994-4850-97a3-3e3717bf755a >>> >>> Example 2 >>> [34mINFO [0;39m 02-10 08:11:52.066 [36mo.a.o.c.r.KStream:160 >>> [ventExec-e2-t65] [0;39m - Media Flow STATE :: *NOT_FLOWING*, type >>> MediaFlowOutStateChange, evt *AUDIO*, sid >>> *b8e9f1a6-d29f-482c-8faf-4ac9b9939527*, uid >>> eb6a7aa7-b913-4d51-a34f-ab1ac684f27d >>> [34mINFO [0;39m 02-10 08:12:03.501 [36mo.a.o.c.r.KStream:160 >>> [ventExec-e2-t51] [0;39m - Media Flow STATE :: *FLOWING*, type >>> MediaFlowOutStateChange, evt *AUDIO*, sid >>> *b8e9f1a6-d29f-482c-8faf-4ac9b9939527*, uid >>> eb6a7aa7-b913-4d51-a34f-ab1ac684f27d >>> >>> Example 3 >>> [34mINFO [0;39m 02-10 08:12:16.099 [36mo.a.o.c.r.KStream:160 >>> [ventExec-e2-t40] [0;39m - Media Flow STATE :: *NOT_FLOWING*, type >>> MediaFlowOutStateChange, evt *AUDIO*, sid >>> *46420fab-fcaa-493f-9283-1b2fbf9f3170*, uid >>> b996144c-9ab1-43d5-8500-da359446ad13 >>> [34mINFO [0;39m 02-10 08:12:19.305 [36mo.a.o.c.r.KStream:160 >>> [ventExec-e2-t40] [0;39m - Media Flow STATE :: *FLOWING*, type >>> MediaFlowOutStateChange, evt *AUDIO*, sid >>> *46420fab-fcaa-493f-9283-1b2fbf9f3170*, uid >>> b996144c-9ab1-43d5-8500-da359446ad13 >>> >>> I have copied the full log of all MediaFlow State events when doing the >>> 140 >>> users test here: https://pastebin.com/ZEmtZVyH >>> >>> => You can find another 20 examples of FLOWING after NOT_FLOWING for type >>> AUDIO. You can _never_ find it for VIDEO. But type AUDIO can indeed start >>> flowing again! >>> >>> I understand that this might be a challenge to detect stop events if we >>> can't use this one. But just to be clear, I'm talking about type AUDIO. >>> There must be a way to filter them out to not kill the stream in those >>> cases. >>> >> >> Great! >> Thanks for the clarifications! >> I'll add this check to the master ASAP! >> >> >>> >>> Thanks >>> Sebastian >>> >>> Sebastian Wagner >>> Director Arrakeen Solutions, OM-Hosting.com >>> http://arrakeen-solutions.co.nz/ >>> https://om-hosting.com - Cloud & Server Hosting for HTML5 >>> Video-Conferencing OpenMeetings >>> < >>> https://www.youracclaim.com/badges/da4e8828-743d-4968-af6f-49033f10d60a/public_url >>> > >>> < >>> https://www.youracclaim.com/badges/b7e709c6-aa87-4b02-9faf-099038475e36/public_url >>> > >>> >>> >>> On Wed, 10 Feb 2021 at 23:53, Maxim Solodovnik <[email protected]> >>> wrote: >>> >>> > On Wed, 10 Feb 2021 at 16:22, [email protected] < >>> [email protected] >>> > > >>> > wrote: >>> > >>> > > I was able to fix the missing Video Pods issue. And I think it was a >>> > > successful test with 140 users. >>> > > >>> > > See: >>> > > >>> > > >>> > >>> https://cwiki.apache.org/confluence/display/OPENMEETINGS/OpenMeetings+140+users+test+10-02-2021+with+256crypt+and+FLOW_STATE+fix+for+not+terminating+AUDIO >>> > > >>> > > Fixes in the branch I was testing: >>> > > >>> > > - The previously described fix to reduce the N factor in SCrypt >>> > > calculation >>> > > - The previous fix for adding some DB INdex on Address.email >>> column >>> > > >>> > > And also in this one: Fix for not calling "stopBroadcast" when >>> > > MediaFlowState.NOT_FLOWING is of type AUDIO. >>> > > => I discovered in previous tests that there are almost exactly the >>> > amount >>> > > of video pods missing compared to the log event: >>> > > [34mINFO [0;39m 02-10 08:11:38.220 [36mo.a.o.c.r.KStream:160 >>> > > [ventExec-e2-t48] [0;39m - Media Flow STATE :: NOT_FLOWING, type >>> > > MediaFlowOutStateChange, evt *AUDIO*, sid >>> > > d6c5db69-0d30-4ba6-b863-aa3c297e17ac, uid >>> > > 9ad475f3-e994-4850-97a3-3e3717bf755a >>> > > >>> > > I think AUDIO can stop flowing! there is no need to kill the stream >>> in >>> > this >>> > > scenario. >>> > > >>> > > To test and verify I updated my performance investigating branch to >>> > modify >>> > > and add && MediaType.VIDEO == evt.getMediaType(): before >>> > > >>> > > >>> > >>> https://github.com/apache/openmeetings/blob/feature/OPENMEETINGS-2567-investigate-performance-monitoring/openmeetings-core/src/main/java/org/apache/openmeetings/core/remote/KStream.java#L162 >>> > > >>> > > And I repeated the 140 user test and it's fine now! (well a lot >>> better!) >>> > > >>> > >>> > This code was added because clients thought everything is OK while >>> other >>> > people in the room were not seeing/hearing >>> > And such misunderstanding is much worse than just reloading :( >>> > I believe the change like this need to be tested extremely carefully >>> > >>> > According to my observations: if stream is NOT_FLOWING it will never >>> flow >>> > again :( >>> > this is why the connection is dropped >>> > >>> > >>> > > During this test there were almost 100 "Media Flow STATE :: >>> NOT_FLOWING, >>> > > type AUDIO" events BUT as you can see in the screenshots most video >>> pods >>> > > are fine: >>> > > >>> > > >>> > >>> https://cwiki.apache.org/confluence/display/OPENMEETINGS/OpenMeetings+140+users+test+10-02-2021+with+256crypt+and+FLOW_STATE+fix+for+not+terminating+AUDIO#OpenMeetings140userstest10022021with256cryptandFLOW_STATEfixfornotterminatingAUDIO-ScreenshotsfromeachroomwithVideoPods >>> > > >>> > > There are other issues :) There is a low percentage of video pods >>> hanging >>> > > (see screenshots on some pods). But previously those pods were >>> completely >>> > > missing and the ratio of pods missing was 50-70%!! >>> > > => So I still call this a success :) Since it's a massive improvement >>> > from >>> > > the results before. >>> > > >>> > > I will raise a separated PR for discussing lowering the SCrypt value >>> (1) >>> > > and how to address the issue around MediaState.NOT_FLOWING and type >>> AUDIO >>> > > to not close the broadCast in those cases (2). >>> > > >>> > > Unfortunate NOT_FLOWING for type AUDIO, I'm not sure but we may not >>> be >>> > able >>> > > to use this. I can also remember from the past that gaps in Audio >>> streams >>> > > can happen in a normal streaming scenario. Cause streaming tries to >>> save >>> > > bandwidth. So NOT_FLOWING for Audio may actually not mean anything. >>> > > >>> > > Thanks, >>> > > Seb >>> > > >>> > > Sebastian Wagner >>> > > Director Arrakeen Solutions, OM-Hosting.com >>> > > http://arrakeen-solutions.co.nz/ >>> > > https://om-hosting.com - Cloud & Server Hosting for HTML5 >>> > > Video-Conferencing OpenMeetings >>> > > < >>> > > >>> > >>> https://www.youracclaim.com/badges/da4e8828-743d-4968-af6f-49033f10d60a/public_url >>> > > > >>> > > < >>> > > >>> > >>> https://www.youracclaim.com/badges/b7e709c6-aa87-4b02-9faf-099038475e36/public_url >>> > > > >>> > > >>> > >>> > >>> > -- >>> > Best regards, >>> > Maxim >>> > >>> >> >> >> -- >> Best regards, >> Maxim >> > > > -- > Best regards, > Maxim > -- Best regards, Maxim
