My sincerest apologies. I have had quite a few covid distractions this week... and last. But that doesn’t mean I haven’t been reviewing all the forum messages ... and direct emails (:)) that I have received over the last week or so. I am usually very quick to respond / reply... but for personal reasons... it has been difficult... and again, I apologize for not answering emails to date. Saying that, I had previously mentioned that I did a test on the OM demo site regarding the number of users per room etc. @Maxim, I know there has been a lot of discussion regarding the next release and so on... but if you can hold off for a day or two... I would like to give you a little feedback on a few things. I promise to get back in touch with you all later on today (tonight where I am).
All the best, Denis. Sent from my iPhone > On Nov 17, 2020, at 7:54 AM, dww <[email protected]> wrote: > > I apologize for accidently sending this twice. > >> On Tue, 2020-11-17 at 08:40 -0500, dww wrote: >> Hello Denis, >> >> I was looking forward to your feedback soon. >> >> It seems there are mainly 2 major stress points. >> >> One, is the stream number limit per kms instance of about 200. That should >> allow about 10 or 11 users in a room with both video and microphone but this >> can be overcome by clustering starting with a docker swarm and the >> containers can be spread over multiple servers. >> >> Two, is the question of how many simultanious pods from users with both >> video and microphone can be accommodated on the client browser for the room. >> When I ran the bash script on a laptop I created 4 guests so that was 4 >> instances of Chrome each with 5 pods on them. Things got unstable after >> that. So I surmise that 20-25 would be the limit there. I don't see how that >> limit could be passed without some design changes. >> >> Any ideas would be welcome. >> >> I think Maxim mentioned that Zoom draws the entire screen on their servers >> and streams that to the clients. >> >> Dennis >> >>> On Fri, 2020-11-13 at 01:23 -0600, Denis Noctor wrote: >>> Hi there Maxim... I did a test with 8 computers and 2 tablets last night >>> (spread across 2 WiFis)... please don’t delete the logs on the OM demo >>> server (next)... I will come back to you all with some feedback and pics >>> later tomorrow (if that’s okay)... however, for reference... I started the >>> process in the public room #7...start time around 8.22pm (12th Nov) >>> (México... 6 hrs behind) and end time 9.50pm... (if you want to check the >>> logs) .... the short version is that 8 users experienced relatively stable >>> performance. Will give you a more detailed feedback once I deal with a >>> personal issue. All the best, Denis. >>> >>> Sent from my iPhone >>> >>>> On Nov 11, 2020, at 9:09 PM, Maxim Solodovnik <[email protected]> wrote: >>>> >>>> Hello All, >>>> >>>> I'll try to answer in one email :) >>>> >>>>> On Wed, 11 Nov 2020 at 20:32, dww <[email protected]> wrote: >>>>> However, Denis, I think your experiment with multiple devices would be >>>>> valuable as then there is only one browser tab or window with the OM >>>>> room open as a guest on each device. Perhaps that will make a >>>>> difference. >>>> >>>> yes, this would be better test (even if "fake" camera is used) >>>> >>>>> >>>>> Dennis >>>>> >>>>> On Wed, 2020-11-11 at 08:24 -0500, dww wrote: >>>>> > Thanks, Denis, >>>>> > >>>>> > Back on Oct. 17 Maxim provided the following Bash script to be run on >>>>> > the machine with a client side browser for the psuedo guest users. ( >>>>> > Use another machine to create the room administratively and send >>>>> > invitations) This is a far simpler way to stress test the client side >>>>> > browser. >>>>> > >>>>> > Dennis >>>>> > >>>>> > Hello, >>>>> > >>>>> > i just have tried the following script >>>>> > started as `./run10.sh 5` >>>>> > >>>>> > everything seems to work, but my CPU was 800% busy (all cores were >>>>> > 100% >>>>> > busy) >>>>> > >>>>> > without `--use-fake-device-for-media-stream` parameter I had lots of >>>>> > permission errors due to camera was "captured" by first browser >>>>> > other have reported "Camera busy" error >>>>> > >>>>> > >>>>> > _HASH_HERE_ - should be replaced with real hash (I have created >>>>> > endless >>>>> > invitation hash to the private conference room) >>>>> > >>>>> > the script >>>>> > =============================================== >>>>> > #!/bin/bash >>>>> > >>>>> > i=$1 >>>>> > >>>>> > if [ -z "${i}" ]; then >>>>> > i=30 >>>>> > fi >>>>> > let "i += 0" >>>>> > >>>>> > rm -rf /tmp/delme* >>>>> > >>>>> > while ((i--)); do >>>>> > #echo "${i}" >>>>> > mkdir /tmp/delme${i} >>>>> > >>>>> > #local conference >>>>> > chromium-browser --user-data-dir=/tmp/delme${i} --disable-infobars >>>>> > --no-default-browser-check --allow-insecure-localhost >>>>> > --use-fake-device-for-media-stream ' >>>>> > https://localhost:5443/openmeetings/hash?invitation=_HASH_HERE_&language=1' >>>>> > & >>>>> > done >>>>> > >>>>> > >>>>> > On Wed, 2020-11-11 at 01:53 -0600, Denis Noctor wrote: >>>>> > > Hi there everyone, this seems to be the “elephant in the room” >>>>> > > discussion, while there has been a HUGE amount of development and >>>>> > > progress in OM since March (thank you so much @Maxim) ... there is >>>>> > > the whole issue of, for example, the number of users per room... >>>>> > > which seems to be about 5-6 (and maybe even to 7) when pushed to >>>>> > > the >>>>> > > limit... with both audio and video being broadcasted from all >>>>> > > users... and, something else.. if there are simultaneous >>>>> > > classes/sessions being held on the same server... will this >>>>> > > restrict >>>>> > > things even further? Is this an overall limitation >>>> >>>> Sebastian did some AWS based testing >>>> And, if i'm not mistaken, the server with 4GB RAM was able to handle at >>>> least 3 rooms of 5 people >>>> (5.1.0-SNAPSHOT should behave better than 5.0.1) >>>> >>>> to increase the number of rooms you can use cluster >>>> >>>>> to using a >>>>> > > browser >>>>> > > based approach... or should we be taking approach? >>>> >>>> well, >>>> there is "The Limit" >>>> KMS can handle only certain amount of multimedia connections >>>> additionally there are other limits: >>>> - bandwidth >>>> - CPU >>>> - RAM >>>> - open files (network socket is a file) >>>> >>>> "The Limit" is something I'm not sure how to deal with (yet) >>>> >>>>> > > >>>>> > > It was my intention to test out the OM “demo servers” over the last >>>>> > > 2 >>>>> > > weeks but will take today off and try to test 10 real device >>>>> > > connections... with a combination of desktops, laptops, android >>>>> > > tablets and maybe even the odd iPhone or two. >>>> >>>> Apple devices has issues with sound (outgoing) >>>> I'm still investigating this one >>>> >>>>> > > >>>>> > > My million dollar question is... prior to WebRTC and Kurento... was >>>>> > > it possible to have 5-10 users in a room with audio and video >>>>> > > working >>>>> > > seamlessly in previous versions (for example, the old “flash” setup >>>>> > > (which will be redundant after Christmas... Chrome etc >>>>> > > notifications) >>>>> > > and if so, what has changed? >>>> >>>> Yes this was possible >>>> OM_before_5 was based on Red5 media server >>>> Unfortunately it's open source version has no WebRTC support >>>> >>>>> > > >>>>> > > If there is anyone out there that has no problem with user numbers >>>>> > > (using audio and vid)... exceeding a body of 7-10+, please let us >>>>> > > know. >>>>> > > >>>>> > > In the meantime, I’ll give you my feedback on my tests. >>>>> > > >>>>> > > I really appreciate everything that has been done to date. >>>>> > > >>>>> > > Thanks. >>>>> > > >>>>> > > Sent from my iPhone >>>>> > > >>>>> > > > On Nov 9, 2020, at 4:50 PM, dww <[email protected]> wrote: >>>>> > > > >>>>> > > > Hello Maxim, >>>>> > > > >>>>> > > > A couple of weeks ago there was an email thread about the 5 total >>>>> > > > users >>>>> > > > for one room, each user with video/microphone under the >>>>> > > > Subject: "docker container clustering experiments #1". >>>> >>>> For whatever reason you love to start new mail threads :)))) >>>> >>>>> In this >>>>> > > > case >>>>> > > > it >>>>> > > > appears the bottleneck is the CPU usage on the client machine >>>>> > > > with >>>>> > > > the >>>>> > > > browser. >>>>> > > > >>>>> > > > In a response to Denis Noctor on a similar thread you mentioned >>>>> > > > to >>>>> > > > try >>>>> > > > the following: >>>>> > > > >>>>> > > > "please check allowed amount of opened files for the user who >>>>> > > > starts >>>>> > > > OM/KMS/TURN >>>>> > > > increasing it might help" >>>>> > > > >>>>> > > > Might this help with the issue we discussed? Where approximately >>>>> > > > do >>>>> > > > I >>>>> > > > set the allowed amount of opened files? >>>> >>>> KMS seems to drop connections when there is not enough files >>>> (network socket is a file) >>>> you can check the limit for current user using `ulimit -n` (`ulimit -a` to >>>> see all limits) >>>> >>>> to check limit for `nobody` user `su nobody --shell /bin/bash --command >>>> "ulimit -n"` >>>> >>>> to increase the limit i'm changing `/etc/security/limits.conf` file >>>> https://github.com/openmeetings/openmeetings-docker/blob/48b72f4d0f38a0fab2021a0a2e4d6693c61c00be/scripts/om_euser.sh#L35 >>>> >>>> (seems to work at Ubuntu) >>>> >>>> >>>>> > > > >>>>> > > > Also are there any other things that can be tried to improve this >>>>> > > > scalability? Are there areas in the code that can be examined to >>>>> > > > investigate how to improve this? >>>> >>>> KMS cluster would be ultimate solution, I guess >>>> >>>>> > > > >>>>> > > > Thanks, >>>>> > > > Dennis >>>>> > > > >>>>> > > > >>>>> >>>> >>>>
