OK Please file JIRA On Mon, 30 Mar 2020 at 22:02, Rodolfo Gonzalez <[email protected]> wrote:
> Hmmmmm, did I post that? :o > > Anyway, I'm not sure it it's useful. This customer is using old 4.0.5, and > the one with the reported problem was using 4.0.10 (I had to move it back > to 4.0.5). > > > > El lun., 30 mar. 2020 a las 8:54, Maxim Solodovnik (<[email protected]>) > escribió: > >> BTW what version is this? >> >> On Mon, 30 Mar 2020 at 21:52, Maxim Solodovnik <[email protected]> >> wrote: >> >>> You already post this one 4 days ago >>> and I have asked for JIRA :)) >>> Will try to address ASAP >>> >>> On Mon, 30 Mar 2020 at 21:51, Rodolfo Gonzalez < >>> [email protected]> wrote: >>> >>>> I've just found this exception in the output, I'm not quite sure if >>>> this is related to the reported problem, but the users are complaining of >>>> not being able to connect: >>>> >>>> ERROR 03-30 08:49:19.686 o.a.o.w.a.WebSession:208 [.0-5451-exec-50] - >>>> Unexpected exception while checking hashes >>>> java.lang.NullPointerException: null >>>> at >>>> org.apache.openmeetings.db.entity.basic.Client.getUserId(Client.java:124) >>>> at >>>> org.apache.openmeetings.web.app.ClientManager.getByKeys(ClientManager.java:296) >>>> at >>>> org.apache.openmeetings.web.app.ClientManager.invalidate(ClientManager.java:305) >>>> at >>>> org.apache.openmeetings.web.app.WebSession.invalidate(WebSession.java:122) >>>> at org.apache.wicket.Session.invalidateNow(Session.java:513) >>>> at >>>> org.apache.openmeetings.web.app.WebSession.checkHashes(WebSession.java:183) >>>> at >>>> org.apache.openmeetings.web.pages.HashPage.onInitialize(HashPage.java:102) >>>> at >>>> org.apache.wicket.Component.fireInitialize(Component.java:871) >>>> at >>>> org.apache.wicket.MarkupContainer.internalInitialize(MarkupContainer.java:1037) >>>> at org.apache.wicket.Page.isPageStateless(Page.java:461) >>>> at >>>> org.apache.wicket.core.request.mapper.AbstractBookmarkableMapper.getPageInfo(AbstractBookmarkableMapper.java:473) >>>> at >>>> org.apache.wicket.core.request.mapper.AbstractBookmarkableMapper.mapHandler(AbstractBookmarkableMapper.java:420) >>>> at >>>> org.apache.wicket.core.request.mapper.MountedMapper.mapHandler(MountedMapper.java:155) >>>> at >>>> org.apache.openmeetings.web.app.Application$NoVersionMapper.mapHandler(Application.java:282) >>>> at >>>> org.apache.wicket.request.mapper.CompoundRequestMapper.mapHandler(CompoundRequestMapper.java:210) >>>> at >>>> org.apache.wicket.request.cycle.RequestCycle.mapUrlFor(RequestCycle.java:449) >>>> at >>>> org.apache.wicket.request.handler.render.WebPageRenderer.respond(WebPageRenderer.java:233) >>>> at >>>> org.apache.wicket.core.request.handler.RenderPageRequestHandler.respond(RenderPageRequestHandler.java:202) >>>> at >>>> org.apache.wicket.request.cycle.RequestCycle$HandlerExecutor.respond(RequestCycle.java:912) >>>> at >>>> org.apache.wicket.request.RequestHandlerExecutor.execute(RequestHandlerExecutor.java:65) >>>> at >>>> org.apache.wicket.request.cycle.RequestCycle.execute(RequestCycle.java:283) >>>> at >>>> org.apache.wicket.request.cycle.RequestCycle.processRequest(RequestCycle.java:253) >>>> at >>>> org.apache.wicket.request.cycle.RequestCycle.processRequestAndDetach(RequestCycle.java:221) >>>> at >>>> org.apache.wicket.protocol.ws.AbstractUpgradeFilter.processRequestCycle(AbstractUpgradeFilter.java:70) >>>> at >>>> org.apache.wicket.protocol.http.WicketFilter.processRequest(WicketFilter.java:204) >>>> at >>>> org.apache.wicket.protocol.http.WicketFilter.doFilter(WicketFilter.java:286) >>>> at >>>> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) >>>> at >>>> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) >>>> at >>>> org.red5.logging.LoggerContextFilter.doFilter(LoggerContextFilter.java:84) >>>> at >>>> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) >>>> at >>>> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) >>>> at >>>> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:199) >>>> at >>>> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96) >>>> at >>>> org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:608) >>>> at >>>> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:139) >>>> at >>>> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92) >>>> at >>>> org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:668) >>>> at >>>> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87) >>>> at >>>> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343) >>>> at >>>> org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:408) >>>> at >>>> org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66) >>>> at >>>> org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:764) >>>> at >>>> org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1388) >>>> at >>>> org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49) >>>> at >>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) >>>> at >>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) >>>> at >>>> org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) >>>> at java.lang.Thread.run(Thread.java:748) >>>> >>>> >>>> El lun., 30 mar. 2020 a las 8:42, Maxim Solodovnik (< >>>> [email protected]>) escribió: >>>> >>>>> Would appreciate if you could share your steps >>>>> And/or write short video of what is happening >>>>> Unfortunately I can't help until I'll be able to reproduce the issue :( >>>>> >>>>> On Mon, 30 Mar 2020 at 21:39, Rodolfo Gonzalez < >>>>> [email protected]> wrote: >>>>> >>>>>> Hi Maxim, >>>>>> >>>>>> no, it happens in one tab AFAIK, randomly. It's a bit hard to know >>>>>> more details from the user's side, since they're at home (they us OM >>>>>> during >>>>>> #StayAtHome). But it has happened to me too, with one tab/one window. >>>>>> >>>>>> I'll ask support to inquire more details from my users. >>>>>> >>>>>> El vie., 27 mar. 2020 a las 23:56, Maxim Solodovnik (< >>>>>> [email protected]>) escribió: >>>>>> >>>>>>> Hello Rodolfo, >>>>>>> >>>>>>> In the other thread it turn out to be "multi-tab" issue >>>>>>> Is this your case? >>>>>>> >>>>>>> If not could you share your steps? >>>>>>> >>>>>>> On Thu, 26 Mar 2020 at 14:21, Maxim Solodovnik <[email protected]> >>>>>>> wrote: >>>>>>> >>>>>>>> most probably not :) >>>>>>>> >>>>>>>> On Thu, 26 Mar 2020 at 12:13, Rodolfo Gonzalez < >>>>>>>> [email protected]> wrote: >>>>>>>> >>>>>>>>> The version of java is not the cause, right? :) >>>>>>>>> >>>>>>>>> Thanks, have a good day. >>>>>>>>> >>>>>>>>> El mié., 25 mar. 2020 a las 20:55, Maxim Solodovnik (< >>>>>>>>> [email protected]>) escribió: >>>>>>>>> >>>>>>>>>> Well >>>>>>>>>> In the ticket "Users log-in to OM from Moodle" >>>>>>>>>> Moodle is CMS with OM plugin working via REST >>>>>>>>>> >>>>>>>>>> So it seems to be same report >>>>>>>>>> Will try to to some tests later >>>>>>>>>> >>>>>>>>>> It is now day-time-job-time >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Thu, 26 Mar 2020 at 09:13, Rodolfo Gonzalez < >>>>>>>>>> [email protected]> wrote: >>>>>>>>>> >>>>>>>>>>> Hi Maxim, >>>>>>>>>>> >>>>>>>>>>> I read the ticket. It's not exactly the same problem. In my >>>>>>>>>>> case, users do not login from the OM's login window, but the API is >>>>>>>>>>> used >>>>>>>>>>> with a generic user with SOAP rights. Also, the problem is not a >>>>>>>>>>> constant, >>>>>>>>>>> which makes harder to reproduce. I have goten the error randomly. >>>>>>>>>>> I'll try >>>>>>>>>>> to do a test program to list rooms hashes in loop and concurently, >>>>>>>>>>> to see >>>>>>>>>>> if I can find something. >>>>>>>>>>> >>>>>>>>>>> El mié., 25 mar. 2020 a las 20:00, Maxim Solodovnik (< >>>>>>>>>>> [email protected]>) escribió: >>>>>>>>>>> >>>>>>>>>>>> The problem is not reproducible in the clean environment :( >>>>>>>>>>>> I have re-opened >>>>>>>>>>>> https://issues.apache.org/jira/browse/OPENMEETINGS-2192 >>>>>>>>>>>> But we need to figure out reproducible steps for this :( >>>>>>>>>>>> >>>>>>>>>>>> On Thu, 26 Mar 2020 at 07:43, Rodolfo Gonzalez < >>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> Hi Maxim, the problem is, in the server output I can not find >>>>>>>>>>>>> exceptions other than those refused logins. In the client (PHP) >>>>>>>>>>>>> side, I >>>>>>>>>>>>> only get that the object which is returned by the login API call >>>>>>>>>>>>> is null >>>>>>>>>>>>> (so the call to get the room hash crashes, as the method is >>>>>>>>>>>>> called from a >>>>>>>>>>>>> null). >>>>>>>>>>>>> >>>>>>>>>>>>> I mean, I find other kind of exceptions, concerning to video >>>>>>>>>>>>> stream and other things. But the problem seems to be that the >>>>>>>>>>>>> login is >>>>>>>>>>>>> being rejected. Another user posted a similar (maybe the same) >>>>>>>>>>>>> problem just >>>>>>>>>>>>> a moment ago. >>>>>>>>>>>>> >>>>>>>>>>>>> Regards. >>>>>>>>>>>>> >>>>>>>>>>>>> El mié., 25 mar. 2020 a las 18:21, Maxim Solodovnik (< >>>>>>>>>>>>> [email protected]>) escribió: >>>>>>>>>>>>> >>>>>>>>>>>>>> Hello Rodolfo, >>>>>>>>>>>>>> >>>>>>>>>>>>>> Debug logging you quoted provide zero info :( >>>>>>>>>>>>>> There are no limits for logins in the system >>>>>>>>>>>>>> >>>>>>>>>>>>>> Could you share error stack trace? >>>>>>>>>>>>>> >>>>>>>>>>>>>> I might write test foe this >>>>>>>>>>>>>> Am I right thinking the use case is: 100 users are trying to >>>>>>>>>>>>>> get room hash in parallel? >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> On Thu, 26 Mar 2020 at 00:58, Rodolfo Gonzalez < >>>>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> currently I run an OM 4.0.10 server with a pretty default >>>>>>>>>>>>>>> configuration, using >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> # java -version >>>>>>>>>>>>>>> openjdk version "1.8.0_242" >>>>>>>>>>>>>>> OpenJDK Runtime Environment (build >>>>>>>>>>>>>>> 1.8.0_242-8u242-b08-0ubuntu3~16.04-b08) >>>>>>>>>>>>>>> OpenJDK 64-Bit Server VM (build 25.242-b08, mixed mode) >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> on Ubuntu 16.04 (default values for file handles limits). >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I integrated OM with a web app made in PHP with the Yii >>>>>>>>>>>>>>> framework, using the SOAP API. I use this library: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> https://github.com/hakeem23/open-meetings-soap >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> The server is on AWS, being a c4d.4xlarge instance, with >>>>>>>>>>>>>>> htop reporting 16 cores. CPU usage is reported by AWS to be >>>>>>>>>>>>>>> around 50%, >>>>>>>>>>>>>>> with a system load maxing up at the 5-7 range reported by htop. >>>>>>>>>>>>>>> The thing >>>>>>>>>>>>>>> is that as many as 100 users try to list rooms or connect to >>>>>>>>>>>>>>> the server at >>>>>>>>>>>>>>> a given time. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I'm having several problems with the service, the main one >>>>>>>>>>>>>>> being that the users sometimes can not get the room hashes or >>>>>>>>>>>>>>> enter a room. >>>>>>>>>>>>>>> Given the flow of the process, my best guess is that the >>>>>>>>>>>>>>> library can not >>>>>>>>>>>>>>> login to the OM server. The errors in the output are like >>>>>>>>>>>>>>> these, I think >>>>>>>>>>>>>>> (I'm not sure if these are the cause of the problem): >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> DEBUG 03-25 11:44:24.577 o.a.o.d.u.AuthLevelUtil:42 >>>>>>>>>>>>>>> [0.0-5453-exec-7] - Level Admin :: [DENIED] >>>>>>>>>>>>>>> DEBUG 03-25 11:44:24.632 o.a.o.d.u.AuthLevelUtil:42 >>>>>>>>>>>>>>> [0.0-5453-exec-7] - Level Admin :: [DENIED] >>>>>>>>>>>>>>> DEBUG 03-25 11:44:24.633 o.a.o.d.u.AuthLevelUtil:42 >>>>>>>>>>>>>>> [0.0-5453-exec-7] - Level GroupAdmin :: [DENIED] >>>>>>>>>>>>>>> DEBUG 03-25 11:44:24.642 o.a.o.d.u.AuthLevelUtil:42 >>>>>>>>>>>>>>> [0.0-5453-exec-7] - Level Admin :: [DENIED] >>>>>>>>>>>>>>> DEBUG 03-25 11:44:24.642 o.a.o.d.u.AuthLevelUtil:42 >>>>>>>>>>>>>>> [0.0-5453-exec-7] - Level Admin :: [DENIED] >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> and so on. But I'm unsure if this is the error which causes >>>>>>>>>>>>>>> a cascade exceptions which render the web app unable to get the >>>>>>>>>>>>>>> room >>>>>>>>>>>>>>> hashes and so (those process relay on being loged in, anyway). >>>>>>>>>>>>>>> The thing >>>>>>>>>>>>>>> is, the SOAP library currently does the login process from the >>>>>>>>>>>>>>> app, using >>>>>>>>>>>>>>> and user "admin" with SOAP and ADMIN level credentials. And the >>>>>>>>>>>>>>> problem is >>>>>>>>>>>>>>> intermitent. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Do you consider that my guess is right? Is there a limit on >>>>>>>>>>>>>>> "logins" at any given time? If so, can it be increased? Does >>>>>>>>>>>>>>> the version of >>>>>>>>>>>>>>> java affect? >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Any other ideas are welcome. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Regards, >>>>>>>>>>>>>>> Rodolfo. >>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> -- >>>>>>>>>>>>>> WBR >>>>>>>>>>>>>> Maxim aka solomax >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> -- >>>>>>>>>>>> WBR >>>>>>>>>>>> Maxim aka solomax >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> WBR >>>>>>>>>> Maxim aka solomax >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> WBR >>>>>>>> Maxim aka solomax >>>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> WBR >>>>>>> Maxim aka solomax >>>>>>> >>>>>> >>>>> >>>>> -- >>>>> WBR >>>>> Maxim aka solomax >>>>> >>>> >>> >>> -- >>> WBR >>> Maxim aka solomax >>> >> >> >> -- >> WBR >> Maxim aka solomax >> > -- WBR Maxim aka solomax
