I found more. Users reported that they could not connect, and in fact I
couldn't either. This was in the logs:
ERROR 03-30 12:23:13.623 o.a.w.p.w.j.WicketEndpoint:86 [.0-5453-exec-16] -
An error occurred in web socket connection with id : 5b0
java.io.EOFException: null
at
org.apache.tomcat.util.net.NioEndpoint$NioSocketWrapper.fillReadBuffer(NioEndpoint.java:1179)
at
org.apache.tomcat.util.net.NioEndpoint$NioSocketWrapper.fillReadBuffer(NioEndpoint.java:1150)
at
org.apache.tomcat.util.net.NioEndpoint$NioSocketWrapper.read(NioEndpoint.java:1120)
at
org.apache.tomcat.websocket.server.WsFrameServer.onDataAvailable(WsFrameServer.java:72)
at
org.apache.tomcat.websocket.server.WsFrameServer.doOnDataAvailable(WsFrameServer.java:171)
at
org.apache.tomcat.websocket.server.WsFrameServer.notifyDataAvailable(WsFrameServer.java:151)
at
org.apache.tomcat.websocket.server.WsHttpUpgradeHandler.upgradeDispatch(WsHttpUpgradeHandler.java:148)
at
org.apache.coyote.http11.upgrade.UpgradeProcessorInternal.dispatch(UpgradeProcessorInternal.java:54)
at
org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:53)
at
org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:764)
at
org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1388)
at
org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at
org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:748)
El lun., 30 mar. 2020 a las 9:03, Maxim Solodovnik (<[email protected]>)
escribió:
> OK
> Please file JIRA
>
> On Mon, 30 Mar 2020 at 22:02, Rodolfo Gonzalez <[email protected]>
> wrote:
>
>> Hmmmmm, did I post that? :o
>>
>> Anyway, I'm not sure it it's useful. This customer is using old 4.0.5,
>> and the one with the reported problem was using 4.0.10 (I had to move it
>> back to 4.0.5).
>>
>>
>>
>> El lun., 30 mar. 2020 a las 8:54, Maxim Solodovnik (<[email protected]>)
>> escribió:
>>
>>> BTW what version is this?
>>>
>>> On Mon, 30 Mar 2020 at 21:52, Maxim Solodovnik <[email protected]>
>>> wrote:
>>>
>>>> You already post this one 4 days ago
>>>> and I have asked for JIRA :))
>>>> Will try to address ASAP
>>>>
>>>> On Mon, 30 Mar 2020 at 21:51, Rodolfo Gonzalez <
>>>> [email protected]> wrote:
>>>>
>>>>> I've just found this exception in the output, I'm not quite sure if
>>>>> this is related to the reported problem, but the users are complaining of
>>>>> not being able to connect:
>>>>>
>>>>> ERROR 03-30 08:49:19.686 o.a.o.w.a.WebSession:208 [.0-5451-exec-50] -
>>>>> Unexpected exception while checking hashes
>>>>> java.lang.NullPointerException: null
>>>>> at
>>>>> org.apache.openmeetings.db.entity.basic.Client.getUserId(Client.java:124)
>>>>> at
>>>>> org.apache.openmeetings.web.app.ClientManager.getByKeys(ClientManager.java:296)
>>>>> at
>>>>> org.apache.openmeetings.web.app.ClientManager.invalidate(ClientManager.java:305)
>>>>> at
>>>>> org.apache.openmeetings.web.app.WebSession.invalidate(WebSession.java:122)
>>>>> at org.apache.wicket.Session.invalidateNow(Session.java:513)
>>>>> at
>>>>> org.apache.openmeetings.web.app.WebSession.checkHashes(WebSession.java:183)
>>>>> at
>>>>> org.apache.openmeetings.web.pages.HashPage.onInitialize(HashPage.java:102)
>>>>> at
>>>>> org.apache.wicket.Component.fireInitialize(Component.java:871)
>>>>> at
>>>>> org.apache.wicket.MarkupContainer.internalInitialize(MarkupContainer.java:1037)
>>>>> at org.apache.wicket.Page.isPageStateless(Page.java:461)
>>>>> at
>>>>> org.apache.wicket.core.request.mapper.AbstractBookmarkableMapper.getPageInfo(AbstractBookmarkableMapper.java:473)
>>>>> at
>>>>> org.apache.wicket.core.request.mapper.AbstractBookmarkableMapper.mapHandler(AbstractBookmarkableMapper.java:420)
>>>>> at
>>>>> org.apache.wicket.core.request.mapper.MountedMapper.mapHandler(MountedMapper.java:155)
>>>>> at
>>>>> org.apache.openmeetings.web.app.Application$NoVersionMapper.mapHandler(Application.java:282)
>>>>> at
>>>>> org.apache.wicket.request.mapper.CompoundRequestMapper.mapHandler(CompoundRequestMapper.java:210)
>>>>> at
>>>>> org.apache.wicket.request.cycle.RequestCycle.mapUrlFor(RequestCycle.java:449)
>>>>> at
>>>>> org.apache.wicket.request.handler.render.WebPageRenderer.respond(WebPageRenderer.java:233)
>>>>> at
>>>>> org.apache.wicket.core.request.handler.RenderPageRequestHandler.respond(RenderPageRequestHandler.java:202)
>>>>> at
>>>>> org.apache.wicket.request.cycle.RequestCycle$HandlerExecutor.respond(RequestCycle.java:912)
>>>>> at
>>>>> org.apache.wicket.request.RequestHandlerExecutor.execute(RequestHandlerExecutor.java:65)
>>>>> at
>>>>> org.apache.wicket.request.cycle.RequestCycle.execute(RequestCycle.java:283)
>>>>> at
>>>>> org.apache.wicket.request.cycle.RequestCycle.processRequest(RequestCycle.java:253)
>>>>> at
>>>>> org.apache.wicket.request.cycle.RequestCycle.processRequestAndDetach(RequestCycle.java:221)
>>>>> at
>>>>> org.apache.wicket.protocol.ws.AbstractUpgradeFilter.processRequestCycle(AbstractUpgradeFilter.java:70)
>>>>> at
>>>>> org.apache.wicket.protocol.http.WicketFilter.processRequest(WicketFilter.java:204)
>>>>> at
>>>>> org.apache.wicket.protocol.http.WicketFilter.doFilter(WicketFilter.java:286)
>>>>> at
>>>>> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
>>>>> at
>>>>> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
>>>>> at
>>>>> org.red5.logging.LoggerContextFilter.doFilter(LoggerContextFilter.java:84)
>>>>> at
>>>>> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
>>>>> at
>>>>> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
>>>>> at
>>>>> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:199)
>>>>> at
>>>>> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96)
>>>>> at
>>>>> org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:608)
>>>>> at
>>>>> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:139)
>>>>> at
>>>>> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92)
>>>>> at
>>>>> org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:668)
>>>>> at
>>>>> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87)
>>>>> at
>>>>> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343)
>>>>> at
>>>>> org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:408)
>>>>> at
>>>>> org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)
>>>>> at
>>>>> org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:764)
>>>>> at
>>>>> org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1388)
>>>>> at
>>>>> org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
>>>>> at
>>>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>>>>> at
>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>>>>> at
>>>>> org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
>>>>> at java.lang.Thread.run(Thread.java:748)
>>>>>
>>>>>
>>>>> El lun., 30 mar. 2020 a las 8:42, Maxim Solodovnik (<
>>>>> [email protected]>) escribió:
>>>>>
>>>>>> Would appreciate if you could share your steps
>>>>>> And/or write short video of what is happening
>>>>>> Unfortunately I can't help until I'll be able to reproduce the issue
>>>>>> :(
>>>>>>
>>>>>> On Mon, 30 Mar 2020 at 21:39, Rodolfo Gonzalez <
>>>>>> [email protected]> wrote:
>>>>>>
>>>>>>> Hi Maxim,
>>>>>>>
>>>>>>> no, it happens in one tab AFAIK, randomly. It's a bit hard to know
>>>>>>> more details from the user's side, since they're at home (they us OM
>>>>>>> during
>>>>>>> #StayAtHome). But it has happened to me too, with one tab/one window.
>>>>>>>
>>>>>>> I'll ask support to inquire more details from my users.
>>>>>>>
>>>>>>> El vie., 27 mar. 2020 a las 23:56, Maxim Solodovnik (<
>>>>>>> [email protected]>) escribió:
>>>>>>>
>>>>>>>> Hello Rodolfo,
>>>>>>>>
>>>>>>>> In the other thread it turn out to be "multi-tab" issue
>>>>>>>> Is this your case?
>>>>>>>>
>>>>>>>> If not could you share your steps?
>>>>>>>>
>>>>>>>> On Thu, 26 Mar 2020 at 14:21, Maxim Solodovnik <
>>>>>>>> [email protected]> wrote:
>>>>>>>>
>>>>>>>>> most probably not :)
>>>>>>>>>
>>>>>>>>> On Thu, 26 Mar 2020 at 12:13, Rodolfo Gonzalez <
>>>>>>>>> [email protected]> wrote:
>>>>>>>>>
>>>>>>>>>> The version of java is not the cause, right? :)
>>>>>>>>>>
>>>>>>>>>> Thanks, have a good day.
>>>>>>>>>>
>>>>>>>>>> El mié., 25 mar. 2020 a las 20:55, Maxim Solodovnik (<
>>>>>>>>>> [email protected]>) escribió:
>>>>>>>>>>
>>>>>>>>>>> Well
>>>>>>>>>>> In the ticket "Users log-in to OM from Moodle"
>>>>>>>>>>> Moodle is CMS with OM plugin working via REST
>>>>>>>>>>>
>>>>>>>>>>> So it seems to be same report
>>>>>>>>>>> Will try to to some tests later
>>>>>>>>>>>
>>>>>>>>>>> It is now day-time-job-time
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Thu, 26 Mar 2020 at 09:13, Rodolfo Gonzalez <
>>>>>>>>>>> [email protected]> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hi Maxim,
>>>>>>>>>>>>
>>>>>>>>>>>> I read the ticket. It's not exactly the same problem. In my
>>>>>>>>>>>> case, users do not login from the OM's login window, but the API
>>>>>>>>>>>> is used
>>>>>>>>>>>> with a generic user with SOAP rights. Also, the problem is not a
>>>>>>>>>>>> constant,
>>>>>>>>>>>> which makes harder to reproduce. I have goten the error randomly.
>>>>>>>>>>>> I'll try
>>>>>>>>>>>> to do a test program to list rooms hashes in loop and concurently,
>>>>>>>>>>>> to see
>>>>>>>>>>>> if I can find something.
>>>>>>>>>>>>
>>>>>>>>>>>> El mié., 25 mar. 2020 a las 20:00, Maxim Solodovnik (<
>>>>>>>>>>>> [email protected]>) escribió:
>>>>>>>>>>>>
>>>>>>>>>>>>> The problem is not reproducible in the clean environment :(
>>>>>>>>>>>>> I have re-opened
>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OPENMEETINGS-2192
>>>>>>>>>>>>> But we need to figure out reproducible steps for this :(
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Thu, 26 Mar 2020 at 07:43, Rodolfo Gonzalez <
>>>>>>>>>>>>> [email protected]> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hi Maxim, the problem is, in the server output I can not find
>>>>>>>>>>>>>> exceptions other than those refused logins. In the client (PHP)
>>>>>>>>>>>>>> side, I
>>>>>>>>>>>>>> only get that the object which is returned by the login API call
>>>>>>>>>>>>>> is null
>>>>>>>>>>>>>> (so the call to get the room hash crashes, as the method is
>>>>>>>>>>>>>> called from a
>>>>>>>>>>>>>> null).
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I mean, I find other kind of exceptions, concerning to video
>>>>>>>>>>>>>> stream and other things. But the problem seems to be that the
>>>>>>>>>>>>>> login is
>>>>>>>>>>>>>> being rejected. Another user posted a similar (maybe the same)
>>>>>>>>>>>>>> problem just
>>>>>>>>>>>>>> a moment ago.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Regards.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> El mié., 25 mar. 2020 a las 18:21, Maxim Solodovnik (<
>>>>>>>>>>>>>> [email protected]>) escribió:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hello Rodolfo,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Debug logging you quoted provide zero info :(
>>>>>>>>>>>>>>> There are no limits for logins in the system
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Could you share error stack trace?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I might write test foe this
>>>>>>>>>>>>>>> Am I right thinking the use case is: 100 users are trying to
>>>>>>>>>>>>>>> get room hash in parallel?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Thu, 26 Mar 2020 at 00:58, Rodolfo Gonzalez <
>>>>>>>>>>>>>>> [email protected]> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> currently I run an OM 4.0.10 server with a pretty default
>>>>>>>>>>>>>>>> configuration, using
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> # java -version
>>>>>>>>>>>>>>>> openjdk version "1.8.0_242"
>>>>>>>>>>>>>>>> OpenJDK Runtime Environment (build
>>>>>>>>>>>>>>>> 1.8.0_242-8u242-b08-0ubuntu3~16.04-b08)
>>>>>>>>>>>>>>>> OpenJDK 64-Bit Server VM (build 25.242-b08, mixed mode)
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> on Ubuntu 16.04 (default values for file handles limits).
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I integrated OM with a web app made in PHP with the Yii
>>>>>>>>>>>>>>>> framework, using the SOAP API. I use this library:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> https://github.com/hakeem23/open-meetings-soap
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> The server is on AWS, being a c4d.4xlarge instance, with
>>>>>>>>>>>>>>>> htop reporting 16 cores. CPU usage is reported by AWS to be
>>>>>>>>>>>>>>>> around 50%,
>>>>>>>>>>>>>>>> with a system load maxing up at the 5-7 range reported by
>>>>>>>>>>>>>>>> htop. The thing
>>>>>>>>>>>>>>>> is that as many as 100 users try to list rooms or connect to
>>>>>>>>>>>>>>>> the server at
>>>>>>>>>>>>>>>> a given time.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I'm having several problems with the service, the main one
>>>>>>>>>>>>>>>> being that the users sometimes can not get the room hashes or
>>>>>>>>>>>>>>>> enter a room.
>>>>>>>>>>>>>>>> Given the flow of the process, my best guess is that the
>>>>>>>>>>>>>>>> library can not
>>>>>>>>>>>>>>>> login to the OM server. The errors in the output are like
>>>>>>>>>>>>>>>> these, I think
>>>>>>>>>>>>>>>> (I'm not sure if these are the cause of the problem):
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> DEBUG 03-25 11:44:24.577 o.a.o.d.u.AuthLevelUtil:42
>>>>>>>>>>>>>>>> [0.0-5453-exec-7] - Level Admin :: [DENIED]
>>>>>>>>>>>>>>>> DEBUG 03-25 11:44:24.632 o.a.o.d.u.AuthLevelUtil:42
>>>>>>>>>>>>>>>> [0.0-5453-exec-7] - Level Admin :: [DENIED]
>>>>>>>>>>>>>>>> DEBUG 03-25 11:44:24.633 o.a.o.d.u.AuthLevelUtil:42
>>>>>>>>>>>>>>>> [0.0-5453-exec-7] - Level GroupAdmin :: [DENIED]
>>>>>>>>>>>>>>>> DEBUG 03-25 11:44:24.642 o.a.o.d.u.AuthLevelUtil:42
>>>>>>>>>>>>>>>> [0.0-5453-exec-7] - Level Admin :: [DENIED]
>>>>>>>>>>>>>>>> DEBUG 03-25 11:44:24.642 o.a.o.d.u.AuthLevelUtil:42
>>>>>>>>>>>>>>>> [0.0-5453-exec-7] - Level Admin :: [DENIED]
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> and so on. But I'm unsure if this is the error which causes
>>>>>>>>>>>>>>>> a cascade exceptions which render the web app unable to get
>>>>>>>>>>>>>>>> the room
>>>>>>>>>>>>>>>> hashes and so (those process relay on being loged in, anyway).
>>>>>>>>>>>>>>>> The thing
>>>>>>>>>>>>>>>> is, the SOAP library currently does the login process from the
>>>>>>>>>>>>>>>> app, using
>>>>>>>>>>>>>>>> and user "admin" with SOAP and ADMIN level credentials. And
>>>>>>>>>>>>>>>> the problem is
>>>>>>>>>>>>>>>> intermitent.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Do you consider that my guess is right? Is there a limit on
>>>>>>>>>>>>>>>> "logins" at any given time? If so, can it be increased? Does
>>>>>>>>>>>>>>>> the version of
>>>>>>>>>>>>>>>> java affect?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Any other ideas are welcome.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>> Rodolfo.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>> WBR
>>>>>>>>>>>>>>> Maxim aka solomax
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> --
>>>>>>>>>>>>> WBR
>>>>>>>>>>>>> Maxim aka solomax
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> WBR
>>>>>>>>>>> Maxim aka solomax
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> WBR
>>>>>>>>> Maxim aka solomax
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> WBR
>>>>>>>> Maxim aka solomax
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>> --
>>>>>> WBR
>>>>>> Maxim aka solomax
>>>>>>
>>>>>
>>>>
>>>> --
>>>> WBR
>>>> Maxim aka solomax
>>>>
>>>
>>>
>>> --
>>> WBR
>>> Maxim aka solomax
>>>
>>
>
> --
> WBR
> Maxim aka solomax
>
