Re: [prosody-dev] possible issue with event loop block
On Tuesday, September 19, 2017 at 5:00:21 AM UTC-4, Matthew Wild wrote:
>
> Hi,
>
> On 18 September 2017 at 19:33, Harsha Bellur <hbe...@gmail.com
> > wrote:
> > Hi Mathew,
> >
> > I guess I should have mentioned this earlier - when websocket is closed,
> I
> > send a http request to a microservice. I believe I am establishing
> multiple
> > connections that is either slowing things down or causing segmentation
> fault
> > based on strace.
>
> How are you sending this HTTP request? and where? Did you modify Prosody's
> code?
>
I am using the request method in net/http.lua to post HTTPS/HTTP requests
to an external http server. I did modify the method to handle POST as well
as PUT - that's about it.
>
> >
> > 15:16:50.130413 --- SIGSEGV {si_signo=SIGSEGV, si_code=SEGV_MAPERR,
> > si_addr=0x10} ---
> > 15:16:50.238181 +++ killed by SIGSEGV +++
> > -
>
> A segmentation fault is a very different thing to just a slow server.
> Is this reproducible?
>
At my end, it is intermittent but reproducible. The way I do it is setup
say 40 thousand WS connections and stop nginx. Basically, I am simulating a
use case where the web socket server has lost connectivity to all clients.
>
> Regards,
> Matthew
>
--
You received this message because you are subscribed to the Google Groups
"prosody-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to prosody-dev+unsubscr...@googlegroups.com.
To post to this group, send email to prosody-dev@googlegroups.com.
Visit this group at https://groups.google.com/group/prosody-dev.
For more options, visit https://groups.google.com/d/optout.
Re: [prosody-dev] possible issue with event loop block
To clarify, each websocket disconnect results in a HTTP request being sent
out. Since I am sending the request to the same endpoint, I am wondering if
I can stay connected. Please advice if I can do that.
On Monday, September 18, 2017 at 2:33:52 PM UTC-4, Harsha Bellur wrote:
>
> Hi Mathew,
>
> I guess I should have mentioned this earlier - when websocket is closed, I
> send a http request to a microservice. I believe I am establishing multiple
> connections that is either slowing things down or causing segmentation
> fault based on strace.
>
>
> 15:16:50.130413 --- SIGSEGV {si_signo=SIGSEGV, si_code=SEGV_MAPERR,
> si_addr=0x10} ---
> 15:16:50.238181 +++ killed by SIGSEGV +++
> -
>
> Again, this happens when a large number of websockets close at the same
> time
>
> Thanks
>
> On Saturday, September 16, 2017 at 4:16:57 AM UTC-4, Matthew Wild wrote:
>>
>> Hey,
>>
>> On 16 September 2017 at 01:27, Harsha Bellur <hbe...@gmail.com> wrote:
>> > Well, besides setting use_libevent to true in prosody config, is there
>> > anything else to do?
>> > I am using Jitsi components for audio/video conferencing - so NGINX
>> proxies
>> > APIs to these components as well.
>> > When connected the clients send ping messages (XEP-0199) periodically -
>> that
>> > is about it
>> > Basically, I am validating how the websocket server handles loss of
>> > connectivity from all clients at once - this is where I am seeing the
>> issue.
>>
>> Can you provide the output of `strace -Ttt -p PROSODY_PID` for the
>> duration of the socket-closing operation?
>>
>> Regards,
>> Matthew
>>
>
--
You received this message because you are subscribed to the Google Groups
"prosody-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to prosody-dev+unsubscr...@googlegroups.com.
To post to this group, send email to prosody-dev@googlegroups.com.
Visit this group at https://groups.google.com/group/prosody-dev.
For more options, visit https://groups.google.com/d/optout.
Re: [prosody-dev] possible issue with event loop block
Well, besides setting use_libevent to true in prosody config, is there anything else to do? I am using Jitsi components for audio/video conferencing - so NGINX proxies APIs to these components as well. When connected the clients send ping messages (XEP-0199) periodically - that is about it Basically, I am validating how the websocket server handles loss of connectivity from all clients at once - this is where I am seeing the issue. - Harsha On Monday, August 28, 2017 at 6:46:55 AM UTC-4, Kim Alvefur wrote: > > Hey, > > On Mon, Aug 28, 2017 at 01:52:42AM -0700, Harsha Bellur wrote: > > I am using nginx as the websocket proxy. Prosody version 0.9.8 > > > > I am making 1500 websocket connections using a script and when all > > connections are done - I close them all. At this point, based on the > logs, > > it seems like the event loop is (presumably) blocked till all the > > connections are closed (until listener's ondisconnect function is called > > for all connections). The *main* observation is that there is a delay in > > handling the events - *about 3 - 5 seconds between handling each > disconnect*. > > This renders prosody unusable for a long time until all the disconnect > > events are processed, > > > > What I want to understand is the delay between each disconnect. Is there > a > > way to reduce this delay? > > > > Note that I do not observe this delay (between disconnects) if I run the > > same test with say up to 500 connections. The *total* delay was about 2 > - 3 > > seconds during which time the event loop is ( presumably) blocked till > all > > the connections are closed. > > This all sounds very weird. Can you describe your (Prosody) > configuration in more detail? Especially whether you use libevent or the > default select network backed, and if you have any custom settings for > it. Have you tried connecting directly to Prosody, without nginx? Have > you tried something other than websocket connections? What was sent on > the connections? > > > -- > Zash > -- You received this message because you are subscribed to the Google Groups "prosody-dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to prosody-dev+unsubscr...@googlegroups.com. To post to this group, send email to prosody-dev@googlegroups.com. Visit this group at https://groups.google.com/group/prosody-dev. For more options, visit https://groups.google.com/d/optout.
Re: [prosody-dev] possible issue with event loop block
Well, besides setting use_libevent to true in prosody config, is there anything else to do? I am using Jitsi components for audio/video conferencing - so NGINX proxies APIs to these components as well. When connected the clients send ping messages (XEP-0199) periodically - that is about it Basically, I am validating how the websocket server handles loss of connection connectivity from all clients at once - this is where I am seeing the issue. - Harsha On Monday, August 28, 2017 at 6:46:55 AM UTC-4, Kim Alvefur wrote: > > Hey, > > On Mon, Aug 28, 2017 at 01:52:42AM -0700, Harsha Bellur wrote: > > I am using nginx as the websocket proxy. Prosody version 0.9.8 > > > > I am making 1500 websocket connections using a script and when all > > connections are done - I close them all. At this point, based on the > logs, > > it seems like the event loop is (presumably) blocked till all the > > connections are closed (until listener's ondisconnect function is called > > for all connections). The *main* observation is that there is a delay in > > handling the events - *about 3 - 5 seconds between handling each > disconnect*. > > This renders prosody unusable for a long time until all the disconnect > > events are processed, > > > > What I want to understand is the delay between each disconnect. Is there > a > > way to reduce this delay? > > > > Note that I do not observe this delay (between disconnects) if I run the > > same test with say up to 500 connections. The *total* delay was about 2 > - 3 > > seconds during which time the event loop is ( presumably) blocked till > all > > the connections are closed. > > This all sounds very weird. Can you describe your (Prosody) > configuration in more detail? Especially whether you use libevent or the > default select network backed, and if you have any custom settings for > it. Have you tried connecting directly to Prosody, without nginx? Have > you tried something other than websocket connections? What was sent on > the connections? > > > -- > Zash > -- You received this message because you are subscribed to the Google Groups "prosody-dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to prosody-dev+unsubscr...@googlegroups.com. To post to this group, send email to prosody-dev@googlegroups.com. Visit this group at https://groups.google.com/group/prosody-dev. For more options, visit https://groups.google.com/d/optout.
[prosody-dev] possible issue with event loop block
I am using nginx as the websocket proxy. Prosody version 0.9.8 I am making 1500 websocket connections using a script and when all connections are done - I close them all. At this point, based on the logs, it seems like the event loop is (presumably) blocked till all the connections are closed (until listener's ondisconnect function is called for all connections). The *main* observation is that there is a delay in handling the events - *about 3 - 5 seconds between handling each disconnect*. This renders prosody unusable for a long time until all the disconnect events are processed, What I want to understand is the delay between each disconnect. Is there a way to reduce this delay? Note that I do not observe this delay (between disconnects) if I run the same test with say up to 500 connections. The *total* delay was about 2 - 3 seconds during which time the event loop is ( presumably) blocked till all the connections are closed. Please advise -- You received this message because you are subscribed to the Google Groups "prosody-dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to prosody-dev+unsubscr...@googlegroups.com. To post to this group, send email to prosody-dev@googlegroups.com. Visit this group at https://groups.google.com/group/prosody-dev. For more options, visit https://groups.google.com/d/optout.
[prosody-dev] websocket connections
In net/server_select.lua, comments around max file descriptor value (set to 1024) suggest limiting the value to 1024 to prevent glibc buffer overflow. I have installed prosody on centos 7 which has a patch applied to mitigate the issue. That said, I wanted to check if I can increase the max file descriptor count -- You received this message because you are subscribed to the Google Groups "prosody-dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to prosody-dev+unsubscr...@googlegroups.com. To post to this group, send email to prosody-dev@googlegroups.com. Visit this group at https://groups.google.com/group/prosody-dev. For more options, visit https://groups.google.com/d/optout.
[prosody-dev] Milli second time
Hi I am not sure if this topic has been broached before but is there a way I can get time in miilisecond granularity? Thanks Harsha -- You received this message because you are subscribed to the Google Groups "prosody-dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to prosody-dev+unsubscr...@googlegroups.com. To post to this group, send email to prosody-dev@googlegroups.com. Visit this group at https://groups.google.com/group/prosody-dev. For more options, visit https://groups.google.com/d/optout.
Re: [prosody-dev] events
Hi Mathew, Perhaps I should not be using fire_event(). What I am attempting to do is capture all "muc_events" (such as join room, leave room...) in a MUC room and send it via HTTP. I have written a module that incorporates a handler which does a HTTP post of the events received. This, in my opinion, must be an asynchronous operation. Your help will be much appreciated. Thanks Harsha On Wednesday, January 20, 2016 at 3:26:44 PM UTC-5, Matthew Wild wrote: > > On 20 January 2016 at 20:24, Matthew Wild <mwi...@gmail.com > > wrote: > > On 20 January 2016 at 19:13, Harsha Bellur <hbe...@gmail.com > > wrote: > >> Hi > >> > >> In situations where events (same event_name) are getting fired > >> (module:fire_event()) quicker than the handler function (defined in > >> module:hook()), is it safe to assume that the events are queued? > > > > There is no queue - module:fire_event() d > > Sorry, hit send somehow too early. > > There is no queue - module:fire_event() does not return until the > handler has finished. > > Handlers should never take long to execute, they should return > instantly. If you're having trouble with slow handlers, they need > fixing (we can help you figure out how, if you need). > > Regards, > Matthew > -- You received this message because you are subscribed to the Google Groups "prosody-dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to prosody-dev+unsubscr...@googlegroups.com. To post to this group, send email to prosody-dev@googlegroups.com. Visit this group at https://groups.google.com/group/prosody-dev. For more options, visit https://groups.google.com/d/optout.
[prosody-dev] events
Hi In situations where events (same event_name) are getting fired (module:fire_event()) quicker than the handler function (defined in module:hook()), is it safe to assume that the events are queued? Thanks Harsha -- You received this message because you are subscribed to the Google Groups "prosody-dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to prosody-dev+unsubscr...@googlegroups.com. To post to this group, send email to prosody-dev@googlegroups.com. Visit this group at https://groups.google.com/group/prosody-dev. For more options, visit https://groups.google.com/d/optout.
