Hi Pablo,
You can send a SIGQUIT signal to mono (killall -SIGQUIT mono or ctrl+\
might work as well) to get a stacktraces for all live threads. This
might show possible problems with threadpool exhaustion or other
blocking calls.
Have you checked with netstat if there's data in de receive queue?
Ivo
Op 13-9-2016 om 0:45 schreef [email protected]:
Hi Alan,
I'll try to create a small test case. We are experiencing this running
a Plastic SCM server on raspberry, and a client on a different
machine. It will take me a few hours to isolate a test case taking
just the required parts from the plastic network protocol.
Meanwhile: do you have any suggestion to try to check whether my
assumption is correct? I mean, I suspect somehow the BeginReceive
never awakes. I guess it is using epoll down the stairs. Any log or
something I can build to make sure the issue is on my code and not on
Mono?
Thanks,
pablo
On 9/12/2016 11:37, Alan McGovern wrote:
Can you share the code to repro the issue?
Sent from my iPhone
On 12 Sep 2016, at 09:39, "[email protected]
<mailto:[email protected]>" <[email protected]
<mailto:[email protected]>> wrote:
Thanks for the hint Chris,
We are actually enqueuing the request and doing the actual read on a
different threadpool.
That being said, we are reproducing this issue with a single client,
I mean, single thread attending on the server. Not sure what can go
wrong, tcp connection is still there, client can send, it simply
looks like the server never wakes up to attend it, randomly.
Any other hints would be appreciated.
Thanks,
pablo
On 9/9/2016 18:11, Chris Swiedler wrote:
From what I understand it’s dangerous to do blocking reads on the
callback from something like BeginReceive. The callback will happen
on a threadpool thread, and the blocking reads could then cause the
threadpool to be exhausted. I don’t know if that’s causing your
specific problem (it could, if the reads really do block and you
have enough of them) but it’s something to watch out for.
chris
*From:*Mono-devel-list
[mailto:[email protected]] *On Behalf Of
*[email protected]
*Sent:* Friday, September 09, 2016 2:42 AM
*To:* [email protected]
*Cc:* dude <[email protected]>
*Subject:* [Mono-dev] Potential issue with async sockets in
Raspberry ARM – Mono 4.4.0
Hi,
I’m posting here hoping someone can throw some light into the
problem. I'm a little bit lost now.
Our server code (Plastic SCM) running on Raspberry on Mono 4.4.0
(exact version below) ends up not awaking from socket.BeginReceive
after a while.
I mean, client connects and requests data in 4MB chunks, and
depending on the run, it can transfer a few GB but it ends up not
awaking. The client just sits waiting on a “socket recv” but the
server doesn’t answer. Connection is established (can be checked at
OS level).
The code could be simplified as follows:
mSocket.BeginReceive(buffer, 0, 0, SocketFlags.None,
OnMessageReceived, null);
OnMessageReceived => does the EndReceive and then reads data and
enqueues the request for a threadpool to attend it. Once the
request is attented and the response sent, BeginReceive is invoked
again. Important: all “BeginReceive()” calls are done from the same
thread which NEVER dies.
All we use the BeginReceive for is to decouple socket from thread,
so we do not have a 1-1. You see we do pass “zero” as bytes to
read, because all we want to achieve is to get awakened when new
data is received, then just read using blocking calls (no async).
I’m asking if there could be something about Mono because I read
4.6 release notes
(http://www.mono-project.com/docs/about-mono/releases/4.6.0/) and
the “atomic fixes for ARM64”. Could it be related somehow.
The same code runs on Windows and Linux PCs (even Macs) without
issues. We use the same code on all our production servers and even
our Cloud roles, and we are not aware of issues.
Now we are testing a new faster storage and using Raspberry to
check how fast it goes on slower hardware.
Complete Mono version:
Mono JIT compiler version 4.4.0 (tarball Tue Jun 14 13:44:06 UTC 2016)
Copyright (C) 2002-2014 Novell, Inc, Xamarin Inc and Contributors.
www.mono-project.com <http://www.mono-project.com>
TLS: __thread
SIGSEGV: normal
Notifications: epoll
Architecture: armel,vfp+hard
Disabled: none
Misc: softdebug
LLVM: supported, not enabled.
GC: sgen
.
Thanks!
Pablo Santos
www.plasticscm.com <http://www.plasticscm.com>
_______________________________________________
Mono-devel-list mailing list
[email protected] <mailto:[email protected]>
http://lists.dot.net/mailman/listinfo/mono-devel-list
_______________________________________________
Mono-devel-list mailing list
[email protected]
http://lists.dot.net/mailman/listinfo/mono-devel-list
_______________________________________________
Mono-devel-list mailing list
[email protected]
http://lists.dot.net/mailman/listinfo/mono-devel-list
