We dealt with an issue similar to this long ago that was due to threadpool requests being dropped occasionally. How reproducible is your issue?
On Mon, Sep 12, 2016 at 11:45 PM, [email protected] <[email protected]> wrote: > Hi Alan, > > I'll try to create a small test case. We are experiencing this running a > Plastic SCM server on raspberry, and a client on a different machine. It > will take me a few hours to isolate a test case taking just the required > parts from the plastic network protocol. > > Meanwhile: do you have any suggestion to try to check whether my assumption > is correct? I mean, I suspect somehow the BeginReceive never awakes. I guess > it is using epoll down the stairs. Any log or something I can build to make > sure the issue is on my code and not on Mono? > > Thanks, > > pablo > > > On 9/12/2016 11:37, Alan McGovern wrote: > > Can you share the code to repro the issue? > > Sent from my iPhone > > On 12 Sep 2016, at 09:39, "[email protected]" > <[email protected]> wrote: > > Thanks for the hint Chris, > > We are actually enqueuing the request and doing the actual read on a > different threadpool. > > That being said, we are reproducing this issue with a single client, I mean, > single thread attending on the server. Not sure what can go wrong, tcp > connection is still there, client can send, it simply looks like the server > never wakes up to attend it, randomly. > > Any other hints would be appreciated. > > Thanks, > > pablo > > On 9/9/2016 18:11, Chris Swiedler wrote: > > From what I understand it’s dangerous to do blocking reads on the callback > from something like BeginReceive. The callback will happen on a threadpool > thread, and the blocking reads could then cause the threadpool to be > exhausted. I don’t know if that’s causing your specific problem (it could, > if the reads really do block and you have enough of them) but it’s something > to watch out for. > > > > chris > > > > From: Mono-devel-list [mailto:[email protected]] On > Behalf Of [email protected] > Sent: Friday, September 09, 2016 2:42 AM > To: [email protected] > Cc: dude <[email protected]> > Subject: [Mono-dev] Potential issue with async sockets in Raspberry ARM – > Mono 4.4.0 > > > > Hi, > > I’m posting here hoping someone can throw some light into the problem. I'm a > little bit lost now. > > Our server code (Plastic SCM) running on Raspberry on Mono 4.4.0 (exact > version below) ends up not awaking from socket.BeginReceive after a while. > > I mean, client connects and requests data in 4MB chunks, and depending on > the run, it can transfer a few GB but it ends up not awaking. The client > just sits waiting on a “socket recv” but the server doesn’t answer. > Connection is established (can be checked at OS level). > > The code could be simplified as follows: > > mSocket.BeginReceive(buffer, 0, 0, SocketFlags.None, > OnMessageReceived, null); > > OnMessageReceived => does the EndReceive and then reads data and enqueues > the request for a threadpool to attend it. Once the request is attented and > the response sent, BeginReceive is invoked again. Important: all > “BeginReceive()” calls are done from the same thread which NEVER dies. > > All we use the BeginReceive for is to decouple socket from thread, so we do > not have a 1-1. You see we do pass “zero” as bytes to read, because all we > want to achieve is to get awakened when new data is received, then just read > using blocking calls (no async). > > I’m asking if there could be something about Mono because I read 4.6 release > notes (http://www.mono-project.com/docs/about-mono/releases/4.6.0/) and the > “atomic fixes for ARM64”. Could it be related somehow. > > The same code runs on Windows and Linux PCs (even Macs) without issues. We > use the same code on all our production servers and even our Cloud roles, > and we are not aware of issues. > Now we are testing a new faster storage and using Raspberry to check how > fast it goes on slower hardware. > > Complete Mono version: > Mono JIT compiler version 4.4.0 (tarball Tue Jun 14 13:44:06 UTC 2016) > Copyright (C) 2002-2014 Novell, Inc, Xamarin Inc and Contributors. > www.mono-project.com > TLS: __thread > SIGSEGV: normal > Notifications: epoll > Architecture: armel,vfp+hard > Disabled: none > Misc: softdebug > LLVM: supported, not enabled. > GC: sgen > . > > Thanks! > > Pablo Santos > www.plasticscm.com > > > _______________________________________________ > Mono-devel-list mailing list > [email protected] > http://lists.dot.net/mailman/listinfo/mono-devel-list > > > > _______________________________________________ > Mono-devel-list mailing list > [email protected] > http://lists.dot.net/mailman/listinfo/mono-devel-list > -- Studying for the Turing test _______________________________________________ Mono-devel-list mailing list [email protected] http://lists.dot.net/mailman/listinfo/mono-devel-list
