I created a PR for this: https://github.com/mono/mono/pull/3603 <https://github.com/mono/mono/pull/3603>
I submitted the PR for the 4.6 branch, although I would hope we can also put this in the current GA 4.4 branch (and master). I assume multiple PR’s would be necessary. > On Sep 21, 2016, at 4:08 AM, [email protected] wrote: > > Thanks Dave, > > I was only able to reproduce it (even on LAN) on slow hw like the raspberry > and doing parallel requests. Single thread was unable to reproduce the error, > I sent several GBs this way. But with 4 threads, it fails in like 8 seconds. > > pablo > > On 9/20/2016 14:30, David Curylo wrote: >> Pablo, >> >> I couldn’t find evidence of anyone already working on this issue, so I >> started on a PR yesterday. I was a little blind after 8 hours of reading >> through tcpdump .pcap files :) but will submit it today with as good of test >> coverage as I can get without a remote system. I’ll post here when I submit >> it. >> >> -Dave >> >>> On Sep 20, 2016, at 3:44 AM, >>> <mailto:[email protected]>[email protected] >>> <mailto:[email protected]> wrote: >>> >>> Hi David, >>> >>> Great. >>> >>> So, will you make a pull request? Or is there one already in progress? >>> >>> Thanks, >>> >>> pablo >>> >>> On 9/20/2016 3:47, David Curylo wrote: >>>> I’m glad it helps. AFAICT a slow network actually _helps_ trigger the >>>> issue, guessing it allows the kernel TCP buffers to fill and leads to this >>>> issue. >>>> >>>> Here is an older mono NetworkStream.Write, which was making up for the >>>> issue by continuing to push data through when the buffers filled: >>>> https://github.com/mono/mono/blob/mono-3.12.1/mcs/class/System/System.Net.Sockets/NetworkStream.cs#L414 >>>> >>>> <https://github.com/mono/mono/blob/mono-3.12.1/mcs/class/System/System.Net.Sockets/NetworkStream.cs#L414> >>>> >>>> So I expect that will work around your issue, although it needs to be >>>> fixed in Socket.cs like the comments >>>> <https://bugzilla.xamarin.com/show_bug.cgi?id=43902#c3> in the bug. >>>> >>>>> On Sep 19, 2016, at 6:12 PM, >>>>> <mailto:[email protected]>[email protected] >>>>> <mailto:[email protected]> wrote: >>>>> >>>>> Hi David, >>>>> >>>>> Thanks for the info!! >>>>> >>>>> So, does this fix the issue? >>>>> >>>>> while (sent < count) >>>>> { >>>>> sent += mSocket.Send(buffer, offset + sent, count - sent, >>>>> SocketFlags.None); >>>>> } >>>>> >>>>> Or am I simply having good luck?? >>>>> >>>>> I'm testing through VPN myself, so maybe the slow network helps. But, we >>>>> are able to see the same issue on LAN... always using the raspberry as >>>>> server. >>>>> >>>>> Thanks, >>>>> >>>>> pablo >>>>> >>>>> On 9/19/2016 23:38, David Curylo wrote: >>>>>> Pablo, >>>>>> >>>>>> I believe I’m experiencing a similar issue, also reported by others >>>>>> here: >>>>>> <https://bugzilla.xamarin.com/show_bug.cgi?id=43902>https://bugzilla.xamarin.com/show_bug.cgi?id=43902 >>>>>> <https://bugzilla.xamarin.com/show_bug.cgi?id=43902> >>>>>> >>>>>> Mono’s Socket.Send doesn’t block until all data is sent. The old >>>>>> NetworkStream in mono used to account for this with a while loop to >>>>>> continue sending, but the NetworkStream from MS Reference Source expects >>>>>> the Socket to block until all data is sent. When the reference source >>>>>> was pulled in, the Socket.Send could sometimes fail to send all the >>>>>> data. I can’t ever make this failure occur on localhost, but it will >>>>>> occur for me communicating with remote hosts. >>>>>> >>>>>> I’m working on a PR now to have Socket.Send_internal loop until all data >>>>>> is sent so Socket.Send will block as the reference source expects it to, >>>>>> unless anyone else has any suggestions. >>>>>> >>>>>> -Dave >>>>>> >>>>>>> On Sep 19, 2016, at 9:33 AM, >>>>>>> <mailto:[email protected]>[email protected] >>>>>>> <mailto:[email protected]> wrote: >>>>>>> >>>>>>> Hi, >>>>>>> >>>>>>> I have found a potential issue with NetworkStream on ARM (Raspberry). >>>>>>> >>>>>>> It might be a stupid bug on my code too, but I don't see it. >>>>>>> >>>>>>> >>>>>>> In short: a simple multi-threaded server running on Raspberry (same >>>>>>> behavior on Mono 4.4.0 as Mono 4.6.0), client on W10: >>>>>>> >>>>>>> 1. Fails to correctly send data using NetworkStream (server seems to >>>>>>> send the data, but it never arrives to client). >>>>>>> <https://github.com/psantosl/mono-raspberry-socket-issue/blob/master/Program.cs>https://github.com/psantosl/mono-raspberry-socket-issue/blob/master/Program.cs >>>>>>> >>>>>>> <https://github.com/psantosl/mono-raspberry-socket-issue/blob/master/Program.cs> >>>>>>> 1.1. Fails *only* using NetworkStream (no BufferedStream or >>>>>>> BinaryReader/Writer around, and it still fails): >>>>>>> <https://github.com/psantosl/mono-raspberry-socket-issue/blob/master-only-network-stream/Program.cs>https://github.com/psantosl/mono-raspberry-socket-issue/blob/master-only-network-stream/Program.cs >>>>>>> >>>>>>> <https://github.com/psantosl/mono-raspberry-socket-issue/blob/master-only-network-stream/Program.cs> >>>>>>> >>>>>>> 2. Works correctly using Sockets (no streams on top). >>>>>>> <https://github.com/psantosl/mono-raspberry-socket-issue/blob/master-use-socket-only/Program.cs>https://github.com/psantosl/mono-raspberry-socket-issue/blob/master-use-socket-only/Program.cs >>>>>>> >>>>>>> <https://github.com/psantosl/mono-raspberry-socket-issue/blob/master-use-socket-only/Program.cs> >>>>>>> >>>>>>> 3. Works correctly using a custom NetworkStream: >>>>>>> 3.1. >>>>>>> <https://github.com/psantosl/mono-raspberry-socket-issue/blob/master-use-alternative-network-stream/Program.cs>https://github.com/psantosl/mono-raspberry-socket-issue/blob/master-use-alternative-network-stream/Program.cs >>>>>>> >>>>>>> <https://github.com/psantosl/mono-raspberry-socket-issue/blob/master-use-alternative-network-stream/Program.cs> >>>>>>> 3.2. >>>>>>> <https://github.com/psantosl/mono-raspberry-socket-issue/blob/master-use-alternative-network-stream/MyNetworkStream.cs>https://github.com/psantosl/mono-raspberry-socket-issue/blob/master-use-alternative-network-stream/MyNetworkStream.cs >>>>>>> >>>>>>> <https://github.com/psantosl/mono-raspberry-socket-issue/blob/master-use-alternative-network-stream/MyNetworkStream.cs> >>>>>>> 3.3. Although I just saw it timing out after transferring several >>>>>>> hundred MB on each connection (I'm on a slow VPN connected to the >>>>>>> Raspberry, although we reproduced it on LAN too). Interestingly, using >>>>>>> the regular NetworkStream it fails inmediately. >>>>>>> >>>>>>> >>>>>>> When it fails, what happens is that at a given point in time (in my >>>>>>> tests like in a few seconds) one of the client threads (or more than >>>>>>> one) NEVER exits the Read, data never arrives although the server log >>>>>>> says the data was written, but it doesn't. Since I've set a timeout for >>>>>>> the read operation, you see how the program stops working. >>>>>>> >>>>>>> >>>>>>> >>>>>>> == Additional tests performed == >>>>>>> I tried with different buffer sizes, it always fails with the >>>>>>> NetworkStream. >>>>>>> >>>>>>> I ran the same server code on other setups: >>>>>>> >>>>>>> * Window 10/.NET: OK. >>>>>>> * OpenSuse 13.2 + mono 4.0.4 (Stable 4.0.4.1/5ab4c0d): OK. >>>>>>> * Raspberry: failed both with mono 4.4.0 and mono-4.6.0.245 (built from >>>>>>> GitHub on the 746756c commit. >>>>>>> >>>>>>> I originally discovered the issue running a Plastic SCM server on >>>>>>> BananaPi <http://www.bananapi.org/> (like a Raspberry + SATA). >>>>>>> >>>>>>> == Extra notes == >>>>>>> * I know the multi-threaded server is not optimal and >>>>>>> thread-per-connection is really bad, this is just a test case. >>>>>>> * Initially I thought we had an issue with async sockets, but after a >>>>>>> few days I just isolated the problem in NetworkStream. >>>>>>> >>>>>>> >>>>>>> Any hints would be appreciated. >>>>>>> >>>>>>> Thanks, >>>>>>> >>>>>>> pablo >>>>>>> >>>>>>> www.plasticscm.com <http://www.plasticscm.com/> >>>>>>> _______________________________________________ >>>>>>> Mono-devel-list mailing list >>>>>>> [email protected] <mailto:[email protected]> >>>>>>> http://lists.dot.net/mailman/listinfo/mono-devel-list >>>>>>> <http://lists.dot.net/mailman/listinfo/mono-devel-list> >>>>>> >>>>> >>>> >>> >> >
_______________________________________________ Mono-devel-list mailing list [email protected] http://lists.dot.net/mailman/listinfo/mono-devel-list
