Thanks Dave,
I was only able to reproduce it (even on LAN) on slow hw like the
raspberry and doing parallel requests. Single thread was unable to
reproduce the error, I sent several GBs this way. But with 4
threads, it fails in like 8 seconds.
pablo
On 9/20/2016 14:30, David Curylo wrote:
Pablo,
I couldn’t find evidence of anyone already working
on this issue, so I started on a PR yesterday. I was a little
blind after 8 hours of reading through tcpdump .pcap files :)
but will submit it today with as good of test coverage as I can
get without a remote system. I’ll post here when I submit it.
-Dave
Hi David,
Great.
So, will you make a pull request? Or is there one
already in progress?
Thanks,
pablo
On 9/20/2016 3:47, David
Curylo wrote:
I’m glad it helps. AFAICT a
slow network actually _helps_ trigger the issue,
guessing it allows the kernel TCP buffers to fill and
leads to this issue.
Here is an older mono
NetworkStream.Write, which was making up for the
issue by continuing to push data through when the
buffers filled:
So I expect that will work around your
issue, although it needs to be fixed in Socket.cs
like the comments in the bug.
Hi
David,
Thanks for the info!!
So, does this fix the issue?
while (sent < count)
{
sent +=
mSocket.Send(buffer, offset + sent,
count - sent, SocketFlags.None);
}
Or am I simply having good luck??
I'm testing through VPN myself, so
maybe the slow network helps. But, we
are able to see the same issue on
LAN... always using the raspberry as
server.
Thanks,
pablo
On
9/19/2016 23:38, David Curylo wrote:
Pablo,
Mono’s Socket.Send
doesn’t block until all data is
sent. The old NetworkStream in mono
used to account for this with a
while loop to continue sending, but
the NetworkStream from MS Reference
Source expects the Socket to block
until all data is sent. When the
reference source was pulled in, the
Socket.Send could sometimes fail to
send all the data. I can’t ever
make this failure occur on
localhost, but it will occur for me
communicating with remote hosts.
I’m working on a PR now
to have Socket.Send_internal loop
until all data is sent so
Socket.Send will block as the
reference source expects it to,
unless anyone else has any
suggestions.
-Dave
Hi,
I have found a potential
issue with NetworkStream
on ARM (Raspberry).
It might be a stupid bug
on my code too, but I
don't see it.
In short: a simple
multi-threaded server
running on Raspberry (same
behavior on Mono 4.4.0 as
Mono 4.6.0), client on
W10:
1. Fails
to correctly send data
using NetworkStream
(server seems to send the
data, but it never arrives
to client). https://github.com/psantosl/mono-raspberry-socket-issue/blob/master/Program.cs
1.1. Fails
*only* using NetworkStream
(no BufferedStream or
BinaryReader/Writer
around, and it still
fails): https://github.com/psantosl/mono-raspberry-socket-issue/blob/master-only-network-stream/Program.cs
2. Works correctly using
Sockets (no streams on
top). https://github.com/psantosl/mono-raspberry-socket-issue/blob/master-use-socket-only/Program.cs
3. Works correctly using a
custom NetworkStream:
3.1. https://github.com/psantosl/mono-raspberry-socket-issue/blob/master-use-alternative-network-stream/Program.cs
3.2. https://github.com/psantosl/mono-raspberry-socket-issue/blob/master-use-alternative-network-stream/MyNetworkStream.cs
3.3. Although I just saw
it timing out after
transferring several
hundred MB on each
connection (I'm on a slow
VPN connected to the
Raspberry, although we
reproduced it on LAN too).
Interestingly, using the
regular NetworkStream it
fails inmediately.
When it fails, what
happens is that at a given
point in time (in my tests
like in a few seconds) one
of the client threads (or
more than one) NEVER exits
the Read, data never
arrives although the
server log says the data
was written, but it
doesn't. Since I've set a
timeout for the read
operation, you see how the
program stops working.
== Additional tests
performed ==
I tried with different
buffer sizes, it always
fails with the
NetworkStream.
I ran the same server code
on other setups:
* Window 10/.NET: OK.
* OpenSuse 13.2 + mono
4.0.4 (Stable
4.0.4.1/5ab4c0d): OK.
* Raspberry: failed both
with mono 4.4.0 and
mono-4.6.0.245 (built from
GitHub on the 746756c
commit.
I originally discovered
the issue running a
Plastic SCM server on BananaPi
(like a Raspberry + SATA).
== Extra notes
==
* I know the
multi-threaded server is
not optimal and
thread-per-connection is
really bad, this is just a
test case.
* Initially I thought we
had an issue with async
sockets, but after a few
days I just isolated the
problem in NetworkStream.
Any hints would be
appreciated.
Thanks,
pablo
www.plasticscm.com
_______________________________________________
Mono-devel-list mailing list
Mono-devel-list@lists.dot.net
http://lists.dot.net/mailman/listinfo/mono-devel-list
|