Re: [Factor-talk] concurrency.distributed under Windows

[Alexander Ilin]

Hello! 08.01.2018, 02:41, "John Benediktsson" :If you look at ``M\ remote-thread send``, it uses ``send-remote-message`` which opens a TCP connection using ``with-client``, which sends a message, then closes the connection. The code is 10's of lines to implement concurrency.distributed so I'm sure you could modify it in minor ways to be specific to your application and maybe useful to contribute back.   Thank you for believing in me! I think I'll give it a shot while I still have some holiday time left. If you need to get more familiar with network programming, I recommend Beej's guide: https://beej.us/guide/bgnet/ Huh? I'd better read the ZeroMQ manual, seems roughly the same volume. ---=---Александр --
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
Factor-talk mailing list
Factor-talk@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/factor-talk

Re: [Factor-talk] concurrency.distributed under Windows

[John Benediktsson]

If you look at ``M\ remote-thread send``, it uses ``send-remote-message``
which opens a TCP connection using ``with-client``, which sends a message,
then closes the connection.

For a high-throughput messaging application, you would probably want to
re-use a TCP connections, or use UDP, and probably allow for out-of-order
messages since you could have different synchronous responses taking
different lengths of time from client to server to client.

I'm not surprised that a busy loop is causing problems because TCP
connections aren't intended to be spammed like that without causing
resource constraints and potentially exposing some bugs in handling those
situations.

The code is 10's of lines to implement concurrency.distributed so I'm sure
you could modify it in minor ways to be specific to your application and
maybe useful to contribute back.

If you need to get more familiar with network programming, I recommend
Beej's guide:

https://beej.us/guide/bgnet/

Best,
John.

On Sun, Jan 7, 2018 at 4:41 AM, Alexander Ilin  wrote:

> > So, it doesn't seem to be a runaway buffer issue, unless Factor does
> something completely wrong, like peeking in the buffer and never removing
> messages from it.
>
>   Also, is it OK that I'm seeing hundreds of TCP connections being made
> between the processes?
>   It seems like every individual message is sent in a separate connection,
> which brought up and then torn down instantly.
>
>   I would expect a longer-lived TCP channel to be established, but what do
> I know about how networks (and Factor) work...
>
> ---=---
>  Александр
>
> 
> --
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> ___
> Factor-talk mailing list
> Factor-talk@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/factor-talk
>
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
Factor-talk mailing list
Factor-talk@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/factor-talk

Re: [Factor-talk] concurrency.distributed under Windows

[Alexander Ilin]

> So, it doesn't seem to be a runaway buffer issue, unless Factor does 
> something completely wrong, like peeking in the buffer and never removing 
> messages from it.

  Also, is it OK that I'm seeing hundreds of TCP connections being made between 
the processes?
  It seems like every individual message is sent in a separate connection, 
which brought up and then torn down instantly.

  I would expect a longer-lived TCP channel to be established, but what do I 
know about how networks (and Factor) work...

---=--- 
 Александр

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Factor-talk mailing list
Factor-talk@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/factor-talk

Re: [Factor-talk] concurrency.distributed under Windows

[Alexander Ilin]

Hello!

07.01.2018, 07:21, "Chris Double" :
> On Sat, Jan 6, 2018 at 1:18 PM, Alexander Ilin  wrote:
>>  It seems that I overloaded the system somehow, but I don't see me doing 
>> anything illegal. Can you guys help me figure out what's going on?
>
> I suspect you are overloading the network stack on your machine since
> you are sending data as fast as possible and receiving it as fast as
> possible. Maybe the sender is getting ahead of the receiver and
> queuing more data in the network stack over time. Try adding a delay
> in the sender and see if that improves things.

Thank you for the suggestion!

OK, may be the sender was getting ahead of the receiver and overflooded the 
network stack buffers in the previous example. (I would have expected a 
different error message text in that case, but maybe that's what you get.) Here 
is a modified version of the same code, this time using send-synchronous and 
reply-synchronous. There is no way to overflood the network buffers in this 
code, because the two processes are moving in lockstep, exchanging one message 
at a time and waiting for the other to respond before sending anything.

Factor instance 1 (codename "Blackhole"):

USING: concurrency.distributed concurrency.messaging io.servers threads ;
9010 local-server start-node [
[ receive t swap reply-synchronous t ] loop
] "black-hole" [ spawn ] keep register-remote-thread

Factor instance 2 (codename "Pulsar"):

USING: concurrency.distributed concurrency.messaging io.servers ;
9011 local-server start-node ! a port to receive synchronous replies
9010 local-server "black-hole" 
[ f swap send-synchronous drop t ] curry loop

Unfortunately, the resulting behavior is the same. For about 14 seconds both 
instances run at 100% CPU, then both run at about 50% CPU, and after 60 seconds 
from the experiment start the same error string is thrown by the Pulsar:
"You were not connected because a duplicate name exists on the network. If 
joining a domain, go to System in Control Panel to change the computer name and 
try again. If joining a workgroup, choose another workgroup name."

So, it doesn't seem to be a runaway buffer issue, unless Factor does something 
completely wrong, like peeking in the buffer and never removing messages from 
it.

---=--- 
 Александр

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Factor-talk mailing list
Factor-talk@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/factor-talk