On Friday, 20 June 2014 at 08:05:01 UTC, Bienlein wrote:
What data load profile do you expect? Vibe is tuned to handle
thousands simultaneous incoming light requests (milliseconds),
while distributed computing works better with exclusive heavy
requests, at least minutes of work worth, BOINC uses hours
worth work items.
Communication will be bi-directional as some thread sends
computation along with the data to be computed to some remote
thread and somewhen waits for the answer. The amount of data
will be relatively small, that is only consisting of numbers to
be processed creating some result, because Julia is geared
towards scientific computing. In that way there is little data
compared to enterprise computing.
How often remote computations are initiated depends to a large
extend on the user. So that is hard to say. But from the
typical use case scenario it will be anything but few large
messages to go across the wire. I might have to do a bit more
research looking at Julia to get a better understanding. But
it's all about performance, e.g. crunching as many numbers as
quickly as possible. So I will have to stick to a
high-performance solution anyway. For that reason it looks more
like Swift or ZeroMQ. Depends a lot which one can do
marshalling and unmarshalling of the function invocation and
the data more transparently than the other.
application performance != networking performance. Sounds like in
your use case actual communication overhead has low impact
compared to actual computations done remotely. In that case
vibe.d solution can actually be very helpful because it will
provide async communication model out of the box (your nodes will
be able to do some other useful work while waiting for responses
from other nodes).
