We were happily using PiCloud for several long calculations and we very happy 
with with it. With their realtime cores, we could take really large 
calculations set and run through fairly quickly.

Now that PiCloud is going away, we ran a few tests on Mutlyvac but so far, we 
are struggling to accomplish the same thing we had on PiCloud.

I have several "pieces" of my puzzle but can't seem to be able to put it 
together. I've seen and tried StarCluster and also various parallel python 
options but all options seem challenging to put together.

The goal is to mimic PiCloud, ie. loop through a function:

def some_NP_func(x, y):
   return z

some_cloud.call(some_NP_func, a1, a2)

Which computes the function on the cloud. We use this often in for loops with 
arrays of arguments. The other scenario is:

some_cloud.map(some_NP_intense_func, [...], [...])

Which iterates through and returns results. We need to run a lot of this in 
batch from a scheduler so I always try to avoid interactive environment (how 
does iPython parallel work in batch?).

What is the preferred approach or method right now for heavy parallel 
computation like this?


Reply via email to