If you add processes on all of those machines, pmap will just work. Is that what you're thinking of?
On Fri, Jun 13, 2014 at 2:28 PM, John Drummond <[email protected]> wrote: > Hi, > Are there any recommendations for running embarrassingly parallel problems > across multiple windows machines? > > I have to do the same piece of work on multiple multiple GB files - which > takes hours on a single machine spun out across the cores with parallel map. > > I'm lucky enough to have access to several windows machines and also a > couple of debian ones and was wondering what the best way to split the job > across them > > I've copied the source data across all the machines. > > Is there some equivalent of pmap that anyone has running across multiple > machines (e.g. with ssh or mpi)? > > or should I just write my own start up and wind down scripts and in that > split the work into chunks for each machine communicating via ssh or > whatever? > > Set up linux virtual machines and then use the current addprocs with extra > machines? > > Thanks for any thoughts. > > >
