thanks for your post erik. could you please elaborate why Julia does not scale well to larger clusters?
also, what would it take to get Julia to utilize InfiniBand were it available. thanks, ben On Wednesday, October 1, 2014 12:26:11 PM UTC-4, Erik Schnetter wrote: > > For example, when running Julia on a cluster, there may be special > high-speed low-latency communication hardware such as InfiniBand > interconnects. Currently, Julia would not use these, but MPI would, > giving MPI an unfair advantage until Julia is using such hardware as > well. > > Similarly, Julia's current cluster manager should work fine and > efficiently if you are using (say) 10 workstations. However, if you > are using many more -- say 1000 -- then Julia's current cluster > manager implementation will not scale. Again, this is only a > limitation of the current implementation, not of the high-level API > that Julia is offering, and I'm sure this will be improved in time. >
