Hi Gus, > How do you intend to use Rmpi?
The problem I want to solve/attack is in highly embarrasingly parallel nature. According to a couple of brief tutorials I read, Rmpi favors LamMpi and, as I understand, even with emergence of openMPI it hasn't been modified in the way to work with openMPI. What I want to do, in concept, is to spawn a master and a bunch of slaves, and have master issue appropriate R commands/functions to slaves. I have read some R users (including Rmpi author) tell that it does work with openMPI, though I cannot get really critical information. In some cases, I find completely erroneous information as well. That's a bit discouraging, but... If I cannot get Rmpi to work with openMPI, I am planning to do MPMD with openMPI via mpirun command. There is an open-source software called r (pronounced little-r) which provides hash-bang capability for R and it is nice for command line and piping use. I am hoping that I can use r with mpirun. My long-term goal is to launch a bunch of instances on cloud and do the (embarrasingly) parallel computing. Ie., master issues R directives to slaves and when all slaves finish it (the master) collects output from each slave run and put them together. The cost of using Amazon's HPC cluster is much higher than that of non-hpc machines and since my problem is embarrassingly parallel, I don't really need high-speed inter-node communications. I have heard words like Torque, Slurg(?) that I can use to do mpi resource control, but I don't know enough about mpi and don't want to bite more than I can chew. What I want to do at the moment is to get to know openMpi to the best I can. Tena Sakai tsa...@gallo.ucsf.edu On 1/12/11 6:30 PM, "Gus Correa" <g...@ldeo.columbia.edu> wrote: > Tena Sakai wrote: >> Thank you, Gus. >> >> I agree with what you say about location of OpenMPI software. >> Indeed, /usr/local is nfs-mounted to all cluster nodes, albeit >> a bit unfortunate name "local." If/when I have a chance to >> Set up machines, I will make local really local to each node. >> >> Regards, >> >> Tena Sakai >> tsa...@gallo.ucsf.edu >> >> >> On 1/12/11 4:20 PM, "Gus Correa" <g...@ldeo.columbia.edu> wrote: >> >>> Tena Sakai wrote: >>>> Hi, >>>> >>>> On a FAQ page ( >>>> http://www.open-mpi.org/faq/?category=running#adding-ompi-to-path), >>>> (under 1. What pre-requisites are necessary for running an Open MPI >>>> job?), I see an example >>>> of LD_LIBRARY_PATH environment variable: >>>> LD_LIBRARY_PATH: /opt/openmpi/lib >>>> >>>> When I compiled the source, a directory /usr/local/lib/openmpi was >>>> generated (with many >>>> files in it). Given that, do I set D_LIBRARY_PATH to /usr/local/lib or >>>> do I set >>>> it to /usr/local/lib/openmpi? >>>> >>>> Thank you. >>>> >>>> Tena Sakai >>>> tsa...@gallo.ucsf.edu >>>> >>> To /usr/local/lib. >>> >>> I would suggest not using the default /usr/local to install OpenMPI, >>> since it tends to be really local to the machine where you built OpenMPI. >>> This will require that you install OpenMPI on all nodes/machines if >>> you want to run programs across a network. >>> >>> Instead, a simpler way to get OpenMPI available to all nodes, >>> although installing only on one of them (say the head node of your >>> cluster) is to do it in a directory that is shared, typically via NFS. >>> To do so, use the --prefix=/my/shared/OpenMPI/directory option of >>> the configure script. >>> >>> There are FAQs about this too. >>> >>> Anyway, it may depend on your environment also, whether it is a cluster >>> with a private subnet (where my suggestion is typically used), >>> a bunch of separate computers on a LAN (where the suggestion won't work >>> unless you have a shared NFS directory), or other. >>> >>> Gus Correa > Hi Tena > > Just out of curiosity, this is not really an OpenMPI issue, > and I don't know anything about Rmpi either: > > How do you intend to use Rmpi? > > Interactively without any resource manager control (Torque,SGI,etc), > as in a standalone PC, although utilizing several hosts via MPI? > > Interactively but under resource manager control? > (E.g., Torque has the -I directive for this type of thing, > and I guess other resource managers have similar mechanisms.) > > In batch mode, where a full R script runs and eventually exits? > (Which can be with or without a resource manager.) > > Maybe it is something else. > > Matlab (parallel or not), IDL, etc, just like R[mpi], > bring about the same type of questions for us here. > > Gus Correa > _______________________________________________ > users mailing list > us...@open-mpi.org > http://www.open-mpi.org/mailman/listinfo.cgi/users
