Re: [R] Status of Rmpi--Good with tweaks
On 24 Mar 2004, Ross Boylan outgrape: > By the way, I'm in the dark about (r)sprng. From some of the snow docs, > I thought that was really snow's business. But your site notes that the > Rmpi packages are withough SPRNG support. And Tony Rossini's pages (I > think) had a slightly cryptic remark that SPRNG under MPI was a bit > dicey. rsprng works like this: each process calls the function init.sprng, with three pieces of information: 1) the total number of processes; 2) the rank of current process; 3) a seed that is the same for all processes. An alternative is that the master generates say ten streams, and send one stream to each slave (and keep track of who gets what, to ensure reproducibility). Only very little (and simple) inter-process communication is required for SPRNG. The communication layer is orthogonal of SPRNG itself. rsprng can be used with Rmpi just fine. (SPRNG can be used in serial code as well.) If you look at SPRNG source, it has some sort of MPI support, which is disabled when I compile SPRNG. See this note: http://www.biostat.umn.edu/~nali/RsprngNotes.php Cheers, Michael __ [EMAIL PROTECTED] mailing list https://www.stat.math.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
Re: [R] Status of Rmpi--Good with tweaks
Ross Boylan <[EMAIL PROTECTED]> writes:
> By the way, I'm in the dark about (r)sprng. From some of the snow docs,
> I thought that was really snow's business. But your site notes that the
> Rmpi packages are withough SPRNG support. And Tony Rossini's pages (I
> think) had a slightly cryptic remark that SPRNG under MPI was a bit
> dicey.
SPRNG doesn't need to be invoked via MPI, but can be configured to use
it as the invocation/transport layer.
It shouldn't matter to SNOW in the least, or rpvm. I think it
shouldn't matter to Rmpi, but we won't have a cluster to play with for
a month or so.
Luke -- I've seen the gentle hints.
best,
-tony
--
[EMAIL PROTECTED]http://www.analytics.washington.edu/
Biomedical and Health Informatics University of Washington
Biostatistics, SCHARP/HVTN Fred Hutchinson Cancer Research Center
UW (Tu/Th/F): 206-616-7630 FAX=206-543-3461 | Voicemail is unreliable
FHCRC (M/W): 206-667-7025 FAX=206-667-4812 | use Email
CONFIDENTIALITY NOTICE: This e-mail message and any attachme...{{dropped}}
__
[EMAIL PROTECTED] mailing list
https://www.stat.math.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
Re: [R] Status of Rmpi--Good with tweaks
I have some more good news and some questions. On Tue, 2004-03-23 at 20:50, Hao Yu wrote: > Sorry. I have not been able to update Rmpi since the version > 0.4-4 on R site. I don't think any version of Rmpi is on the R site at the moment. Minor aside: Also, it would be nice if the packages starting with "R" were consistent about whether it's "R" or "r." Rmpi but rpvm is a little dissonant. Unfortunately, there seems to be no agreement. > However, I have been using and testing Rmpi > internally since 0.4-4. Now it is version 0.4-7. See the > attached package. It requires R 1.8.1 (no more serialize > package requirement) and works with the newest MPI-LAM > versions 7. It will configure automatically if a rpm package > from www.lam-mpi.org is used. I tested it on both Redhat 9 > and Debian. At least it passed the package check without any > warning on Redhat 9. Now the problem is that it may have > some problems with the default lam coming with Debian > system. This is the reason why I hesitate to release it to > R. I'm happy to report the Rmpi 0.4-7 installs and works without a problem (or any necessary modification) on my Debian mostly testing system (it does have some unstable stuff, but in particular it has the older lam/mpi 6.5.8-2. version 7 has been held in unstable for 60 days because of some problems on alpha hardware). Note this is not quite the lam in the current stable Debian distro, which is at 6.5.6-6. > > I am also working on it to see if #ifdef can be used to work > MPI 1.2 specs other than LAM-MPI. > Hopefully in a couple of months, I am able to submit a > stable version to R. Great. Thanks. I think it would be well worth making 0.4-7 available at http://www.stats.uwo.ca/faculty/yu/Rmpi/. I'll send you, off list, the hacks I made to get the thing (starting with 0.4-6) to compile with the MPI 1.2ish mpich. It didn't run, but at least it did compile and load at the end. By the way, I'm in the dark about (r)sprng. From some of the snow docs, I thought that was really snow's business. But your site notes that the Rmpi packages are withough SPRNG support. And Tony Rossini's pages (I think) had a slightly cryptic remark that SPRNG under MPI was a bit dicey. > > Regards, > > Hao > > PS: Rmpi should still work without serializing as long as > native MPI calls are used. The serializing is mainly used to > help moving an arbitrarily R object around. I don't completely follow that, since I don't control whether or not native MPI calls are used, as far as I know. Do you mean that Rmpi can be used alone OK, but if you use snow (or, I guess Rmpi calls oriented toward snow and the transmission of R objects) it will need the serialize facility (prior to R 1.8)? __ [EMAIL PROTECTED] mailing list https://www.stat.math.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
Re: [R] Status of Rmpi--Good with tweaks
Thanks--I'll try this new version when I get back to working on
this--hopefully soon.
luke
On Tue, 23 Mar 2004, Hao Yu wrote:
> Sorry. I have not been able to update Rmpi since the version
> 0.4-4 on R site. However, I have been using and testing Rmpi
> internally since 0.4-4. Now it is version 0.4-7. See the
> attached package. It requires R 1.8.1 (no more serialize
> package requirement) and works with the newest MPI-LAM
> versions 7. It will configure automatically if a rpm package
> from www.lam-mpi.org is used. I tested it on both Redhat 9
> and Debian. At least it passed the package check without any
> warning on Redhat 9. Now the problem is that it may have
> some problems with the default lam coming with Debian
> system. This is the reason why I hesitate to release it to
> R.
>
> I am also working on it to see if #ifdef can be used to work
> MPI 1.2 specs other than LAM-MPI.
> Hopefully in a couple of months, I am able to submit a
> stable version to R.
>
> Regards,
>
> Hao
>
> PS: Rmpi should still work without serializing as long as
> native MPI calls are used. The serializing is mainly used to
> help moving an arbitrarily R object around.
>
> Ross Boylan wrote:
> >
> > On Tue, 2004-03-23 at 18:26, Ross Boylan wrote:
> > > On Tue, 2004-03-23 at 17:35, Luke Tierney wrote:
> > > > The serialize package should no longer be needed since the
> > > > functionality is now in R itself. I haven't run snow with a new
> > > > version of Rmpi newer than 0.4-4; with that version things worked on
> > > > my systems the last time I tried. We need to revize the paper you
> > > > cite; when we get to that we'll hopefully have a chance to give a
> > > > newer version of Rmpi a go.
> > > >
> > > > Best,
> > > >
> > > > luke
> > >
> > > I got 0.4-6 to work, with one tweak to eliminate the loading of
> > > serialize and one fix of my own setup.
> > >
> > > I changed the .First.Lib code in Rmpi to be
> > > if (!exists("serialize") && !require(serialize))
> > > stop("serialize package cannot be loaded. Exit")
> > >
> > > !exists("serialize") && is new. This is effort to check if the
> > ^an
> > > serialize function exists (R 1.8) and skip loading the library in that
> > > case. Since there are various ways this might be fooled, perhaps an
> > > explicit check of the R version would be better.
> > >
> > > I'm also not sure if the R packaging mechanism makes this easy, or
> > "this" refers to the modification to the .First.Lib that I made.
> > > autogenerates it. This is a bit of a tricky situation, since serialize
> > > is required for some versions of R but should not be used for the most
> > > recent.
> > >
> > > I suspect that without this the cluster would not come up because Rmpi
> > > would not load properly. I have not tested that.
>
>
--
Luke Tierney
University of Iowa Phone: 319-335-3386
Department of Statistics andFax: 319-335-3017
Actuarial Science
241 Schaeffer Hall email: [EMAIL PROTECTED]
Iowa City, IA 52242 WWW: http://www.stat.uiowa.edu
__
[EMAIL PROTECTED] mailing list
https://www.stat.math.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
Re: [R] Status of Rmpi--Good with tweaks
On Tue, 2004-03-23 at 18:26, Ross Boylan wrote:
> On Tue, 2004-03-23 at 17:35, Luke Tierney wrote:
> > The serialize package should no longer be needed since the
> > functionality is now in R itself. I haven't run snow with a new
> > version of Rmpi newer than 0.4-4; with that version things worked on
> > my systems the last time I tried. We need to revize the paper you
> > cite; when we get to that we'll hopefully have a chance to give a
> > newer version of Rmpi a go.
> >
> > Best,
> >
> > luke
>
> I got 0.4-6 to work, with one tweak to eliminate the loading of
> serialize and one fix of my own setup.
>
> I changed the .First.Lib code in Rmpi to be
> if (!exists("serialize") && !require(serialize))
> stop("serialize package cannot be loaded. Exit")
>
> !exists("serialize") && is new. This is effort to check if the
^an
> serialize function exists (R 1.8) and skip loading the library in that
> case. Since there are various ways this might be fooled, perhaps an
> explicit check of the R version would be better.
>
> I'm also not sure if the R packaging mechanism makes this easy, or
"this" refers to the modification to the .First.Lib that I made.
> autogenerates it. This is a bit of a tricky situation, since serialize
> is required for some versions of R but should not be used for the most
> recent.
>
> I suspect that without this the cluster would not come up because Rmpi
> would not load properly. I have not tested that.
__
[EMAIL PROTECTED] mailing list
https://www.stat.math.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
Re: [R] Status of Rmpi--Good with tweaks
On Tue, 2004-03-23 at 17:35, Luke Tierney wrote:
> The serialize package should no longer be needed since the
> functionality is now in R itself. I haven't run snow with a new
> version of Rmpi newer than 0.4-4; with that version things worked on
> my systems the last time I tried. We need to revize the paper you
> cite; when we get to that we'll hopefully have a chance to give a
> newer version of Rmpi a go.
>
> Best,
>
> luke
I got 0.4-6 to work, with one tweak to eliminate the loading of
serialize and one fix of my own setup.
I changed the .First.Lib code in Rmpi to be
if (!exists("serialize") && !require(serialize))
stop("serialize package cannot be loaded. Exit")
!exists("serialize") && is new. This is effort to check if the
serialize function exists (R 1.8) and skip loading the library in that
case. Since there are various ways this might be fooled, perhaps an
explicit check of the R version would be better.
I'm also not sure if the R packaging mechanism makes this easy, or
autogenerates it. This is a bit of a tricky situation, since serialize
is required for some versions of R but should not be used for the most
recent.
I suspect that without this the cluster would not come up because Rmpi
would not load properly. I have not tested that.
The other problem was of my own making. I had Rmpi in a local
directory, loaded with lib.loc. The children don't know about this, and
automatically try to load Rmpi. I solved this by setting .Renviron with
the proper R_LIBS path. (By the way, I wanted to say
R_LIBS=~/lib:${R_LIBS}, but that syntax doesn't work. This is
consistent with the documentation, just awkward.)
I also ran into a couple of snow things, really documentation issues,
that I'll put in a separate message.
__
[EMAIL PROTECTED] mailing list
https://www.stat.math.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
