On 02/07/2018 05:01 PM, Armen Abnousi wrote:
Hi Valerie,
Thanks for your reply. I can make it work using snow I believe. But the
problem is that all functions in these packages that you have suggested
(including snow; as far as I have seen) return the answer to one node and
then if I want all nodes to have the same answer I will need to broadcast
it to all nodes or repeat the same operation. This is very prohibitive,
requiring a lot of communication. On the other hand if I could use MPI
functionality, allGather would cut the operation to half (if not more,
given the size of the communication that will be needed. I haven't been
able to find a package with a functionality similar to allGather that will
work without MPI. With larger dataset sizes, this problem becomes more
threatening and that is just the opposite of what drives us to use
parallelism. I don't know if there is any other option, maybe we can't do
better, but I thought I'll let you know what my problem is and maybe you'll
suggest a better way to deal with it. Thank you again.
as a sub-optimal solution one could use the file system for data
transfer, sending tempfile() or app-specific
BiocFileCache::BiocFileCache() file paths between nodes; access could be
synchronized using BiocParallel::ipc* mutex.
Although the build system does not support MPI, sufficiently motivated
users might go to the trouble of installing it; you could make your
parallel code conditional on presence of MPI, defaulting to serial
(maybe with an encouraging message()) otherwise. MPI depedencies would
be moved to the Enhances: field, I think...
Martin
Thanks,
Armen
On Wed, Jan 31, 2018 at 2:37 PM, Obenchain, Valerie <
valerie.obench...@roswellpark.org> wrote:
Hi Armen,
MPI is not installed on the Windows builders so pbdMPI is not supported.
Was there a specific reason you chose pbdMPI? There are several other
options for parallel evaluation which are supported across all 3 platforms,
e.g., BiocParallel, doParallel and batchtools.
I took a brief look at what naddaR is importing from pdbMPI and didn't see
anything unusual - I believe you can find all functionality in one of the
alternatives. Please have a look and let us know if one of these will work
for you.
Thanks.
Valerie
On 01/27/2018 12:35 PM, Armen Abnousi wrote:
Hi,
I'm submitting a package for build, but for windows it goes to tokay2 and
fails during the build with the following message:
----------------------------------------------------------------------------------
* checking for file 'naddaR/DESCRIPTION' ... OK
* preparing 'naddaR':
* checking DESCRIPTION meta-information ... OK
* installing the package to build vignettes
-----------------------------------
* installing *source* package 'naddaR' ...
** R
** inst
** preparing package for lazy loading
Error : .onLoad failed in loadNamespace() for 'pbdMPI', details:
call: inDL(x, as.logical(local), as.logical(now), ...)
error: unable to load shared object
'C:/Users/pkgbuild/packagebuilder/workers/jobs/626/R-libs/pbdMPI/libs/x64/pbdMPI.dll':
LoadLibrary failure: The specified module could not be found.
ERROR: lazy loading failed for package 'naddaR'
* removing
'C:/Users/pkgbuild/AppData/Local/Temp/Rtmp4kcgED/Rinst1ffc2775216d/naddaR'
In R CMD INSTALL
-----------------------------------
ERROR: package installation failed
------------------------------------------------------------
-----------------------
How can I change this? It builds for Linux and Mac on Merida2 and Malbec2.
Thanks,
Armen
[[alternative HTML version deleted]]
_______________________________________________bioc-de...@r-project.org mailing
listhttps://stat.ethz.ch/mailman/listinfo/bioc-devel
This email message may contain legally privileged and/or confidential
information. If you are not the intended recipient(s), or the employee or
agent responsible for the delivery of this message to the intended
recipient(s), you are hereby notified that any disclosure, copying,
distribution, or use of this email message is prohibited. If you have
received this message in error, please notify the sender immediately by
e-mail and delete this email message from your computer. Thank you.
[[alternative HTML version deleted]]
_______________________________________________
Bioc-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/bioc-devel
This email message may contain legally privileged and/or...{{dropped:2}}
_______________________________________________
Bioc-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/bioc-devel
