Dear Professor Ripley,
PkgB does not need to be on the search path---importing into the namespace
is fine. I did not realize that namespace scoping ensured that if a
cluster is created from within a package, that packages entire environment
tree is available on all the workers.
I tried to apply how makeCluster works from an interactive R session, where
functions from packages that are loaded when the cluster is created are not
available on the workers, to how it works from within a package.
Thanks for your reply,
Josh
On Fri, Aug 8, 2014 at 4:47 PM, Prof Brian Ripley <rip...@stats.ox.ac.uk>
wrote:
> The safe, elegant way to do this is to use namespace scoping: it is still
> not at all clear why 'other code' needs PkgB *on the search path*.
>
> In other cases seen in CRAN submissions, 'other code' has been in PkgA's
> namespace, and hence things in PkgB's exports have been visible as it was
> imported by PkgA and hence in the environment tree for functions in PkgA.
> Then namespace scoping will ensure that PkgB's namespace is loaded on the
> cluster workers.
>
>
>
> On 08/08/2014 00:58, Joshua Wiley wrote:
>
>> Someone kindly pointed out that it is not clear from my email why Depends
>> will not work. A more complete example is:
>>
>> PkgA:
>> f <- function(ncores) {
>> cl <- makeCluster(ncores)
>>
>> clusterEvalQ(cl, {
>> require(PkgB)
>> })
>> [other code]
>>
>> ### this is the code I want to work and need to be able to call
>> ### PkgB functions on each of the cluster slaves
>> output <- parLapply(cl, 1:n, function(i) {
>> [code from my package and using some functions from PkgB]
>> })
>>
>> }
>>
>> As far as I know, just because I add PkgB to the Depends (or imports,
>> whatever) of PkgA, does not mean that the cluster started by PkgA will
>> automatically have PkgB loaded and functions available.
>>
>> Thanks!
>>
>>
>>
>> On Fri, Aug 8, 2014 at 9:35 AM, Joshua Wiley <jwiley.ps...@gmail.com>
>> wrote:
>>
>> Dear All,
>>>
>>> What is the preferred way for Package A, to initialize a cluster, and
>>> load
>>> Package B on all nodes?
>>>
>>> I am writing a package that parallelizes some functions through the use
>>> of
>>> a cluster if useRs are on a Windows machine (using parLapply and family).
>>> I also make use of another package in some of my code, so it is
>>> necessary
>>> to load the required packages on each slave once the cluster is started.
>>>
>>> Right now, I have done this, by evaluating require(packages) on each
>>> slave; however, Rcmd check has a note that I should remove the "require"
>>> in
>>> my code.
>>>
>>> Thanks!
>>>
>>> Josh
>>>
>>> --
>>> Joshua F. Wiley
>>> Ph.D. Student, UCLA Department of Psychology
>>> http://joshuawiley.com/
>>> Senior Analyst, Elkhart Group Ltd.
>>> http://elkhartgroup.com
>>> Office: 260.673.5518
>>>
>>>
>>
>>
>>
>
> --
> Brian D. Ripley, rip...@stats.ox.ac.uk
> Professor of Applied Statistics, http://www.stats.ox.ac.uk/~ripley/
> University of Oxford, Tel: +44 1865 272861 (self)
> 1 South Parks Road, +44 1865 272866 (PA)
> Oxford OX1 3TG, UK Fax: +44 1865 272595
>
--
Joshua F. Wiley
Ph.D. Student, UCLA Department of Psychology
http://joshuawiley.com/
Senior Analyst, Elkhart Group Ltd.
http://elkhartgroup.com
Office: 260.673.5518
[[alternative HTML version deleted]]
______________________________________________
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
