Patches to parallel could certainly be *considered*, but people will
probably want to argue about the details quite a bit. I would suggest
presenting/discussing your ideas for modification on r-devel before
posting them as a 'wishlist' item to the R bug tracker. (If an R-core
member participates in the discussion on r-devel and is convinced of the
utility and harmlessness of your modifications the last step might be
unnecessary).
cheers
Ben Bolker
On 7/23/21 1:17 PM, David Norris wrote:
My understanding about this issue with progressr is that it is nontrivial, and will
take author Henrik Bengtsson some time & thought to resolve.
OTOH if patches to 'parallel' were welcome, I could embed my solution in a
modified version of parallel::mclapply by introducing 1 or 2 new parameters
(e.g., mc.prog=NULL) to support progress reporting consistently with the
existing interface.
From: Duncan Murdoch <murdoch.dun...@gmail.com>
Date: Friday, July 23, 2021 at 11:26 AM
To: Hugh Parsonage <hugh.parson...@gmail.com>, David Norris
<da...@precisionmethods.guru>
Cc: "r-package-devel@r-project.org" <r-package-devel@r-project.org>
Subject: Re: [R-pkg-devel] Using parallel:::sendMaster &c. despite warnings
On 23/07/2021 10:59 a.m., Hugh Parsonage wrote:
Happily, package parallel is open-source. If you would like to make
use of the unexported functions, you could copy the source code with
acknowledgement into your package.
There's a good chance that wouldn't work, because parallel is a base
package. Base packages can work with R internals and user-written
packages can't.
Probably David's best course of action is to submit patches where
necessary so that he doesn't need ::: access. The original motivation
appeared to be avoiding inefficiency in a contributed package; if
there's a patch that can fix that, it could be available in a few days.
If it needs changes to the parallel package (e.g. exposing functions
that are currently internal), that will take much longer, and needs a
strong argument why the current API isn't sufficient.
Alternatively, he can go ahead and use :::, but just not expect his
package to be on CRAN. There are other ways to distribute packages.
Duncan Murdoch
Naturally this might be a lot of
work and risks different behaviour between your package and parallel,
but neither might be that terrible for you. And certainly would be
better than trying to pretend that the functions you want are exported
from parallel.
On Fri, 23 Jul 2021 at 23:09, David Norris <mailto:da...@precisionmethods.guru>
wrote:
I can confirm that getFromNamespace() does indeed circumvent the WARNING.
I might otherwise prefer ':::', however, for making its 'bad intentions' so
clear.
I was very much aware of the cautionary language under `?readChild`, but
wondered whether *in practice* more liberal community policies might be in
force pending the maturation of the futureverse.
Although the sparse history of the relevant file hardly supports
"might-break-at-any-time" alarmism, I do note that the last commit (in May
2020) introduced an argument sendMaster(raw.asis=TRUE) that I am actually using.
https://github.com/wch/r-source/commits/trunk/src/library/parallel/R/unix/mcfork.R
(Also agree with Yohann that ideally one might support Windows with a warning
such as data.table issues regarding my Mac's non-support for OpenMP.)
Best, David
From: Sebastian Meyer <mailto:seb.me...@fau.de>
Organization: Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU)
Date: Friday, July 23, 2021 at 8:05 AM
To: David Norris <mailto:da...@precisionmethods.guru>,
"mailto:r-package-devel@r-project.org"; <mailto:r-package-devel@r-project.org>
Subject: Re: [R-pkg-devel] Using parallel:::sendMaster &c. despite warnings
Am 23.07.21 um 13:19 schrieb David Norris:
Because parallelized progress reporting in the futureverse.org incurs latencies
too great for my application
(https://github.com/HenrikBengtsson/progressr/issues/118), I have found it
necessary to implement my own progress reporting using some of the non-exported
functionality from `parallel`. (I do appreciate that Windows lacks the fork()
system call, and will not support this. But am willing to make this an OS_type:
unix-only package.)
Of course, I get a WARNING for this:
── R CMD check results ──────────────────────────────── precautionary 0.2.6 ────
Duration: 6m 41.8s
❯ checking dependencies in R code ... WARNING
Unexported objects imported by ':::' calls:
‘parallel:::readChild’ ‘parallel:::selectChildren’
‘parallel:::sendMaster’
See the note in ?`:::` about the use of this operator.
Including base/recommended package(s):
‘parallel’
Is this warning an absolute deal-killer on CRAN? Is there a 'correct' way to do
`:::` that avoids the warning altogether?
The 'parallel' functions your package intends to access seem to be
intentionally unexported. Their help page says: "They are not available
on Windows, and not exported from the namespace", and "This is a very
low-level interface for expert use only: it not regarded as part of the
R API and subject to change without notice."
Correspondingly, the CRAN Repository Policy says
Also, ::: should not be used to access undocumented/internal objects in base
packages (nor should other means of access be employed). Such usages can cause
packages to break at any time, even in patched versions of R.
which kind of answers both of your questions. The policy thus implicitly
advises against using getFromNamespace().
Best regards,
Sebastian Meyer
Kind regards,
David Norris
______________________________________________
mailto:R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel
______________________________________________
mailto:R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel
______________________________________________
mailto:R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel
______________________________________________
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel
______________________________________________
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel
