On 2020-12-12 19:50, Duncan Murdoch wrote:
On 12/12/2020 6:01 p.m., Ben Bolker wrote:
On 12/12/20 5:50 PM, Duncan Murdoch wrote:
On 12/12/2020 4:08 p.m., Spencer Graves wrote:
Hi, Ben et al.:
On 2020-12-12 13:43, Ben Bolker wrote:
Apologies if I'm telling you something you already know:
By default, fda::CRAN() uses the presence of environment
variables
matched by the regexp "^_R_" as a heuristic to decide whether it's
being
running on CRAN.
testthat::skip_on_cran() calls testthat::on_cran() to look
for an
environment variable called NOT_CRAN equal to "true". The
devtools::check() machinery automatically sets this variable.
> testthat::on_cran
Error: 'on_cran' is not an exported object from 'namespace:testthat'
Besides, on my Mac, I get:
> testthat:::on_cran()
[1] TRUE
My Mac is NOT CRAN, and I don't want that function to return
TRUE on
my computer unless I explicitly run "R CMD check --on-cran".
So: fda::CRAN() depends on breakable assumptions, defaults to
FALSE
in an empty environment. skip_on_cran() defaults to TRUE in an empty
environment (but defaults to FALSE in a devtools::check()
environment).
If future changes break fda::CRAN, I will have to deal with it
then.
I'd be happier if the CRAN maintainers would develop a
procedure to
make it easier for package maintainers do two things:
* Include tests in their package that run longer than the
time
limit permitted on CRAN.
That's very easy now. Just put them in a "slowtests" directory, and
tell R CMD check to use that. How could it be easier?
How would you do that? In "R CMD check --help" I see that one can
use --test-dir= to specify the test directory, but I don't see a way to
specify _additional_ test directories; short of setting a tests/
directory with CRAN-specific tests and a slowtests/ directory with
*both* CRAN-specific and CRAN-excluded tests (thus duplicating files,
which seems clunky), I don't see how to do this within a standard R CMD
check framework (without testing a CRAN-indicating environment variable,
which gets us back where we started ...) Or would you run R CMD check
twice, once without and once with --test-dir=slowtests ?
What I would do is have the slowtests run the regular tests. So if I
want both, I run slowtests. If I want just the fast ones, I don't
specify. I can't think why I wouldn't want to run the slow ones without
the fast ones, but if I did, it's not too hard to figure out a scheme
that runs fast by default, slow when requested, and both if you request
that instead.
There doesn't seem to be very much in "Writing R Extensions" about
testing - a little bit in
https://cran.r-project.org/doc/manuals/r-release/R-exts.html#Package-subdirectories
What am I missing?
Just to clarify, the ideal would be to be able to designate a separate
set of tests that were *not* run on CRAN, and to be able to run them in
the same "R CMD check" pass as the CRAN-specific tests.
Yes, do that as described above.
There are a
bunch of ways to achieve this, but I think Spencer is saying (and I
agree) that it would be nice if it were there an official mechanism that
made this easier (and it seems pretty easy if the CRAN maintainers were
agreeable to the idea ...
There is such a mechanism, and I've just described it (and not for the
first time; it's also described in WRE). I think the problem is that
you and Spencer are looking for something that's more complicated. It
doesn't need to be complicated.
I want to put all the tests of a particular function in the
"\examples" section. If some things are too pedantic to show to a user,
I can put them in "\dontshow". If they run too long for CRAN, I wrap
them in "if(!fda::CRAN()){...}", as I previously noted.
Putting slow tests in a "slowtest" directory to me violates a
sensible rule of documentation, because it makes it harder to think
about how complete a test suite is.
I probably should not broaden this discussion to include "\dontrun",
but I will: I think any example in "\dontrun" should be made to work
and wrapped in something like "if(!fda::CRAN()){...}" if you don't want
it to be run on CRAN, where you don't care if it breaks or not. I've
read too many books with examples that didn't work! The "fda" package
has 76 reverse dependencies. I think most of those are attributable to
the quality of the fundamental ideas, but I'd like to think that some of
them are because I insisted in included decent unit tests in the
"\examples" -- AND because I insisted on make sure all but a couple of
the examples in the book actually worked!
Thanks very much to everyone who has contributed to this thread. I
don't think we've reached a consensus, but we've had a good discussion
and may eventually help improve package documentation and testing
practices in the future.
Spencer
Duncan Murdoch
______________________________________________
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel
______________________________________________
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel
