[moved to R-devel]
I am a bit concerned by the fact that package developers would be removing all tests. The tests seem like one of the main mechanisms for ensuring that CRAN does not descend to the almost useless state of statlib: there is an automatic mechanism for removing packages that no longer work.
I have several tests that I don't put in packages, because they are very long or they don't work without locally available databases, but I think it is important that there should be tests that check the main functionality of packages. For very long tests like monte carlo simulations it is usually fairly easy to devise some short version (a few iterations) which checks that the software actually works and gives results that are the same as they always have been, even though it does not check the more serious statistical problems one would actually want to do.
Perhaps there should be another directory, called something like longTests, that is not run daily but is run occasionally.
On r-help I suggested that users might contribute tests to package developers, but another possibility is that there is some mechanism for them to contribute suites of package tests directly to CRAN. This would provide a mechanism for more complete checking of packages, and perhaps a just system for removing packages when the developer is no longer active and the package is broken.
Paul Gilbert
Uwe Ligges wrote:
Paulo Justiniano Ribeiro Jr wrote:
One problem with distributing packages with the test sub-directory is that this can overload the daily tests in the CRAN machines
My workaround for that is: For the geoR package I run the tests in my machine but remove the tests directory when submitting to CRAN. Also I mantain a package web-page where I make the version with tests available for downloading
[moved to R-devel, chosen a more sensible subject]
If tests are running in a reasonable amount of time, it does not make sense to remove them. People want to install the package on a huge number of platforms you probably have not tested the package on yourself. The only solution is to distribute the tests as well.
Overloading the daily tests on CRAN is another issue (strong related, though). Installing and checking (in maintainer's mode, i.e. without double-installing) all CRAN packages under R-release for Windows takes 8 hours on a (dual, but using only one) Xeon 3.06GHz machine these days.
The checks (without isntallation) of the 15 packages with most intensive tests take ~1 hour.
Uwe Ligges
P.J.
On Tue, 1 Feb 2005, Spencer Graves wrote:
Hi, Paul:
How can I access "the package tests directory" you mentioned? Only one of the 52 subdirectories of "library" in my current installation of R 2.0.1 has a "test" folder.
Thanks, Spencer Graves
Paul Gilbert wrote:
One point that did not get mentioned in this discussion, and I believe
deserves
much more publicity, is the impact of packages tests. The design of
the package
system allows package developers to put tests in packages, and these
are checked
regularly (see <http://cran.at.r-project.org/contrib/checkSummary.html>).
These are intended to test the package functionality, but also give R
what is
perhaps the largest test suite of any statistical software (certainly
the most
quickly growing). While any single package's test will never guarantee
that
the package works perfectly, the ensemble goes a long way toward ensuring
that core R functionality behaves as intended. It seems unlikely to me
that any
commercial effort will ever be able to catch up.
There are several ways that tests can add to our confidence that calculations can be trusted. They can - check against theoretical results - check against published results - check against results from other software - check that calculations done in different ways give the same result - check that monte carlo experiments give distributions that are consistent with expected results
Some of these are relatively time consuming to set up and check the first time, but after that they can be automatic.
If you have particular calculations with specific packages that you are especially concerned about, I encourage you to participate by devising good tests and sending them to the package developers. (But first check the tests they are already doing in the package tests directory.)
Paul Gilbert
[EMAIL PROTECTED] wrote:
Dear all, I am beginner using R. I have a question about it. When you use it, since it is written by so many authors, how do you know that the results are trustable?(I don't want to affend anyone, also I trust people). But I think this should be a question.
Thanks, Ming
______________________________________________ R-help@stat.math.ethz.ch mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
______________________________________________ R-help@stat.math.ethz.ch mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
______________________________________________
R-help@stat.math.ethz.ch mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
Paulo Justiniano Ribeiro Jr LEG (Laboratório de Estatística e Geoinformação) Departamento de Estatística Universidade Federal do Paraná Caixa Postal 19.081 CEP 81.531-990 Curitiba, PR - Brasil Tel: (+55) 41 361 3573 Fax: (+55) 41 361 3141 e-mail: [EMAIL PROTECTED] http://www.est.ufpr.br/~paulojus
______________________________________________
R-help@stat.math.ethz.ch mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
______________________________________________ R-devel@stat.math.ethz.ch mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
