On 21/08/2023 14:34, Dirk Eddelbuettel wrote:
On 21 August 2023 at 15:16, Ivan Krylov wrote:
| On Mon, 21 Aug 2023 12:02:55 +0100
| Scott Ritchie <sritchi...@gmail.com> wrote:
|
| > remotes::install_github("sritchie73/ukbnmr")
| > library(ukbnmr)
| > system.time({ remove_technical_variation(test_data) })
|
| data.tables, you say? Can you show us the NOTE message you're getting?
| It could be that your example takes too much CPU time (as opposed to
| "real", "wallclock" time) due to running too many threads started by
| data.table.
Yep, and that is a new test AFAIK.
| It's not obvious why data.table would start too many threads (it's
| supposed to honour the limits that CRAN expresses in environment
| variables), but at least it should be easy to check and discount.
It grabs all it can get which is what you want for performance (I am on a
six-core machine here):
$ R -q
> library(data.table)
data.table 1.14.8 using 6 threads (see ?getDTthreads). Latest news:
r-datatable.com
>
and it honors variables if set
$ OMP_THREAD_LIMIT=2 R -q
> library(data.table)
data.table 1.14.8 using 2 threads (see ?getDTthreads). Latest news:
r-datatable.com
>
so I presume that variable is NOT set by CRAN. It might help if it were.
Dirk
I had to update a package recently to get around this by putting
explicit 'data.table::setDTthreads(1)' in all examples, tests, and
vignettes. The incoming checks now do these CPU/elapsed tests, so you
can test by submitting, and if you're still over the ratio it will
auto-reject and tell you there. That was the only way to get my
submission to pass incoming.
______________________________________________
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel
