Re: [R-pkg-devel] Unexpected multi-core CPU usage in package tests

2024-02-28 Thread Jon Clayden
On Wed, 28 Feb 2024 at 17:36, Ivan Krylov  wrote:
>
> В Tue, 27 Feb 2024 11:14:19 +
> Jon Clayden  пишет:
>
> > My testing route is to install the packages within the
> > 'rocker/r-devel' Docker container, which is Debian-based, then use
> > 'time' to evaluate CPU usage. Note that, even though 'RNifti' does not
> > use OpenMP, setting OMP_NUM_THREADS changes its CPU usage
>
> I think that's because rocker/r-devel uses parallel OpenBLAS:
>
> $ podman run --rm -it docker.io/rocker/r-devel \
>  R -q -s -e 'sessionInfo()' | grep -A1 BLAS
> BLAS:   /usr/lib/x86_64-linux-gnu/openblas-pthread/libblas.so.3
> LAPACK: /usr/lib/x86_64-linux-gnu/openblas-pthread/libopenblasp-r0.3.24.so;  
> LAPACK version 3.11.0
>
> The incoming CRAN check machine either sets the BLAS parallellism to 1
> or uses a non-parallel BLAS. With rocker/r-devel, you can run R with
> the environment variable OPENBLAS_NUM_THREADS set to 1. It's been
> effective in the past to run R -d gdb and set a breakpoint on
> pthread_create before launching the test. (In theory, it may be
> required to set a breakpoint on every system call that may be used to
> create threads, including various variations of clone(), subject to
> variations between operating systems, but pthread_create has been
> enough for me so far.)
>
> With OPENBLAS_NUM_THREADS=1, I'm only seeing OpenMP threads created by
> the mmand package during tests for your package tractor.base, and the
> latest commit (that temporary disables testing of mmand) doesn't hit
> the breakpoint or raise any NOTEs at all.

Very many thanks for testing this, Ivan. I did come across the
breakpoint suggestion when searching on this, and tried it, but I ran
into some problems with gdb within the container. Your experiments
confirm that my yardstick was just misleading me; I will add support
for explicitly setting the thread count within mmand, which should
give me a longer-term solution.

Thanks again,
Jon

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel


Re: [R-pkg-devel] Unexpected multi-core CPU usage in package tests

2024-02-28 Thread Ivan Krylov via R-package-devel
В Tue, 27 Feb 2024 11:14:19 +
Jon Clayden  пишет:

> My testing route is to install the packages within the
> 'rocker/r-devel' Docker container, which is Debian-based, then use
> 'time' to evaluate CPU usage. Note that, even though 'RNifti' does not
> use OpenMP, setting OMP_NUM_THREADS changes its CPU usage

I think that's because rocker/r-devel uses parallel OpenBLAS:

$ podman run --rm -it docker.io/rocker/r-devel \
 R -q -s -e 'sessionInfo()' | grep -A1 BLAS
BLAS:   /usr/lib/x86_64-linux-gnu/openblas-pthread/libblas.so.3 
LAPACK: /usr/lib/x86_64-linux-gnu/openblas-pthread/libopenblasp-r0.3.24.so;  
LAPACK version 3.11.0

The incoming CRAN check machine either sets the BLAS parallellism to 1
or uses a non-parallel BLAS. With rocker/r-devel, you can run R with
the environment variable OPENBLAS_NUM_THREADS set to 1. It's been
effective in the past to run R -d gdb and set a breakpoint on
pthread_create before launching the test. (In theory, it may be
required to set a breakpoint on every system call that may be used to
create threads, including various variations of clone(), subject to
variations between operating systems, but pthread_create has been
enough for me so far.)

With OPENBLAS_NUM_THREADS=1, I'm only seeing OpenMP threads created by
the mmand package during tests for your package tractor.base, and the
latest commit (that temporary disables testing of mmand) doesn't hit
the breakpoint or raise any NOTEs at all.

-- 
Best regards,
Ivan

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel