Re: [R] Number of Cores limited to two in CRAN

2023-07-03 Thread Ravi Varadhan via R-help 
Dear Henrik,
Thank you!  This is quite helpful, especially your longer blog post.

Best regards,
Ravi

From: Henrik Bengtsson 
Sent: Sunday, July 2, 2023 4:33 AM
To: Ravi Varadhan 
Cc: R-Help 
Subject: Re: [R] Number of Cores limited to two in CRAN


  External Email - Use Caution



Short answer: You don't want to override that limit in your R package.
Don't do it.

Long answer: You'll find the reason for this in the 'CRAN Repository
Policy' 
(https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcran.r-project.org%2Fweb%2Fpackages%2Fpolicies.html=05%7C01%7Cravi.varadhan%40jhu.edu%7Cbf56c90173504853907408db7ad70f08%7C9fa4f438b1e6473b803f86f8aedf0dec%7C0%7C0%7C638238836311718317%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C=KDB6%2BqMgn4KrKtVEgWezfRY9isT926wLwZGg5yuDKI4%3D=0).
Specifically, the following passage:

"Checking the package should take as little CPU time as possible, as
the CRAN check farm is a very limited resource and there are thousands
of packages. Long-running tests and vignette code can be made optional
for checking, but do ensure that the checks that are left do exercise
all the features of the package.

**If running a package uses multiple threads/cores it must never use
more than two simultaneously: the check farm is a shared resource and
will typically be running many checks simultaneously.**

Examples should run for no more than a few seconds each: they are
intended to exemplify to the would-be user how to use the functions in
the package."

Basically, you can use two cores to demonstrate or validate (e.g. in
package tests) that your code *can* run in parallel, but you must not
use more than that to demonstrate that your code can "run super fast".

Even-longer answer: See my blog post 'Please Avoid detectCores() in
your R Packages' 
(https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.jottr.org%2F2022%2F12%2F05%2Favoid-detectcores%2F=05%7C01%7Cravi.varadhan%40jhu.edu%7Cbf56c90173504853907408db7ad70f08%7C9fa4f438b1e6473b803f86f8aedf0dec%7C0%7C0%7C638238836311718317%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C=xPkA6JCccHZcST76lWa6xFYWWKBOZYQCOezF5t4YVPM%3D=0)
from 2022-12-05 for even more reasons.

/Henrik

On Sun, Jul 2, 2023 at 9:55 AM Ravi Varadhan via R-help
 wrote:
>
> This is the specific error messsage from R CMD check --as-cran
>
>
> Error in .check_ncores(length(names)) : 16 simultaneous processes spawned
>   Calls: prepost -> makeCluster -> makePSOCKcluster -> .check_ncores
>   Execution halted
>
>
> Thanks,
> Ravi
>
> 
> From: Ravi Varadhan
> Sent: Saturday, July 1, 2023 1:15 PM
> To: R-Help 
> Subject: Number of Cores limited to two in CRAN
>
> Hi,
> I am developing a package where I would like to utilize multiple cores for 
> parallel computing.  However, I get an error message when I run R CMD check 
> --as-cran.
>
> I read that CRAN limits the number of cores to 2.  Is this correct? Is there 
> any way to overcome this limitation?
>
> Thank you,
> Ravi
>
> [[alternative HTML version deleted]]
>
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fstat.ethz.ch%2Fmailman%2Flistinfo%2Fr-help=05%7C01%7Cravi.varadhan%40jhu.edu%7Cbf56c90173504853907408db7ad70f08%7C9fa4f438b1e6473b803f86f8aedf0dec%7C0%7C0%7C638238836311873942%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C=KTNwXurNvbEhviVf15RF0la%2FvUVGBW60yuyVVy9Umxc%3D=0
> PLEASE do read the posting guide 
> https://nam02.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.r-project.org%2Fposting-guide.html=05%7C01%7Cravi.varadhan%40jhu.edu%7Cbf56c90173504853907408db7ad70f08%7C9fa4f438b1e6473b803f86f8aedf0dec%7C0%7C0%7C638238836311873942%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C=HvGb20UiUey9O3plynLiRmHQtqpjfrndk2iPfvleT3Q%3D=0
> and provide commented, minimal, self-contained, reproducible code.

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Number of Cores limited to two in CRAN

2023-07-02 Thread Henrik Bengtsson 
Short answer: You don't want to override that limit in your R package.
Don't do it.

Long answer: You'll find the reason for this in the 'CRAN Repository
Policy' (https://cran.r-project.org/web/packages/policies.html).
Specifically, the following passage:

"Checking the package should take as little CPU time as possible, as
the CRAN check farm is a very limited resource and there are thousands
of packages. Long-running tests and vignette code can be made optional
for checking, but do ensure that the checks that are left do exercise
all the features of the package.

**If running a package uses multiple threads/cores it must never use
more than two simultaneously: the check farm is a shared resource and
will typically be running many checks simultaneously.**

Examples should run for no more than a few seconds each: they are
intended to exemplify to the would-be user how to use the functions in
the package."

Basically, you can use two cores to demonstrate or validate (e.g. in
package tests) that your code *can* run in parallel, but you must not
use more than that to demonstrate that your code can "run super fast".

Even-longer answer: See my blog post 'Please Avoid detectCores() in
your R Packages' (https://www.jottr.org/2022/12/05/avoid-detectcores/)
from 2022-12-05 for even more reasons.

/Henrik

On Sun, Jul 2, 2023 at 9:55 AM Ravi Varadhan via R-help
 wrote:
>
> This is the specific error messsage from R CMD check --as-cran
>
>
> Error in .check_ncores(length(names)) : 16 simultaneous processes spawned
>   Calls: prepost -> makeCluster -> makePSOCKcluster -> .check_ncores
>   Execution halted
>
>
> Thanks,
> Ravi
>
> 
> From: Ravi Varadhan
> Sent: Saturday, July 1, 2023 1:15 PM
> To: R-Help 
> Subject: Number of Cores limited to two in CRAN
>
> Hi,
> I am developing a package where I would like to utilize multiple cores for 
> parallel computing.  However, I get an error message when I run R CMD check 
> --as-cran.
>
> I read that CRAN limits the number of cores to 2.  Is this correct? Is there 
> any way to overcome this limitation?
>
> Thank you,
> Ravi
>
> [[alternative HTML version deleted]]
>
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Number of Cores limited to two in CRAN

2023-07-02 Thread Ravi Varadhan via R-help 
This is the specific error messsage from R CMD check --as-cran


Error in .check_ncores(length(names)) : 16 simultaneous processes spawned
  Calls: prepost -> makeCluster -> makePSOCKcluster -> .check_ncores
  Execution halted


Thanks,
Ravi


From: Ravi Varadhan
Sent: Saturday, July 1, 2023 1:15 PM
To: R-Help 
Subject: Number of Cores limited to two in CRAN

Hi,
I am developing a package where I would like to utilize multiple cores for 
parallel computing.  However, I get an error message when I run R CMD check 
--as-cran.

I read that CRAN limits the number of cores to 2.  Is this correct? Is there 
any way to overcome this limitation?

Thank you,
Ravi

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] Number of Cores limited to two in CRAN

2023-07-02 Thread Ravi Varadhan via R-help 
Hi,
I am developing a package where I would like to utilize multiple cores for 
parallel computing.  However, I get an error message when I run R CMD check 
--as-cran.

I read that CRAN limits the number of cores to 2.  Is this correct? Is there 
any way to overcome this limitation?

Thank you,
Ravi

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.