Re: [Bioc-devel] long tests results in package RSS feed

[Hervé Pagès]

The feeds for the long tests are here:

  https://bioconductor.org/rss/build/longtests/packages/

I also added feeds for the workflow and book builds:

  https://bioconductor.org/rss/build/workflows/packages/

  https://bioconductor.org/rss/build/books/packages/

I'll update the
page at https://bioconductor.org/developers/rss-feeds/ shortly.

Cheers,
H.


On 21/10/2021 09:36, Hervé Pagès wrote:

On 20/10/2021 22:40, Pierrick Roger wrote:

It seems the results of the long tests are not sent to the package RSS
feed. Am I missing something? Is there another feed?


We only have RSS feeds for Software builds and Data Experiment builds:

   https://bioconductor.org/developers/rss-feeds/


If not, would it be possible to send those results to the package RSS
feed?


We'll add this.

Best,
H.



Regards,





--
Hervé Pagès

Bioconductor Core Team
hpages.on.git...@gmail.com

___
Bioc-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/bioc-devel

Re: [Rd] stats::fft produces inconsistent results

[Ben Bolker]

  Nice!

On 10/21/21 4:26 PM, Dipterix Wang wrote:
Thank you for such detailed and plain explanation. It is much clearer to 
me now w.r.t. the R internal memory management and how PROTECT should be 
used.


Also after diving into the documentation of FFTW3 library, I think I 
found why the data was centered.


https://www.fftw.org/fftw3_doc/Planner-Flags.html 



Basically
1. FFTW3 modifies the input data by default
2. one has to initialize the data after planning fft (except for some 
special situations). This “subtle” detail is buried in their 
documentation and is very hard to debug once a mistake is made.


The second one actually causes CRAN package fftwtools to produce 
inconsistent results on osx 
(https://github.com/krahim/fftwtools/issues/15 
)


Best,
Dipterix

On Oct 21, 2021, at 6:32 AM, GILLIBERT, Andre 
mailto:andre.gillib...@chu-rouen.fr>> 
wrote:


> Haha, thanks : ) I guess I will probably be grouchy too if seeing so many 
people making the same mistakes again and again. It just happened to be me.

Fortunately, you did not get offensed. :)

This is nice to have a large community of developers for R packages, 
even if, sometimes, buggy packages are annoying R developers because 
any small change in R may "break" them even though they were actually 
broken from the begining.


>Indeed, I found myself often confused about when to PROTECT and when not.

A (relatively) quick explanation.
There are several “pools” of data objects that have different rules. 
The most common “pool” is the pool of garbage collectable R objects, 
that can be allocated with allocVector and is passed from R to C code 
and vice versa. Another pool is the malloc/free pool, that works with 
explicit allocation/deallocation. R does not modify the malloc/free 
implementation in any way, and memory leaks may happen. Operating 
systems may have other pools of memory (e.g. mmap'ed memory) that are 
not handled by R either. There is also a transient storage 
(R_alloc/vmaxset/vmaxget) that is automatically freed when returning 
from C to R, and should be used for temporary storage but not for 
objects returned to R code.


The PROTECT system is needed for garbage collectable objects.
The garbage collector may trigger whenever a R internal function is 
called. Typically, when some memory is internally allocated.
The garbage collector frees objects that are neither referenced 
directly nor indirectly from R code and from the PROTECT stack.
The PROTECT stack is used by C code to make sure objects that are not 
yet (or will never be) referenced by R code, are not destroyed when 
the garbage collector runs.


The functions allocating new R objects, such as allocVector(), but 
also coerceVector(), duplicate(),return unprotected objects, that may 
be destroyed the next time an internal R function is called, unless it 
is explicitly PROTECT'ed before. Indeed, such objects would have no 
reference from R code and so, would be deleted.


The PROTECT stack must be balanced on a call from R to a C function. 
There must be as many UNPROTECT'ions than PROTECT'ions.


The typical C code PROTECTs any object allocated as soon as it is 
allocated (e.g. call to allocVector or coerceVector). It UNPROTECTs 
temporary objects to "free" them (the actual memory release may be 
delayed to the next garbage collection). It UNPROTECTs the object it 
returns to R code. Indeed, in pure C code, there will be no garbage 
collection between the time the object is UNPROTECTed and the time R 
grabs the object. You must be very careful if you are using C++, 
because destructors must not call any R internal function that may 
trigger a garbage collection.
The arguments to the C code, do not have to be PROTECT'ed, unless they 
are re-allocated. For instance, it is frequent to call coerceVector or 
arguments and re-assign them to the C variable that represents the 
argument. The new object must be PROTECT'ed.


Actually, you do not need to *directly* PROTECT all objects that are 
allocated in the C function, but you must make sure that all objects 
are *indirectly* PROTECT'ed. For instance, you may allocate a VECSXP 
(a "list" in R) and fill the slots with newly allocated objects. You 
only need to PROTECT the VECSXP, since its slots are indirectly protected.


If you have any doubt, it is not a bug to over-PROTECT objects. It may 
slightly slow down garbage collection and use space on the PROTECTion 
stack, but that is rarely a big deal. You should only avoid that when 
that would lead to thousands or millions of protections.


As I said, the PROTECT stack must be balanced between the entry and 
exit of the C code. This is not a problem for 99% of functions that 
free all the memory they use internally except the object that is 
returned. Sometimes, some "background" memory, hidden to R code, may 
have to be allocated for more time. A call to R_PreserveObject 

Re: [Rd] stats::fft produces inconsistent results

[Dipterix Wang]

Thank you for such detailed and plain explanation. It is much clearer to me now 
w.r.t. the R internal memory management and how PROTECT should be used. 

Also after diving into the documentation of FFTW3 library, I think I found why 
the data was centered.

https://www.fftw.org/fftw3_doc/Planner-Flags.html

Basically 
1. FFTW3 modifies the input data by default 
2. one has to initialize the data after planning fft (except for some special 
situations). This “subtle” detail is buried in their documentation and is very 
hard to debug once a mistake is made. 

The second one actually causes CRAN package fftwtools to produce inconsistent 
results on osx (https://github.com/krahim/fftwtools/issues/15)

Best,
Dipterix

> On Oct 21, 2021, at 6:32 AM, GILLIBERT, Andre  
> wrote:
> 
> > Haha, thanks : ) I guess I will probably be grouchy too if seeing so many 
> > people making the same mistakes again and again. It just happened to be me.
> 
> Fortunately, you did not get offensed. :)
> 
> This is nice to have a large community of developers for R packages, even if, 
> sometimes, buggy packages are annoying R developers because any small change 
> in R may "break" them even though they were actually broken from the begining.
> 
> > Indeed, I found myself often confused about when to PROTECT and when not. 
>  
> A (relatively) quick explanation.
> There are several “pools” of data objects that have different rules. The most 
> common “pool” is the pool of garbage collectable R objects, that can be 
> allocated with allocVector and is passed from R to C code and vice versa. 
> Another pool is the malloc/free pool, that works with explicit 
> allocation/deallocation. R does not modify the malloc/free implementation in 
> any way, and memory leaks may happen. Operating systems may have other pools 
> of memory (e.g. mmap'ed memory) that are not handled by R either. There is 
> also a transient storage (R_alloc/vmaxset/vmaxget) that is automatically 
> freed when returning from C to R, and should be used for temporary storage 
> but not for objects returned to R code.
>  
> The PROTECT system is needed for garbage collectable objects.
> The garbage collector may trigger whenever a R internal function is called. 
> Typically, when some memory is internally allocated.
> The garbage collector frees objects that are neither referenced directly nor 
> indirectly from R code and from the PROTECT stack.
> The PROTECT stack is used by C code to make sure objects that are not yet (or 
> will never be) referenced by R code, are not destroyed when the garbage 
> collector runs.
>  
> The functions allocating new R objects, such as allocVector(), but also 
> coerceVector(), duplicate(),return unprotected objects, that may be destroyed 
> the next time an internal R function is called, unless it is explicitly 
> PROTECT'ed before. Indeed, such objects would have no reference from R code 
> and so, would be deleted.
>   
> The PROTECT stack must be balanced on a call from R to a C function. There 
> must be as many UNPROTECT'ions than PROTECT'ions.
> 
> The typical C code PROTECTs any object allocated as soon as it is allocated 
> (e.g. call to allocVector or coerceVector). It UNPROTECTs temporary objects 
> to "free" them (the actual memory release may be delayed to the next garbage 
> collection). It UNPROTECTs the object it returns to R code. Indeed, in pure C 
> code, there will be no garbage collection between the time the object is 
> UNPROTECTed and the time R grabs the object. You must be very careful if you 
> are using C++, because destructors must not call any R internal function that 
> may trigger a garbage collection.
> The arguments to the C code, do not have to be PROTECT'ed, unless they are 
> re-allocated. For instance, it is frequent to call coerceVector or arguments 
> and re-assign them to the C variable that represents the argument. The new 
> object must be PROTECT'ed.
> 
> Actually, you do not need to *directly* PROTECT all objects that are 
> allocated in the C function, but you must make sure that all objects are 
> *indirectly* PROTECT'ed. For instance, you may allocate a VECSXP (a "list" in 
> R) and fill the slots with newly allocated objects. You only need to PROTECT 
> the VECSXP, since its slots are indirectly protected.
> 
> If you have any doubt, it is not a bug to over-PROTECT objects. It may 
> slightly slow down garbage collection and use space on the PROTECTion stack, 
> but that is rarely a big deal. You should only avoid that when that would 
> lead to thousands or millions of protections.
> 
> As I said, the PROTECT stack must be balanced between the entry and exit of 
> the C code. This is not a problem for 99% of functions that free all the 
> memory they use internally except the object that is returned. Sometimes, 
> some "background" memory, hidden to R code, may have to be allocated for more 
> time. A call to R_PreserveObject protects the object, even after the C code 
> returns to R, 

Re: [Bioc-devel] How to simplify install when package dependency requires compilation from source

[Hervé Pagès]

Hi Shraddha,

I don't think you want to start the business of building and 
distributing Mac binary packages from other people to your users. Even 
if you were doing this, BiocManager::install() would not know about 
these binaries, and I'm not sure that there would be a simple way to 
make your users aware that they can get these binaries from a repository 
that you host and manage.


Also that would mean that you would need to rebuild all the binaries 
that netDx depends on each time they get updated. Would you be able to 
commit to that?


There is no Mac binary for clusterExperiment because the package has a 
TIMEOUT on this platform. The maintainer was contacted recently and is 
working on it.


Thanks for your patience.

Cheers,
H.


On 21/10/2021 06:21, Shraddha Pai wrote:

Hi all,

How do I simplify install when a dependency is available only in source
form and requires compilation using C/C++/Fortran?

My package netDx has "clusterExperiment" as a dependency (Suggests).

When I run BiocManager::install("netDx",dependencies=TRUE), I get:
"Package which is only available in source form, and may need compilation of
   C/C++/Fortran: ‘clusterExperiment’
Do you want to attempt to install these from sources? (Yes/no/cancel)"

If I say yes, it fails because the laptop doesn't have gcc installed, and
then I have to go run "brew install gcc" (this is OS X). This is getting
too complicated for your average user.

Is there a way I could precompile needed packages and just make them
available to my users? Are there instructions for how to do this somewhere?
I can host the compiled version on a lab server.

Thanks,
Shraddha


--

*Shraddha Pai, PhD*
Principal Investigator, OICR
Assistant Professor, Department of Medical Biophysics, University of Toronto
shraddhapai.com; @spaiglass on Twitter
https://pailab.oicr.on.ca


*Ontario Institute for Cancer Research*
MaRS Centre, 661 University Avenue, Suite 510, Toronto, Ontario, Canada M5G
0A3
*@OICR_news*  | *www.oicr.on.ca*




*Collaborate. Translate. Change lives.*



This message and any attachments may contain confident...{{dropped:13}}


___
Bioc-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/bioc-devel

Re: [Bioc-devel] mosbi package: Suggested dependency not available

[Hervé Pagès]

Hi Tim,

runibic has a compilation error on Mac because its native code uses 
OpenMP. However package code should not use OpenMP on Mac (see 
https://stat.ethz.ch/pipermail/r-sig-mac/2019-June/012995.html).


I just marked runibic as unsupported on Mac.

It seems that mosbi only uses runibic in one function, 
mosbi::run_unibic(), but that function is never called (the only place 
where I see a call to it is in the example section of its man page but 
the call is commented out). So the only reason that 'R CMD check' fails 
on Mac is because, by default, 'R CMD check' wants all the deps, even 
the suggested ones (i.e. soft deps), to be installed. To prevent this, I 
added a .BBSoptions file to mosbi, with the following line in it:


  CHECKprepend.mac: _R_CHECK_FORCE_SUGGESTS_=0

See for example the build report for piano 
https://bioconductor.org/checkResults/3.14/bioc-LATEST/piano/merida1-checksrc.html

for the effect that this will have on the exact command used during CHECK.

Cheers,
H.


On 21/10/2021 06:06, Kern, Lori wrote:

Yes you will be fine. It looks like runibic is failing possibly of a missing 
dependency on our end and we will investigate.  Once the package builds your 
package should also build correctly.


Cheers



Lori Shepherd

Bioconductor Core Team

Roswell Park Comprehensive Cancer Center

Department of Biostatistics & Bioinformatics

Elm & Carlton Streets

Buffalo, New York 14263


From: Bioc-devel  on behalf of Tim Daniel Rose 

Sent: Thursday, October 21, 2021 9:03 AM
To: bioc-devel@r-project.org 
Subject: [Bioc-devel] mosbi package: Suggested dependency not available

Hi everyone,

My new package mosbi
(https://bioconductor.org/packages/devel/bioc/html/mosbi.html) gets a
MacOS build 'ERROR', because a suggested dependency 'runibic' is not
available for that platform.

The reviewers stated on github
(https://secure-web.cisco.com/14eSzrEiAByl2rDPyDqX7C6ssESP9c7weoJHn1WbC4-92uBIuqNNPKwxibQdqC29ln5mUT8DswyAnmxgVprBPBgce_IfGc99QCllfD0NtVv_WcxcB4gLzIqjV8OIdaFeayLxT2e4SPC63XWfMCFjIjHhDfxs6_l4dA1LyUtVWMFBZPFikPZsUwBQ5AbSneNpm3ZgSugHP8xkrf2F-o1e725NfvIyFFKdRIk4lAQXie3mgLAsgX84leu4uiWSSdRzTClVu2kAWkHwDUD34pJlRs2tKgBDo7MC8CuhKL62uKugxJEQyVCHDBpYWtbZv0THZ/https%3A%2F%2Fgithub.com%2FBioconductor%2FContributions%2Fissues%2F2252)
 that it
should be okay, but they could not reproduce the error locally.

On the release schedule site
(https://bioconductor.org/developers/release-schedule/) it says, that
packages need to pass builds and checks without errors until tomorrow.

I want to check here again, if it is okay like this or if I need to take
further action.

Best,
Tim

--
Tim Daniel Rose, MSc

PhD candidate at
Junior Research Group LipiTUM
Chair of Experimental Bioinformatics
TUM School of Life Sciences
Technical University of Munich (TUM)

Maximus-von-Imhof-Forum 3
85354 Freising-Weihenstephan
Germany

mail: tim.r...@wzw.tum.de
web: www.lipitum.de

___
Bioc-devel@r-project.org mailing list
https://secure-web.cisco.com/1YLC8ocJPS7PoYgSR9H-3gWC9z_Ho1lnVHhX3_drWyo1Irn-BQs4tR6mR2pcRZnUQrUSRbQ8ZMYtG6RZoKob7-W5QjKySO2hJ1KOKRxse3tUJj9OSyILOaviJV4EsTi4qu7N04Gpbf3JvNPsf1YsP2SuzjfBqFi0UFPFLGt56KRUltNH90fXIBoZiECd_RXTOCuxOrNrkXj5PmbilmjV02vlIjgoWspqJxy2tfN2dz_118TCrY4pymh_4u9sQcq6ZMgeJW5i-25cJY4l8snUNtc6-jDXXFwOJznXoRYWAYhzBb3ah4INu5uOqkngyYsI9/https%3A%2F%2Fstat.ethz.ch%2Fmailman%2Flistinfo%2Fbioc-devel



This email message may contain legally privileged and/or confidential 
information.  If you are not the intended recipient(s), or the employee or 
agent responsible for the delivery of this message to the intended 
recipient(s), you are hereby notified that any disclosure, copying, 
distribution, or use of this email message is prohibited.  If you have received 
this message in error, please notify the sender immediately by e-mail and 
delete this email message from your computer. Thank you.
[[alternative HTML version deleted]]

___
Bioc-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/bioc-devel



--
Hervé Pagès

Bioconductor Core Team
hpages.on.git...@gmail.com

___
Bioc-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/bioc-devel

[Bioc-devel] Deprecation genphen

[Simo Kitanovski]

Dears,

we would like to deprecate genphen as its functionality has
been superseded by other R packages.

Would it be possible to begin the deprecation process?

Best,
Simo

___
Bioc-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/bioc-devel

Re: [R-pkg-devel] failing S3 dispatch

[Duncan Murdoch]

On 21/10/2021 1:23 p.m., Jens Oehlschlägel wrote:

Thanks Duncan,

I finally found the reason for the mysterious dispatch-failure: I had an
unwanted and unexported replicated definition of the 'clone' generic in
ff's namespace (a left-over).

I still don't understand how this prevented the proper dispatch since
the duplicate in ff's namespace it was *not* called. I further
experimented and *any* non-exported object matching the name of the
generic caused the problem. Scary, maybe worth a check!


Your NAMESPACE file contains

  S3method("clone",ff)

When R installs your package, it has to figure out where "clone" lives. 
 You imported a copy of it from bit, and then overwrote that imported 
copy with the replicated definition.


That created a new generic belonging to ff, and that's the one that your 
clone.ff method was attached to.


When you called clone(x), you called the bit::clone generic which never 
received the registration, so dispatch to clone.ff never happened.


It makes sense:  you don't want a generic in one package to interfere 
with an unrelated generic in another package that happens to have the 
same name.


Duncan Murdoch




Anyhow, removing the non-exported object solved the problem.

Best regards

Jens


On 20.10.21 13:43, Jens Oehlschlägel wrote:

Thank you Duncan,

bit NAMESPACE has

S3method(clone,default)
export(clone)

ff NAMESPACE has

import(bit)
# wish of CRAN maintainers: export another time here (now maintained
and exported in bit)
# without this R CMD CHECK complained, but with it R CMD CHECK
complains also, how to export again and why?
# clone
#,clone.default
  clone.ff
,clone.ffdf
S3method("clone",ff)
S3method(clone, ffdf)
# wish of CRAN maintainers: export another time here (now maintained
and exported in bit)
#S3method(clone, default)

Best

Jens



On 20.10.21 11:02, Duncan Murdoch wrote:

On 19/10/2021 3:43 p.m., Jens Oehlschlägel wrote:

I didn't find an answer elsewhere:

My package 'bit' creates a S3 generic 'clone' and exports it.
Furthermore it registers a S3 method 'clone.default' (not exported).

My package 'ff' imports package 'bit' and exports and registers a
new S3
method 'clone.ff'. However, calling 'clone(ffobj)' dispatches to
clone.default instead of clone.ff !? Why?


You should show us the NAMESPACE entries involving clone and clone.ff
from ff.

Some comments that may or may not be relevant:

  - Normally you wouldn't export clone.ff, it's enough to register it
using S3method().

  - You may have created a new generic named clone, and that's what
clone.ff would attach itself to.  You can have bit::clone and
ff::clone as different generics and that would cause problems.



What is the recommended way to create new S3-methods that get
dispatched? In earlier versions of the packages I simply exported
everything - that worked.


I import the generic and use S3method(generic, method).  I don't
export the methods, so I wouldn't be able to call z <- clone.ff(a).

Duncan Murdoch



Best


Jens


   > require(ff)
   >
   > a <- as.ff(0:9)
   > class(x)
[1] "ff_vector" "ff"
   >
   > x <- clone(a)
   > y <- bit:::clone.default(a)
   > z <- clone.ff(a)
   >
   > # cloned ffobjects should have different filenames>

   > filename(a)  # original
[1] "/tmp/Rtmpk17JRZ/ff/clone1ed54cbb5060.ff"
   >
   > filename(x)  # unexpected equal (dispatch to clone.default)
[1] "/tmp/Rtmpk17JRZ/ff/clone1ed54cbb5060.ff"
   >
   > filename(y)  # expected equal
[1] "/tmp/Rtmpk17JRZ/ff/clone1ed54cbb5060.ff"
   >
   > filename(z)  # OK
[1] "/tmp/Rtmpk17JRZ/ff/clone1ed551d3ee66.ff"

   > version
      _
platform   x86_64-pc-linux-gnu
arch   x86_64
os linux-gnu
system x86_64, linux-gnu
status
major  4
minor  1.1
year   2021
month  08
day    10
svn rev    80725
language   R
version.string R version 4.1.1 (2021-08-10)
nickname   Kick Things

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel





__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel


__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel

Re: [R-pkg-devel] failing S3 dispatch

[Jens Oehlschlägel]

Thanks Duncan,

I finally found the reason for the mysterious dispatch-failure: I had an 
unwanted and unexported replicated definition of the 'clone' generic in 
ff's namespace (a left-over).


I still don't understand how this prevented the proper dispatch since 
the duplicate in ff's namespace it was *not* called. I further 
experimented and *any* non-exported object matching the name of the 
generic caused the problem. Scary, maybe worth a check!


Anyhow, removing the non-exported object solved the problem.

Best regards

Jens


On 20.10.21 13:43, Jens Oehlschlägel wrote:

Thank you Duncan,

bit NAMESPACE has

S3method(clone,default)
export(clone)

ff NAMESPACE has

import(bit)
# wish of CRAN maintainers: export another time here (now maintained 
and exported in bit)
# without this R CMD CHECK complained, but with it R CMD CHECK 
complains also, how to export again and why?

# clone
#,clone.default
 clone.ff
,clone.ffdf
S3method("clone",ff)
S3method(clone, ffdf)
# wish of CRAN maintainers: export another time here (now maintained 
and exported in bit)

#S3method(clone, default)

Best

Jens



On 20.10.21 11:02, Duncan Murdoch wrote:

On 19/10/2021 3:43 p.m., Jens Oehlschlägel wrote:

I didn't find an answer elsewhere:

My package 'bit' creates a S3 generic 'clone' and exports it.
Furthermore it registers a S3 method 'clone.default' (not exported).

My package 'ff' imports package 'bit' and exports and registers a 
new S3

method 'clone.ff'. However, calling 'clone(ffobj)' dispatches to
clone.default instead of clone.ff !? Why?


You should show us the NAMESPACE entries involving clone and clone.ff 
from ff.


Some comments that may or may not be relevant:

 - Normally you wouldn't export clone.ff, it's enough to register it 
using S3method().


 - You may have created a new generic named clone, and that's what 
clone.ff would attach itself to.  You can have bit::clone and 
ff::clone as different generics and that would cause problems.




What is the recommended way to create new S3-methods that get
dispatched? In earlier versions of the packages I simply exported
everything - that worked.


I import the generic and use S3method(generic, method).  I don't 
export the methods, so I wouldn't be able to call z <- clone.ff(a).


Duncan Murdoch



Best


Jens


  > require(ff)
  >
  > a <- as.ff(0:9)
  > class(x)
[1] "ff_vector" "ff"
  >
  > x <- clone(a)
  > y <- bit:::clone.default(a)
  > z <- clone.ff(a)
  >
  > # cloned ffobjects should have different filenames>

  > filename(a)  # original
[1] "/tmp/Rtmpk17JRZ/ff/clone1ed54cbb5060.ff"
  >
  > filename(x)  # unexpected equal (dispatch to clone.default)
[1] "/tmp/Rtmpk17JRZ/ff/clone1ed54cbb5060.ff"
  >
  > filename(y)  # expected equal
[1] "/tmp/Rtmpk17JRZ/ff/clone1ed54cbb5060.ff"
  >
  > filename(z)  # OK
[1] "/tmp/Rtmpk17JRZ/ff/clone1ed551d3ee66.ff"

  > version
     _
platform   x86_64-pc-linux-gnu
arch   x86_64
os linux-gnu
system x86_64, linux-gnu
status
major  4
minor  1.1
year   2021
month  08
day    10
svn rev    80725
language   R
version.string R version 4.1.1 (2021-08-10)
nickname   Kick Things

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel





__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel


__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel

Re: [Bioc-devel] long tests results in package RSS feed

[Hervé Pagès]

On 20/10/2021 22:40, Pierrick Roger wrote:

It seems the results of the long tests are not sent to the package RSS
feed. Am I missing something? Is there another feed?


We only have RSS feeds for Software builds and Data Experiment builds:

  https://bioconductor.org/developers/rss-feeds/


If not, would it be possible to send those results to the package RSS
feed?


We'll add this.

Best,
H.



Regards,



--
Hervé Pagès

Bioconductor Core Team
hpages.on.git...@gmail.com

___
Bioc-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/bioc-devel

Re: [Rd] Fwd: Using existing envars in Renviron on friendly Windows

[Martin Maechler]

> Michał Bojanowski 
> on Wed, 20 Oct 2021 16:31:08 +0200 writes:

> Hello Tomas,
> Yes, that's accurate although rather terse, which is perhaps the
> reason why I did not realize it applies to my case.

> How about adding something in the direction of:

> 1. Continuing the cited paragraph with:
> In particular, on Windows it may be necessary to quote references to
> existing environment variables, especially those containing file paths
> (which include backslashes). For example: `"${WINVAR}"`.

> 2. Add an example (not run):

> # On Windows do quote references to variables containing paths, e.g.:
> # If APPDATA=C:\Users\foobar\AppData\Roaming
> # to point to a library tree inside APPDATA in .Renviron use
> R_LIBS_USER="${APPDATA}"/R-library

> Incidentally the last example is on backslashes too.


> What do you think?

I agree that adding an example really helps a lot in such cases,
in my experience, notably if it's precise enough to be used +/- directly.



> On Mon, Oct 18, 2021 at 5:02 PM Tomas Kalibera  
wrote:
>> 
>> 
>> On 10/15/21 6:44 PM, Michał Bojanowski wrote:
>> > Perhaps a small update to ?.Renviron would be in order to mention 
that...
>> 
>> Would you have a more specific suggestion how to update the
>> documentation? Please note that it already says
>> 
>> "‘value’ is then processed in a similar way to a Unix shell: in
>> particular the outermost level of (single or double) quotes is stripped,
>> and backslashes are removed except inside quotes."
>> 
>> Thanks,
>> Tomas
>> 
>> > On Fri, Oct 15, 2021 at 6:43 PM Michał Bojanowski 
 wrote:
>> >> Indeed quoting works! Kevin suggested the same, but he didnt reply to 
the list.
>> >> Thank you all!
>> >> Michal
>> >>
>> >> On Fri, Oct 15, 2021 at 6:40 PM Ivan Krylov  
wrote:
>> >>> Sorry for the noise! I wasn't supposed to send my previous message.
>> >>>
>> >>> On Fri, 15 Oct 2021 16:44:28 +0200
>> >>> Michał Bojanowski  wrote:
>> >>>
>>  AVAR=${APPDATA}/foo/bar
>> 
>>  Which is a documented way of referring to existing environment
>>  variables. Now, with that in R I'm getting:
>> 
>>  Sys.getenv("APPDATA")# That works OK
>>  [1] "C:\\Users\\mbojanowski\\AppData\\Roaming"
>> 
>>  so OK, but:
>> 
>>  Sys.getenv("AVAR")
>>  [1] "C:UsersmbojanowskiAppDataRoaming/foo/bar"
>> >>> Hmm, a function called by readRenviron does seem to remove 
backslashes,
>> >>> but not if they are encountered inside quotes:
>> >>>
>> >>> 
https://github.com/r-devel/r-svn/blob/3f8b75857fb1397f9f3ceab6c75554e1a5386adc/src/main/Renviron.c#L149
>> >>>
>> >>> Would AVAR="${APPDATA}"/foo/bar work?
>> >>>
>> >>> --
>> >>> Best regards,
>> >>> Ivan
>> > __
>> > R-devel@r-project.org mailing list
>> > https://stat.ethz.ch/mailman/listinfo/r-devel

> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [R-pkg-devel] Is there a better way ...?

[Andrew Simmons]

Duncan's version is much clearer than my solution, and the only reason I
use my version is so that the source reference of the function looks
neater, and so that auto-code-indentation won't mess up my source reference
either.
If none of that made sense, don't worry about it, use Duncan's approach.
I think the advantage of Duncan's solution is that getting and setting
.fooInfo is more compact.

It's not clear exactly how you're modifying .fooInfo, but I'll assume
plot.foo is modifying it. Using Duncan's approach, you might do something
like:


plot.foo <- local({


.fooInfo <- 0


function (...)
{
value <- .fooInfo  # get .fooInfo
.fooInfo <<- .fooInfo + 1  # set .fooInfo, you must use <<- here
instead of <-
return(value)
}


})


and Deepayan's approach:


..foo.env <- local({


.fooInfo <- 0


environment()
})


plot.foo <- function (...)
{
value <- .foo.env$.fooInfo  # get .fooInfo
.foo.env$.fooInfo <- .foo.env$.fooInfo + 1  # set .fooInfo
return(value)
}


Both of these work perfectly fine, so you don't have to worry too much
about which you implement. The differences are mostly just visual
appearance, they have nearly equivalent functionality and performance.

On Thu, Oct 21, 2021 at 2:45 AM Rolf Turner  wrote:

>
> On Thu, 21 Oct 2021 02:03:41 -0400
> Duncan Murdoch  wrote:
>
> > On 21/10/2021 12:40 a.m., Andrew Simmons wrote:
> > > I think the simplest answer is to store the variable in the
> > > functions frame. I'm assuming here that the only plot.foo needs
> > > access to .fooInfo, if not this can be changed.
> > >
> > >
> > > plot.foo <- function (...)
> > > {
> > >  .fooInfo
> > > }
> > > environment(plot.foo) <- new.env()
> > > evalq({
> > >  .fooInfo <- NULL
> > > }, environment(plot.foo))
> > >
> > >
> > > Make your function, and do whatever you need with .fooInfo within
> > > said function. Whenever you previously updated .fooInfo in the
> > > global environment, update .fooInfo in plot.foo environment instead.
> > > Also, because .fooInfo is not stored in the package's frame, it
> > > won't be locked when the namespace is sealed. If you created it at
> > > the toplevel, that would create some issues. But this works fine.
> >
> > I agree with the final result, but I'd write the code differently:
> >
> > plot.foo <- local({
> >
> >.fooInfo <- NULL
> >
> >function (...) { ... }
> > })
> >
> > creates an environment, puts .fooInfo into it with value NULL, then
> > creates a function with that environment attached and returns it.
> >
> > I think Andrew's approach will work, but changing a function's
> > environment always worries me.  Using local(), the function assigned
> > to plot.foo never has a different environment than the one it ends up
> > with (and I don't need to remember how evalq() works).
>
> Thanks everyone for these suggestions.  They seem a great deal
> less shaganappi/kludgy than my previous approaches.
>
> I've never really felt totally comfortable with the environment
> concept, despite have used it quite a bit (basically in a
> hammer-and-hope style.)
>
> Can anyone comment on the difference between Deepayan's suggestion
> (create a new environment in the package) and Duncan's suggestion
> (create an environment that is local to plot.foo())?  Are there pros
> and cons between the two?
>
> And Deepayan:  what is the rationale for not exporting the new
> environment that you suggest creating?  Presumably this guards against
> something.  What?  I'd just like to extend my (currently minimal)
> comprehension of the issues.
>
> I must admit that Andrew's suggestion kind of overwhelms and bewilders
> me.  I really have no idea what evalq() does.  I guess I could RTFM,
> but the thought of doing that scares me! :-)
>
> Thanks again everybody.
>
> cheers,
>
> Rolf
>
> --
> Honorary Research Fellow
> Department of Statistics
> University of Auckland
> Phone: +64-9-373-7599 ext. 88276
>
>

[[alternative HTML version deleted]]

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel

[Bioc-devel] How to simplify install when package dependency requires compilation from source

[Shraddha Pai]

Hi all,

How do I simplify install when a dependency is available only in source
form and requires compilation using C/C++/Fortran?

My package netDx has "clusterExperiment" as a dependency (Suggests).

When I run BiocManager::install("netDx",dependencies=TRUE), I get:
"Package which is only available in source form, and may need compilation of
  C/C++/Fortran: ‘clusterExperiment’
Do you want to attempt to install these from sources? (Yes/no/cancel)"

If I say yes, it fails because the laptop doesn't have gcc installed, and
then I have to go run "brew install gcc" (this is OS X). This is getting
too complicated for your average user.

Is there a way I could precompile needed packages and just make them
available to my users? Are there instructions for how to do this somewhere?
I can host the compiled version on a lab server.

Thanks,
Shraddha


--

*Shraddha Pai, PhD*
Principal Investigator, OICR
Assistant Professor, Department of Medical Biophysics, University of Toronto
shraddhapai.com; @spaiglass on Twitter
https://pailab.oicr.on.ca


*Ontario Institute for Cancer Research*
MaRS Centre, 661 University Avenue, Suite 510, Toronto, Ontario, Canada M5G
0A3
*@OICR_news*  | *www.oicr.on.ca*




*Collaborate. Translate. Change lives.*



This message and any attachments may contain confidentia...{{dropped:11}}

___
Bioc-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/bioc-devel

Re: [Bioc-devel] mosbi package: Suggested dependency not available

[Kern, Lori]

Yes you will be fine. It looks like runibic is failing possibly of a missing 
dependency on our end and we will investigate.  Once the package builds your 
package should also build correctly.


Cheers



Lori Shepherd

Bioconductor Core Team

Roswell Park Comprehensive Cancer Center

Department of Biostatistics & Bioinformatics

Elm & Carlton Streets

Buffalo, New York 14263


From: Bioc-devel  on behalf of Tim Daniel 
Rose 
Sent: Thursday, October 21, 2021 9:03 AM
To: bioc-devel@r-project.org 
Subject: [Bioc-devel] mosbi package: Suggested dependency not available

Hi everyone,

My new package mosbi
(https://bioconductor.org/packages/devel/bioc/html/mosbi.html) gets a
MacOS build 'ERROR', because a suggested dependency 'runibic' is not
available for that platform.

The reviewers stated on github
(https://secure-web.cisco.com/14eSzrEiAByl2rDPyDqX7C6ssESP9c7weoJHn1WbC4-92uBIuqNNPKwxibQdqC29ln5mUT8DswyAnmxgVprBPBgce_IfGc99QCllfD0NtVv_WcxcB4gLzIqjV8OIdaFeayLxT2e4SPC63XWfMCFjIjHhDfxs6_l4dA1LyUtVWMFBZPFikPZsUwBQ5AbSneNpm3ZgSugHP8xkrf2F-o1e725NfvIyFFKdRIk4lAQXie3mgLAsgX84leu4uiWSSdRzTClVu2kAWkHwDUD34pJlRs2tKgBDo7MC8CuhKL62uKugxJEQyVCHDBpYWtbZv0THZ/https%3A%2F%2Fgithub.com%2FBioconductor%2FContributions%2Fissues%2F2252)
 that it
should be okay, but they could not reproduce the error locally.

On the release schedule site
(https://bioconductor.org/developers/release-schedule/) it says, that
packages need to pass builds and checks without errors until tomorrow.

I want to check here again, if it is okay like this or if I need to take
further action.

Best,
Tim

--
Tim Daniel Rose, MSc

PhD candidate at
Junior Research Group LipiTUM
Chair of Experimental Bioinformatics
TUM School of Life Sciences
Technical University of Munich (TUM)

Maximus-von-Imhof-Forum 3
85354 Freising-Weihenstephan
Germany

mail: tim.r...@wzw.tum.de
web: www.lipitum.de

___
Bioc-devel@r-project.org mailing list
https://secure-web.cisco.com/1YLC8ocJPS7PoYgSR9H-3gWC9z_Ho1lnVHhX3_drWyo1Irn-BQs4tR6mR2pcRZnUQrUSRbQ8ZMYtG6RZoKob7-W5QjKySO2hJ1KOKRxse3tUJj9OSyILOaviJV4EsTi4qu7N04Gpbf3JvNPsf1YsP2SuzjfBqFi0UFPFLGt56KRUltNH90fXIBoZiECd_RXTOCuxOrNrkXj5PmbilmjV02vlIjgoWspqJxy2tfN2dz_118TCrY4pymh_4u9sQcq6ZMgeJW5i-25cJY4l8snUNtc6-jDXXFwOJznXoRYWAYhzBb3ah4INu5uOqkngyYsI9/https%3A%2F%2Fstat.ethz.ch%2Fmailman%2Flistinfo%2Fbioc-devel



This email message may contain legally privileged and/or confidential 
information.  If you are not the intended recipient(s), or the employee or 
agent responsible for the delivery of this message to the intended 
recipient(s), you are hereby notified that any disclosure, copying, 
distribution, or use of this email message is prohibited.  If you have received 
this message in error, please notify the sender immediately by e-mail and 
delete this email message from your computer. Thank you.
[[alternative HTML version deleted]]

___
Bioc-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/bioc-devel

Re: [R-pkg-devel] Is there a better way ...?

[Martin Maechler]

> Duncan Murdoch 
> on Thu, 21 Oct 2021 08:09:02 -0400 writes:

> I agree with almost everything Deepayan said, but would add one thing:
> On 21/10/2021 3:41 a.m., Deepayan Sarkar wrote:
> ...

>> My suggestion is having a package-specific environment, and Duncan's
>> is to have a function-specific environment. If you only need this for
>> this one function, then that should be good enough. If you eventually
>> want to access the persistent information from multiple functions,
>> having a package-specific environment would be more useful.

> I agree with that statement, but those aren't the only two choices. 
> Your local() call can create several functions and return them in a 
> list; then just those functions have access to the local variables.  For 
> example,

> createFns <- local({

> .fooInfo <- NULL

> fn1 <- function (...) { ... }
> fn2 <- function (...) { ... }

> list(fn1 = fn1, fn2 = fn2)
> })

> fns <- createFns()
> fn1 <- fns$fn1
> fn2 <- fns$fn2

> Now fn1 and fn2 are functions that can see .fooInfo, and nobody else can 
> (without going through contortions).

> One other difference between this approach and the package-specific 
> environment:  there's only one package-specific environment in 
> Deepayan's formulation, but I could call createFns() several times, 
> creating several pairs of functions, each pair with its own independent 
> version of .fooInfo.

> I don't know if that's something that would be useful to you, but 
> conceivably you'd want to maintain partial plots in several different 
> windows, and that would allow you to do so.

Note that the above approach has been how  nls()  has been
implemented for R ... a very long time ago {before R 1.0.0}

e.g. from  example(nls) :

DNase1 <- subset(DNase, Run == 1)
fm1 <- nls(density ~ SSlogis(log(conc), Asym, xmid, scal), DNase1)
str(fm1 $ m)
> List of 16
>  $ resid :function ()  
>  $ fitted:function ()  
>  $ formula   :function ()  
>  $ deviance  :function ()  
>  $ lhs   :function ()  
>  $ gradient  :function ()  
>  $ conv  :function ()  
>  $ incr  :function ()  
>  $ setVarying:function (vary = rep_len(TRUE, np))  
>  $ setPars   :function (newPars)  
>  $ getPars   :function ()  
>  $ getAllPars:function ()  
>  $ getEnv:function ()  
>  $ trace :function ()  
>  $ Rmat  :function ()  
>  $ predict   :function (newdata = list(), qr = FALSE)  
>  - attr(*, "class")= chr "nlsModel"

## so 16 functions, all sharing the *same* environment very
## efficiently and nicely

## this is *the* environment for the fitted model :
fmE <- environment(fm1$m[[1]])
ls.str(fmE)
> convCrit : function ()  
> dev :  num 0.00479
> env :  
> form : Class 'formula'  language density ~ SSlogis(log(conc), Asym, xmid, 
> scal)
> getPars : function ()  
>  
>  
>  

so the environment "contains" the functions themselves (but quite
a few more things) and for an environment that means it only
has pointers to the same function objects which are *also* in  `fm1$m`.

So, there has been a nice convincing and important example on
how to do this - inside R for more than two decennia.

Martin Maechler

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel

[Bioc-devel] mosbi package: Suggested dependency not available

[Tim Daniel Rose]

Hi everyone,

My new package mosbi 
(https://bioconductor.org/packages/devel/bioc/html/mosbi.html) gets a 
MacOS build 'ERROR', because a suggested dependency 'runibic' is not 
available for that platform.


The reviewers stated on github 
(https://github.com/Bioconductor/Contributions/issues/2252) that it 
should be okay, but they could not reproduce the error locally.


On the release schedule site 
(https://bioconductor.org/developers/release-schedule/) it says, that 
packages need to pass builds and checks without errors until tomorrow.


I want to check here again, if it is okay like this or if I need to take 
further action.


Best,
Tim

--
Tim Daniel Rose, MSc

PhD candidate at
Junior Research Group LipiTUM
Chair of Experimental Bioinformatics
TUM School of Life Sciences
Technical University of Munich (TUM)

Maximus-von-Imhof-Forum 3
85354 Freising-Weihenstephan
Germany

mail: tim.r...@wzw.tum.de
web: www.lipitum.de

___
Bioc-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/bioc-devel

Re: [R-pkg-devel] Is there a better way ...?

[Duncan Murdoch]

I agree with almost everything Deepayan said, but would add one thing:


On 21/10/2021 3:41 a.m., Deepayan Sarkar wrote:
 ...


My suggestion is having a package-specific environment, and Duncan's
is to have a function-specific environment. If you only need this for
this one function, then that should be good enough. If you eventually
want to access the persistent information from multiple functions,
having a package-specific environment would be more useful.


I agree with that statement, but those aren't the only two choices. 
Your local() call can create several functions and return them in a 
list; then just those functions have access to the local variables.  For 
example,


createFns <- local({

   .fooInfo <- NULL

   fn1 <- function (...) { ... }
   fn2 <- function (...) { ... }

   list(fn1 = fn1, fn2 = fn2)
})

fns <- createFns()
fn1 <- fns$fn1
fn2 <- fns$fn2

Now fn1 and fn2 are functions that can see .fooInfo, and nobody else can 
(without going through contortions).


One other difference between this approach and the package-specific 
environment:  there's only one package-specific environment in 
Deepayan's formulation, but I could call createFns() several times, 
creating several pairs of functions, each pair with its own independent 
version of .fooInfo.


I don't know if that's something that would be useful to you, but 
conceivably you'd want to maintain partial plots in several different 
windows, and that would allow you to do so.


And there are other choices too:  there are several packages 
implementing object systems that allow objects to maintain persistent 
data.   I haven't used those, so this list may contain omissions and 
errors:  R6, R.oo, proto.


Duncan Murdoch

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel

Re: [Bioc-devel] Package name

[Kern, Lori]

Good point.  I'll open an issue on the github to fix.


Lori Shepherd

Bioconductor Core Team

Roswell Park Comprehensive Cancer Center

Department of Biostatistics & Bioinformatics

Elm & Carlton Streets

Buffalo, New York 14263


From: Bioc-devel  on behalf of Laurent Gatto 

Sent: Thursday, October 21, 2021 12:53 AM
To: bioc-devel@r-project.org 
Subject: [Bioc-devel] Package name

The Package Guidelines for Developers and Reviewers say that:

A package name should be descriptive and should not already exist as a current 
package (case-insensitive) in Bioconductor nor CRAN.

The sentences says current packages - does that imply that names of packages 
that have been archived (on CRAN) or deprecated (on Bioconductor) are 
available? This is likely to lead to serious confusion.

Laurent

___
Bioc-devel@r-project.org mailing list
https://secure-web.cisco.com/18tLjfrOdSZ-K_8neKbEy5VWz_fgbNJthSRI3zRVyXXtc-p9kCgNhG51wWXnY7UGhy4yP_imTwLGoP4BCIicB_fqzg9U937WF_IJiOPJh7NnfQXFLeEV-SiiJJ1eCyN2vaJFacWPvahAlN135mDHZNw_peW0Yl4BOq8m2QBMh4i952Nt6oghMQpSWSjaP_2bN4VKIBT2ZP-A7pDqddlOSeCCaMEKJZp_6w1WthdY69MB6lAbsF-i9uX3JVNSCmAlXW3YMNOfVEBijto4EJaGIUJMJwGX_vec9kTf9gtFiYztotSHNfquFZ4GlaHmXeHwPaBEtazOY5fPiuzLjzDK52Q/https%3A%2F%2Fstat.ethz.ch%2Fmailman%2Flistinfo%2Fbioc-devel



This email message may contain legally privileged and/or confidential 
information.  If you are not the intended recipient(s), or the employee or 
agent responsible for the delivery of this message to the intended 
recipient(s), you are hereby notified that any disclosure, copying, 
distribution, or use of this email message is prohibited.  If you have received 
this message in error, please notify the sender immediately by e-mail and 
delete this email message from your computer. Thank you.
[[alternative HTML version deleted]]

___
Bioc-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/bioc-devel

Re: [Rd] BUG?: R CMD check with --as-cran *disables* checks for unused imports otherwise performed

[Jeffrey Dick]

FWIW, I also encountered this issue and posted on R-pkg-devel about it,
with no resolution at the time (May 2020). See "Dependencies NOTE lost with
--as-cran" (
https://stat.ethz.ch/pipermail/r-package-devel/2020q2/005467.html)

On Wed, Oct 20, 2021 at 11:55 PM Henrik Bengtsson <
henrik.bengts...@gmail.com> wrote:

> ISSUE:
>
> Using 'R CMD check' with --as-cran,
> set_R_CHECK_PACKAGES_USED_IGNORE_UNUSED_IMPORTS_=TRUE, whereas the
> default is FALSE, which you get if you don't add --as-cran.
> I would expect --as-cran to check more things and more be conservative
> than without.  So, is this behavior a mistake?  Could it be a thinko
> around the negating "IGNORE", and the behavior is meant to be vice
> verse?
>
> Example:
>
> $ R CMD check QDNAseq_1.29.4.tar.gz
> ...
> * using R version 4.1.1 (2021-08-10)
> * using platform: x86_64-pc-linux-gnu (64-bit)
> ...
> * checking dependencies in R code ... NOTE
> Namespace in Imports field not imported from: ‘future’
>   All declared Imports should be used.
>
> whereas, if I run with --as-cran, I don't get that NOTE;
>
> $ R CMD check --as-cran QDNAseq_1.29.4.tar.gz
> ...
> * checking dependencies in R code ... OK
>
>
> TROUBLESHOOTING:
>
> In src/library/tools/R/check.R [1], the following is set if --as-cran is
> used:
>
>   Sys.setenv("_R_CHECK_PACKAGES_USED_IGNORE_UNUSED_IMPORTS_" = "TRUE")
>
> whereas, if not set, the default is:
>
> ignore_unused_imports <-
>
> config_val_to_logical(Sys.getenv("_R_CHECK_PACKAGES_USED_IGNORE_UNUSED_IMPORTS_",
> "FALSE"))
>
> [1]
> https://github.com/wch/r-source/blob/b50e3f755674cbb697a4a7395b766647a5cfeea2/src/library/tools/R/check.R#L6335
> [2]
> https://github.com/wch/r-source/blob/b50e3f755674cbb697a4a7395b766647a5cfeea2/src/library/tools/R/QC.R#L5954-L5956
>
> /Henrik
>
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>

[[alternative HTML version deleted]]

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] stats::fft produces inconsistent results

[GILLIBERT, Andre]

> Haha, thanks : ) I guess I will probably be grouchy too if seeing so many 
> people making the same mistakes again and again. It just happened to be me.


Fortunately, you did not get offensed. :)


This is nice to have a large community of developers for R packages, even if, 
sometimes, buggy packages are annoying R developers because any small change in 
R may "break" them even though they were actually broken from the begining.


> Indeed, I found myself often confused about when to PROTECT and when not.



A (relatively) quick explanation.

There are several �pools� of data objects that have different rules. The most 
common �pool� is the pool of garbage collectable R objects, that can be 
allocated with allocVector and is passed from R to C code and vice versa. 
Another pool is the malloc/free pool, that works with explicit 
allocation/deallocation. R does not modify the malloc/free implementation in 
any way, and memory leaks may happen. Operating systems may have other pools of 
memory (e.g. mmap'ed memory) that are not handled by R either. There is also a 
transient storage (R_alloc/vmaxset/vmaxget) that is automatically freed when 
returning from C to R, and should be used for temporary storage but not for 
objects returned to R code.



The PROTECT system is needed for garbage collectable objects.

The garbage collector may trigger whenever a R internal function is called. 
Typically, when some memory is internally allocated.

The garbage collector frees objects that are neither referenced directly nor 
indirectly from R code and from the PROTECT stack.

The PROTECT stack is used by C code to make sure objects that are not yet (or 
will never be) referenced by R code, are not destroyed when the garbage 
collector runs.



The functions allocating new R objects, such as allocVector(), but also 
coerceVector(), duplicate(),return unprotected objects, that may be destroyed 
the next time an internal R function is called, unless it is explicitly 
PROTECT'ed before. Indeed, such objects would have no reference from R code and 
so, would be deleted.


The PROTECT stack must be balanced on a call from R to a C function. There must 
be as many UNPROTECT'ions than PROTECT'ions.

The typical C code PROTECTs any object allocated as soon as it is allocated 
(e.g. call to allocVector or coerceVector). It UNPROTECTs temporary objects to 
"free" them (the actual memory release may be delayed to the next garbage 
collection). It UNPROTECTs the object it returns to R code. Indeed, in pure C 
code, there will be no garbage collection between the time the object is 
UNPROTECTed and the time R grabs the object. You must be very careful if you 
are using C++, because destructors must not call any R internal function that 
may trigger a garbage collection.
The arguments to the C code, do not have to be PROTECT'ed, unless they are 
re-allocated. For instance, it is frequent to call coerceVector or arguments 
and re-assign them to the C variable that represents the argument. The new 
object must be PROTECT'ed.


Actually, you do not need to *directly* PROTECT all objects that are allocated 
in the C function, but you must make sure that all objects are *indirectly* 
PROTECT'ed. For instance, you may allocate a VECSXP (a "list" in R) and fill 
the slots with newly allocated objects. You only need to PROTECT the VECSXP, 
since its slots are indirectly protected.


If you have any doubt, it is not a bug to over-PROTECT objects. It may slightly 
slow down garbage collection and use space on the PROTECTion stack, but that is 
rarely a big deal. You should only avoid that when that would lead to thousands 
or millions of protections.


As I said, the PROTECT stack must be balanced between the entry and exit of the 
C code. This is not a problem for 99% of functions that free all the memory 
they use internally except the object that is returned. Sometimes, some 
"background" memory, hidden to R code, may have to be allocated for more time. 
A call to R_PreserveObject protects the object, even after the C code returns 
to R, until R_ReleaseObject is called. Without an explicit call to 
R_ReleaseObject, memory is leaked!


There is another mechanism in R that must be known. If you call any R function 
from C code, or any internal R function that may fail with an error, or any 
internal R function that can be stopped by the user (see R_CheckUserInterrupt), 
then, R may call a longjmp to exit all the C code. This is very much 
incompatible with C++ exceptions or constructors/destructors. Rcpp can avoid, 
to some extent, that problem.


With C code, this means that some malloc'ed memory or allocated resources (file 
descriptors, sockets, etc.) may be leaked unless something is done to prevent 
that. All PROTECT'ed objects are automatically unprotected, so there is no 
problem with memory leak of garbage collectable objects. There is a 
R_UnwindProtect() mechanism to free temporary resources (e.g. a socket you 
allocated) when a 

Re: [R-pkg-devel] Is there a better way ...?

[Deepayan Sarkar]

On Thu, Oct 21, 2021 at 12:15 PM Rolf Turner  wrote:
>
>
> On Thu, 21 Oct 2021 02:03:41 -0400
> Duncan Murdoch  wrote:
>
> > On 21/10/2021 12:40 a.m., Andrew Simmons wrote:
> > > I think the simplest answer is to store the variable in the
> > > functions frame. I'm assuming here that the only plot.foo needs
> > > access to .fooInfo, if not this can be changed.
> > >
> > >
> > > plot.foo <- function (...)
> > > {
> > >  .fooInfo
> > > }
> > > environment(plot.foo) <- new.env()
> > > evalq({
> > >  .fooInfo <- NULL
> > > }, environment(plot.foo))
> > >
> > >
> > > Make your function, and do whatever you need with .fooInfo within
> > > said function. Whenever you previously updated .fooInfo in the
> > > global environment, update .fooInfo in plot.foo environment instead.
> > > Also, because .fooInfo is not stored in the package's frame, it
> > > won't be locked when the namespace is sealed. If you created it at
> > > the toplevel, that would create some issues. But this works fine.
> >
> > I agree with the final result, but I'd write the code differently:
> >
> > plot.foo <- local({
> >
> >.fooInfo <- NULL
> >
> >function (...) { ... }
> > })
> >
> > creates an environment, puts .fooInfo into it with value NULL, then
> > creates a function with that environment attached and returns it.
> >
> > I think Andrew's approach will work, but changing a function's
> > environment always worries me.  Using local(), the function assigned
> > to plot.foo never has a different environment than the one it ends up
> > with (and I don't need to remember how evalq() works).
>
> Thanks everyone for these suggestions.  They seem a great deal
> less shaganappi/kludgy than my previous approaches.
>
> I've never really felt totally comfortable with the environment
> concept, despite have used it quite a bit (basically in a
> hammer-and-hope style.)
>
> Can anyone comment on the difference between Deepayan's suggestion
> (create a new environment in the package) and Duncan's suggestion
> (create an environment that is local to plot.foo())?  Are there pros
> and cons between the two?

My suggestion is having a package-specific environment, and Duncan's
is to have a function-specific environment. If you only need this for
this one function, then that should be good enough. If you eventually
want to access the persistent information from multiple functions,
having a package-specific environment would be more useful.

I'm not sure what you are trying to do, but I can't see how you can do
something sensible with a function-specific environment if someone
does

plot.foo(something)
plot.default(1:10)
plot.foo(something else, add = TRUE)

So maybe you would eventually want to set a hook (?setHook) for
plot.new to ensure that no other plot has been created in between,
which could write into this package-specific environment.

> And Deepayan:  what is the rationale for not exporting the new
> environment that you suggest creating?  Presumably this guards against
> something.  What?  I'd just like to extend my (currently minimal)
> comprehension of the issues.

Nothing other than the usual reason for not exporting things
unnecessarily, which is to not pollute the user workspace.

> I must admit that Andrew's suggestion kind of overwhelms and bewilders
> me.  I really have no idea what evalq() does.  I guess I could RTFM,
> but the thought of doing that scares me! :-)

Andrew's suggestion looks more complicated than it is. Think of
.fooInfo as a "global" variable, just in your package namespace rather
than .GlobalEnv, so you could do (in your package code)

.fooInfo <- NULL

plot.foo <- function(...)
{
   if (is.null(.fooInfo)) ... # use .fooInfo
   .fooInfo <<- something # set .fooInfo
}

Andrew suggested a separate (and unnamed) environment to store both
.fooInfo and plot.foo, so the setting part becomes a bit more
complicated (but accessing becomes safer in the sense that no other
function can access .fooInfo).

My suggestion is essentially similar, except that you can use <-
instead of <<- because it's an environment.

.fooEnv <- new.env()

plot.foo <- function(...)
{
   if (is.null(.fooEnv$info)) ... # use .fooEnv$info
   .fooEnv$info <- something # set .fooEnv$info
}

Best,
-Deepayan

> Thanks again everybody.
>
> cheers,
>
> Rolf
>
> --
> Honorary Research Fellow
> Department of Statistics
> University of Auckland
> Phone: +64-9-373-7599 ext. 88276
>

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel

Re: [R-pkg-devel] Is there a better way ...?

[Rolf Turner]

On Thu, 21 Oct 2021 02:03:41 -0400
Duncan Murdoch  wrote:

> On 21/10/2021 12:40 a.m., Andrew Simmons wrote:
> > I think the simplest answer is to store the variable in the
> > functions frame. I'm assuming here that the only plot.foo needs
> > access to .fooInfo, if not this can be changed.
> > 
> > 
> > plot.foo <- function (...)
> > {
> >  .fooInfo
> > }
> > environment(plot.foo) <- new.env()
> > evalq({
> >  .fooInfo <- NULL
> > }, environment(plot.foo))
> > 
> > 
> > Make your function, and do whatever you need with .fooInfo within
> > said function. Whenever you previously updated .fooInfo in the
> > global environment, update .fooInfo in plot.foo environment instead.
> > Also, because .fooInfo is not stored in the package's frame, it
> > won't be locked when the namespace is sealed. If you created it at
> > the toplevel, that would create some issues. But this works fine.
> 
> I agree with the final result, but I'd write the code differently:
> 
> plot.foo <- local({
> 
>.fooInfo <- NULL
> 
>function (...) { ... }
> })
> 
> creates an environment, puts .fooInfo into it with value NULL, then 
> creates a function with that environment attached and returns it.
> 
> I think Andrew's approach will work, but changing a function's 
> environment always worries me.  Using local(), the function assigned
> to plot.foo never has a different environment than the one it ends up
> with (and I don't need to remember how evalq() works).

Thanks everyone for these suggestions.  They seem a great deal
less shaganappi/kludgy than my previous approaches.

I've never really felt totally comfortable with the environment
concept, despite have used it quite a bit (basically in a
hammer-and-hope style.)

Can anyone comment on the difference between Deepayan's suggestion
(create a new environment in the package) and Duncan's suggestion
(create an environment that is local to plot.foo())?  Are there pros
and cons between the two?

And Deepayan:  what is the rationale for not exporting the new
environment that you suggest creating?  Presumably this guards against
something.  What?  I'd just like to extend my (currently minimal)
comprehension of the issues.

I must admit that Andrew's suggestion kind of overwhelms and bewilders
me.  I really have no idea what evalq() does.  I guess I could RTFM,
but the thought of doing that scares me! :-)

Thanks again everybody.

cheers,

Rolf

-- 
Honorary Research Fellow
Department of Statistics
University of Auckland
Phone: +64-9-373-7599 ext. 88276

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel

Re: [R-pkg-devel] Is there a better way ...?

[Duncan Murdoch]

On 21/10/2021 12:40 a.m., Andrew Simmons wrote:

I think the simplest answer is to store the variable in the functions
frame. I'm assuming here that the only plot.foo needs access to .fooInfo,
if not this can be changed.


plot.foo <- function (...)
{
 .fooInfo
}
environment(plot.foo) <- new.env()
evalq({
 .fooInfo <- NULL
}, environment(plot.foo))


Make your function, and do whatever you need with .fooInfo within said
function. Whenever you previously updated .fooInfo in the global
environment, update .fooInfo in plot.foo environment instead.
Also, because .fooInfo is not stored in the package's frame, it won't be
locked when the namespace is sealed. If you created it at the toplevel,
that would create some issues. But this works fine.


I agree with the final result, but I'd write the code differently:

plot.foo <- local({

  .fooInfo <- NULL

  function (...) { ... }
})

creates an environment, puts .fooInfo into it with value NULL, then 
creates a function with that environment attached and returns it.


I think Andrew's approach will work, but changing a function's 
environment always worries me.  Using local(), the function assigned to 
plot.foo never has a different environment than the one it ends up with 
(and I don't need to remember how evalq() works).


Duncan Murdoch




On Thu, Oct 21, 2021 at 12:29 AM Rolf Turner 
wrote:



I have a plot method (say plot.foo()) that I want to be able to call so
that if argument "add" is set equal to TRUE, then further structure will
be added to the same plot.  This is to be used *only* in the context in
which the plot being added to was created using plot.foo().

[Please don't ask me why I don't do everything in a single call to
plot.foo().  There *are* reasons.]

In order to make sure that the "further structure" has the appropriate
form, the call to plot.foo(...,add=TRUE,...) needs to access information
about what was done in the previous call to plot.foo().

Initially I tried to effect this by creating, in a call of the form
plot.foo(...,add=FALSE,...), an object, say ".fooInfo", in the global
environment, and then having the call plot.foo(...,add=TRUE,...)
access the necessary information from ".fooInfo".

It all worked OK, but when I built my package for Windoze, using
win-builder, I got a note of the form:


NOTE
Found the following assignment to the global environment:
File 'pckgename/R/plot.foo.R':
   assign(".fooInfo", fooInfo, pos = 1)


I thought uh-oh; CRAN will kick up a stink if/when I submit my package.

I think I've figured out a work-around using tempfile().  There
are difficulties in that tempfile() creates unique names by tacking
on an alpha-numeric string such as "38348397e595c" to the file name
that I specify, so the call to plot.foo(...,add=TRUE,...) cannot know
the *exact* file name.  I think I can get around that by grepping on
"fooInfo" in list.files(tempdir()).  I also need to make sure that
I unlink() any old instances of files in tempdir() with the string
"fooInfo" in their names before creating a new instance.  Elsewise
ambiguities will ensue.

As I say --- I think this can all be made to work.  But 
Am I missing some "obvious" strategy?  I.e. is there a
better/simpler/less convoluted way of handling this problem?

Grateful for any pearls of wisdom.

cheers,

Rolf Turner

--
Honorary Research Fellow
Department of Statistics
University of Auckland
Phone: +64-9-373-7599 ext. 88276

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel



[[alternative HTML version deleted]]

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel



__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel