Re: [Bioc-devel] EXTERNAL: LazyData: false and accessing data in the YAML section of an Rmd document

[McDavid, Andrew]

How about in the YAML:

params:
  configuration: NULL
  project_conf: NULL
  data: NULL


Then in the first chunk something like:
```{r data-config}
if(is.null(params$data)){
data(data_ionstar, package = "prolfqua")
 data =  data_ionstar
configuration = data$filtered()$config
}
```


You can use a more ornate expression for these parameters when rendering 
otherwise `rmarkdown::render()`, which I suppose you could call from a Makefile 
if needed when rendering package the vignettes.

Andrew McDavid

Biostatistics and Computational Biology
University of Rochester Medical Center

On Nov 19, 2021, at 6:00 AM, 
bioc-devel-requ...@r-project.org wrote:

If anyone wonders, why do I need it. By this I can reuse the vignette
with a different dataset.


[[alternative HTML version deleted]]

___
Bioc-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/bioc-devel

Re: [Bioc-devel] EXTERNAL: Not possible to create runnable examples for package

[McDavid, Andrew]

Hi Kelsey,

1. If you are writing a wrapper, then do all the intermediate steps need to be 
exposed and exported to the end user?  If you don't @export a function, you 
don't need an example (though it can still be useful to write one for your own 
purposes).   If the intermediate steps do need to be exported, then users 
(including, in my own experience, myself 6 months from now ;) ) will have a 
hard time understanding the format or state requirements if there aren't 
self-contained examples they can run.
2.  If the data is large, can you find a way to use a very minimal, perhaps 
even simulated, toy example to illustrate the functionality?  The package 
scater does a good job of this, IMO.  They use a purpose-built function for 
generate small, but illustrative example data. For instance, 
https://rdrr.io/github/davismcc/scater/src/R/plotRowData.R. You might also read 
about ways to mock objects, though I think this would be more appropriate to 
mimic external services rather than the scenario you describe here.
3.  With roxygen2, you can factor out repeated code using the @example tag, 
which includes code from an external file. For instance see 
https://github.com/amcdavid/CellaRepertorium/search?q=small_cluster_example.R
4.  If it's difficult to write examples for functions that are part of the 
exported interface, it can be a sign that the interface needs to redesigned.

So in summary, I don't think it's a good idea to waive this requirement if you 
want others to be able to easily use your package.  If you share a repository, 
others might be able to chime in with more specific suggestions about the 
interface design.


Andrew McDavid

Biostatistics and Computational Biology
University of Rochester Medical Center

On 8/10/21 11:48 AM, Kelsey Chetnik via Bioc-devel wrote:
I am preparing a package for submission to Bioconductor. One of the errors I 
got from running BioCheck() was "At least 80%of man pages documenting exported 
objects must have runnable examples". The way our package is constructed makes 
meeting this requirement practically impossible.

Our package is designed to create pipelines by chaining functions and saving 
intermediate results in an object. These functions are not meant to be run in 
small examples - many require a series of other functions to be run before they 
can be called. It wouldn't be possible to make the examples runnable without 
either large intermediate objects that would cause our package to exceed the 
size limit of 5 MB or long examples that could consist of 20+ lines of code. It 
would also be very complicated and difficult to maintain these examples.

Is it possible for our package to be accepted without resolving this issue if 
we provide justification for it?


[[alternative HTML version deleted]]

___
Bioc-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/bioc-devel

Re: [Bioc-devel] Build issues under macOS

[McDavid, Andrew]

FWIW I am getting the same error in my builds on merida1 for a package that is 
under review:
http://bioconductor.org/spb_reports/CellaRepertorium_buildreport_20200908175926.html#merida1_buildbin_anchor

Similar to Matt, I have Rcpp code, but no fortran code so I am perplexed why 
there is any linking to gfortran happening in the first place, what 
install_name_tool is trying to do to the .so, and how I could attempt to 
reproduce this issue locally to investigate it myself.

Andrew McDavid
Biostatistics and Computational Biology
Office: SRB 4.206 Ph: 585.275.5983

On Sep 9, 2020, at 6:00 AM, 
bioc-devel-requ...@r-project.org wrote:

Date: Wed, 9 Sep 2020 01:03:10 +
From: "Stone, Matt" mailto:mst...@fredhutch.org>>
To: "bioc-devel@r-project.org" 
mailto:bioc-devel@r-project.org>>
Subject: [Bioc-devel] Build issues under macOS
Message-ID: 
<294fbff9-0972-4f77-8b8c-1b8e5a7f5...@fredhutch.org>
Content-Type: text/plain; charset="utf-8"

Hi,

I've begun the submission of a new package [1], but I'm encountering some 
errors in the automated build under macOS on merida1 and would greatly 
appreciate any assistance.

I've currently tried to build twice, once upon initial submission and once 
after updating the DESCRIPTION to remove a redundant Maintainer field. I'm a 
bit confused that each build failed with different errors, as the only 
difference between the two was the removal of the Maintainer field.

In the first build [2], the binary build failed because the installer could not 
link to the fortran libraries. I have previously encountered a similar issue on 
my own macbook and was able to work around it by installing gfortran directly 
[3] rather than through homebrew. I have this noted in the package's README, 
but is there a better solution to ensure compatibility in more environments, 
including merida1?

In the second build [4], the check failed because the package was 
uninstallable, and the binary build failed because edgeR could not be found. 
I'm assuming the latter is the cause of the former error. edgeR is an indirect 
dependency via scran; is it necessary to make it an explicit import?

Before submission, I set up the BiocCheck github action supported by r-lib [5], 
and was able to build the package and run a mostly clean (minus a few NOTEs) 
check/BiocCheck under Ubuntu, Windows, and macOS. Is anyone familiar with the 
distinction between the environments on merida1 and the docker image in this 
workflow? I'd be happy to help update the latter to be consistent with merida1.

Thank you!

Matt


[[alternative HTML version deleted]]

___
Bioc-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/bioc-devel

Re: [Bioc-devel] HDF5Array failure on windows

[McDavid, Andrew]

For what it's worth, I have been getting seemingly the same error from 
HDF5Array, only on windows, on the package builder: 
https://master.bioconductor.org/checkResults/3.11/bioc-LATEST/MAST/tokay2-buildsrc.html

I have been hoping it will magically go away, as I do not have access to any 
windows systems to troubleshoot.  Maybe my wish will be come true :-)?


Andrew McDavid
Biostatistics and Computational Biology
Office: SRB 4.206 Ph: 585.275.5983


Hi Kasper, Martin,

About bsseq's timeout: An important recent change to
DelayedArray/HDF5Array is that, starting with DelayedArray 0.13.8,
nothing in these packages uses parallel evaluation **by default**.
Concretely this means that getAutoBPPARAM() now returns NULL on a fresh
session instead of one of the parallelization backends defined in
BiocParallel (e.g. MulticoreParam on Linux/Mac, SnowParam on Windows).
This could slow down some code in bsseq or other packages that were
benefiting from the automatic parallel evaluation. Now it's the
responsibility of the user to set the parallelization backend (with
setAutoBPPARAM) if they wish things like matrix multiplication, rowsum()
or rowSums() use parallel evaluation again.

Also BiocParallel has been moved from Depends to Suggests.

About bsseq error on Windows: That seems indeed to be related to the
HDF5Array error on the same platform. The unit tests in bsseq and
HDF5Array both fail with the same error ("HDF5 dataset
'./.HDF5ArrayAUTO00014_dimnames/2' does not exist"). I'll take a look.
What's puzzling is that we see this error on both Windows archs for
bsseq (i.e. on 64-bit and 32-bit) while we only see it on 64-bit Windows
for HDF5Array. This suggests something nasty and hard to troubleshoot..
sigh!

Cheers,
H.




[[alternative HTML version deleted]]

___
Bioc-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/bioc-devel

Re: [Bioc-devel] Updating for 3.11

[McDavid, Andrew]

I agree with Martin that the R administration guide should be frequently 
consulted on Mac OS and other systems (and would have save me a couple hours of 
trouble shooting the other week).  However it's not always updated for the most 
recent Mac OS perversions, as I found:  
https://amcdavid.github.io/2019/12/Compiling-R-From-Source. The prebuilt binary 
didn't work for me this time, in that I was getting segfaults from it last 
month that went away once I compiled from source.

And I also agree that there is probably a silver lining in that it helps 
discipline us to avoid unnecessary dependencies, which is of course somewhat in 
tension with the bioconductor philosophy of reuse of components.

Andrew McDavid
Biostatistics and Computational Biology
Office: SRB 4.206 Ph: 585.275.5983

On Jan 8, 2020, at 6:00 AM, 
bioc-devel-requ...@r-project.org wrote:

Date: Tue, 7 Jan 2020 19:40:02 +
From: Martin Morgan mailto:mtmorgan.b...@gmail.com>>
To: Karl Stamm mailto:karl.st...@gmail.com>>, 
"bioc-devel@r-project.org"
mailto:bioc-devel@r-project.org>>
Subject: Re: [Bioc-devel] Updating for 3.11
Message-ID:
mailto:bn8pr04mb552273d7ecc8e6df79c688c0f9...@bn8pr04mb5522.namprd04.prod.outlook.com>>

Content-Type: text/plain; charset="utf-8"

I'm not an expert on macOS, but generally one wants to follow documented R 
procedures.

This changes (corrects?) some of the contortions you indicate below, for 
instance it is not necessary to fiddle with clang4. Also, by using the 
CRAN-distributed binary (from 
https://urldefense.proofpoint.com/v2/url?u=http-3A__mac.r-2Dproject.org_=DwIGaQ=4sF48jRmVAe_CH-k9mXYXEGfSnM3bY53YSKuLUQRxhA=YOujD4cGNGWN0ZskTj96dUHU2a4CkraklBd4BndS21sf3_emyYLIG1llnUarWCda=qhzUtwVNpidVS2ld_L6Ge6zDl1AOT0itOnwoD4dSkG0=qrbZDJn7R1U4XJ9npcLJ0HO7HTEgM2SdHo-uO_6n6ho=
 ) for R-devel one does not need to compile packages (other than your own) from 
source but rather can install the binaries (see pkgType on ?options and 
?install.packages).

I think it's helpful too to ask whether dependencies are all strictly 
necessary, or only convenient (stringr and stringi useage often seems to fall 
into this category, where base string manipulation functionality is often 
sufficient) or providing very marginal functionality that adds minimally to our 
packages -- perhaps, in general and not necessarily your use of DESeq2, our 
packages could adopt a New Year's resolution and go on a dependency diet?

Martin


[[alternative HTML version deleted]]

___
Bioc-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/bioc-devel

Re: [Bioc-devel] Bioc-devel Digest, Vol 182, Issue 7

[McDavid, Andrew]

Off topic, but I am curious how you maintain the parallel installs in your home 
directory given the Mac binary package's proclivities to dump things into 
/Library/Frameworks/?   Do you compile from source, with appropriate --prefix 
to keep your installation sandboxed in ~/bin?

Andrew McDavid
Biostatistics and Computational Biology
Office: SRB 4.206 Ph: 585.275.5983

Message: 1
Date: Tue, 7 May 2019 10:46:49 +
From: Martin Morgan mailto:mtmorgan.b...@gmail.com>>
To: Miles Benton mailto:miles.bento...@gmail.com>>, 
Felix Ernst
mailto:felix.gm.er...@outlook.com>>
Cc: "bioc-devel@r-project.org" 
mailto:bioc-devel@r-project.org>>
Subject: Re: [Bioc-devel] BiocManager for BioC 3.9/R 3.6
Message-ID:
mailto:mn2pr05mb649688873eee7284e89e9c9ef9...@mn2pr05mb6496.namprd05.prod.outlook.com>>

Content-Type: text/plain; charset="utf-8"

On my macOS I do this using command-line aliases and the R_LIBS_USER 
environment variable

 $ alias bioc-3.9
 alias bioc-3.9='R_LIBS_USER=~/Library/R/3.6/Bioc/3.9 ~/bin/R-3-6-branch/bin/R'

but other solutions are possible

Martin Morgan


[[alternative HTML version deleted]]

___
Bioc-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/bioc-devel

Re: [Bioc-devel] Pushing towards a better home for matrix generics

[McDavid, Andrew]

As a casual observer of this thread, my takeaway is that 1. the current 
situation is untenable (e.g. it means that Aaron would have to essentially 
reimplement S4 method dispatch) 2.  Given the long history, and extent of 
reverse dependencies to Matrix, there are good reasons to not mess with the 
signature of its implicit generic (though I don't see much utility in multiple 
dispatch here) and 3.  therefore the least-bad alternative may be to eliminate 
the call to setGeneric('colSums'), etc, in BiocGenerics, hopefully with some 
fixes to the help system to make it more tolerant to S4 method naming.  I 
appreciate that until these fixes are forthcoming it means more work 
maintaining the help aliases for some methods.  How often do we think the 
aliases would be break?

Andrew McDavid
Biostatistics and Computational Biology
Office: SRB 4.206 Ph: 585.275.5983

Message: 1
Date: Sun, 10 Feb 2019 13:36:43 +
From: Aaron Lun 
mailto:infinite.monkeys.with.keyboa...@gmail.com>>
To: "Pages, Herve" mailto:hpa...@fredhutch.org>>, Martin 
Maechler
mailto:maech...@stat.math.ethz.ch>>
Cc: Michael Lawrence 
mailto:lawrence.mich...@gene.com>>,
"bioc-devel@r-project.org" 
mailto:bioc-devel@r-project.org>>
Subject: Re: [Bioc-devel] Pushing towards a better home for matrix
generics
Message-ID: 
<1549805803.3935.11.ca...@gmail.com>
Content-Type: text/plain; charset="utf-8"

Returning to this topic:

It's good to hear some of the rationale behind the current state of
affairs. That said, the set-up we have now is quite difficult to work
with; as mentioned before, I've had to hack around it like:

# Example from "BiocSingular", 
https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_LTLA_BiocSingular=DwIGaQ=4sF48jRmVAe_CH-k9mXYXEGfSnM3bY53YSKuLUQRxhA=YOujD4cGNGWN0ZskTj96dUHU2a4CkraklBd4BndS21sf3_emyYLIG1llnUarWCda=VBZgu2tHS9vBNQ2d8RSG4ijIcp3jyZkfCsRnA1mk7XI=FN4WdWx8gfnjFoLRQuBaIHaKrAjONVa9hsbAyIXSwGo=
.safe_colSums <- function(x) {
if (is(x, "Matrix")) {
Matrix::colSums(x)
} else {
colSums(x)
}
}

... which is ugly, and even worse, still incorrect, e.g., for non-
Matrix classes that have methods for the implicit colSums generic. This
situation is not sustainable for further package development.

Is there a path forward that is palatable to everyone? Or perhaps these
conversations are already happening on R-devel?

-A

On Tue, 2019-01-29 at 18:46 +, Pages, Herve wrote:
Yes the help system could enforce the full signature for the aliases
but
that means the end user then will have to always do
?`colSums,SomeClass,ANY,ANY-method`, which feels unnecessary
complicated
and confusing in the case of a generic where dispatching on the 2nd
and
3rd arguments hardly makes sense.

Or are you saying that the help system should enforce an alias that
strictly matches the signature explicitly used in the setMethod
statement? Problem with this is that then there is no easy way for
the
end user to know a priori which form to use to access the man page.
Is
it ?`colSums,dgCMatrix,ANY,ANY-method` or is it
?`colSums,dgCMatrix-method`. Right now when you type colSums
(after loading the Matrix package), you get this:

   > library(Matrix)
   > colSums
   standardGeneric for "colSums" defined from package "base"

   function (x, na.rm = FALSE, dims = 1, ...)
standardGeneric("colSums")
   
   
   Methods may be defined for arguments: x, na.rm, dims
   Use  showMethods("colSums")  for currently available ones.

This suggests that the correct form is ?`colSums,dgCMatrix,ANY,ANY-
method`.

All this confusion can be avoided by specifying signature="x" in the
definition of the implicit generic. It formalizes where dispatch
really
happens and sets expectations upfront. No loose ends.

Hope this makes sense,

H.


[[alternative HTML version deleted]]

___
Bioc-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/bioc-devel

Re: [Bioc-devel] git transition for projects with prior git history

[McDavid, Andrew]

Hi Nitesh,

Schematically, my git repo started with commit D, while bioconductor's started 
with A.  It's possible this was because I did something wrong managing the 
bioconductor repo, but since I can't rewrite history, there's not anything I 
can do about that now.  Their "founding" commits are distinct.


 BioconductorA---B---C
   / /
 MasterD---E---F---G---I

Since then, I have I have been cherry-picking changes from master onto 
bioconductor per my understanding of recommended practice.  
If I try to merge bioconductor onto master, or vice versa, I get the unrelated 
histories warning.  Vlad's suggestion works, but results in replaying ~700 
commits onto the bioconductor repo...not so nice maybe.

The https://github.com/Bioconductor-mirror/MAST.git repo is to make SVN commits 
from the git tree.


> On Jul 28, 2017, at 11:33 AM, Turaga, Nitesh <nitesh.tur...@roswellpark.org> 
> wrote:
> 
> I would be careful before using the --allow-unrelated-histories flag. Please 
> investigate where there is a difference.
> 
> Also, i don't understand why you are using the bioconductor-git-mirror? Your 
> non-zero commit history should be related to bioconductor git server. 
> 
> Best
> 
> Nitesh
> 
> Get Outlook for Android
> 
> 
> 
> From: Vladimir Kiselev
> Sent: Thursday, July 27, 5:11 PM
> Subject: Re: [Bioc-devel] git transition for projects with prior git history
> To: McDavid, Andrew, bioc-devel@r-project.org
> 
> 
> Hi Andrew, I solved it by just adding '--allow-unrelated-histories' to force 
> the merge: 
> https://stackoverflow.com/questions/37937984/git-refusing-to-merge-unrelated-histories
>  Cheers, Vlad On Thu, Jul 27, 2017 at 9:53 PM McDavid, Andrew < 
> andrew_mcda...@urmc.rochester.edu> wrote: > Is there a recommended recipe to 
> utilize the git.bioconductor.org< > http://git.bioconductor.org> remote with 
> an existing git repo that has > non-zero history? I tried adding the 
> git.bioconductor.org< > http://git.bioconductor.org> as a remote, making a 
> branch, and then > checking out a branch on that remote, but it gave my 
> computer sad. Do I > need to clone a new repo instead? > > Example: > $ git 
> remote -vv > bioc https://github.com/Bioconductor-mirror/MAST.git (fetch) > 
> bioc https://github.com/Bioconductor-mirror/MAST.git (push) > biocgit 
> g...@git.bioconductor.org:packages/MAST > (fetch) > biocgit 
> g...@git.bioconductor.org:packages/MAST > (push) > origin 
> g...@github.com:RGLab/MAST.git (fetch) > origin git
 @github.com:RGLab/MAST.git (push > > $ git fetch biocgit > $ git checkout -b 
bgMaster --track biocgit/master > ... > > ... > $ git merge master bgMaster > 
fatal: refusing to merge unrelated histories > > [[alternative HTML version 
deleted]] > > ___ > 
Bioc-devel@r-project.org mailing list > 
https://stat.ethz.ch/mailman/listinfo/bioc-devel > -- http://genat.uk 
[[alternative HTML version deleted]] 
___ Bioc-devel@r-project.org 
mailing list https://stat.ethz.ch/mailman/listinfo/bioc-devel 
> 
> 
> This email message may contain legally privileged and/or confidential 
> information. If you are not the intended recipient(s), or the employee or 
> agent responsible for the delivery of this message to the intended 
> recipient(s), you are hereby notified that any disclosure, copying, 
> distribution, or use of this email message is prohibited. If you have 
> received this message in error, please notify the sender immediately by 
> e-mail and delete this email message from your computer. Thank you.

___
Bioc-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/bioc-devel

[Bioc-devel] git transition for projects with prior git history

[McDavid, Andrew]

Is there a recommended recipe to utilize the 
git.bioconductor.org remote with an existing git 
repo that has non-zero history?  I tried adding the 
git.bioconductor.org as a remote, making a branch, 
and then checking out a branch on that remote, but it gave my computer sad.  Do 
I need to clone a new repo instead?

Example:
$ git remote -vv
bioc https://github.com/Bioconductor-mirror/MAST.git (fetch)
bioc https://github.com/Bioconductor-mirror/MAST.git (push)
biocgit 
g...@git.bioconductor.org:packages/MAST 
(fetch)
biocgit 
g...@git.bioconductor.org:packages/MAST (push)
origin g...@github.com:RGLab/MAST.git (fetch)
origin g...@github.com:RGLab/MAST.git (push

$ git fetch biocgit
$ git checkout -b bgMaster --track biocgit/master
...

...
$ git merge master bgMaster
fatal: refusing to merge unrelated histories

[[alternative HTML version deleted]]

___
Bioc-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/bioc-devel

Re: [Bioc-devel] Problem pushing new versions of xcms via git svn dcommit

[McDavid, Andrew]

Hi Steffen,

Option 2 worked for me recently with a repo that had previous git history.  I 
think you after you run

git branch --track biocmaster bioc/master

You need to run

git checkout biocmaster

Then you may run git svn dcommit, which should just be a null op at this point. 
 I will add that if your repo has previous git history (like yours) you 
absolutely must cherry pick onto the svn repo or great suffering will ensue.  
This is because, I believe, the histories of the svn repo and git repo are 
different.  Rebasing or merging tries to replay from the first commit of your 
git repo

-Andrew



On Jan 3, 2017, at 6:00 AM, 
bioc-devel-requ...@r-project.org wrote:

Hi all,?and a happy new year!

I am experiencing a lot of frustration getting changes from github
pushed to BioC svn, and that includes some fixes for build issues
in packages depending on xcms. I have tried several options:

1) git svn checkout, git remote add, git merge and git svn dcommit.
2) git clone github repo, update_remotes.sh?and git svn rebase

both of which seem to fail due to the same reason,
which seems to be somewhere around


[[alternative HTML version deleted]]

___
Bioc-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/bioc-devel

[Bioc-devel] Why can't NAMESPACE be a strict subset of DESCRIPTION Imports?

[McDavid, Andrew]

BiocCheck throws a warning when a package is listed as Imports: in DESCRIPTION 
but not used in the NAMESPACE.  This may happen when a developer uses the fully 
qualified names of objects within a package.  I am unclear as to the rationale 
for this warning.

Per Hadley:

It's common for packages to be listed in Imports in DESCRIPTION, but not in 
NAMESPACE. In fact, this is what I recommend: list the package in DESCRIPTION 
so that it's installed, then always refer to it explicitly with pkg::fun(). 
Unless there is a strong reason not to, it's better to be explicit. It's a 
little more work to write, but a lot easier to read when you come back to the 
code in the future.

Per R extensions:

If a package only needs a few objects from another package it can use a fully 
qualified variable reference in the code instead of a formal import. A fully 
qualified reference to the function f in package foo is of the form foo::f.

-Andrew


[[alternative HTML version deleted]]

___
Bioc-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/bioc-devel