Re: [Bioc-devel] Controlling vignette compilation order

Agreed. And the BioC build system doesn’t even CHECK workflow packages, so r75944 actually wouldn’t have an effect at all. P.S. simpleSingleCell has successfully built on the BioC-devel with my custom inter-vignette compilation set-up, so that’s a relief. > On 7 Jan 2019, at 20:13, Pages,

Re: [Bioc-devel] Controlling vignette compilation order

This changes the default for _R_CHECK_BUILD_VIGNETTES_SEPARATELY_ from false to true so only affects the re-built of the vignettes during 'R CMD check'. While this is a step in the right direction, it would be good if  'R CMD build' was modified accordingly i.e. to also build vignettes in

Re: [Bioc-devel] Controlling vignette compilation order

Thanks for the update Martin. At least separate processes will avoid the MAX_DLL problems we had earlier. “simpleSingleCell” is now set up so that each vignette will try to compile any upstream vignettes that it depends on (if those upstream targets haven’t already been built). This is done in

Re: [Bioc-devel] Controlling vignette compilation order

r75944 | ripley | 2019-01-02 03:37:21 -0500 (Wed, 02 Jan 2019) | 1 line making re-building vignettes in separate processes the default from R-devel suggests that stand-alone vignettes are now necessary. Martin On 12/24/18, 3:02 AM, "Bioc-devel on behalf of Aaron Lun" wrote: A working

Re: [Bioc-devel] Controlling vignette compilation order

A working example of knitr caching across workflows is now available at https://github.com/LTLA/BiocWorkCache . It uses “~/chipseq.log” as a log to demonstrate that the code in the most-upstream workflow (“test1.Rmd”) is indeed only executed once during

Re: [Bioc-devel] Controlling vignette compilation order

Yes, I had noticed the vignettes.rds as well, and I figured that would be a problem. I just tried setting set cache=TRUE in my vignettes, implemented such that BUILDing each downstream vignette will also run all upstream vignettes on which it depends (that haven’t already been compiled). If an

Re: [Bioc-devel] Controlling vignette compilation order

Yes, that is the simplest solution, and it’s what I’m doing now. It’s not overly confusing for a reader, but it’s awkward to add new vignettes in the middle of the compilation order, as I then have to rename the others (or give the new vignette a weird name, e.g., “xtra-3b-de.Rmd” to get it to

Re: [Bioc-devel] Controlling vignette compilation order

On Sat, Dec 22, 2018 at 10:56 AM Michael Lawrence wrote: > > Anything that eventually lands in inst/doc is a vignette, I think, so > there might be a hack around that. Just so this is not misread - it's *not* possible to just hack your vignette "product" files (PDF or HTML) into inst/doc and

Re: [Bioc-devel] Controlling vignette compilation order

...but in the end isn't it just simpler to name your vignettes in collation order? Who other than you will be able to parse what you've done? Martin On 12/22/18, 1:56 PM, "Bioc-devel on behalf of Michael Lawrence" wrote: Anything that eventually lands in inst/doc is a vignette, I think,

Re: [Bioc-devel] Controlling vignette compilation order

Anything that eventually lands in inst/doc is a vignette, I think, so there might be a hack around that. On Fri, Dec 21, 2018 at 11:26 PM Aaron Lun wrote: > > I gave it a shot: > > https://github.com/LTLA/DrakeTest > > This uses a single “controller” Rmd file

Re: [Bioc-devel] Controlling vignette compilation order

Sounds like a use case for drake... On Tue, Dec 18, 2018 at 6:58 AM Aaron Lun < infinite.monkeys.with.keyboa...@gmail.com> wrote: > @Michael In this case, the resource produced by vignette X is a > SingleCellExperiment object containing the results of various processing > steps (normalization,

Re: [Bioc-devel] Controlling vignette compilation order

Hi Aaron, Right now 'R CMD build' evaluates all vignettes in the same R session. Personally I see this as an undesirable feature and hope that it will change in the future. Problem with this is that when a vignette hits the max DLL limit, breaking it down into smaller vignettes doesn't help.

Re: [Bioc-devel] Controlling vignette compilation order

@Michael In this case, the resource produced by vignette X is a SingleCellExperiment object containing the results of various processing steps (normalization, clustering, etc.) described in that vignette. I can imagine a lazy evaluation model for this, but it wouldn’t be pretty. If I had

Re: [Bioc-devel] Controlling vignette compilation order

Also perhaps using BiocFileCache so that the result object is only generated once, then cached for future (different session) use. On 12/18/18, 8:35 AM, "Bioc-devel on behalf of Michael Lawrence" wrote: I would recommend against dependencies across vignettes. Ideally someone can pick

Re: [Bioc-devel] Controlling vignette compilation order

I would recommend against dependencies across vignettes. Ideally someone can pick up a vignette and execute the code independently of any other documentation. Perhaps you could move the code generating those shared resources to the package. They could behave lazily, only generating the resource if