[Bioc-devel] DMRcate: TIMEOUT
Hi bioc, Recently, under R/3.4.0, the newest version of DMRcate is taking 47mins to build on my local machine and I am getting TIMEOUTs on the build on the checkResults page http://master.bioconductor.org/checkResults/3.5/bioc-LATEST/DMRcate/malbec2-buildsrc.html. I have not made any changes to the attached data package or routines that warrant this increase in time. The vignette building in particular takes up the bulk of the time and this is where the build hangs. Building the same source on R/3.3.3 only takes 4mins 53secs by comparison. Screendump below. Can someone provide an insight into what they think is happening here? timpet@clark-lab:~/Documents$ time R CMD build DMRcate * checking for file ‘DMRcate/DESCRIPTION’ ... OK * preparing ‘DMRcate’: * checking DESCRIPTION meta-information ... OK * installing the package to build vignettes * creating vignettes ... OK * checking for LF line-endings in source and make files * checking for empty or unneeded directories * building ‘DMRcate_1.12.1.tar.gz’ real47m33.472s user46m55.023s sys0m4.647s timpet@clark-lab:~/Documents$ R -e "sessionInfo()" R version 3.4.0 (2017-04-21) -- "You Stupid Darkness" Copyright (C) 2017 The R Foundation for Statistical Computing Platform: x86_64-pc-linux-gnu (64-bit) R is free software and comes with ABSOLUTELY NO WARRANTY. You are welcome to redistribute it under certain conditions. Type 'license()' or 'licence()' for distribution details. Natural language support but running in an English locale R is a collaborative project with many contributors. Type 'contributors()' for more information and 'citation()' on how to cite R or R packages in publications. Type 'demo()' for some demos, 'help()' for on-line help, or 'help.start()' for an HTML browser interface to help. Type 'q()' to quit R. > sessionInfo() R version 3.4.0 (2017-04-21) Platform: x86_64-pc-linux-gnu (64-bit) Running under: Ubuntu 14.04.5 LTS Matrix products: default BLAS: /usr/lib/libblas/libblas.so.3.0 LAPACK: /usr/lib/lapack/liblapack.so.3.0 locale: [1] LC_CTYPE=en_AU.UTF-8 LC_NUMERIC=C [3] LC_TIME=en_AU.UTF-8LC_COLLATE=en_AU.UTF-8 [5] LC_MONETARY=en_AU.UTF-8LC_MESSAGES=en_AU.UTF-8 [7] LC_PAPER=en_AU.UTF-8 LC_NAME=C [9] LC_ADDRESS=C LC_TELEPHONE=C [11] LC_MEASUREMENT=en_AU.UTF-8 LC_IDENTIFICATION=C attached base packages: [1] stats graphics grDevices utils datasets methods base loaded via a namespace (and not attached): [1] compiler_3.4.0 Best, Tim -- Tim Peters, PhD Bioinformatics Research Officer | Epigenetics Research Laboratory | Genomics and Epigenetics Division Garvan Institute of Medical Research 384 Victoria St., Darlinghurst, NSW, Australia 2010 Tel: +61 (2) 9295 8319 [[alternative HTML version deleted]] ___ Bioc-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/bioc-devel
Re: [Bioc-devel] DMRcate: TIMEOUT
Thanks Martin and Sean, found the culprit. It's DSS::DMLtest(). Looks
like it takes much longer under the new version. I should be able to
attenuate the vignette input considerably so hopefully it will now beat
the timeout.
For R/3.3.3, on a bsseq object with 5000 CpG sites, DMLtest() under
DSS_2.14.0 takes roughly 4 seconds:
> system.time(DSSres <- DMLtest(obj_bsseq, group1=sampnames[1:3],
group2=sampnames[4:6], smoothing=FALSE))
Estimating dispersion for each CpG site, this will take a while ...
user system elapsed
3.724 0.017 3.738
> sessionInfo()
R version 3.3.3 (2017-03-06)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: CentOS release 6.8 (Final)
locale:
[1] LC_CTYPE=en_US.iso885915 LC_NUMERIC=C
[3] LC_TIME=en_US.iso885915 LC_COLLATE=en_US.iso885915
[5] LC_MONETARY=en_US.iso885915 LC_MESSAGES=en_US.iso885915
[7] LC_PAPER=en_US.iso885915 LC_NAME=C
[9] LC_ADDRESS=C LC_TELEPHONE=C
[11] LC_MEASUREMENT=en_US.iso885915 LC_IDENTIFICATION=C
attached base packages:
[1] splines parallel stats4stats graphics grDevices utils
[8] datasets methods base
other attached packages:
[1] DSS_2.14.0 bsseq_1.10.0
[3] limma_3.30.13 SummarizedExperiment_1.4.0
[5] Biobase_2.34.0 DMRcatedata_1.10.1
[7] GenomicRanges_1.26.4 GenomeInfoDb_1.10.3
[9] IRanges_2.8.2 S4Vectors_0.12.2
[11] BiocGenerics_0.20.0
loaded via a namespace (and not attached):
[1] Rcpp_0.12.11 XVector_0.14.1 zlibbioc_1.20.0 munsell_0.4.3
[5] colorspace_1.3-2 lattice_0.20-35plyr_1.8.4 tools_3.3.3
[9] grid_3.3.3 data.table_1.10.4 R.oo_1.21.0 gtools_3.5.0
[13] matrixStats_0.52.2 permute_0.9-4 Matrix_1.2-10 R.utils_2.5.0
[17] bitops_1.0-6 RCurl_1.95-4.8 R.methodsS3_1.7.1 scales_0.4.1
[21] locfit_1.5-9.1
And under R/3.4.0 under DSS_2.16.0 it takes 108 seconds:
> system.time(DSSres <- DMLtest(obj_bsseq, group1=sampnames[1:3],
group2=sampnames[4:6], smoothing=FALSE))
Estimating dispersion for each CpG site, this will take a while ...
user system elapsed
108.305 0.184 108.596
> sessionInfo()
R version 3.4.0 (2017-04-21)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 14.04.5 LTS
Matrix products: default
BLAS: /usr/lib/libblas/libblas.so.3.0
LAPACK: /usr/lib/lapack/liblapack.so.3.0
locale:
[1] LC_CTYPE=en_AU.UTF-8 LC_NUMERIC=C LC_TIME=en_AU.UTF-8
LC_COLLATE=en_AU.UTF-8 LC_MONETARY=en_AU.UTF-8
[6] LC_MESSAGES=en_AU.UTF-8LC_PAPER=en_AU.UTF-8
LC_NAME=C LC_ADDRESS=C LC_TELEPHONE=C
[11] LC_MEASUREMENT=en_AU.UTF-8 LC_IDENTIFICATION=C
attached base packages:
[1] splines parallel stats4stats graphics grDevices
utils datasets methods base
other attached packages:
[1] DSS_2.16.0 bsseq_1.12.1 SummarizedExperiment_1.6.2
DelayedArray_0.2.4 matrixStats_0.52.2
[6] Biobase_2.36.2 DMRcatedata_1.12.0
GenomicRanges_1.28.3 GenomeInfoDb_1.12.1 IRanges_2.10.2
[11] S4Vectors_0.14.2 BiocGenerics_0.22.0
loaded via a namespace (and not attached):
[1] Rcpp_0.12.11XVector_0.16.0 zlibbioc_1.22.0
munsell_0.4.3 colorspace_1.3-2lattice_0.20-35
[7] plyr_1.8.4 tools_3.4.0 grid_3.4.0
data.table_1.10.4 R.oo_1.21.0 gtools_3.5.0
[13] permute_0.9-4 Matrix_1.2-10 GenomeInfoDbData_0.99.0
R.utils_2.5.0 bitops_1.0-6RCurl_1.95-4.8
[19] limma_3.32.2compiler_3.4.0 R.methodsS3_1.7.1
scales_0.4.1 locfit_1.5-9.1
Thanks for your recommendations, I should be fine from here on.
Cheers,
Tim
On 31/05/17 21:36, Martin Morgan wrote:
> On 05/31/2017 06:18 AM, Sean Davis wrote:
>> Hi, Tim.
>>
>> Have you tried building the vignette independently, separate from the
>> package build process? Doing so might give you some hints about which
>> code
>> blocks are the culprits.
>
> Also
>
> > Stangle("DMRcate.Rnw")
> Writing to file DMRcate.R
> > source("DMRcate.R", echo=TRUE, max=Inf)
>
> to generate the R code and process it.
>
> Martin
>
>>
>> Sean
>>
>>
>> On Wed, May 31, 2017 at 1:24 AM, Tim Peters <t.pet...@garvan.org.au>
>> wrote:
>>
>>> Hi bioc,
>>>
>>> Recently, under R/3.4.0, the newest version of DMRcate is taking 47mins
>>> to build on my local machine and I am getting TIMEOUTs on the build on
>>> the checkResults page
>>> http://master.bioconductor.org/checkResults/3.5/bioc-
>>> LATEST/DMRcate/malbec2-buildsrc.html.
>>> I have not made any changes to the attached data package or routines
>>> that warrant this increase in time. The vignette building in particular
>>> takes up the bulk of the time and this is where the build hangs
[Bioc-devel] Access permissions for DMRcate
Hi BioC, I'm attempting to sync the my repo https://github.com/timpeters82/DMRcate-devel with Bioconductor using the process here: http://bioconductor.org/developers/how-to/git/sync-existing-repositories/. However I'm encountering the following error at Step 9: timpet@goodnow-lab:~/Documents/DMRcate$ git push upstream master FATAL: W any packages/DMRcate timpeters82 DENIED by fallthru (or you mis-spelled the reponame) fatal: Could not read from remote repository. Please make sure you have the correct access rights and the repository exists Here are my remotes: timpet@goodnow-lab:~/Documents/DMRcate$ git remote -v originhttps://github.com/timpeters82/DMRcate-devel/ (fetch) originhttps://github.com/timpeters82/DMRcate-devel/ (push) upstream g...@git.bioconductor.org:packages/DMRcate.git<mailto:g...@git.bioconductor.org:packages/DMRcate.git> (fetch) upstream g...@git.bioconductor.org:packages/DMRcate.git<mailto:g...@git.bioconductor.org:packages/DMRcate.git> (push) Possibly related, I noticed that my Bioconductor Git Credentials page https://git.bioconductor.org/BiocCredentials/permissions_by_user/ only gives me access to my newer package "consensus", but not "DMRcate". Can I add DMRcate myself through some process, or alternatively can I get an admin to do so please? Best, Tim -- === Tim Peters, PhD Bioinformatics Research Officer | Immunogenomics Laboratory | Immunology Division Garvan Institute of Medical Research 384 Victoria St., Darlinghurst, NSW, Australia 2010 E: t.pet...@garvan.org.au<mailto:t.pet...@garvan.org.au> | W: http://www.garvan.org.au | P: +612 9295 8325 NOTICE Please consider the environment before printing this email. This message and any attachments are intended for the addressee named and may contain legally privileged/confidential/copyright information. If you are not the intended recipient, you should not read, use, disclose, copy or distribute this communication. If you have received this message in error please notify us at once by return email and then delete both messages. We accept no liability for the distribution of viruses or similar in electronic communications. This notice should not be removed. [[alternative HTML version deleted]] ___ Bioc-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/bioc-devel
Re: [Bioc-devel] Access permissions for DMRcate
Thanks Valerie, much obliged. Cheers, Tim On 14/09/18 06:06, Obenchain, Valerie wrote: Hi Tim, We had 2 accounts for you, one under tim.pet...@csiro.au<mailto:tim.pet...@csiro.au> and one under t.pet...@garvan.org.au<mailto:t.pet...@garvan.org.au>. I've removed the @csiro.au account and your information (i.e., package permissions) should by synced up. Valerie On 09/11/2018 08:14 PM, Tim Peters wrote: Hi BioC, I'm attempting to sync the my repo https://github.com/timpeters82/DMRcate-devel with Bioconductor using the process here: http://bioconductor.org/developers/how-to/git/sync-existing-repositories/. However I'm encountering the following error at Step 9: timpet@goodnow-lab:~/Documents/DMRcate$ git push upstream master FATAL: W any packages/DMRcate timpeters82 DENIED by fallthru (or you mis-spelled the reponame) fatal: Could not read from remote repository. Please make sure you have the correct access rights and the repository exists Here are my remotes: timpet@goodnow-lab:~/Documents/DMRcate$ git remote -v originhttps://github.com/timpeters82/DMRcate-devel/ (fetch) originhttps://github.com/timpeters82/DMRcate-devel/ (push) upstream g...@git.bioconductor.org:packages/DMRcate.git<mailto:g...@git.bioconductor.org:packages/DMRcate.git><mailto:g...@git.bioconductor.org:packages/DMRcate.git><mailto:g...@git.bioconductor.org:packages/DMRcate.git> (fetch) upstream g...@git.bioconductor.org:packages/DMRcate.git<mailto:g...@git.bioconductor.org:packages/DMRcate.git><mailto:g...@git.bioconductor.org:packages/DMRcate.git><mailto:g...@git.bioconductor.org:packages/DMRcate.git> (push) Possibly related, I noticed that my Bioconductor Git Credentials page https://git.bioconductor.org/BiocCredentials/permissions_by_user/ only gives me access to my newer package "consensus", but not "DMRcate". Can I add DMRcate myself through some process, or alternatively can I get an admin to do so please? Best, Tim -- === Tim Peters, PhD Bioinformatics Research Officer | Immunogenomics Laboratory | Immunology Division Garvan Institute of Medical Research 384 Victoria St., Darlinghurst, NSW, Australia 2010 E: t.pet...@garvan.org.au<mailto:t.pet...@garvan.org.au><mailto:t.pet...@garvan.org.au><mailto:t.pet...@garvan.org.au> | W: http://www.garvan.org.au | P: +612 9295 8325 NOTICE Please consider the environment before printing this email. This message and any attachments are intended for the addressee named and may contain legally privileged/confidential/copyright information. If you are not the intended recipient, you should not read, use, disclose, copy or distribute this communication. If you have received this message in error please notify us at once by return email and then delete both messages. We accept no liability for the distribution of viruses or similar in electronic communications. This notice should not be removed. [[alternative HTML version deleted]] ___ Bioc-devel@r-project.org<mailto:Bioc-devel@r-project.org> mailing list https://stat.ethz.ch/mailman/listinfo/bioc-devel This email message may contain legally privileged and/or confidential information. If you are not the intended recipient(s), or the employee or agent responsible for the delivery of this message to the intended recipient(s), you are hereby notified that any disclosure, copying, distribution, or use of this email message is prohibited. If you have received this message in error, please notify the sender immediately by e-mail and delete this email message from your computer. Thank you. -- === Tim Peters, PhD Bioinformatics Research Officer | Immunogenomics Laboratory | Immunology Division Garvan Institute of Medical Research 384 Victoria St., Darlinghurst, NSW, Australia 2010 E: t.pet...@garvan.org.au<mailto:t.pet...@garvan.org.au> | W: http://www.garvan.org.au | P: +612 9295 8325 NOTICE Please consider the environment before printing this email. This message and any attachments are intended for the addressee named and may contain legally privileged/confidential/copyright information. If you are not the intended recipient, you should not read, use, disclose, copy or distribute this communication. If you have received this message in error please notify us at once by return email and then delete both messages. We accept no liability for the distribution of viruses or similar in electronic communications. This notice should not be removed. [[alternative HTML version deleted]] ___ Bioc-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/bioc-devel
Re: [Bioc-devel] Access permissions for DMRcate
Sorry Valerie one more thing: looks like I have access to the data package, but not DMRcate itself - could you sync it as well please? 't.pet...@garvan.org.au<mailto:t.pet...@garvan.org.au>' has access to the following packages: DMRcatedata consensus Cheers, Tim On 14/09/18 15:38, Tim Peters wrote: Thanks Valerie, much obliged. Cheers, Tim On 14/09/18 06:06, Obenchain, Valerie wrote: Hi Tim, We had 2 accounts for you, one under tim.pet...@csiro.au<mailto:tim.pet...@csiro.au><mailto:tim.pet...@csiro.au><mailto:tim.pet...@csiro.au> and one under t.pet...@garvan.org.au<mailto:t.pet...@garvan.org.au><mailto:t.pet...@garvan.org.au><mailto:t.pet...@garvan.org.au>. I've removed the @csiro.au account and your information (i.e., package permissions) should by synced up. Valerie On 09/11/2018 08:14 PM, Tim Peters wrote: Hi BioC, I'm attempting to sync the my repo https://github.com/timpeters82/DMRcate-devel with Bioconductor using the process here: http://bioconductor.org/developers/how-to/git/sync-existing-repositories/. However I'm encountering the following error at Step 9: timpet@goodnow-lab:~/Documents/DMRcate$ git push upstream master FATAL: W any packages/DMRcate timpeters82 DENIED by fallthru (or you mis-spelled the reponame) fatal: Could not read from remote repository. Please make sure you have the correct access rights and the repository exists Here are my remotes: timpet@goodnow-lab:~/Documents/DMRcate$ git remote -v originhttps://github.com/timpeters82/DMRcate-devel/ (fetch) originhttps://github.com/timpeters82/DMRcate-devel/ (push) upstream g...@git.bioconductor.org:packages/DMRcate.git<mailto:g...@git.bioconductor.org:packages/DMRcate.git><mailto:g...@git.bioconductor.org:packages/DMRcate.git><mailto:g...@git.bioconductor.org:packages/DMRcate.git><mailto:g...@git.bioconductor.org:packages/DMRcate.git><mailto:g...@git.bioconductor.org:packages/DMRcate.git><mailto:g...@git.bioconductor.org:packages/DMRcate.git><mailto:g...@git.bioconductor.org:packages/DMRcate.git> (fetch) upstream g...@git.bioconductor.org:packages/DMRcate.git<mailto:g...@git.bioconductor.org:packages/DMRcate.git><mailto:g...@git.bioconductor.org:packages/DMRcate.git><mailto:g...@git.bioconductor.org:packages/DMRcate.git><mailto:g...@git.bioconductor.org:packages/DMRcate.git><mailto:g...@git.bioconductor.org:packages/DMRcate.git><mailto:g...@git.bioconductor.org:packages/DMRcate.git><mailto:g...@git.bioconductor.org:packages/DMRcate.git> (push) Possibly related, I noticed that my Bioconductor Git Credentials page https://git.bioconductor.org/BiocCredentials/permissions_by_user/ only gives me access to my newer package "consensus", but not "DMRcate". Can I add DMRcate myself through some process, or alternatively can I get an admin to do so please? Best, Tim -- === Tim Peters, PhD Bioinformatics Research Officer | Immunogenomics Laboratory | Immunology Division Garvan Institute of Medical Research 384 Victoria St., Darlinghurst, NSW, Australia 2010 E: t.pet...@garvan.org.au<mailto:t.pet...@garvan.org.au><mailto:t.pet...@garvan.org.au><mailto:t.pet...@garvan.org.au><mailto:t.pet...@garvan.org.au><mailto:t.pet...@garvan.org.au><mailto:t.pet...@garvan.org.au><mailto:t.pet...@garvan.org.au> | W: http://www.garvan.org.au | P: +612 9295 8325 NOTICE Please consider the environment before printing this email. This message and any attachments are intended for the addressee named and may contain legally privileged/confidential/copyright information. If you are not the intended recipient, you should not read, use, disclose, copy or distribute this communication. If you have received this message in error please notify us at once by return email and then delete both messages. We accept no liability for the distribution of viruses or similar in electronic communications. This notice should not be removed. [[alternative HTML version deleted]] ___ Bioc-devel@r-project.org<mailto:Bioc-devel@r-project.org><mailto:Bioc-devel@r-project.org><mailto:Bioc-devel@r-project.org> mailing list https://stat.ethz.ch/mailman/listinfo/bioc-devel This email message may contain legally privileged and/or confidential information. If you are not the intended recipient(s), or the employee or agent responsible for the delivery of this message to the intended recipient(s), you are hereby notified that any disclosure, copying, distribution, or use of this email message is prohibited. If you have received this message in error, please notify the sender immediately by e-mail and delete this email message from your computer. Thank you. -- === Tim Peters, PhD Bioinformatics Research O
[Bioc-devel] minfi::makeGenomicRatioSetFromMatrix() Windows mem alloc error
Hi Bioc, There seems to be a Windows-specific memory allocation error when running makeGenomicRatioSetFromMatrix() from within DMRcate::DMR.plot() calling the IlluminaHumanMethylationEPICanno.ilm10b4.hg19 annotation . You can find the offending error in the most recent build here: http://bioconductor.org/checkResults/3.9/bioc-LATEST/DMRcate/tokay2-checksrc.html > DMR.plot(ranges=results.ranges, dmr=1, CpGs=myBetas, phen.col=cols, > genome="hg19", samps=samps) Loading required package: IlluminaHumanMethylationEPICanno.ilm10b4.hg19 Failed with error: 'cannot allocate vector of size 510.7 Mb' Error in makeGenomicRatioSetFromMatrix(CpGs, array = "IlluminaHumanMethylationEPIC", : cannot load annotation package IlluminaHumanMethylationEPICanno.ilm10b4.hg19 Calls: DMR.plot -> makeGenomicRatioSetFromMatrix Execution halted Can the minfi developers investigate this please? The purpose of calling makeGenomicRatioSetFromMatrix() is to access the chromosome position and ID from a methylation matrix with probe IDs as rownames. DMR.plot() also needs the beta values so I am killing 2 birds with 1 stone with this call. Best, Tim -- === Tim Peters, PhD Bioinformatics Research Officer | Immunogenomics Laboratory | Immunology Division Garvan Institute of Medical Research 384 Victoria St., Darlinghurst, NSW, Australia 2010 E: t.pet...@garvan.org.au<mailto:t.pet...@garvan.org.au> | W: http://www.garvan.org.au | P: +612 9295 8325 NOTICE Please consider the environment before printing this email. This message and any attachments are intended for the addressee named and may contain legally privileged/confidential/copyright information. If you are not the intended recipient, you should not read, use, disclose, copy or distribute this communication. If you have received this message in error please notify us at once by return email and then delete both messages. We accept no liability for the distribution of viruses or similar in electronic communications. This notice should not be removed. [[alternative HTML version deleted]] ___ Bioc-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/bioc-devel
[Bioc-devel] DMRcate_2.0.0 and updated DMRcatedata
Hello, I have a major update of DMRcate and its associated data package, DMRcatedata. They both pass R CMD check and BiocCheck for R 3.6.1. Because the new version of DMRcate depends on the updated DMRcatedata package, can I host DMRcatedata (it's ~28MB) somewhere for you to download and update Bioconductor, and then I can sync the new version of DMRcate please? Best, Tim -- === Tim Peters, PhD Bioinformatics Research Officer | Immunogenomics Laboratory | Immunology Division Garvan Institute of Medical Research 384 Victoria St., Darlinghurst, NSW, Australia 2010 E: t.pet...@garvan.org.au | W: http://www.garvan.org.au | P: +612 9295 8325 NOTICE Please consider the environment before printing this email. This message and any attachments are intended for the addressee named and may contain legally privileged/confidential/copyright information. If you are not the intended recipient, you should not read, use, disclose, copy or distribute this communication. If you have received this message in error please notify us at once by return email and then delete both messages. We accept no liability for the distribution of viruses or similar in electronic communications. This notice should not be removed. ___ Bioc-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/bioc-devel
Re: [Bioc-devel] DMRcate_2.0.0 and updated DMRcatedata
Thanks Nitesh, The only reason DMRcatedata isn't passing BiocCheck is because of package numbering and the fact that the old version already exists - which this new version is supposed to overwrite (http://bioconductor.org/spb_reports/DMRcatedata_buildreport_20190912205035.html)<http://bioconductor.org/spb_reports/DMRcatedata_buildreport_20190912205035.html#tokay1_buildsrc_anchor>. I understand the reviewer won't look at the package until it passes, but because this is an update of a (now) ExperimentData package can you ask the reviewer to overlook this please? Best, Tim On 12/9/19 11:51 pm, Turaga, Nitesh wrote: Thanks Tim. The assigned reviewer will review your package and provide suggestions as needed. Best, Nitesh On Sep 12, 2019, at 3:15 AM, Tim Peters <mailto:t.pet...@garvan.org.au> wrote: Hi Nitesh, Thanks, I have transformed DMRcatedata into an ExperimentHub package and started an issue here https://github.com/Bioconductor/Contributions/issues/1247. Cheers, Tim On 23/8/19 3:46 am, Turaga, Nitesh wrote: Hi Tim, Based on what your have mentioned, it seems that DMRcatedata should become an experiment hub package. https://bioconductor.org/packages/devel/bioc/vignettes/ExperimentHub/inst/doc/CreateAnExperimentHubPackage.html Please take a look that vignette. Let me know if you have any questions. Nitesh On Aug 21, 2019, at 7:49 PM, Tim Peters <mailto:t.pet...@garvan.org.au> wrote: Hello, I have a major update of DMRcate and its associated data package, DMRcatedata. They both pass R CMD check and BiocCheck for R 3.6.1. Because the new version of DMRcate depends on the updated DMRcatedata package, can I host DMRcatedata (it's ~28MB) somewhere for you to download and update Bioconductor, and then I can sync the new version of DMRcate please? Best, Tim -- =========== Tim Peters, PhD Bioinformatics Research Officer | Immunogenomics Laboratory | Immunology Division Garvan Institute of Medical Research 384 Victoria St., Darlinghurst, NSW, Australia 2010 E: t.pet...@garvan.org.au<mailto:t.pet...@garvan.org.au> | W: http://www.garvan.org.au | P: +612 9295 8325 NOTICE Please consider the environment before printing this email. This message and any attachments are intended for the addressee named and may contain legally privileged/confidential/copyright information. If you are not the intended recipient, you should not read, use, disclose, copy or distribute this communication. If you have received this message in error please notify us at once by return email and then delete both messages. We accept no liability for the distribution of viruses or similar in electronic communications. This notice should not be removed. ___ Bioc-devel@r-project.org<mailto:Bioc-devel@r-project.org> mailing list https://stat.ethz.ch/mailman/listinfo/bioc-devel This email message may contain legally privileged and/or confidential information. If you are not the intended recipient(s), or the employee or agent responsible for the delivery of this message to the intended recipient(s), you are hereby notified that any disclosure, copying, distribution, or use of this email message is prohibited. If you have received this message in error, please notify the sender immediately by e-mail and delete this email message from your computer. Thank you. -- === Tim Peters, PhD Bioinformatics Research Officer | Immunogenomics Laboratory | Immunology Division Garvan Institute of Medical Research 384 Victoria St., Darlinghurst, NSW, Australia 2010 E: t.pet...@garvan.org.au<mailto:t.pet...@garvan.org.au> | W: http://www.garvan.org.au | P: +612 9295 8325 NOTICE Please consider the environment before printing this email. This message and any attachments are intended for the addressee named and may contain legally privileged/confidential/copyright information. If you are not the intended recipient, you should not read, use, disclose, copy or distribute this communication. If you have received this message in error please notify us at once by return email and then delete both messages. We accept no liability for the distribution of viruses or similar in electronic communications. This notice should not be removed. This email message may contain legally privileged and/or confidential information. If you are not the intended recipient(s), or the employee or agent responsible for the delivery of this message to the intended recipient(s), you are hereby notified that any disclosure, copying, distribution, or use of this email message is prohibited. If you have received this message in error, please notify the sender immediately by e-mail and delete this email message from your computer. Thank you. -- === Tim Peters, PhD Bioinformatics Research Officer | Immunogenomics Laboratory | Immunology Division Garva
Re: [Bioc-devel] check fails on arch 'i386'
Many thanks Kasper, this would be much appreciated.
Best,
Tim
On 18/10/19 12:58 am, Kasper Daniel Hansen wrote:
I see these packages may have something to do with mini. There is a a (very
old, nothing new) issue with the annotation objects in minfi where loading them
uses a _lot_ of ram. For a long time I couldn't understand it, but now I get
it. I should really fix this, and probably _very_soon. It won't have any real
user-visible changes (unless you're using the SNP functionality which is very
nice but not used by many people I think), but it will require a new set of
annotation packages.
Best,
Kasper
On Thu, Oct 17, 2019 at 4:08 AM Tim Peters
mailto:t.pet...@garvan.org.au>> wrote:
Hi Hervé (and Reka),
I am having much the same problem on tokay1 with DMRcate
(https://bioconductor.org/checkResults/3.10/bioc-LATEST/DMRcate/tokay1-checksrc.html).
I've gone through a number of iterations trying to minimise the amount
of memory run by the examples (including your suggestions such as using
GRanges as a container for everything in a CpGannotated object, and
restricting analysis to 100 CpG sites) but the memory allocation error
is persistent, failing to allocate a quite modest ~500Mb.
I suspect it's being caused by minfi::makeGenomicRatioSetFromMatrix()
which is called by cpg.annotate() and DMR.plot() - I started a thread
here around the time of the April release concerning the problem but it
wasn't responded to:
https://stat.ethz.ch/pipermail/bioc-devel/2019-April/015012.html
This leaves me in a bind as minfi::makeGenomicRatioSetFromMatrix() has
exactly the functionality cpg.annotate() and DMR.plot() needs, plus it
provides minimal code and I don't really want to reinvent the wheel here!
Would it be possible to waive the 32-bit Windows build for DMRcate?
Best,
Tim
On 15/10/19 4:34 am, Pages, Herve wrote:
> Hi Toth,
>
> Not clear to me what's going on but I kind of suspect this might have
> something to do with the use of data.table.
>
> A few things to keep in mind:
>
> - 'R CMD check' runs all the example in the same R session. This means
> that the outcome of the examples of a given man page can be affected by
> the examples from other man pages.
>
> - A process cannot allocate more than 3 Gb of memory on 32-bit Windows.
> There is no such limit on 64-bit Windows or on the other platforms we
> support (Linux and Mac). On these platforms the maximum amount of memory
> a process can allocate only depends on what's available on the system.
>
> - Generally speaking, sticking to the Bioconductor way of doing things
> (e.g use of GRanges objects instead of data.table objects, use of
> findOverlaps() instead of data.table::foverlaps()) will lead to faster
> code and reduced memory usage. Therefore I would strongly encourage you
> to consider switching to GRanges/findOverlaps() and to avoid the use of
> data.table in methrix.
>
> Best,
> H.
>
>
> On 10/11/19 09:36, Toth, Reka wrote:
>> Dear BioC developers,
>>
>> The most recent build of methrix fails on Tokay1 server:
>> https://urldefense.proofpoint.com/v2/url?u=https-3A__bioconductor.org_spb-5Freports_methrix-5Fbuildreport-5F20191009174210.html=DwIF-g=eRAMFD45gAfqt84VtBcfhQ=BK7q3XeAvimeWdGbWY_wJYbW0WYiZvSXAJJKaaPhzWA=doO3ej8vD58ZBQWVxkHTaCLk-xipbskav8uofJL2syI=zs3uDnFqyAl-hr_7StB3Yr-tGZW8VPfclbPY_JKtSt8=
>> The error message is the following:
>>
>> ** running examples for arch 'i386' ... ERROR
>> Running examples in 'methrix-Ex.R' failed
>> The error most likely occurred in:
>>
>>> base::assign(".ptime", proc.time(), pos = "CheckExEnv")
>>> ### Name: get_region_summary
>>> ### Title: Extract and summarize methylation or coverage info by regions of
>>> ### interest
>>> ### Aliases: get_region_summary
>>>
>>> ### ** Examples
>>>
>>> data("methrix_data")
>>> get_region_summary(m = methrix_data, regions = data.table(chr = "chr21",
>>> start = 27867971, end = 27868103), type = "M", how = "mean")
>> -Checking for overlaps..
>> Error: cannot allocate vector of size 1.4 Gb
>> Execution halted
>>
>> I am a bit puzzled what the reasons could be, since it passed this step
>> during the previous builds and I didn’t change any of the functions in my
>> last commit.
>> When I run the check on our local Windows, I don’t get any error. We use a
>> very small dataset as example, it definitely should not use a vector of size
>> 1.4 Gb.
>>
>> I would appreciate every input or idea on how to tackle this error.
>> Thanks a lot!
>>
>> Best,
>> Reka
>>
>> Dr. Reka Toth
>> Computational Epigenomics Group
>> Division of Epigenomics and Ca
Re: [Bioc-devel] check fails on arch 'i386'
Hi Hervé (and Reka),
I am having much the same problem on tokay1 with DMRcate
(https://bioconductor.org/checkResults/3.10/bioc-LATEST/DMRcate/tokay1-checksrc.html).
I've gone through a number of iterations trying to minimise the amount
of memory run by the examples (including your suggestions such as using
GRanges as a container for everything in a CpGannotated object, and
restricting analysis to 100 CpG sites) but the memory allocation error
is persistent, failing to allocate a quite modest ~500Mb.
I suspect it's being caused by minfi::makeGenomicRatioSetFromMatrix()
which is called by cpg.annotate() and DMR.plot() - I started a thread
here around the time of the April release concerning the problem but it
wasn't responded to:
https://stat.ethz.ch/pipermail/bioc-devel/2019-April/015012.html
This leaves me in a bind as minfi::makeGenomicRatioSetFromMatrix() has
exactly the functionality cpg.annotate() and DMR.plot() needs, plus it
provides minimal code and I don't really want to reinvent the wheel here!
Would it be possible to waive the 32-bit Windows build for DMRcate?
Best,
Tim
On 15/10/19 4:34 am, Pages, Herve wrote:
Hi Toth,
Not clear to me what's going on but I kind of suspect this might have
something to do with the use of data.table.
A few things to keep in mind:
- 'R CMD check' runs all the example in the same R session. This means
that the outcome of the examples of a given man page can be affected by
the examples from other man pages.
- A process cannot allocate more than 3 Gb of memory on 32-bit Windows.
There is no such limit on 64-bit Windows or on the other platforms we
support (Linux and Mac). On these platforms the maximum amount of memory
a process can allocate only depends on what's available on the system.
- Generally speaking, sticking to the Bioconductor way of doing things
(e.g use of GRanges objects instead of data.table objects, use of
findOverlaps() instead of data.table::foverlaps()) will lead to faster
code and reduced memory usage. Therefore I would strongly encourage you
to consider switching to GRanges/findOverlaps() and to avoid the use of
data.table in methrix.
Best,
H.
On 10/11/19 09:36, Toth, Reka wrote:
Dear BioC developers,
The most recent build of methrix fails on Tokay1 server:
https://urldefense.proofpoint.com/v2/url?u=https-3A__bioconductor.org_spb-5Freports_methrix-5Fbuildreport-5F20191009174210.html=DwIF-g=eRAMFD45gAfqt84VtBcfhQ=BK7q3XeAvimeWdGbWY_wJYbW0WYiZvSXAJJKaaPhzWA=doO3ej8vD58ZBQWVxkHTaCLk-xipbskav8uofJL2syI=zs3uDnFqyAl-hr_7StB3Yr-tGZW8VPfclbPY_JKtSt8=
The error message is the following:
** running examples for arch 'i386' ... ERROR
Running examples in 'methrix-Ex.R' failed
The error most likely occurred in:
base::assign(".ptime", proc.time(), pos = "CheckExEnv")
### Name: get_region_summary
### Title: Extract and summarize methylation or coverage info by regions of
### interest
### Aliases: get_region_summary
### ** Examples
data("methrix_data")
get_region_summary(m = methrix_data, regions = data.table(chr = "chr21", start = 27867971, end =
27868103), type = "M", how = "mean")
-Checking for overlaps..
Error: cannot allocate vector of size 1.4 Gb
Execution halted
I am a bit puzzled what the reasons could be, since it passed this step during
the previous builds and I didn’t change any of the functions in my last commit.
When I run the check on our local Windows, I don’t get any error. We use a very
small dataset as example, it definitely should not use a vector of size 1.4 Gb.
I would appreciate every input or idea on how to tackle this error.
Thanks a lot!
Best,
Reka
Dr. Reka Toth
Computational Epigenomics Group
Division of Epigenomics and Cancer Risk Factors /C010
Postdoctoral Fellow
German Cancer Research Center (DKFZ)
Foundation under Public Law
Im Neuenheimer Feld 280
69120 Heidelberg
Germany
r.t...@dkfz-heidelberg.de
https://urldefense.proofpoint.com/v2/url?u=http-3A__www.dkfz.de=DwIF-g=eRAMFD45gAfqt84VtBcfhQ=BK7q3XeAvimeWdGbWY_wJYbW0WYiZvSXAJJKaaPhzWA=doO3ej8vD58ZBQWVxkHTaCLk-xipbskav8uofJL2syI=U7YGBonX7WmBqfyEneCtEsuRUKl1fbTS6Esi3e9XGAI=
___
Bioc-devel@r-project.org mailing list
https://urldefense.proofpoint.com/v2/url?u=https-3A__stat.ethz.ch_mailman_listinfo_bioc-2Ddevel=DwIF-g=eRAMFD45gAfqt84VtBcfhQ=BK7q3XeAvimeWdGbWY_wJYbW0WYiZvSXAJJKaaPhzWA=doO3ej8vD58ZBQWVxkHTaCLk-xipbskav8uofJL2syI=2BO1kFpXuH1gpxwLxHae_ELnR7g_k9JV-qtslgowQMQ=
--
===
Tim Peters, PhD
Bioinformatics Research Officer | Immunogenomics Laboratory | Immunology
Division
Garvan Institute of Medical Research
384 Victoria St., Darlinghurst, NSW, Australia 2010
E: t.pet...@garvan.org.au | W: http://www.garvan.org.au | P: +612 9295 8325
NOTICE
Please consider the environment before printing this email. This message and
any attachments are intended for the addressee named and may contain legally
privileged/confidential/c
