Re: [Bioc-devel] BiocParallel: Best standards for passing locally assigned variables/functions, e.g. a bpExport()?
On 11/04/2013 11:34 AM, Michael Lawrence wrote:
The dynamic nature of R limits the extent of these checks. But as Ryan has
noted, a simple sanity check goes a long way. If what he has done could be
extended to the rest of the search path (people always forget to attach
packages), I think we've hit the 80% with 20%. Got a 404 on that URL btw.
I added three issues to BiocParallel on github.
1. bpexport
2. a function to check for non-local use. I think this should use codetools (to
avoid adding additional dependencies) but I'm a little flexible. Contributions
welcome on github, especially as a pull request with code formatted
consistently, a man page, and especially unit tests to provide a clear
understanding of circumstances covered or not. Michel Lang's Recall and the
implementation in foreach also sound releavant here.
3. integration of (2) into bplapply etc.
Please feel free to address these further on github.
Martin
Michael
On Mon, Nov 4, 2013 at 11:05 AM, Gabriel Becker gmbec...@ucdavis.eduwrote:
Hey guys,
Here is code that I have written which resolves library names into a full
list of symbols:
https://github.com/duncantl/CodeDepends/blob/forCRAN_0.3.5/R/librarySymbols.RNote
this does not require that the packages actually be loaded at the time
of the check, and does not load them (or rather, it loads them but does not
attach them, so no searchpath muddying occurs). You do need a list of
packages to check though (it adds the base ones automatically). It handles
dependency and could be easily extended to handle suggests as well I think.
When CodeDepends gets pushed to cran (not my call and not high on my
priority list to push for currently) it will actually do exactly what you
want. (the forCRAN_0.3.5 branch already does and I believe it is
documented, so you could use devtools to install it now).
As a side note, I'm not sure that existence of a symbol is sufficient (it
certainly is necessary). What about situations where the symbol exists but
is stale compared to the value in the parent? Are we sure that can never
happen?
~G
On Mon, Nov 4, 2013 at 7:29 AM, Michel Lang michell...@gmail.com wrote:
You might want to consider using Recall() for recursion which should
solve
this. Determining the required variables using heuristics as codetools
will
probably lead to some confusion when using functions which include calls
to, e.g., with():
f = function() {
with(iris, Sepal.Length + Sepal.Width)
}
codetools:::findGlobals(f)
I would suggest to write up some documentation on what the function's
environment contains and how to to define variables accordingly - or why
it
can generally be considered a good idea to pass everything essential as
an
argument. Nevertheless a bpExport function would be a good addition for
some rare corner cases in my opinion.
Michel
2013/11/3 Henrik Bengtsson h...@biostat.ucsf.edu
Hi,
in BiocParallel, is there a suggested (or planned) best standards for
making *locally* assigned variables (e.g. functions) available to the
applied function when it runs in a separate R process (which will be
the most common use case)? I understand that avoid local variables
should be avoided and it's preferred to put as mush as possible in
packages, but that's not always possible or very convenient.
EXAMPLE:
library('BiocParallel')
library('BatchJobs')
# Here I pick a recursive functions to make the problem a bit harder,
i.e.
# the function needs to call itself (itself = see below)
fib - function(n=0) {
if (n 0) stop(Invalid 'n': , n)
if (n == 0 || n == 1) return(1)
fib(n-2) + fib(n-1)
}
# Executing in the current R session
cluster.functions - makeClusterFunctionsInteractive()
bpParams - BatchJobsParam(cluster.functions=cluster.functions)
register(bpParams)
values - bplapply(0:9, FUN=fib)
## SubmitJobs |++| 100% (00:00:00)
## Waiting [S:0 R:0 D:10 E:0] |+++| 100% (00:00:00)
# Executing in a separate R process, where fib() is not defined
# (not specific to BiocParallel)
cluster.functions - makeClusterFunctionsLocal()
bpParams - BatchJobsParam(cluster.functions=cluster.functions)
register(bpParams)
values - bplapply(0:9, FUN=fib)
## SubmitJobs |++| 100% (00:00:00)
## Waiting [S:0 R:0 D:10 E:0] |+++| 100% (00:00:00)
Error in LastError$store(results = results, is.error = !ok,
throw.error =
TRUE)
:
Errors occurred during execution. First error message:
Error in FUN(...): could not find function fib
[...]
# The following illustrates that the solution is not always
straightforward.
# (not specific to BiocParallel; must have been discussed previously)
values - bplapply(0:9, FUN=function(n, fib) {
fib(n)
}, fib=fib)
Error in LastError$store(results = results, is.error = !ok,
throw.error = TRUE) :
Errors occurred during execution. First error message:
Error in fib(n): could not find function fib
[...]
# Workaround; make fib() aware of
[Bioc-devel] Commit problem in the release branch Re: BioC 2.13 branch created
Dear all, I just fixed some bugs in my pathview package. And would like to update my release version (with Bioc 2.13) too. I checked out the release version, and modified it. But I can’t commit the changes somehow. I did so for the 2.12 release version, it worked fine. Bellows is the error message I got: svn commit -m 'vignette 1.2.1' SendingDESCRIPTION svn: Commit failed (details follow): svn: access to '/bioconductor/!svn/ver/81643/branches/RELEASE_2_13/madman/Rpacks/pathview/DESCRIPTION' forbidden Please let me know if there is any issue. Thank you! Weijun On Mon, 10/14/13, Dan Tenenbaum dtene...@fhcrc.org wrote: Subject: [Bioc-devel] BioC 2.13 branch created To: bioc-devel@r-project.org Date: Monday, October 14, 2013, 5:49 PM The BioC 2.13 branch is now ready. Remember, you always have access to 2 versions of your package: the release and the devel versions. Right now the release version of your package (which is not officially released yet but will be in the next 72 hours if everything goes well) is in the 2.13 branch and accessible at: https://hedgehog.fhcrc.org/bioconductor/branches/RELEASE_2_13/madman/Rpacks/PKGNAME Only bug fixes and documentation improvements should go here. As always the devel version of your package is at: https://hedgehog.fhcrc.org/bioconductor/trunk/madman/Rpacks/PKGNAME Normal development of your package can now resume here. Please let us know if you have any questions. Thanks! Dan ___ Bioc-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/bioc-devel ___ Bioc-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/bioc-devel
Re: [Bioc-devel] Commit problem in the release branch Re: BioC 2.13 branch created
Dan, That works! Somehow, it always prompted me for username/passwd if needed in the past,but not this time. Anyway, thanks! Weijun On Wed, 11/6/13, Dan Tenenbaum dtene...@fhcrc.org wrote: Subject: Re: Commit problem in the release branch Re: [Bioc-devel] BioC 2.13 branch created Cc: bioc-devel@r-project.org Date: Wednesday, November 6, 2013, 11:31 AM - Original Message - To: Dan Tenenbaum dtene...@fhcrc.org Cc: bioc-devel@r-project.org Sent: Wednesday, November 6, 2013 8:18:22 AM Subject: Commit problem in the release branch Re: [Bioc-devel] BioC 2.13 branch created Dear all, I just fixed some bugs in my pathview package. And would like to update my release version (with Bioc 2.13) too. I checked out the release version, and modified it. But I can’t commit the changes somehow. I did so for the 2.12 release version, it worked fine. Bellows is the error message I got: svn commit -m 'vignette 1.2.1' Sending DESCRIPTION svn: Commit failed (details follow): svn: access to '/bioconductor/!svn/ver/81643/branches/RELEASE_2_13/madman/Rpacks/pathview/DESCRIPTION' forbidden [[elided Yahoo spam]] I just checked the permissions file and you do have permission to commit to both the release and devel branches of pathview as user w.luo. Try explicitly specifying your username: svn commit --username w.luo Dan Weijun On Mon, 10/14/13, Dan Tenenbaum dtene...@fhcrc.org wrote: Subject: [Bioc-devel] BioC 2.13 branch created To: bioc-devel@r-project.org Date: Monday, October 14, 2013, 5:49 PM The BioC 2.13 branch is now ready. Remember, you always have access to 2 versions of your package: the release and the devel versions. Right now the release version of your package (which is not officially released yet but will be in the next 72 hours if everything goes well) is in the 2.13 branch and accessible at: https://hedgehog.fhcrc.org/bioconductor/branches/RELEASE_2_13/madman/Rpacks/PKGNAME Only bug fixes and documentation improvements should go here. As always the devel version of your package is at: https://hedgehog.fhcrc.org/bioconductor/trunk/madman/Rpacks/PKGNAME Normal development of your package can now resume here. Please let us know if you have any questions. Thanks! Dan ___ Bioc-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/bioc-devel ___ Bioc-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/bioc-devel
Re: [Bioc-devel] Commit problem in the release branch Re: BioC 2.13 branch created
- Original Message - From: Luo Weijun luo_wei...@yahoo.com To: Dan Tenenbaum dtene...@fhcrc.org Cc: bioc-devel@r-project.org Sent: Wednesday, November 6, 2013 8:38:18 AM Subject: Re: Commit problem in the release branch Re: [Bioc-devel] BioC 2.13 branch created Dan, That works! Somehow, it always prompted me for username/passwd if needed in the past,but not this time. Anyway, thanks! This can happen if you used other svn credentials in the meantime since you last worked on your package. For example, maybe you checked out a different BioC package using the credentials readonly/readonly. svn will use the last credentials you gave it unless you tell it otherwise. Dan Weijun On Wed, 11/6/13, Dan Tenenbaum dtene...@fhcrc.org wrote: Subject: Re: Commit problem in the release branch Re: [Bioc-devel] BioC 2.13 branch created To: Luo Weijun luo_wei...@yahoo.com Cc: bioc-devel@r-project.org Date: Wednesday, November 6, 2013, 11:31 AM - Original Message - From: Luo Weijun luo_wei...@yahoo.com To: Dan Tenenbaum dtene...@fhcrc.org Cc: bioc-devel@r-project.org Sent: Wednesday, November 6, 2013 8:18:22 AM Subject: Commit problem in the release branch Re: [Bioc-devel] BioC 2.13 branch created Dear all, I just fixed some bugs in my pathview package. And would like to update my release version (with Bioc 2.13) too. I checked out the release version, and modified it. But I can’t commit the changes somehow. I did so for the 2.12 release version, it worked fine. Bellows is the error message I got: svn commit -m 'vignette 1.2.1' Sending DESCRIPTION svn: Commit failed (details follow): svn: access to '/bioconductor/!svn/ver/81643/branches/RELEASE_2_13/madman/Rpacks/pathview/DESCRIPTION' forbidden Please let me know if there is any issue. Thank you! I just checked the permissions file and you do have permission to commit to both the release and devel branches of pathview as user w.luo. Try explicitly specifying your username: svn commit --username w.luo Dan Weijun On Mon, 10/14/13, Dan Tenenbaum dtene...@fhcrc.org wrote: Subject: [Bioc-devel] BioC 2.13 branch created To: bioc-devel@r-project.org Date: Monday, October 14, 2013, 5:49 PM The BioC 2.13 branch is now ready. Remember, you always have access to 2 versions of your package: the release and the devel versions. Right now the release version of your package (which is not officially released yet but will be in the next 72 hours if everything goes well) is in the 2.13 branch and accessible at: https://hedgehog.fhcrc.org/bioconductor/branches/RELEASE_2_13/madman/Rpacks/PKGNAME Only bug fixes and documentation improvements should go here. As always the devel version of your package is at: https://hedgehog.fhcrc.org/bioconductor/trunk/madman/Rpacks/PKGNAME Normal development of your package can now resume here. Please let us know if you have any questions. Thanks! Dan ___ Bioc-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/bioc-devel ___ Bioc-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/bioc-devel
Re: [Bioc-devel] cleaning up obsolete warnings
Hi Michael, On 11/05/2013 04:30 PM, Michael Lawrence wrote: I'm getting a little tired of this one popping up: In x %in% other : Starting with BioC 2.12, the behavior of %in% on GenomicRanges objects has changed to use *equality* instead of *overlap* for comparing elements between GenomicRanges objects 'x' and 'table'. Now 'x[i]' and 'table[j]' are considered to match when they are equal (i.e. 'x[i] == table[j]'), instead of when they overlap. This new behavior is consistent with base::`%in%`(). If you need the old behavior, please use: query %over% subject Now that even release is Bioc 2.13, can we remove it? And there might be others. If you use %over% you don't get the warning ;-) Anyway this is gone in BioC 2.14 as part of the normal process of removing features that were defunct in BioC 2.13. Cheers, H. Thanks, Michael [[alternative HTML version deleted]] ___ Bioc-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/bioc-devel -- Hervé Pagès Program in Computational Biology Division of Public Health Sciences Fred Hutchinson Cancer Research Center 1100 Fairview Ave. N, M1-B514 P.O. Box 19024 Seattle, WA 98109-1024 E-mail: hpa...@fhcrc.org Phone: (206) 667-5791 Fax:(206) 667-1319 ___ Bioc-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/bioc-devel
Re: [Bioc-devel] cleaning up obsolete warnings
Great, thanks for updating. I legitimately wanted to use %in%, not %over%. Michael On Wed, Nov 6, 2013 at 10:59 AM, Hervé Pagès hpa...@fhcrc.org wrote: Hi Michael, On 11/05/2013 04:30 PM, Michael Lawrence wrote: I'm getting a little tired of this one popping up: In x %in% other : Starting with BioC 2.12, the behavior of %in% on GenomicRanges objects has changed to use *equality* instead of *overlap* for comparing elements between GenomicRanges objects 'x' and 'table'. Now 'x[i]' and 'table[j]' are considered to match when they are equal (i.e. 'x[i] == table[j]'), instead of when they overlap. This new behavior is consistent with base::`%in%`(). If you need the old behavior, please use: query %over% subject Now that even release is Bioc 2.13, can we remove it? And there might be others. If you use %over% you don't get the warning ;-) Anyway this is gone in BioC 2.14 as part of the normal process of removing features that were defunct in BioC 2.13. Cheers, H. Thanks, Michael [[alternative HTML version deleted]] ___ Bioc-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/bioc-devel -- Hervé Pagès Program in Computational Biology Division of Public Health Sciences Fred Hutchinson Cancer Research Center 1100 Fairview Ave. N, M1-B514 P.O. Box 19024 Seattle, WA 98109-1024 E-mail: hpa...@fhcrc.org Phone: (206) 667-5791 Fax:(206) 667-1319 [[alternative HTML version deleted]] ___ Bioc-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/bioc-devel
Re: [Bioc-devel] cleaning up obsolete warnings
On 11/06/2013 11:29 AM, Michael Lawrence wrote: Great, thanks for updating. I legitimately wanted to use %in%, not %over%. Great to hear you have a use case for %in% (with the new behavior). So I'm not the only one ;-) H. Michael On Wed, Nov 6, 2013 at 10:59 AM, Hervé Pagès hpa...@fhcrc.org mailto:hpa...@fhcrc.org wrote: Hi Michael, On 11/05/2013 04:30 PM, Michael Lawrence wrote: I'm getting a little tired of this one popping up: In x %in% other : Starting with BioC 2.12, the behavior of %in% on GenomicRanges objects has changed to use *equality* instead of *overlap* for comparing elements between GenomicRanges objects 'x' and 'table'. Now 'x[i]' and 'table[j]' are considered to match when they are equal (i.e. 'x[i] == table[j]'), instead of when they overlap. This new behavior is consistent with base::`%in%`(). If you need the old behavior, please use: query %over% subject Now that even release is Bioc 2.13, can we remove it? And there might be others. If you use %over% you don't get the warning ;-) Anyway this is gone in BioC 2.14 as part of the normal process of removing features that were defunct in BioC 2.13. Cheers, H. Thanks, Michael [[alternative HTML version deleted]] _ Bioc-devel@r-project.org mailto:Bioc-devel@r-project.org mailing list https://stat.ethz.ch/mailman/__listinfo/bioc-devel https://stat.ethz.ch/mailman/listinfo/bioc-devel -- Hervé Pagès Program in Computational Biology Division of Public Health Sciences Fred Hutchinson Cancer Research Center 1100 Fairview Ave. N, M1-B514 P.O. Box 19024 Seattle, WA 98109-1024 E-mail: hpa...@fhcrc.org mailto:hpa...@fhcrc.org Phone: (206) 667-5791 tel:%28206%29%20667-5791 Fax: (206) 667-1319 tel:%28206%29%20667-1319 -- Hervé Pagès Program in Computational Biology Division of Public Health Sciences Fred Hutchinson Cancer Research Center 1100 Fairview Ave. N, M1-B514 P.O. Box 19024 Seattle, WA 98109-1024 E-mail: hpa...@fhcrc.org Phone: (206) 667-5791 Fax:(206) 667-1319 ___ Bioc-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/bioc-devel
