Re: [Bioc-devel] Very slow when operate GRangesList
Hi, SimpleLists are slow in this situation, basically because the underlying seqselect is slow, due to this loop: x - do.call(c, lapply(seq_len(length(ir)), function(i) window(x, start = start(ir)[i], width = width(ir)[i]))) Am I missing something or could this become a simple x[as.integer(ir)]? In the meantime, using CompressedLists is the way to go. So for an RleList, you need to pass compress=TRUE to the constructor. On Wed, Aug 21, 2013 at 8:30 AM, Ou, Jianhong jianhong...@umassmed.eduwrote: Hi, When I use big set of GrangesList, I found it become very slow when metadata contain AtomicList. e.g. grll - GRanges(seqnames=chr1, ranges=IRanges(start=1:500, width=2), someInfo=rep(RleList(*), 500)) grr - split(grll, 1:500) grl - as.list(grr) system.time(grl- grl[500:1]) user system elapsed 0 0 0 system.time(grr- grr[500:1]) user system elapsed 1.622 0.013 1.635 grll - GRanges(seqnames=chr1, ranges=IRanges(start=1:500, width=2)) grr - split(grll, 1:500) grl - as.list(grr) system.time(grl- grl[500:1]) user system elapsed 0 0 0 system.time(grr- grr[500:1]) user system elapsed 0.029 0.001 0.030 sessionInfo() R Under development (unstable) (2013-07-23 r63392) Platform: x86_64-apple-darwin12.4.0 (64-bit) locale: [1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8 attached base packages: [1] parallel stats graphics grDevices utils datasets methods base other attached packages: [1] GenomicRanges_1.13.36 XVector_0.1.0 IRanges_1.19.24 BiocGenerics_0.7.3 loaded via a namespace (and not attached): [1] stats4_3.1.0 tools_3.1.0 Is there any method to improve this? Yours sincerely, Jianhong Ou LRB 670A Program in Gene Function and Expression 364 Plantation Street Worcester, MA 01605 [[alternative HTML version deleted]] ___ Bioc-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/bioc-devel [[alternative HTML version deleted]] ___ Bioc-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/bioc-devel
Re: [Rd] legitimate use of :::
On 22.08.2013 07:45, Yihui Xie wrote: Hi, So now R CMD check starts to warn against :::, but I believe sometimes it is legitimate to use it when developing R packages. For example, I have some utils functions that are not exported but I want to share them across the packages that I maintain. I do not need to coordinate with other authors about these internal functions since I'm the only author and I know clearly what I'm doing, and I want to avoid copying and pasting the code across packages just to avoid the NOTE in R CMD check. What should I do in this case? Nothing. The way you describe above seems to be a reasonable usage, iff you are the same maintainer who knows what is going on. Other maintainers should not use one of your not exported (hence non API) functions, of course. Uwe Ligges Regards, Yihui -- Yihui Xie xieyi...@gmail.com Web: http://yihui.name Department of Statistics, Iowa State University 102 Snedecor Hall, Ames, IA __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] Confusion about Depends:, Imports:, Enhances:, import(), inportFrom()
In checking my vcdExtra package, the following NOTE newly appeared (R-Forge, using R version 3.0.1 Patched (2013-08-20 r63635)) Package in Depends field not imported from: ‘gnm’ These packages needs to imported from for the case when this namespace is loaded but not attached. In the DESCRIPTION file, I have Depends: R (= 2.10), vcd, gnm (= 1.0.3) In NAMESPACE: # we are a vcd extension import(vcd) I've read 1.1.1 of R-Exts, but it is not clear to me whether I should also import gnm or change the DESCRIPTION file to use Imports: vcd, gnm (= 1.0.3) R-Exts says: The ‘Imports’ field lists packages whose namespaces are imported from (as specified in the NAMESPACE file) but which do not need to be attached, but how can I tell if gnm needs to be attached? Also, what is the difference between Imports: in DESCRIPTION and imports() in NAMESPACE? -Michael -- Michael Friendly Email: friendly AT yorku DOT ca Professor, Psychology Dept. Chair, Quantitative Methods York University Voice: 416 736-2100 x66249 Fax: 416 736-5814 4700 Keele StreetWeb: http://www.datavis.ca Toronto, ONT M3J 1P3 CANADA __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Confusion about Depends:, Imports:, Enhances:, import(), inportFrom()
On Aug 22, 2013, at 9:09, Michael Friendly frien...@yorku.ca wrote: In checking my vcdExtra package, the following NOTE newly appeared (R-Forge, using R version 3.0.1 Patched (2013-08-20 r63635)) Package in Depends field not imported from: ‘gnm’ These packages needs to imported from for the case when this namespace is loaded but not attached. In the DESCRIPTION file, I have Depends: R (= 2.10), vcd, gnm (= 1.0.3) In NAMESPACE: # we are a vcd extension import(vcd) I've read 1.1.1 of R-Exts, but it is not clear to me whether I should also import gnm or change the DESCRIPTION file to use Imports: vcd, gnm (= 1.0.3) R-Exts says: The ‘Imports’ field lists packages whose namespaces are imported from (as specified in the NAMESPACE file) but which do not need to be attached, but how can I tell if gnm needs to be attached? I think the current best practice is to use Imports unless gnm provides functions the end user needs in order to use your package. Practically, I think this usually comes down to asking whether gnm provides relevant generics -- if you provide methods only, the end user would only be able to call them directly (ick!) unless gnm is both loaded and attached (Depends) If gnm is tools for your code, but not the user to call directly, Imports is cleaner. Does that help? Michael Also, what is the difference between Imports: in DESCRIPTION and imports() in NAMESPACE? One enables the other: Imports in DESCRIPTION allows for both imports() and importsFrom(). It could probably be automatic, but the DESC file is much older than the NAMESPACE file. -Michael -- Michael Friendly Email: friendly AT yorku DOT ca Professor, Psychology Dept. Chair, Quantitative Methods York University Voice: 416 736-2100 x66249 Fax: 416 736-5814 4700 Keele StreetWeb: http://www.datavis.ca Toronto, ONT M3J 1P3 CANADA __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] legitimate use of :::
On 8/22/2013 7:45 AM, Uwe Ligges wrote: On 22.08.2013 07:45, Yihui Xie wrote: Hi, So now R CMD check starts to warn against :::, but I believe sometimes it is legitimate to use it when developing R packages. For example, I have some utils functions that are not exported but I want to share them across the packages that I maintain. I do not need to coordinate with other authors about these internal functions since I'm the only author and I know clearly what I'm doing, and I want to avoid copying and pasting the code across packages just to avoid the NOTE in R CMD check. What should I do in this case? Nothing. The way you describe above seems to be a reasonable usage, iff you are the same maintainer who knows what is going on. Other maintainers should not use one of your not exported (hence non API) functions, of course. Uwe Ligges Related to this is the use of other-package unexported utility functions that don't pass Uwe's iff test, but I, as maintainer, want to use in my package. Cases in point: in heplots, I had used stats:::Pillai, stats:::Wilks, stats:::Roy and stats:::LH for calculation in one of my functions. Similarly, I had a need to use car:::df.terms, also unexported, but don't want to ask John Fox to export it just for my use. Uwe's reply suggests that I should not be using car:::df.terms, however. To avoid the NOTEs (which often triggers a 'pls fix' upon submission to CRAN), I simply copied/pasted these functions to my package, but this seems wasteful. -Michael -- Michael Friendly Email: friendly AT yorku DOT ca Professor, Psychology Dept. Chair, Quantitative Methods York University Voice: 416 736-2100 x66249 Fax: 416 736-5814 4700 Keele StreetWeb: http://www.datavis.ca Toronto, ONT M3J 1P3 CANADA __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Confusion about Depends:, Imports:, Enhances:, import(), inportFrom()
On Thu, Aug 22, 2013 at 10:05 AM, R. Michael Weylandt michael.weyla...@gmail.com michael.weyla...@gmail.com wrote: On Aug 22, 2013, at 9:09, Michael Friendly frien...@yorku.ca wrote: In checking my vcdExtra package, the following NOTE newly appeared (R-Forge, using R version 3.0.1 Patched (2013-08-20 r63635)) Package in Depends field not imported from: ‘gnm’ These packages needs to imported from for the case when this namespace is loaded but not attached. In the DESCRIPTION file, I have Depends: R (= 2.10), vcd, gnm (= 1.0.3) In NAMESPACE: # we are a vcd extension import(vcd) I've read 1.1.1 of R-Exts, but it is not clear to me whether I should also import gnm or change the DESCRIPTION file to use Imports: vcd, gnm (= 1.0.3) R-Exts says: The ‘Imports’ field lists packages whose namespaces are imported from (as specified in the NAMESPACE file) but which do not need to be attached, but how can I tell if gnm needs to be attached? I think the current best practice is to use Imports unless gnm provides functions the end user needs in order to use your package. Practically, I think this usually comes down to asking whether gnm provides relevant generics -- if you provide methods only, the end user would only be able to call them directly (ick!) unless gnm is both loaded and attached (Depends) If gnm is tools for your code, but not the user to call directly, Imports is cleaner. Does that help? Michael Also, what is the difference between Imports: in DESCRIPTION and imports() in NAMESPACE? One enables the other: Imports in DESCRIPTION allows for both imports() and importsFrom(). It could probably be automatic, but the DESC file is much older than the NAMESPACE file. I pretty sure you can also use import()/importFrom() in NAMESPACE for packages listed under Depends in DESCRIPTION. /Henrik -Michael -- Michael Friendly Email: friendly AT yorku DOT ca Professor, Psychology Dept. Chair, Quantitative Methods York University Voice: 416 736-2100 x66249 Fax: 416 736-5814 4700 Keele StreetWeb: http://www.datavis.ca Toronto, ONT M3J 1P3 CANADA __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Confusion about Depends:, Imports:, Enhances:, import(), inportFrom()
On Aug 22, 2013, at 15:33, Henrik Bengtsson h...@biostat.ucsf.edu wrote: On Thu, Aug 22, 2013 at 10:05 AM, R. Michael Weylandt michael.weyla...@gmail.com michael.weyla...@gmail.com wrote: On Aug 22, 2013, at 9:09, Michael Friendly frien...@yorku.ca wrote: In checking my vcdExtra package, the following NOTE newly appeared (R-Forge, using R version 3.0.1 Patched (2013-08-20 r63635)) Package in Depends field not imported from: ‘gnm’ These packages needs to imported from for the case when this namespace is loaded but not attached. In the DESCRIPTION file, I have Depends: R (= 2.10), vcd, gnm (= 1.0.3) In NAMESPACE: # we are a vcd extension import(vcd) I've read 1.1.1 of R-Exts, but it is not clear to me whether I should also import gnm or change the DESCRIPTION file to use Imports: vcd, gnm (= 1.0.3) R-Exts says: The ‘Imports’ field lists packages whose namespaces are imported from (as specified in the NAMESPACE file) but which do not need to be attached, but how can I tell if gnm needs to be attached? I think the current best practice is to use Imports unless gnm provides functions the end user needs in order to use your package. Practically, I think this usually comes down to asking whether gnm provides relevant generics -- if you provide methods only, the end user would only be able to call them directly (ick!) unless gnm is both loaded and attached (Depends) If gnm is tools for your code, but not the user to call directly, Imports is cleaner. Does that help? Michael Also, what is the difference between Imports: in DESCRIPTION and imports() in NAMESPACE? One enables the other: Imports in DESCRIPTION allows for both imports() and importsFrom(). It could probably be automatic, but the DESC file is much older than the NAMESPACE file. I pretty sure you can also use import()/importFrom() in NAMESPACE for packages listed under Depends in DESCRIPTION. Agreed, but you couldn't auto-generate Depends from the namespace file. I suppose if (per my above) Depends was only used for generics... Not sure if the devtools world does anything like this, though it seems logical enough. Michael /Henrik -Michael -- Michael Friendly Email: friendly AT yorku DOT ca Professor, Psychology Dept. Chair, Quantitative Methods York University Voice: 416 736-2100 x66249 Fax: 416 736-5814 4700 Keele StreetWeb: http://www.datavis.ca Toronto, ONT M3J 1P3 CANADA __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] legitimate use of :::
On Aug 22, 2013, at 20:57 , Michael Friendly wrote: Cases in point: in heplots, I had used stats:::Pillai, stats:::Wilks, stats:::Roy and stats:::LH for calculation in one of my functions. That particular case has been on what remains of my conscience for some time -- Peter Dalgaard, Professor, Center for Statistics, Copenhagen Business School Solbjerg Plads 3, 2000 Frederiksberg, Denmark Phone: (+45)38153501 Email: pd@cbs.dk Priv: pda...@gmail.com __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] legitimate use of :::
Dear Michael and Uwe, -Original Message- From: r-devel-boun...@r-project.org [mailto:r-devel-bounces@r- project.org] On Behalf Of Michael Friendly Sent: Thursday, August 22, 2013 2:57 PM To: Uwe Ligges Cc: R-devel Subject: Re: [Rd] legitimate use of ::: On 8/22/2013 7:45 AM, Uwe Ligges wrote: On 22.08.2013 07:45, Yihui Xie wrote: Hi, So now R CMD check starts to warn against :::, but I believe sometimes it is legitimate to use it when developing R packages. For example, I have some utils functions that are not exported but I want to share them across the packages that I maintain. I do not need to coordinate with other authors about these internal functions since I'm the only author and I know clearly what I'm doing, and I want to avoid copying and pasting the code across packages just to avoid the NOTE in R CMD check. What should I do in this case? Nothing. The way you describe above seems to be a reasonable usage, iff you are the same maintainer who knows what is going on. Other maintainers should not use one of your not exported (hence non API) functions, of course. Uwe Ligges Related to this is the use of other-package unexported utility functions that don't pass Uwe's iff test, but I, as maintainer, want to use in my package. Cases in point: in heplots, I had used stats:::Pillai, stats:::Wilks, stats:::Roy and stats:::LH for calculation in one of my functions. Similarly, I had a need to use car:::df.terms, also unexported, but don't want to ask John Fox to export it just for my use. Uwe's reply suggests that I should not be using car:::df.terms, however. To avoid the NOTEs (which often triggers a 'pls fix' upon submission to CRAN), I simply copied/pasted these functions to my package, but this seems wasteful. I think that the ideal solution is for everyone to export functions that somewhat else might want, but it's hard to anticipate what these are, and it would be useful then to differentiate functions that are meant for end users from those meant for developers. Maybe packages could document the latter in something like a Utilities.Rd file. Probably there's a better, more formal, solution. The stats:::Pillai, Wilks, HL, and Roy functions seem reasonable candidates for export -- I too use these functions, in the car package, and have resorted to the fix that Michael adopted. I'd be happy to export df.terms, but would rather segregate it from end-user functions. It's also clear to me that enforcing namespace conventions more consistently, which is certainly desirable in the abstract, opens a can of worms, especially for the CRAN administrators. One hopes that we'll all survive the process and will have better packages in the end. My two cents. John -Michael -- Michael Friendly Email: friendly AT yorku DOT ca Professor, Psychology Dept. Chair, Quantitative Methods York University Voice: 416 736-2100 x66249 Fax: 416 736-5814 4700 Keele StreetWeb: http://www.datavis.ca Toronto, ONT M3J 1P3 CANADA __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] legitimate use of :::
To avoid the NOTEs (which often triggers a 'pls fix' upon submission to CRAN), I simply copied/pasted these functions to my package, but this seems wasteful. Wasteful of disk space, but disk space is cheap. It's less wasteful of your time, if the referenced code breaks in an unexpected time. Your time is much more valuable than disk space. A gigabyte of disk space costs about $0.10, so even if you value your time at a very conservative rate of $100 / hour, you should only spend an hour of your time reducing package size if it saves at least 1 TB of disk space. That's a lot of copies of a function! Hadley -- Chief Scientist, RStudio http://had.co.nz/ __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] legitimate use of :::
r63654 has fixed this particular issue, and R-devel will no longer warn against the use of ::: on packages of the same maintainer. Regards, Yihui -- Yihui Xie xieyi...@gmail.com Web: http://yihui.name Department of Statistics, Iowa State University 102 Snedecor Hall, Ames, IA On Thu, Aug 22, 2013 at 6:45 AM, Uwe Ligges lig...@statistik.tu-dortmund.de wrote: On 22.08.2013 07:45, Yihui Xie wrote: Hi, So now R CMD check starts to warn against :::, but I believe sometimes it is legitimate to use it when developing R packages. For example, I have some utils functions that are not exported but I want to share them across the packages that I maintain. I do not need to coordinate with other authors about these internal functions since I'm the only author and I know clearly what I'm doing, and I want to avoid copying and pasting the code across packages just to avoid the NOTE in R CMD check. What should I do in this case? Nothing. The way you describe above seems to be a reasonable usage, iff you are the same maintainer who knows what is going on. Other maintainers should not use one of your not exported (hence non API) functions, of course. Uwe Ligges __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] legitimate use of :::
On Thu, Aug 22, 2013 at 2:03 PM, Hadley Wickham h.wick...@gmail.com wrote: To avoid the NOTEs (which often triggers a 'pls fix' upon submission to CRAN), I simply copied/pasted these functions to my package, but this seems wasteful. Wasteful of disk space, but disk space is cheap. It's less wasteful of your time, if the referenced code breaks in an unexpected time. Your time is much more valuable than disk space. A gigabyte of disk space costs about $0.10, so even if you value your time at a very conservative rate of $100 / hour, you should only spend an hour of your time reducing package size if it saves at least 1 TB of disk space. That's a lot of copies of a function! A bigger issue is source-code license conflicts; you may cut'n'paste GPL code into a distribution that is under another license. /Henrik Hadley -- Chief Scientist, RStudio http://had.co.nz/ __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] legitimate use of :::
On Thu, Aug 22, 2013 at 2:03 PM, Hadley Wickham h.wick...@gmail.com wrote: To avoid the NOTEs (which often triggers a 'pls fix' upon submission to CRAN), I simply copied/pasted these functions to my package, but this seems wasteful. Wasteful of disk space, but disk space is cheap. It's less wasteful of your time, if the referenced code breaks in an unexpected time. Your time is much more valuable than disk space. On the other hand, it's quite dangerous software design. What if the original author finds a bug and implements a fix, but you don't hear about it? Furthermore, what happens when I come along and need the same functionality? Sure I could make a copy, but maybe I only know about your copy and don't know it is a copy of something else, so now we have a copy of a copy, which is even more problematic. Using ::: prevents this issue. Using ::: also allows us to get the credit for the idea/functionality right, which is important and which our field could stand for some improvement on. I would argue, though, in response to the original question that if you have functions that you, yourself are using in multiple packages, or that you know other people are going to want to use, that is an indication that the function is broadly useful and really should be exported from somewhere (either the package it lives in now or a lightweight dependency/utility package where you collect all such functions you create). The much muddier issue is what to do when someone has written a function that *almost* does what you want, but has a hardcoded limitation in it that prevents you from using it. That situation is harder to deal with imo unless the maintainer of the package is open to patches/pull requests. ~G -- Gabriel Becker Graduate Student Statistics Department University of California, Davis [[alternative HTML version deleted]] __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] legitimate use of :::
Wasteful of disk space, but disk space is cheap. It's less wasteful of your time, if the referenced code breaks in an unexpected time. Your time is much more valuable than disk space. On the other hand, it's quite dangerous software design. What if the original author finds a bug and implements a fix, but you don't hear about it? Furthermore, what happens when I come along and need the same functionality? Sure I could make a copy, but maybe I only know about your copy and don't know it is a copy of something else, so now we have a copy of a copy, which is even more problematic. Using ::: prevents this issue. There are costs and benefits to both approaches. Copy-and-paste also minimises external dependencies which can be important in some cases. I'm not arguing for unmitigated duplication, but there are definitely good reasons to do it. I have quite a few v. simple functions that live in multiple packages. Often I want to keep the dependencies of packages as lightweight as possible (learning from past experiences) and avoid tightly coupling packages together. Hadley -- Chief Scientist, RStudio http://had.co.nz/ __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] legitimate use of :::
Dear Peter, -Original Message- From: r-devel-boun...@r-project.org [mailto:r-devel-bounces@r- project.org] On Behalf Of peter dalgaard Sent: Thursday, August 22, 2013 4:45 PM To: Michael Friendly Cc: R-devel; Uwe Ligges Subject: Re: [Rd] legitimate use of ::: On Aug 22, 2013, at 20:57 , Michael Friendly wrote: Cases in point: in heplots, I had used stats:::Pillai, stats:::Wilks, stats:::Roy and stats:::LH for calculation in one of my functions. That particular case has been on what remains of my conscience for some time Happily, it would be easy to relieve your conscience in this matter. Best, John -- Peter Dalgaard, Professor, Center for Statistics, Copenhagen Business School Solbjerg Plads 3, 2000 Frederiksberg, Denmark Phone: (+45)38153501 Email: pd@cbs.dk Priv: pda...@gmail.com __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] legitimate use of :::
Another point to consider is that copying someone else's code forces you to become a maintainer of the copied code. If there are any bug fixes/enhancements/what-have-you in the original you won't get those updates. So now you own the copied code and need to consider the cost of the codebase diverging (from the original). On Aug 22, 2013, at 5:03 PM, Hadley Wickham h.wick...@gmail.com wrote: To avoid the NOTEs (which often triggers a 'pls fix' upon submission to CRAN), I simply copied/pasted these functions to my package, but this seems wasteful. Wasteful of disk space, but disk space is cheap. It's less wasteful of your time, if the referenced code breaks in an unexpected time. Your time is much more valuable than disk space. A gigabyte of disk space costs about $0.10, so even if you value your time at a very conservative rate of $100 / hour, you should only spend an hour of your time reducing package size if it saves at least 1 TB of disk space. That's a lot of copies of a function! Hadley -- Chief Scientist, RStudio http://had.co.nz/ __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] legitimate use of :::
On Thu, Aug 22, 2013 at 4:52 PM, Brian Rowe r...@muxspace.com wrote: Another point to consider is that copying someone else's code forces you to become a maintainer of the copied code. If there are any bug fixes/enhancements/what-have-you in the original you won't get those updates. So now you own the copied code and need to consider the cost of the codebase diverging (from the original). Sometimes that's a good thing - you're equally insulated from the original maintainer changing the function to work in a way that you don't like. Again, I'm not arguing that copy-and-paste is necessarily the right solution, but it's not necessarily the wrong solution either - it depends on the context. Hadley -- Chief Scientist, RStudio http://had.co.nz/ __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] legitimate use of :::
If ::: is disallowed then its likely that package developers will need to export more functions to satisfy the consumers of those otherwise hidden functions but if more functions are exported then there will be a greater likelihood of conflicts among packages. The problem seems to be that there are potentially three sorts of functions here: 1. a function is hidden 2. a function is accessible via ::: but is not on the search path 3. a function is on the search path The problem arises in attempting to force fit these three concepts into only two categories either by removing the first category (as was done previously) or by removing the second category (which seems to be the new approach). __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] legitimate use of :::
You raise an interesting point that I've mulled over a bit: namespace collisions. How many of these issues would go away if there were a better mechanism for managing namespaces? eg in other languages you can control which objects/modules you wish to import from a library. Under this regime I think package developers would be less concerned about exposing functions that otherwise would be private. On Aug 22, 2013, at 6:27 PM, Gabor Grothendieck ggrothendi...@gmail.com wrote: If ::: is disallowed then its likely that package developers will need to export more functions to satisfy the consumers of those otherwise hidden functions but if more functions are exported then there will be a greater likelihood of conflicts among packages. The problem seems to be that there are potentially three sorts of functions here: 1. a function is hidden 2. a function is accessible via ::: but is not on the search path 3. a function is on the search path The problem arises in attempting to force fit these three concepts into only two categories either by removing the first category (as was done previously) or by removing the second category (which seems to be the new approach). __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] legitimate use of :::
On 23.08.2013 00:36, Brian Lee Yung Rowe wrote: You raise an interesting point that I've mulled over a bit: namespace collisions. How many of these issues would go away if there were a better mechanism for managing namespaces? eg in other languages you can control which objects/modules you wish to import from a library. Under this regime I think package developers would be less concerned about exposing functions that otherwise would be private. Exactly, the corresponding NAMESPACE directive is importFrom() and it should be used. On Aug 22, 2013, at 6:27 PM, Gabor Grothendieck ggrothendi...@gmail.com wrote: If ::: is disallowed then its likely that package developers will need to export more functions to satisfy the consumers of those otherwise hidden functions but if more functions are exported then there will be a greater likelihood of conflicts among packages. The problem seems to be that there are potentially three sorts of functions here: 1. a function is hidden 2. a function is accessible via ::: but is not on the search path 3. a function is on the search path Not entirely right: If the package or only parts of it are imported via importFrom by another package, the package is not loaded, hence not on the search path. Best, Uwe Ligges The problem arises in attempting to force fit these three concepts into only two categories either by removing the first category (as was done previously) or by removing the second category (which seems to be the new approach). __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] legitimate use of :::
On Thu, Aug 22, 2013 at 6:41 PM, Uwe Ligges lig...@statistik.tu-dortmund.de wrote: On 23.08.2013 00:36, Brian Lee Yung Rowe wrote: You raise an interesting point that I've mulled over a bit: namespace collisions. How many of these issues would go away if there were a better mechanism for managing namespaces? eg in other languages you can control which objects/modules you wish to import from a library. Under this regime I think package developers would be less concerned about exposing functions that otherwise would be private. Exactly, the corresponding NAMESPACE directive is importFrom() and it should be used. On Aug 22, 2013, at 6:27 PM, Gabor Grothendieck ggrothendi...@gmail.com wrote: If ::: is disallowed then its likely that package developers will need to export more functions to satisfy the consumers of those otherwise hidden functions but if more functions are exported then there will be a greater likelihood of conflicts among packages. The problem seems to be that there are potentially three sorts of functions here: 1. a function is hidden 2. a function is accessible via ::: but is not on the search path 3. a function is on the search path Not entirely right: If the package or only parts of it are imported via importFrom by another package, the package is not loaded, hence not on the search path. OK but it is still true that under the new rules to use importFrom(B, f) in package A that f must be exported by B. That implies that f could cause a conflict when B is placed on the search path via library(B) by some other package (package C) or by the user. f is either exported by B or not. If f is exported by B then f will be placed on the search path whenever B is placed on the search path and if f is not exported then A can't import it. That is there is no way for B to declare a function to be importable by another package without having that function also placed on the search path whenever B is loaded by a library(B)l or a Depends: B from another package. on the search path -- Statistics Software Consulting GKX Group, GKX Associates Inc. tel: 1-877-GKX-GROUP email: ggrothendieck at gmail.com __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] legitimate use of :::
My understanding is that lookup happens in the imports before moving on to the search path, so if I understand you correctly I don't think that is an issue. If A also *exported* f, that would be a problem... From writing R extensions (talking about functions in a package finding variables, sec 1.6): The namespace controls the search strategy for variables used by functions in the package. If not found locally, R searches the package namespace first, then the imports, then the base namespace and then the normal search path. On Thu, Aug 22, 2013 at 4:45 PM, Gabor Grothendieck ggrothendi...@gmail.com wrote: On Thu, Aug 22, 2013 at 6:41 PM, Uwe Ligges lig...@statistik.tu-dortmund.de wrote: On 23.08.2013 00:36, Brian Lee Yung Rowe wrote: You raise an interesting point that I've mulled over a bit: namespace collisions. How many of these issues would go away if there were a better mechanism for managing namespaces? eg in other languages you can control which objects/modules you wish to import from a library. Under this regime I think package developers would be less concerned about exposing functions that otherwise would be private. Exactly, the corresponding NAMESPACE directive is importFrom() and it should be used. On Aug 22, 2013, at 6:27 PM, Gabor Grothendieck ggrothendi...@gmail.com wrote: If ::: is disallowed then its likely that package developers will need to export more functions to satisfy the consumers of those otherwise hidden functions but if more functions are exported then there will be a greater likelihood of conflicts among packages. The problem seems to be that there are potentially three sorts of functions here: 1. a function is hidden 2. a function is accessible via ::: but is not on the search path 3. a function is on the search path Not entirely right: If the package or only parts of it are imported via importFrom by another package, the package is not loaded, hence not on the search path. OK but it is still true that under the new rules to use importFrom(B, f) in package A that f must be exported by B. That implies that f could cause a conflict when B is placed on the search path via library(B) by some other package (package C) or by the user. f is either exported by B or not. If f is exported by B then f will be placed on the search path whenever B is placed on the search path and if f is not exported then A can't import it. That is there is no way for B to declare a function to be importable by another package without having that function also placed on the search path whenever B is loaded by a library(B)l or a Depends: B from another package. on the search path -- Statistics Software Consulting GKX Group, GKX Associates Inc. tel: 1-877-GKX-GROUP email: ggrothendieck at gmail.com __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel -- Gabriel Becker Graduate Student Statistics Department University of California, Davis [[alternative HTML version deleted]] __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] legitimate use of :::
It would be nice if the functionality of importFrom() and import() were available to user level code, rather than just to people building packages for distribution. One most often encounters namespace conflicts at the user level, when loading two packages that have no logical connection other than both bearing on your problem of the moment. R conflates having namespaces with having a library distribution mechanism and while its library distribution mechanism is top notch, most modern languages do not require you to learn the distribution procedure in order to just have namespaces. For instance, in Python you merely put code in a file called foo.py and then in any other file in the same directory you type import functionName from foo. I.E. using namespaces does not require you to build/install packages. Python namespaces are also hierarchical so that the question of this thread would easily be resolved by putting functions into foo._internal and in other packages typing import * from foo._internal Peter [[alternative HTML version deleted]] __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] legitimate use of :::
On Thu, Aug 22, 2013 at 7:57 PM, Gabriel Becker gmbec...@ucdavis.edu wrote: My understanding is that lookup happens in the imports before moving on to the search path, so if I understand you correctly I don't think that is an issue. If A also *exported* f, that would be a problem... A can only import f from B if f has been exported from B so while its not a problem for A, whenever anyone issues a library(B) f will be visible on the search path -- the problem of potential conflicts with f remains. B really only exported f so that A could import it but a side effect of that is that anyone who puts B on the search path makes f visible. __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] legitimate use of :::
(missed the list the first time) Perhaps an importHiddenFrom directive should be added to the namespace vocabulary which can import a non-exported function. The non-exported functions clearly exist somewhere, as package code can use them, so it would not be impossible to allow that (though I'm not saying it would be easy, as I haven't looked at how the namespace stuff is implemented). On Thu, Aug 22, 2013 at 5:19 PM, Gabor Grothendieck ggrothendi...@gmail.com wrote: On Thu, Aug 22, 2013 at 7:57 PM, Gabriel Becker gmbec...@ucdavis.edu wrote: My understanding is that lookup happens in the imports before moving on to the search path, so if I understand you correctly I don't think that is an issue. If A also *exported* f, that would be a problem... A can only import f from B if f has been exported from B so while its not a problem for A, whenever anyone issues a library(B) f will be visible on the search path -- the problem of potential conflicts with f remains. B really only exported f so that A could import it but a side effect of that is that anyone who puts B on the search path makes f visible. -- Gabriel Becker Graduate Student Statistics Department University of California, Davis [[alternative HTML version deleted]] __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] legitimate use of :::
This is what I was getting at as well. It would be great to have a call like
require(package, c('funtion.1','function.2'))
or similar that gives users granular control over what gets imported in the
shell. I would be drunk with joy if the same mechanism could be used to
automatically populate the package directives.
On Aug 22, 2013, at 8:01 PM, Peter Meilstrup peter.meilst...@gmail.com wrote:
It would be nice if the functionality of importFrom() and import() were
available to user level code, rather than just to people building packages
for distribution. One most often encounters namespace conflicts at the user
level, when loading two packages that have no logical connection other than
both bearing on your problem of the moment.
R conflates having namespaces with having a library distribution
mechanism and while its library distribution mechanism is top notch, most
modern languages do not require you to learn the distribution procedure in
order to just have namespaces.
For instance, in Python you merely put code in a file called foo.py and
then in any other file in the same directory you type import functionName
from foo. I.E. using namespaces does not require you to build/install
packages. Python namespaces are also hierarchical so that the question of
this thread would easily be resolved by putting functions into
foo._internal and in other packages typing import * from foo._internal
Peter
[[alternative HTML version deleted]]
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] legitimate use of :::
Peter Meilstrup: (05:01PM on Thu, Aug 22) One most often encounters namespace conflicts at the user level, when loading two packages that have no logical connection other than both bearing on your problem of the moment. Unless I'm mistaken, you can reassign the hidden functions, ie fna - joespackage:::usefulfunction fnb - janespackage:::usefulfunction which is a little bit of a pain, but makes the user's code unambiguous. This also works with two colons for explicitly exported functions. -- Gray Calhoun, Assistant Professor of Economics at Iowa State http://gray.clhn.co (web) __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] legitimate use of :::
Dear Gray, On Thu, 22 Aug 2013 19:41:58 -0500 Gray g...@clhn.co wrote: Peter Meilstrup: (05:01PM on Thu, Aug 22) One most often encounters namespace conflicts at the user level, when loading two packages that have no logical connection other than both bearing on your problem of the moment. Unless I'm mistaken, you can reassign the hidden functions, ie fna - joespackage:::usefulfunction fnb - janespackage:::usefulfunction This will now generate a note from R CMD check and an objection from the CRAN administrators. Best, John which is a little bit of a pain, but makes the user's code unambiguous. This also works with two colons for explicitly exported functions. -- Gray Calhoun, Assistant Professor of Economics at Iowa State http://gray.clhn.co (web) __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] legitimate use of :::
Hey guys, Because I was curious and had nothing else that I should have been doing (that second part is a lie), I looked into the namespace code. I have a working patch that implements importHiddenFrom. It doesn't currently check whether you then export that symbol (which should not be allowed) but that would be easy to implement. Is there any desire from R-core to have add the importHiddenFrom functionality? If so I can add the export check and submit the patch either here or on bugzilla. I figure its a long shot but hey, at least I now know how the namespace stuff works. I do agree with Peter Meilstrup that poking around at the internals of someone else's code is often not a good idea, but as others have pointed out it is done in practice in some fairly high-profile packages, and if its going to happen it seems like it would be nice to indicate as much in the NAMESPACE file. ~G On Thu, Aug 22, 2013 at 5:41 PM, Gray g...@clhn.co wrote: Peter Meilstrup: (05:01PM on Thu, Aug 22) One most often encounters namespace conflicts at the user level, when loading two packages that have no logical connection other than both bearing on your problem of the moment. Unless I'm mistaken, you can reassign the hidden functions, ie fna - joespackage:::usefulfunction fnb - janespackage:::usefulfunction which is a little bit of a pain, but makes the user's code unambiguous. This also works with two colons for explicitly exported functions. -- Gray Calhoun, Assistant Professor of Economics at Iowa State http://gray.clhn.co (web) __** R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/**listinfo/r-develhttps://stat.ethz.ch/mailman/listinfo/r-devel -- Gabriel Becker Graduate Student Statistics Department University of California, Davis [[alternative HTML version deleted]] __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] legitimate use of :::
Maybe it is not a good idea for R CMD check to check ::: at all, and a
warning in R-exts and ?':::' may well be enough. On the other hand, it
is just so easy to get around :::, because everybody can see its
source code:
`:::`
function (pkg, name)
{
pkg - as.character(substitute(pkg))
name - as.character(substitute(name))
get(name, envir = asNamespace(pkg), inherits = FALSE)
}
Then the package authors who really want to take the risk may start
another hide and seek game, e.g.
`%:::%` = function(pkg, fun) get(fun, envir = asNamespace(pkg),
inherits = FALSE)
'stats' %:::% 'Pillai'
Non-exported functions do not necessarily imply instability. Maybe it
is just because the author is too lazy to document them, or he/she did
not realize these functions happen to be useful for another author.
Although R-devel does not warn against ::: on the packages of the same
maintainer now, these maintainers may change in the future, or one
maintainer can be an author but not maintainer of another package,
from which he/she imports an unexported function, or package authors
have coordinated well with each other about the unexported functions.
I believe there are other legitimate reasons for :::, which might make
it difficult for R to cover all these cases, and also bring additional
communications between package authors and CRAN.
In conclusion, R CMD check cannot really stop :::, and ::: can be
there for good reasons, so how about just let it go?
Regards,
Yihui
--
Yihui Xie xieyi...@gmail.com
Web: http://yihui.name
Department of Statistics, Iowa State University
102 Snedecor Hall, Ames, IA
On Thu, Aug 22, 2013 at 9:02 PM, Gabriel Becker gmbec...@ucdavis.edu wrote:
Hey guys,
Because I was curious and had nothing else that I should have been doing
(that second part is a lie), I looked into the namespace code.
I have a working patch that implements importHiddenFrom. It doesn't
currently check whether you then export that symbol (which should not be
allowed) but that would be easy to implement.
Is there any desire from R-core to have add the importHiddenFrom
functionality? If so I can add the export check and submit the patch either
here or on bugzilla. I figure its a long shot but hey, at least I now know
how the namespace stuff works.
I do agree with Peter Meilstrup that poking around at the internals of
someone else's code is often not a good idea, but as others have pointed
out it is done in practice in some fairly high-profile packages, and if its
going to happen it seems like it would be nice to indicate as much in the
NAMESPACE file.
~G
On Thu, Aug 22, 2013 at 5:41 PM, Gray g...@clhn.co wrote:
Peter Meilstrup: (05:01PM on Thu, Aug 22)
One most often encounters namespace conflicts at the user level, when
loading two packages that have no logical connection other than both
bearing on your problem of the moment.
Unless I'm mistaken, you can reassign the hidden functions, ie
fna - joespackage:::usefulfunction
fnb - janespackage:::usefulfunction
which is a little bit of a pain, but makes the user's code
unambiguous. This also works with two colons for explicitly exported
functions.
--
Gray Calhoun, Assistant Professor of Economics at Iowa State
http://gray.clhn.co (web)
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
