Re: [Rd] Attributes of top level environments clobbered (was Re: [R] possible bug in function 'var' in R 2.7.2?)
On Sat, Oct 4, 2008 at 10:45 AM, laurent [EMAIL PROTECTED] wrote: On Sat, 2008-10-04 at 12:00 +0200, [EMAIL PROTECTED] wrote: Message: 18 Date: Fri, 3 Oct 2008 15:35:18 -0500 (CDT) From: Luke Tierney [EMAIL PROTECTED] Subject: Re: [Rd] Attributes of top level environments clobbered (was Re: [R] possible bug in function 'var' in R 2.7.2?) To: Gabor Grothendieck [EMAIL PROTECTED] Cc: r-devel@r-project.org r-devel@r-project.org, Martin Maechler [EMAIL PROTECTED] Message-ID: [EMAIL PROTECTED] Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed On Fri, 3 Oct 2008, Gabor Grothendieck wrote: On Fri, Oct 3, 2008 at 12:46 PM, Luke Tierney [EMAIL PROTECTED] wrote: On Fri, 3 Oct 2008, Gabor Grothendieck wrote: On Fri, Oct 3, 2008 at 11:43 AM, Luke Tierney [EMAIL PROTECTED] wrote: [...] I do appreciate the excellent R software; however, there are a few points like those addressed on the proto home page which do need to be addressed in R for it to be fully functional. There are some interesting poins on that page that are worth looking into. Over time I suspect all but the current 3. will be addressed, but 3., which is a variant on the unclass issue, is not likely to be. You can call this a deficiency in R if you like, and I would agree in the sense that I think it is inappropriate to allow attributes to be set but not in a reliable way because they can be inadvertenly removed. We should have done this differently. THere were/are two choices: Make reference values, including environments, special in that they may not have attributes. This woud have been fairly easy (modulo one use made in decorating the frames on the search path) and could be done now to clean things up. Make R-visible environments in two parts--a wrapper that is passed by value like standard R objects and could have attributes, and an internal part that is essentially the current environment object. This is analogous to the way that character vectors, even of length 1, consist of an STRSXP wrapper containing CHARSXPs that hold the string. The STRSXP's are visible at the R level, the CHARSXPs are not. This would have been messier to implement, and unfortunately would be very messy to retro-fit at this point, so it isn't likely to happen unless there is some other compelling reason to do so. Couldn't the two options be merged into one for a start ? - Make reference values either attribute-free entities (seems important, as the poor reliability of assigning attributes to environment is probably not widely known), or generate warnings upon assignment of attributes. Some specifics need to be added to the poor reliability phrase relating to of attributed environments. The proto package changes the class attribute of environments (but no other attribute of environments) and proto in turn underlies large widely used packages which likely exercise it thoroughly yet through this experience the only places where this was noticeable were points #1 and #3 of the Avoiding R Bugs section of the proto home page at http://r-proto.googlecode.com Neither of these are two points are deal breakers as - #1 has an easy workaround (just change one line in your DESCRIPTION file) and if the promised fix is made even this won't be necesary, - #3 mostly seems mostly harmless in the context of proto as the only attribute change is from a class of enviornment to c(proto, environment). Artificial examples can be constructed where this is a problem but substantial experience with it suggests that it is not a problem in practice. (The remaining R problems listed are all related to promises, not environments.) - Create an R-level class that contains an environment (Environment ?, envobj ?) and implements an environment-like interface by delegation (somehow like your option 2. above). Gabor could certainly create his own class, but having this administered at the R-core level would have the following potential benefits: - Anyone with similar needs will think twice before starting to implement his own solution. - That one class can be moved to a lower level in the internals (C-level, with a new given SEXPTYPE) if it proves a working solution, and as time permits. Just a thought, Actually, I see the main benefit of this or other approach as providing the missing elements of S3 support to environments thereby potentially streamlining the implementation of every package that needs it (proto, tcltk, R.oo, ...) or more perhaps more accurately allowing tcltk and R.oo and other such packages to become as streamlined as proto already is. Each of these packages could then use inheritance rather than containment thereby leveraging the S3 OO facilities that one really expects R to provide. It would be important that the new class whose objects contain environments is sufficiently indistinguishable in terms of its
Re: [Rd] Attributes of top level environments clobbered (was Re: [R] possible bug in function 'var' in R 2.7.2?)
I will look into fixing it sometime if no one else feels like doing it. The environment aspect is not high priority; some other related issues are more so (locking and active bindings as I recall). But even thoughs may not make it to the top of my queue any time soon. The issue of placing attributes on environments has come up before, many times. It is routinely advised against. For better or worse, the way environments were exposed to the R level is not designed to support this properly. Fixing this so that attributes are supported reliably is non-trivial, and it is hard to justify the effort given that there are standard work-arounds (such as putting the environment in a list and attaching attributes to the list). An alternative way of removing the issues associated with attributes on environments, in line with the way NULL works, is to disable placing attributes on environments, at least from the R level. This option is looking increasingly attractive. luke On Fri, 3 Oct 2008, Gabor Grothendieck wrote: On Fri, Oct 3, 2008 at 3:23 AM, Martin Maechler [EMAIL PROTECTED] wrote: a much better (and much less error-prone) idea would be to install R 2.8.0 alpha even now. It will become 'beta' early next week. We are asking the R community to please install and use pre-release versions of R (if you can / are allowed to) at least from beta onwards, and report problems you see early on *before* the final release. The bug discussed in the following year-old post suggested that the problem of clobbering attributes of top level environment objects would be fixed for 2.7 but its still in R version 2.7.2 (2008-08-25) and also still in R version 2.8.0 alpha (2008-10-01 r46589) https://stat.ethz.ch/pipermail/r-devel/2007-October/047184.html The Avoiding R Bugs section of this page: http://r-proto.googlecode.com has more discussion as well as a list of some other R bugs. This can be tested by creating a package with these two files only: ---DESCRIPTION--- Package: testlazy Version: 1.0-0 Date: 2008-10-03 Title: Test lazy loading Author: G Grothendieck Maintainer: G Grothendieck [EMAIL PROTECTED] Description: Test lazy loading with top level objects. Depends: proto LazyLoad: yes License: GPL-2 ---R/testlazy.R--- TopLevel - proto() --- And then testing it: library(testlazy) class(TopLevel) If its class is environment only then the class attribute was stripped. __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel -- Luke Tierney Chair, Statistics and Actuarial Science Ralph E. Wareham Professor of Mathematical Sciences University of Iowa Phone: 319-335-3386 Department of Statistics andFax: 319-335-3017 Actuarial Science 241 Schaeffer Hall email: [EMAIL PROTECTED] Iowa City, IA 52242 WWW: http://www.stat.uiowa.edu __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Attributes of top level environments clobbered (was Re: [R] possible bug in function 'var' in R 2.7.2?)
On Fri, Oct 3, 2008 at 11:43 AM, Luke Tierney [EMAIL PROTECTED] wrote: I will look into fixing it sometime if no one else feels like doing it. The environment aspect is not high priority; some other related issues are more so (locking and active bindings as I recall). But even thoughs may not make it to the top of my queue any time soon. The issue of placing attributes on environments has come up before, many times. It is routinely advised against. For better or worse, the way environments were exposed to the R level is not designed to support this properly. Fixing this so that attributes are supported reliably is non-trivial, and it is hard to justify the effort given that there are standard work-arounds (such as putting the environment in a list and attaching attributes to the list). An alternative way of removing the issues associated with attributes on environments, in line with the way NULL works, is to disable placing attributes on environments, at least from the R level. This option is looking increasingly attractive. These are not good options: - the workaround does not allow one to inherit methods. This implies tediously rewriting or writing wrappers every inherited method in any such subclass. Its really tantamount to eliminating OO for environments which is not a reasonable solution for a language that is supposed to be OO. - eliminating attributes on environments is even worse since widely used packages such as proto, ggplot2 and other packages would suffer. Maintaining reasonable compatiblity should be a goal of the core development. It would be better to document the current situation than to make such a retrograde change. - if time is a problem perhaps the core group needs to add resources to reasonably address the problems in R. Traditional economics do not apply to an open source project. There is no monetary cost to adding additional developers. luke On Fri, 3 Oct 2008, Gabor Grothendieck wrote: On Fri, Oct 3, 2008 at 3:23 AM, Martin Maechler [EMAIL PROTECTED] wrote: a much better (and much less error-prone) idea would be to install R 2.8.0 alpha even now. It will become 'beta' early next week. We are asking the R community to please install and use pre-release versions of R (if you can / are allowed to) at least from beta onwards, and report problems you see early on *before* the final release. The bug discussed in the following year-old post suggested that the problem of clobbering attributes of top level environment objects would be fixed for 2.7 but its still in R version 2.7.2 (2008-08-25) and also still in R version 2.8.0 alpha (2008-10-01 r46589) https://stat.ethz.ch/pipermail/r-devel/2007-October/047184.html The Avoiding R Bugs section of this page: http://r-proto.googlecode.com has more discussion as well as a list of some other R bugs. This can be tested by creating a package with these two files only: ---DESCRIPTION--- Package: testlazy Version: 1.0-0 Date: 2008-10-03 Title: Test lazy loading Author: G Grothendieck Maintainer: G Grothendieck [EMAIL PROTECTED] Description: Test lazy loading with top level objects. Depends: proto LazyLoad: yes License: GPL-2 ---R/testlazy.R--- TopLevel - proto() --- And then testing it: library(testlazy) class(TopLevel) If its class is environment only then the class attribute was stripped. __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel -- Luke Tierney Chair, Statistics and Actuarial Science Ralph E. Wareham Professor of Mathematical Sciences University of Iowa Phone: 319-335-3386 Department of Statistics andFax: 319-335-3017 Actuarial Science 241 Schaeffer Hall email: [EMAIL PROTECTED] Iowa City, IA 52242 WWW: http://www.stat.uiowa.edu __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Attributes of top level environments clobbered (was Re: [R] possible bug in function 'var' in R 2.7.2?)
On Fri, 3 Oct 2008, Gabor Grothendieck wrote: On Fri, Oct 3, 2008 at 11:43 AM, Luke Tierney [EMAIL PROTECTED] wrote: I will look into fixing it sometime if no one else feels like doing it. The environment aspect is not high priority; some other related issues are more so (locking and active bindings as I recall). But even thoughs may not make it to the top of my queue any time soon. The issue of placing attributes on environments has come up before, many times. It is routinely advised against. For better or worse, the way environments were exposed to the R level is not designed to support this properly. Fixing this so that attributes are supported reliably is non-trivial, and it is hard to justify the effort given that there are standard work-arounds (such as putting the environment in a list and attaching attributes to the list). An alternative way of removing the issues associated with attributes on environments, in line with the way NULL works, is to disable placing attributes on environments, at least from the R level. This option is looking increasingly attractive. These are not good options: - the workaround does not allow one to inherit methods. This implies tediously rewriting or writing wrappers every inherited method in any such subclass. Its really tantamount to eliminating OO for environments which is not a reasonable solution for a language that is supposed to be OO. It would mean sealing environments, making them final, pick your favorite OO terminology. Standard thing to do in many OO languages when it is wararanted. - eliminating attributes on environments is even worse since widely used packages such as proto, ggplot2 and other packages would suffer. Maintaining reasonable compatiblity should be a goal of the core development. It would be better to document the current situation than to make such a retrograde change. The authors of proto would finally need to bite the bullet and address this issue. This would make proto more reliable in the end. luke - if time is a problem perhaps the core group needs to add resources to reasonably address the problems in R. Traditional economics do not apply to an open source project. There is no monetary cost to adding additional developers. luke On Fri, 3 Oct 2008, Gabor Grothendieck wrote: On Fri, Oct 3, 2008 at 3:23 AM, Martin Maechler [EMAIL PROTECTED] wrote: a much better (and much less error-prone) idea would be to install R 2.8.0 alpha even now. It will become 'beta' early next week. We are asking the R community to please install and use pre-release versions of R (if you can / are allowed to) at least from beta onwards, and report problems you see early on *before* the final release. The bug discussed in the following year-old post suggested that the problem of clobbering attributes of top level environment objects would be fixed for 2.7 but its still in R version 2.7.2 (2008-08-25) and also still in R version 2.8.0 alpha (2008-10-01 r46589) https://stat.ethz.ch/pipermail/r-devel/2007-October/047184.html The Avoiding R Bugs section of this page: http://r-proto.googlecode.com has more discussion as well as a list of some other R bugs. This can be tested by creating a package with these two files only: ---DESCRIPTION--- Package: testlazy Version: 1.0-0 Date: 2008-10-03 Title: Test lazy loading Author: G Grothendieck Maintainer: G Grothendieck [EMAIL PROTECTED] Description: Test lazy loading with top level objects. Depends: proto LazyLoad: yes License: GPL-2 ---R/testlazy.R--- TopLevel - proto() --- And then testing it: library(testlazy) class(TopLevel) If its class is environment only then the class attribute was stripped. __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel -- Luke Tierney Chair, Statistics and Actuarial Science Ralph E. Wareham Professor of Mathematical Sciences University of Iowa Phone: 319-335-3386 Department of Statistics andFax: 319-335-3017 Actuarial Science 241 Schaeffer Hall email: [EMAIL PROTECTED] Iowa City, IA 52242 WWW: http://www.stat.uiowa.edu -- Luke Tierney Chair, Statistics and Actuarial Science Ralph E. Wareham Professor of Mathematical Sciences University of Iowa Phone: 319-335-3386 Department of Statistics andFax: 319-335-3017 Actuarial Science 241 Schaeffer Hall email: [EMAIL PROTECTED] Iowa City, IA 52242 WWW: http://www.stat.uiowa.edu __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Attributes of top level environments clobbered (was Re: [R] possible bug in function 'var' in R 2.7.2?)
On Fri, Oct 3, 2008 at 12:46 PM, Luke Tierney [EMAIL PROTECTED] wrote: On Fri, 3 Oct 2008, Gabor Grothendieck wrote: On Fri, Oct 3, 2008 at 11:43 AM, Luke Tierney [EMAIL PROTECTED] wrote: I will look into fixing it sometime if no one else feels like doing it. The environment aspect is not high priority; some other related issues are more so (locking and active bindings as I recall). But even thoughs may not make it to the top of my queue any time soon. The issue of placing attributes on environments has come up before, many times. It is routinely advised against. For better or worse, the way environments were exposed to the R level is not designed to support this properly. Fixing this so that attributes are supported reliably is non-trivial, and it is hard to justify the effort given that there are standard work-arounds (such as putting the environment in a list and attaching attributes to the list). An alternative way of removing the issues associated with attributes on environments, in line with the way NULL works, is to disable placing attributes on environments, at least from the R level. This option is looking increasingly attractive. These are not good options: - the workaround does not allow one to inherit methods. This implies tediously rewriting or writing wrappers every inherited method in any such subclass. Its really tantamount to eliminating OO for environments which is not a reasonable solution for a language that is supposed to be OO. It would mean sealing environments, making them final, pick your favorite OO terminology. Standard thing to do in many OO languages when it is wararanted. I don't think this really addresses the problem. The S3 model is in principle capable of handling environments and almost does so fully now. - eliminating attributes on environments is even worse since widely used packages such as proto, ggplot2 and other packages would suffer. Maintaining reasonable compatiblity should be a goal of the core development. It would be better to document the current situation than to make such a retrograde change. The authors of proto would finally need to bite the bullet and address this issue. This would make proto more reliable in the end. The fact that environment attributes get clobbered at top level when defined in packages with lazy loading (but not outside packages nor in packages without lazy loading) is clearly a deficiency of R, not proto. Furthermore, the suggstion that this deficiency in R somehow reflects any unreliably in proto is likewise not accurate. proto is extremely reliable, particularly in comparison to R. In fact there are no known bugs in the development version of proto and large widely used packages use proto. (If you are aware of any bugs let me know.) The fact that it is so solid is quite understandable because even if some of its code is necessarily complex the code is so short that its readily possible to accomplish this apprent bug-free status with reasonble effort. Furthermore, the proto home page documents the problems -- mostly problems with R itself, not proto. I do appreciate the excellent R software; however, there are a few points like those addressed on the proto home page which do need to be addressed in R for it to be fully functional. luke - if time is a problem perhaps the core group needs to add resources to reasonably address the problems in R. Traditional economics do not apply to an open source project. There is no monetary cost to adding additional developers. luke On Fri, 3 Oct 2008, Gabor Grothendieck wrote: On Fri, Oct 3, 2008 at 3:23 AM, Martin Maechler [EMAIL PROTECTED] wrote: a much better (and much less error-prone) idea would be to install R 2.8.0 alpha even now. It will become 'beta' early next week. We are asking the R community to please install and use pre-release versions of R (if you can / are allowed to) at least from beta onwards, and report problems you see early on *before* the final release. The bug discussed in the following year-old post suggested that the problem of clobbering attributes of top level environment objects would be fixed for 2.7 but its still in R version 2.7.2 (2008-08-25) and also still in R version 2.8.0 alpha (2008-10-01 r46589) https://stat.ethz.ch/pipermail/r-devel/2007-October/047184.html The Avoiding R Bugs section of this page: http://r-proto.googlecode.com has more discussion as well as a list of some other R bugs. This can be tested by creating a package with these two files only: ---DESCRIPTION--- Package: testlazy Version: 1.0-0 Date: 2008-10-03 Title: Test lazy loading Author: G Grothendieck Maintainer: G Grothendieck [EMAIL PROTECTED] Description: Test lazy loading with top level objects. Depends: proto LazyLoad: yes License: GPL-2 ---R/testlazy.R--- TopLevel - proto() --- And then testing it: library(testlazy) class(TopLevel) If its class is
