[Rd] Offer zip builds
If you go here: https://cran.cnr.berkeley.edu/bin/windows/base you see EXE installers for Windows. This contrasts with other programming languages that offer both an executable installer and ZIP files that can be extracted and run. For example Go: https://golang.org/dl and Nim: https://nim-lang.org/install_windows.html Python: https://python.org/downloads/release/python-373 and C#: https://dotnet.microsoft.com/download/dotnet-core/2.2 PHP: https://windows.php.net/download Perl: http://strawberryperl.com/releases.html __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Converting non-32-bit integers from python to R to use bit64: reticulate
Thank you Martin for giving to know and developing 'Rmpfr' library for unlimited size integers (GNU C GMP) and arbitrary precision floats (GNU C MPFR): https://cran.r-project.org/package=Rmpfr My question is: In the long term (For R3.7.0 or R3.8.0): Does it have sense that CMP substitutes INTSXP, and MPFR substitutes REALSXP code? With this we would achieve that an integer is always an integer, and a numeric double precision float always a numeric double precision float, without sometimes casting underneath. And would the R Community / R Ordinary Members would be willing to help R Core on such implementation (If has sense, and wants to be adopted)? Thank you all! :) > > If you are interested in using unlimited size integers, you > could use the CRAN R package 'gmp' which builds on the GMP = GNU > MP = GNU Multi Precision C library. > >https://cran.r-project.org/package=gmp > > (and for arbitrary precision "floats", see CRAN pkg 'Rmpfr' > built on package gmp, and both the GNU C libraries GMP and > MPFR: >https://cran.r-project.org/package=Rmpfr > ) > > [[alternative HTML version deleted]] __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] rbind has confusing result for custom sub-class (possible bug?)
Thanks for following up! In fact that's exactly what was done here: https://github.com/Rdatatable/data.table/pull/3602/files On Sun, Jun 2, 2019 at 8:00 PM Joshua Ulrich wrote: > I thought it would be good to summarize my thoughts, since I made a > few hypotheses that turned out to be false. > > This isn't a bug in base R, in either rbind() or `[<-.Date`. > > To summarize the root cause: > base::rbind.data.frame() calls `[<-` for each column of the > data.frame, and there is no `[<-.IDate` method to ensure the > replacement value is converted to integer. And, in fact, `[<-.Date` > calls as.Date() and data.table::as.Date.IDate() calls as.numeric() on > the IDate object. So the problem exists, and can be fixed, in > data.table. > > Best, > Josh > > On Mon, May 27, 2019 at 9:34 AM Joshua Ulrich > wrote: > > > > Follow-up (inline) on my comment about a potential issue in `[<-.Date`. > > > > On Mon, May 27, 2019 at 9:31 AM Michael Chirico > > wrote: > > > > > > Yes, thanks for following up on thread here. And thanks again for > clearing things up, your email was a finger snap of clarity on the whole > issue. > > > > > > I'll add that actually it was data.table's code at fault on the > storage conversion -- note that if you use an arbitrary sub-class 'foo' > with no methods defined, it'll stay integer. > > > > > > That's because [<- calls as.Date and then as.Date.IDate, and that > method (ours) has as.numeric(); earlier I had recognized that if we > commented that line, the issue was "fixed" but I still wasn't understanding > the root cause. > > > > > > My last curiosity on this issue will be in my follow-up thread. > > > > > > Mike C > > > > > > On Mon, May 27, 2019, 10:25 PM Joshua Ulrich > wrote: > > >> > > >> On Sun, May 26, 2019 at 6:47 AM Joshua Ulrich < > josh.m.ulr...@gmail.com> wrote: > > >> > > > >> > On Sun, May 26, 2019 at 4:06 AM Michael Chirico > > >> > wrote: > > >> > > > > >> > > Have finally managed to come up with a fix after checking out > sys.calls() > > >> > > from within the as.Date.IDate debugger, which shows something > like: > > >> > > > > >> > > [[1]] rbind(DF, DF) > > >> > > [[2]] rbind(deparse.level, ...) > > >> > > [[3]] `[<-`(`*tmp*`, ri, value = 18042L) > > >> > > [[4]] `[<-.Date`(`*tmp*`, ri, value = 18042L) > > >> > > [[5]] as.Date(value) > > >> > > [[6]] as.Date.IDate(value) > > >> > > > > >> > > I'm not sure why [<- is called, I guess the implementation is to > assign to > > >> > > the output block by block? Anyway, we didn't have a [<- method. > And > > >> > > [<-.Date looks like: > > >> > > > > >> > > value <- unclass(as.Date(value)) # <- converts to double > > >> > > .Date(NextMethod(.Generic), oldClass(x)) # <- restores 'IDate' > class > > >> > > > > >> > > So we can fix our bug by defining a [<- class; the question that > I still > > >> > > don't see answered in documentation or source code is, why/where > is [<- > > >> > > called, exactly? > > >> > > > > >> > Your rbind(DF, DF) call dispatches to base::rbind.data.frame(). The > > >> > `[<-` call is this line: > > >> > value[[jj]][ri] <- if (is.factor(xij)) as.vector(xij) else xij > > >> > > > >> > That's where the storage.mode changes from integer to double. > > >> > > > >> > debug: value[[jj]][ri] <- if (is.factor(xij)) as.vector(xij) else > xij > > >> > Browse[2]> > > >> > debug: xij > > >> > Browse[2]> storage.mode(xij) > > >> > [1] "integer" > > >> > Browse[2]> value[[jj]][ri] > > >> > [1] "2019-05-26" > > >> > Browse[2]> storage.mode(value[[jj]][ri]) > > >> > [1] "integer" > > >> > Browse[2]> > > >> > debug: if (!is.null(nm <- names(xij))) names(value[[jj]])[ri] <- nm > > >> > Browse[2]> storage.mode(value[[jj]][ri]) > > >> > [1] "double" > > >> > > > >> To be clear, I don't think this is a bug in rbind() or > > >> rbind.data.frame(). The confusion is that rbind.data.frame() calls > > >> `[<-` for each column of the data.frame, and there is no `[<-.IDate` > > >> method. So the parent class method is dispatched, which converts the > > >> storage mode to double. > > >> > > >> Someone may argue that this is an issue with `[<-.Date`, and that it > > >> shouldn't convert the storage.mode from integer to double. > > > > I don't think this is an issue. The storage mode isn't converted if > > the replacement is the same storage mode. For example: > > > > R> x <- .Date(1:5) > > R> storage.mode(x) > > [1] "integer" > > R> x[1L] <- .Date(0L) > > R> storage.mode(x) > > [1] "integer" > > R> x[1L] <- .Date(0) > > R> storage.mode(x) > > [1] "double" > > > > >> > > > >> > > Mike C > > >> > > > > >> > > On Sun, May 26, 2019 at 1:16 PM Michael Chirico < > michaelchiri...@gmail.com> > > >> > > wrote: > > >> > > > > >> > > > Debugging this issue: > > >> > > > > > >> > > > https://github.com/Rdatatable/data.table/issues/2008 > > >> > > > > > >> > > > We have custom class 'IDate' which inherits from 'Date' (it > just forces > > >> > > > integer storage for efficiency, hence, I). > > >> > > > > > >> > > > The concatenation d
Re: [Rd] rgl install for R 3.7
On 02/06/2019 16:28, Koenker, Roger W wrote: I’ve installed R 3.7.0 on a new laptop running macos 10.14.5 and have managed to get most of my usual packages I presume 'R 3.7.0' is R-devel: it is not released and may never be released under that version. to compile from source with a ~/.R/Makevars file that looks like this: CC=/usr/local/clang8/bin/clang CXX=/usr/local/clang8/bin/clang++ LDFLAGS=-L/usr/local/clang8/lib CPPFLAGS=-I/usr/local/clang8/include -I/opt/X11/include/freetype2 I suspect you don't want the second: if you have pkg-config it should find include paths for you. FC=/usr/local/gfortran/bin/gfortran FLIBS=-L/usr/local/gfortran/lib -lgfortran As usual, the last challenge seems to be to get rgl installed. Compilation _seems_ to go smoothly, but I now see: Error in dyn.load(file, DLLpath = DLLpath, ...) : unable to load shared object '/Library/Frameworks/R.framework/Versions/3.7/Resources/library/00LOCK-rgl/00new/rgl/libs/rgl.so': dlopen(/Library/Frameworks/R.framework/Versions/3.7/Resources/library/00LOCK-rgl/00new/rgl/libs/rgl.so, 6): Symbol not found: _FT_Attach_File Referenced from: /Library/Frameworks/R.framework/Versions/3.7/Resources/library/00LOCK-rgl/00new/rgl/libs/rgl.so Perhaps you should show the linking line. A similar setup works for me: configure: Darwin, so ensuring /opt/X11/bin is at the head of the PATH... checking for pkg-config... yes ... Do you have pkg-config (it is not a standard part of macOS, but is available on SU's site -- see the R-admin manual)? You may need to set its path: for rgl I used PKG_CONFIG_PATH=/usr/local/lib/pkgconfig:/usr/lib/pkgconfig (which should be the default) but for a few packages (Cairo gdtools rsvg) PKG_CONFIG_PATH=/opt/X11/lib/pkgconfig:/usr/local/lib/pkgconfig:/usr/lib/pkgconfig /usr/local/clang8/bin/clang++ -std=gnu++11 -dynamiclib -Wl,-headerpad_max_install_names -undefined dynamic_lookup -single_module -multiply_defined suppress -L/Users/ripley/R/R-devel/lib -L/usr/local/clang8/lib -L/usr/local/lib -o rgl.so ABCLineSet.o BBoxDeco.o Background.o ClipPlane.o Color.o Disposable.o Light.o LineSet.o LineStripSet.o Material.o NULLgui.o PlaneSet.o PointSet.o PrimitiveSet.o RenderContext.o Shape.o SphereMesh.o SphereSet.o SpriteSet.o String.o Surface.o TextSet.o Texture.o Viewpoint.o api.o assert.o callbacks.o device.o devicemanager.o fps.o ftgl.o geom.o gl2ps.o glErrors.o glgui.o gui.o init.o par3d.o pixmap.o platform.o pretty.o render.o rglmath.o rglview.o scene.o select.o subscene.o win32gui.o win32lib.o x11gui.o x11lib.o -lGLU -lGL -framework GLKit -framework OpenGL -dylib_file /System/Library/Frameworks/OpenGL.framework/Versions/A/Libraries/libGL.dylib:/System/Library/Frameworks/OpenGL.framework/Versions/A/Libraries/libGL.dylib -L/usr/local/lib -lpng16 -L/usr/X11/lib -lX11 -L/usr/local/lib -lfreetype -L/Users/ripley/R/R-devel/lib -lR -Wl,-framework -Wl,CoreFoundation so that is statically linking libfreetype from /usr/local/lib and installed from https://mac.r-project.org/libs/freetype-2.5.5-darwin.13-x86_64.tar.gz . And that provides a freetype2.pc file, so % pkg-config freetype2 --cflags -I/usr/local/include/freetype2 -I/usr/local/include/libpng16 % pkg-config freetype2 --libs -L/usr/local/lib -lfreetype There is a comment somewhat after this about rgl requiring XQuartz, but I have XQuartz 2.7.11 so I don’t think that this is the problem. Apparently, there is still a freetype problem, however I’m out of my depth at this point. Any suggestions would be most welcome. Roger Koenker Honorary Professor Department of Economics, UCL London WC1H 0AX. -- Brian D. Ripley, rip...@stats.ox.ac.uk Emeritus Professor of Applied Statistics, University of Oxford __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] rgl install for R 3.7
On 02/06/2019 11:28 a.m., Koenker, Roger W wrote: I’ve installed R 3.7.0 on a new laptop running macos 10.14.5 and have managed to get most of my usual packages to compile from source with a ~/.R/Makevars file that looks like this: CC=/usr/local/clang8/bin/clang CXX=/usr/local/clang8/bin/clang++ LDFLAGS=-L/usr/local/clang8/lib CPPFLAGS=-I/usr/local/clang8/include -I/opt/X11/include/freetype2 FC=/usr/local/gfortran/bin/gfortran FLIBS=-L/usr/local/gfortran/lib -lgfortran As usual, the last challenge seems to be to get rgl installed. Compilation _seems_ to go smoothly, but I now see: Error in dyn.load(file, DLLpath = DLLpath, ...) : unable to load shared object '/Library/Frameworks/R.framework/Versions/3.7/Resources/library/00LOCK-rgl/00new/rgl/libs/rgl.so': dlopen(/Library/Frameworks/R.framework/Versions/3.7/Resources/library/00LOCK-rgl/00new/rgl/libs/rgl.so, 6): Symbol not found: _FT_Attach_File Referenced from: /Library/Frameworks/R.framework/Versions/3.7/Resources/library/00LOCK-rgl/00new/rgl/libs/rgl.so There is a comment somewhat after this about rgl requiring XQuartz, but I have XQuartz 2.7.11 so I don’t think that this is the problem. Apparently, there is still a freetype problem, however I’m out of my depth at this point. Any suggestions would be most welcome. I'm not up to date on MacOS (I have 10.13.6) or clang. I'm compiling with something that calls itself gcc, but gcc --version gives Apple LLVM version 9.0.0 (clang-900.0.39.2) So this isn't very relevant, but for what it's worth, rgl compiles and loads fine in the current R-devel (r76642). Duncan Murdoch __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] rgl install for R 3.7
I’ve installed R 3.7.0 on a new laptop running macos 10.14.5 and have managed to get most of my usual packages to compile from source with a ~/.R/Makevars file that looks like this: CC=/usr/local/clang8/bin/clang CXX=/usr/local/clang8/bin/clang++ LDFLAGS=-L/usr/local/clang8/lib CPPFLAGS=-I/usr/local/clang8/include -I/opt/X11/include/freetype2 FC=/usr/local/gfortran/bin/gfortran FLIBS=-L/usr/local/gfortran/lib -lgfortran As usual, the last challenge seems to be to get rgl installed. Compilation _seems_ to go smoothly, but I now see: Error in dyn.load(file, DLLpath = DLLpath, ...) : unable to load shared object '/Library/Frameworks/R.framework/Versions/3.7/Resources/library/00LOCK-rgl/00new/rgl/libs/rgl.so': dlopen(/Library/Frameworks/R.framework/Versions/3.7/Resources/library/00LOCK-rgl/00new/rgl/libs/rgl.so, 6): Symbol not found: _FT_Attach_File Referenced from: /Library/Frameworks/R.framework/Versions/3.7/Resources/library/00LOCK-rgl/00new/rgl/libs/rgl.so There is a comment somewhat after this about rgl requiring XQuartz, but I have XQuartz 2.7.11 so I don’t think that this is the problem. Apparently, there is still a freetype problem, however I’m out of my depth at this point. Any suggestions would be most welcome. Roger Koenker Honorary Professor Department of Economics, UCL London WC1H 0AX. __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] survival changes
Terry Let me call this things to think about, rather than advice. I went through a similar process twice, once about 30 years ago and once about 20 years ago. I had fewer dependent packages of course, but still enough to cause headaches. I don't recommend doing it often. - I think you need to consider where you would like to end up before deciding how to get there. If you end up having to maintain a lot of legacy stuff I don't think you will be very happy. So then the problem becomes how to help people get off the part you want to abandon, rather then how to help them stay on it. - I know you are very experienced, but I will be really impressed if you get the new approach perfect on the first shot. That argues for having a new package with hardly any users so you can fiddle with the API more easily, and not deprecating the old one until you are really happy with the new one. - There may be a part which is common to both old and new and/or there may be a part which is what most dependent packages use. If you can separate that out as something like survivalBase it would make your life easier. That will be especial true if that part is more stable, so don't put in anything you are experimenting with. Good luck, Paul Gilbert On 6/1/19 8:02 PM, Therneau, Terry M., Ph.D. via R-devel wrote: On 6/1/19 1:32 PM, Marc Schwartz wrote: On Jun 1, 2019, at 12:59 PM, Peter Langfelder wrote: On Sat, Jun 1, 2019 at 3:22 AM Therneau, Terry M., Ph.D. via R-devel wrote: In the next version of the survival package I intend to make a non-upwardly compatable change to the survfit object. With over 600 dependent packages this is not something to take lightly, and I am currently undecided about the best way to go about it. I'm looking for advice. The change: 20+ years ago I had decided not to include the initial x=0,y=1 data point in the survfit object itself. It was not formally an estimand and the plot/points/lines etc routines could add this on themselves. That turns out to have been a mistake, and has led to a steady proliferation of extra bits as I realized that the time axis doesn't always start at 0, and later (with multi state) that y does not always start at 1 (though the states sum to 1), and later the the error doesn't always start at 0, and another realization with cumulative hazard, and ... The new survfit method for multi-state coxph models was going to add yet another special case. Basically every component is turning into a duplicate of "row 1" vs "all the others". (And inconsistently named.) Three possible solutions 1. Current working draft of survival_3.0.3: Add a 'version' element to the survfit object and a 'survfit2.3' function that converts old to new. All my downstream functions (print, plot,...) start with an "if (old) update to new" line. This has allowed me to stage updates to the functions that create survfit objects -- I expect it to happen slowly. There will also be a survfit3.2 function to go backwards. Both the forward and backwards functions leave objects alone if they are currently in the desired format. 2. Make a new class "survfit3" and the necessary 'as' functions. The package would contain plot.survfit and plot.survfit3 methods, the former a two line "convert and call the second" function. 3. Something I haven't thought of. A more "clean break" solution would be to start a whole new package (call it survival2) that would make these changes, and deprecate the current survival. You could add warnings about deprecation and urging users to switch in existing survival functions. You could continue bugfixes for survival but only add new features to survival2. The new survival2 and the current survival could live side by side on CRAN for quite some time, giving maintainers of dependent packages (and just plain users) enough time to switch. This could allow you to change/clean up other parts of the package that you could perhaps also use a rethink/rewrite, without too much concern for backward compatibility. Peter Hi, I would be cautious in going in that direction, bearing in mind that survival is a Recommended package, therefore included in the default R distribution from the R Foundation and other parties. To have two versions can/will result in substantial confusion, and I would argue against that approach. There is language in the CRAN submission policy that covers API changes, which strictly speaking, may or may not be the case here, depending upon which direction Terry elects to go: "If an update will change the package’s API and hence affect packages depending on it, it is expected that you will contact the maintainers of affected packages and suggest changes, and give them time (at least 2 weeks, ideally more) to prepare updates before submitting your updated package. Do mention in the submission email which packages are affected and that their maintainers have been informed. In order to derive the reve
Re: [Rd] rbind has confusing result for custom sub-class (possible bug?)
I thought it would be good to summarize my thoughts, since I made a few hypotheses that turned out to be false. This isn't a bug in base R, in either rbind() or `[<-.Date`. To summarize the root cause: base::rbind.data.frame() calls `[<-` for each column of the data.frame, and there is no `[<-.IDate` method to ensure the replacement value is converted to integer. And, in fact, `[<-.Date` calls as.Date() and data.table::as.Date.IDate() calls as.numeric() on the IDate object. So the problem exists, and can be fixed, in data.table. Best, Josh On Mon, May 27, 2019 at 9:34 AM Joshua Ulrich wrote: > > Follow-up (inline) on my comment about a potential issue in `[<-.Date`. > > On Mon, May 27, 2019 at 9:31 AM Michael Chirico > wrote: > > > > Yes, thanks for following up on thread here. And thanks again for clearing > > things up, your email was a finger snap of clarity on the whole issue. > > > > I'll add that actually it was data.table's code at fault on the storage > > conversion -- note that if you use an arbitrary sub-class 'foo' with no > > methods defined, it'll stay integer. > > > > That's because [<- calls as.Date and then as.Date.IDate, and that method > > (ours) has as.numeric(); earlier I had recognized that if we commented that > > line, the issue was "fixed" but I still wasn't understanding the root cause. > > > > My last curiosity on this issue will be in my follow-up thread. > > > > Mike C > > > > On Mon, May 27, 2019, 10:25 PM Joshua Ulrich > > wrote: > >> > >> On Sun, May 26, 2019 at 6:47 AM Joshua Ulrich > >> wrote: > >> > > >> > On Sun, May 26, 2019 at 4:06 AM Michael Chirico > >> > wrote: > >> > > > >> > > Have finally managed to come up with a fix after checking out > >> > > sys.calls() > >> > > from within the as.Date.IDate debugger, which shows something like: > >> > > > >> > > [[1]] rbind(DF, DF) > >> > > [[2]] rbind(deparse.level, ...) > >> > > [[3]] `[<-`(`*tmp*`, ri, value = 18042L) > >> > > [[4]] `[<-.Date`(`*tmp*`, ri, value = 18042L) > >> > > [[5]] as.Date(value) > >> > > [[6]] as.Date.IDate(value) > >> > > > >> > > I'm not sure why [<- is called, I guess the implementation is to > >> > > assign to > >> > > the output block by block? Anyway, we didn't have a [<- method. And > >> > > [<-.Date looks like: > >> > > > >> > > value <- unclass(as.Date(value)) # <- converts to double > >> > > .Date(NextMethod(.Generic), oldClass(x)) # <- restores 'IDate' class > >> > > > >> > > So we can fix our bug by defining a [<- class; the question that I > >> > > still > >> > > don't see answered in documentation or source code is, why/where is [<- > >> > > called, exactly? > >> > > > >> > Your rbind(DF, DF) call dispatches to base::rbind.data.frame(). The > >> > `[<-` call is this line: > >> > value[[jj]][ri] <- if (is.factor(xij)) as.vector(xij) else xij > >> > > >> > That's where the storage.mode changes from integer to double. > >> > > >> > debug: value[[jj]][ri] <- if (is.factor(xij)) as.vector(xij) else xij > >> > Browse[2]> > >> > debug: xij > >> > Browse[2]> storage.mode(xij) > >> > [1] "integer" > >> > Browse[2]> value[[jj]][ri] > >> > [1] "2019-05-26" > >> > Browse[2]> storage.mode(value[[jj]][ri]) > >> > [1] "integer" > >> > Browse[2]> > >> > debug: if (!is.null(nm <- names(xij))) names(value[[jj]])[ri] <- nm > >> > Browse[2]> storage.mode(value[[jj]][ri]) > >> > [1] "double" > >> > > >> To be clear, I don't think this is a bug in rbind() or > >> rbind.data.frame(). The confusion is that rbind.data.frame() calls > >> `[<-` for each column of the data.frame, and there is no `[<-.IDate` > >> method. So the parent class method is dispatched, which converts the > >> storage mode to double. > >> > >> Someone may argue that this is an issue with `[<-.Date`, and that it > >> shouldn't convert the storage.mode from integer to double. > > I don't think this is an issue. The storage mode isn't converted if > the replacement is the same storage mode. For example: > > R> x <- .Date(1:5) > R> storage.mode(x) > [1] "integer" > R> x[1L] <- .Date(0L) > R> storage.mode(x) > [1] "integer" > R> x[1L] <- .Date(0) > R> storage.mode(x) > [1] "double" > > >> > > >> > > Mike C > >> > > > >> > > On Sun, May 26, 2019 at 1:16 PM Michael Chirico > >> > > > >> > > wrote: > >> > > > >> > > > Debugging this issue: > >> > > > > >> > > > https://github.com/Rdatatable/data.table/issues/2008 > >> > > > > >> > > > We have custom class 'IDate' which inherits from 'Date' (it just > >> > > > forces > >> > > > integer storage for efficiency, hence, I). > >> > > > > >> > > > The concatenation done by rbind, however, breaks this and returns a > >> > > > double: > >> > > > > >> > > > library(data.table) > >> > > > DF = data.frame(date = as.IDate(Sys.Date())) > >> > > > storage.mode(rbind(DF, DF)$date) > >> > > > # [1] "double" > >> > > > > >> > > > This is specific to base::rbind (data.table's rbind returns an > >> > > > integer as > >> > > > expected); in ?rbind we see: > >> > > > > >> > > > The method
