Re: [Rd] string concatenation operator (revisited)

or priority? E.g. how is > > a ! b > c > > parsed? As (a ! b) > c. Their precedence is between that of + and - and that of < and >. So "x" ! 1+2 evalates to "x3" and "x" ! 1+2 < "x4" is TRUE. (Actually, pqR also has a .. operator that fixes the problems with generating sequences with the : operator, and it has precedence lower than + and - and higher than ! and !!, but that's not relevant if you don't have the .. operator.) Radford Neal __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] string concatenation operator (revisited)

g on by having a different operator for string concatenation. Plus ! and !! semm natural for representing paste0 and paste, whereas using ++ for paste (with + for paste0) would look rather strange. Radford Neal __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] svd for Large Matrix

d even aside from whatever LAPACK is doing. Regards, Radford Neal __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Faster sorting algorithm...

this discussion is that the "merge" method is substantially faster than "shell" for these character vectors with a million elements, while retaining the stability and collation properties of "shell" (whereas "radix" only does C collation). It would probably not be too hard to take the merge sort code from pqR and use it in R core's implementation. The merge sort in pqR doesn't exploit parallelism at the moment, but merge sort is potentially quite parallelizable (though I think the storage allocation strategy I use would have to be modified). Regards, Radford Neal __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Bug: time complexity of substring is quadratic

tem elapsed 0.041 0.000 0.041 Some of this may be due to pqR's faster garbage collector - R Core implementatons have a particular GC problem with strings, as explained at https://radfordneal.wordpress.com/2018/11/29/faster-garbage-collection-in-pqr/ But

Re: [Rd] Inefficiency in df$col

is produced for any other list, while still retaining the massive slowdown. There is no need for you to write $.data.frame in C. You just need to delete the version written in R. Radford Neal __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] Inefficiency in df$col

" to the warning message (while making the earlier part of the message less intelligible). I think you might want to just delete the definition of $.data.frame, reverting to the situation before R-3.1.0. Radford Neal __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Subsetting row in single column matrix drops names in resulting vector

name to all the elements would be rather strange...). After v <- a[1,_], the program may well have an expression like v[nc] where nc is a column name. We want this to still work if there happens to be only one column. That will happen only if a[1,_] attaches a column name, not a row name, when a has only one column. Radford Neal __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Subsetting row in single column matrix drops names in resulting vector

-rs-design-flaws-in-a-new-version-of-pqr/ That was written when most of these features where introduced, though getting your specific example right relies on another change introduced in the most recent version. Radford Neal __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Bias in R's random integers?

fast hashing algorithm that produces the same results as the simple algorithm, here: https://stat.ethz.ch/pipermail/r-devel/2017-October/075012.html The latest version of this will be in the soon-to-be new release of pqR, and will of course enabled automatically whenever it seems desirable, for a cons

Re: [Rd] Bias in R's random integers?

t R. > set.seed(123) > m <- (2/5)*2^32 > m > 2^31 [1] FALSE > x <- sample(m, 100, replace = TRUE) > table(x %% 2) 0 1 499412 500588 So I doubt that this has anything to do with bias from using 32-bit random values. Radford Neal __

Re: [Rd] True length - length(unclass(x)) - without having to call unclass()?

isting code than to implement more and more special-case functions like anyNA or some special function to allow length(unclass(x)) to be done quickly. The variant result mechanism has extremely low overhead, and is not hard to implement. Radford Neal

Re: [Rd] Proposed speedup of ifelse

bit silly to plot the distributions of times, which will mostly reflect variations in when garbage collections at various levels occur - just the mean is what is relevant. Regards, Radford Neal __ R-devel@r-project.org mailing list https://st

Re: [Rd] Extreme bunching of random values from runif with Mersenne-Twister seed

rst number in the sequence, but rather (say) the 10th, might be preferable. Radford Neal > > seeds = c(86548915L, 86551615L, 86566163L, 86577411L, 86584144L, > 86584272L, > + 86620568L, 86724613L, 86756002L, 86768593L, 86772411L, 86781516L, > + 86794389L, 86805854L, 86814600L, 868350

Re: [Rd] Illegal Logical Values

> This does work as advertised. Not always. As I reported on bugzilla three years ago (PR#15878), it only works if the logical argument does not have to be copied. The bug has been fixed in pqR since pqR-2014-09-30. Radford Neal __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] uniform sampling without replacement algorithm

tbl[h].pos = j; /* don't set until after search for entry n-i */ } VMAXSET(vmax); } return r; } This will be in a new version of pqR that will have many other performance improvements as well, which I expect to release in a few weeks. Radford Neal __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Revert to R 3.2.x code of logicalSubscript in subscript.c?

andling things like x[-1] and x[2:length(x)], so I added the x[v] test to see what performance is like when this special handling isn't invoked. There's no particular reason pqR's code for these operations couldn't be adapted for use in the R Core implementaton, though there are probably a few issues involving large vectors, and the special handling of x[2:length(x)] would require implementing pqR's internal "variant result" mechanism. pqR also has much faster code for some other subset and subset assignment operations. Radford Neal __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Please avoid direct use of NAMED and SET_NAMED macros

d suggest that you produce clear and complete documention on the new scheme. Radford Neal __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] bug: deparse sometimes omits parentheses for unary operators

or BUG FIXES under VERSION RELEASED 2015-09-14. Some are fixed in R-3.4.0, but most remain. Radford Neal __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] attributes on symbols

on symbols are not saved when the workspace is saved with q("yes"). Radford Neal __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] [WISH / PATCH] possibility to split string literals

> : > I don't think it is reasonable to change the parser this way. This is > currently valid R code: > > a <- "foo" > "bar" > > and with the new syntax, it is also valid, but with a different > meaning. Yes. The meaning of that would certainly need to stay the same.

Re: [Rd] RFC: (in-principle) native unquoting for standard evaluation

@ operator. I'm not necessarily advocating that particular use (my ideas in this respect are still undergoing revisions), but the overall point is that there may well be several good uses of a unary @ operator (and there aren't many other good characters to use for a unary operator besides @). It is unclear t

Re: [Rd] length(unclass(x)) without unclass(x)?

s are implemented using pqR's "variant result" mechanism, which also allows many other optimizations. See https://radfordneal.wordpress.com/2013/06/30/how-pqr-makes-programs-faster-by-not-doing-things/ for some explanation. There is no particular reason this mechanism couldn't be in

Re: [Rd] accelerating matrix multiply

> From: "Cohn, Robert S" > > I am using R to multiply some large (30k x 30k double) matrices on a > 64 core machine (xeon phi). I added some timers to > src/main/array.c to see where the time is going. All of the time is > being spent in the matprod function, most of

Re: [Rd] R (development) changes in arith, logic, relop with (0-extent) arrays

urden on users. A language specification shouldn't really be changing all the time for no particular reason. Radford Neal __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] R (development) changes in arith, logic, relop with (0-extent) arrays

Whereas sample(2:n) behaving differently when n is 2 than when n is greater than 2 is itself a bug, that the programmer has to consciously avoid by being aware of the quirk. Radford Neal __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] R (development) changes in arith, logic, relop with (0-extent) arrays

hat after a<-numeric(0), is has to be allowable to write a<1. To not allow this would be highly destructive of code reliability. And for similar reason, after a<-c(), a<1 needs to be allowed, which means NULL<1 should be allowed (giving logical(0)), since c() i

[Rd] New version of the R parser in pqR

trying to understand the code. I have tested the new parser on the 5018 packages in the pqR repository, but of course it's possible that problems might show up in some other CRAN packages. I'm willing to help in resolving any such problems as well. Radford Neal

[Rd] OpenMP on Windows reset FPU mode, disabling long double

, the output of d is 1e-17 for all threads except thread 1, for which it is 0. That's the new thread for which the __asm__(fninit) wasn't done to reset to long double precision, and for which adding 1e-17 to 1.0 therefore has no effect. Radford Neal

Re: [Rd] Problem with psignal.c for Windows builds

One thing I forgot in my previous message about problems with psignal.c on Rtools for Windows... One also needs to change src/gnuwin32/fixed/h/psignal.h At a minimum, one needs the following changes: @@ -122,8 +129,8 @@ typedef struct /* Prototype stuff

[Rd] Problems with embedded R, ReplDLL

might want to take a look there, or at the next version of pqR, which will be released soon. Radford Neal __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] Getting SSE2 instructions to work in 32-bit builds on Windows

), and hence avoid most of the extra procedure call overhead, but I haven't attempted this. Radford Neal __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] Problem with psignal.c for Windows builds

reason to want to use the type defined in sys/types.h. Radford Neal __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] OpenMP problem with 64-bit Rtools

makes a difference. Radford Neal __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] OpenMP problem with 64-bit Rtools

On Fri, Aug 21, 2015 at 8:38 PM, Radford Neal radf...@cs.toronto.edu wrote: The reason for this appears to be that the omp.h file included is appropriate only for the 32-bit version of the openMP library. As far as I can see, there is no 64-bit version of this include file in the stuff

Re: [Rd] R-devel Digest, Vol 149, Issue 22

is to re-implement R's garbage collector in a way that will avoid this (as well as having various other advantages, including less memory overhead per object). Radford Neal __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] Two bugs showing up mostly on SPARC systems

is an error, depending on details of the compiler. It showed up with gcc 4.9.2 on a SPARC system. The fix slightly changes the error behaviuor, signaling an error on inappropriate reads before reading any data, rather than after reading one (but not all) items as before. Radford Neal

Re: [Rd] Two bugs showing up mostly on SPARC systems

On Tue, Jul 14, 2015 at 07:52:56PM -0400, Duncan Murdoch wrote: On 14/07/2015 6:08 PM, Radford Neal wrote: In testing pqR on Solaris SPARC systems, I have found two bugs that are also present in recent R Core versions. You can see the bugs and fixes at the following URLs: https

Re: [Rd] Add-on argument in sample()

attribute of length zero (presently disallowed) might also be a good idea, because it could be used to explicitly mark a single number as a scalar, not a vector of length one. Radford Neal __ R-devel@r-project.org mailing list https://stat.ethz.ch

Re: [Rd] Add-on argument in sample()

of a solution to several other problems as well, as I propose at http://www.cs.utoronto.ca/~radford/ftp/R-lang-ext.pdf Radford Neal __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Lack of protection bug in current R release candidate

the symptoms a few days before, but hadn't isolated the cause.) Radford Neal __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] Lack of protection bug in current R release candidate

-tests-2.R: x - c(a, NA, b) factor(x) factor(x, exclude=) The solution (kludgy, but the whole concept is kludgy) is to forward R_print.na_string and R_print.na_string_noquote with the other roots in RunGenCollect (after the comment /* forward all roots */). Radford Neal

Re: [Rd] Why is the diag function so slow (for extraction)?

. So you're advocating slowing down all ordinary uses of diag to accommodate a usage that nobody thought was important enough to actually document. Radford Neal __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] iterated lapply

previous message). Radford Neal __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] iterated lapply

. Radford Neal __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] iterated lapply

will take). Radford Neal __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Recycling memory with a small free list

on the RHS. Regards, Radford Neal __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Recycling memory with a small free list

Radford Neal: there's a danger of getting carried away and essentially rewriting malloc. To avoid this, one might try just calling free on the no-longer-needed object, letting malloc then figure out when it can be re-used. Nathan Kurz: Yes, I think that's what I was anticipating: add

[Rd] Unexpected behavior of debug() in step-wise mode

that an inner statement should be debugged. This would also allow one to avoid entering the debugger for if when it is used as an expression rather than a statement (for example, a - if (x0) 3 else 4). Radford Neal __ R-devel@r-project.org mailing list

[Rd] vapply definition question

so I guess you're looking at a modified version of the function... The implementation detail is in the comment -- FUN(X[i], ...) is evaluated in the frame of lapply. Martin Morgan You may find it interesting that in the latest version of pqR, lapply is made faster in the case where no ...

[Rd] Error promise already under evaluation ...

The above question still stands, but otherwise, I overlooked the most obvious solution: dim_1 - function(x) dim(x) Unit: nanoseconds expr minlq mean medianuq max neval cld dim(x) 0 172.941 1 12696 1000 a

[Rd] ScalarLogical and setAttrib

, there are ScalarRealMaybeShared, etc.) Radford Neal __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] New flag bit for serialized used by pqR

) ((v) 12) #define DECODE_TYPE(v) ((v) 255) Please let me know if you see any problem with this, or if for some reason you'd prefer that I use one of the other four available bits (in the top of the 32-bit word). Regards, Radford Neal __ R-devel@r

Re: [Rd] diag(x, n) not preserving integer and logical x

not strike me as too odd. It makes sense if you think of (FALSE, TRUE) as forming a field under operations of sum=XOR and product=AND. And of course it will typically be converted to a 0/1 matrix later if that's what was actually intended. Radford Neal __ R

[Rd] Need help on calling Head from C

. But this is not a safe coding method in general. Radford Neal __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Need help on calling Head from C

Thank you for the wonderful suggestion . do you suggest to protect ScalarInteger(1) before calling lang3 ? Yes, either the result of Scalar Integer(1) or the result of install should be protected (before calling lang3 - don't try to do it inside the argument list of lang3).

Re: [Rd] Unfixed bugs in latest R-patched

than my time, so I should be producing a nice patch for you, then I don't agree. I went to the effort of verifing that the bugs are still present in r66002. That's as much as I'm willing to do. It's up to you what you do. Radford Neal On 23/06/2014, 5:24 PM, Radford Neal wrote: A new

Re: [Rd] Unfixed bugs in latest R-patched

such cases can arise from inadvertant dropping of dimensions when subscriptiong a matrix (eg, with 1:n, with n happening to be 1). Sometimes, this problem gets fixed by the transpose. Radford Neal __ R-devel@r-project.org mailing list https://stat.ethz.ch

[Rd] Unfixed bugs in latest R-patched

did handle them, and both were documented to do so. Regards, Radford Neal __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] R-devel Digest, Vol 133, Issue 23

,by=s)? Maybe we should have glm_binomial, glm_poisson, etc. so we don't have to remember the family argument? This way lies madness... Radford Neal __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] Varying results of package checks due to random seed

environment variable is set, the random seed is initialized to its value, rather than from the time and process id. This was motivated by exactly this problem - I can now just set R_SEED to something before running all the package checks. Radford Neal

Re: [Rd] Dynamic list creation (SEXP in C) returns error unimplemented type (29) in 'duplicate' Dynamic list creation (SEXP in C) returns error

the length right to start with. Radford Neal __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Dynamic list creation (SEXP in C) returns error unimplemented type (29) in 'duplicate' Dynamic list creation (SEXP in C) returns error

On Wed, Nov 06, 2013 at 02:40:59PM -0300, George Vega Yon wrote: Hi! You are right, what I actually use is SET_LENGTH... Is that ok? El 06/11/2013 14:00, Radford Neal radf...@cs.toronto.edu escribi?: Is SET_LENGTH a documented feature of the API? Not that I can see. However, it is indeed

Re: [Rd] Compiler warning: function returns address of local variable

..., my previous dev-lang/R-2.10.1 ebuild package has been upgraded to R-3.0.1. While compiling it, I have got the following compiler warning: * QA Notice: Package triggers severe warnings which indicate that it *may exhibit random runtime failures. * main.c:1548:5:

[Rd] missing PROTECT() in src/main/arithmetic.c

issuing a warning for this is probably not a good idea, but if a warning is issued, it certainly shouldn't be this one. Radford Neal __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] Announcing pqR - a faster version of R

I have released a new, faster, version of R, which I call pqR (for pretty quick R), based on R-2.15.0. Among many other improvements, pqR supports automatic use of multiple cores to perform numerical computations in parallel with other numerical computations, and with the interpretive thread. It

Re: [Rd] call R function from C code

of eval (only if you will be doing further allocations while still using sx). Radford Neal __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] Bugs due to naive copying of list elements

Several bugs are present in R-2.15.3 and R-alpha due to naive copying of list elements. The bug below is due to naive copying in subset.c (with similar bugs for matrices and arrays): a-list(c(1,2),c(3,4),c(5,6)) b-a[2:3] a[[2]][2]-9 print(b[[1]][2]) Naive copying in mapply.c leads to the

[Rd] Performance issues with R2.15.1 built from source on Solaris

Is this a SPARC system? On at least some SPARC systems, the long double type in C is implemented very slowly in software, and it seems that it is used for the sums done when calculating standard deviations with sd. Radford Neal Date: Wed, 8 Aug 2012 18:55:37 -0500 From: Eberle, Anthony

[Rd] Improved version of Rprofmem

The Rprofmem facility is currently enabled only if the configuration option --enable-memory-profiling is used. However, the overhead of having it enabled is negligible when profiling is not actually being done, and can easily be made even smaller. So I think it ought to be enabled all the time.

[Rd] Latent flaw in SEXPREC definition

There seems to be a latent flaw in the definition of struct SEXPREC in Rinternals.h, which likely doesn't cause problems now, but could if the relative sizes of data types changes. The SEXPREC structure contains a union that includes a primsxp, symsxp, etc, but not a vecsxp. However, in

Re: [Rd] Latent flaw in SEXPREC definition

But the whole point of separating VECTOR_SEXPREC from the other SEXPRECs is that they are _shorter_. A vecsxp is only going to be larger than (say) an envsxp if 2 R_len_t's are more than 3 pointers, which is quite unlikely since R_len_t variables holds things that one might add to pointers.

[Rd] Best practices for writing R functions (really copying)

is not, so the modification to A after the call t(A) requires that a copy be made. Radford Neal __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] Patches to speed up R 2.13.0

be of substantial benefit to the R community. The R core team is welcome to incorporate them into versions of R that they release in the future. Radford Neal __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] Surprising behavior of letters[c(NA, NA)]

. Radford Neal __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] make error for R 2.13.0 (and 2.12.0)

-gnu --target=i486-linux-gnu Thread model: posix gcc version 4.2.4 (Ubuntu 4.2.4-1ubuntu4) This R-2.12.0 installation generally works (based on testing a few trivial things). The same error occurs when trying to install 2.12.0 on Intel Solaris. Radford Neal

[Rd] History of seq and seq.int

I wonder what is the history of seq and seq.int? From help(seq), one reads that 'seq.int' is an internal generic which can be much faster but has a few restrictions. And indeed, seq.int(1,99,by=2) is over 40 times faster than seq(1,99,by=2) in a quick test I just did. This is not surprising

Re: [Rd] Fourteen patches to speed up R

Luke - Thanks for your comments on the speed patches I submitted. I'm glad you like patch-transpose, patch-for, patch-evalList, and patch-vec-arith. I'll be interested to hear what you or other people think about patch-dollar and patch-vec-subset after you've had more time to consider them.

[Rd] Speeding up squaring of vectors

I see that some of the speed patches that I posted have been incorporated into the current development version (eg, my patch-for, patch-evalList, and patch-vec-arith). My patch for speeding up x^2 has been addressed in an inadvisable way, however. This was a simple addition of four lines of code

[Rd] Correction to vec-subset speed patch

I found a bug in one of the fourteen speed patches I posted, namely in patch-vec-subset. I've fixed this (I now see one does need to duplicate index vectors sometimes, though one can avoid it most of the time). I also split this patch in two, since it really has two different and independent

[Rd] Pointer to fourteen patches to speed up R

programs, on various machines. Radford Neal --- These patches to the R source for improving speed were written by Radford M. Neal, Sept. 2010. See the README file for how to install them. Below, I describe these patches

[Rd] Speeding up transpose

and divides do not take constant time, but are faster when, for instance, dividing by 1.) I've appended below the new version of the modified part of the do_transpose function in src/main/array.c. Radford Neal -- PROTECT

[Rd] Speeding up sum and prod

]; if (n0) updated = TRUE; } *value = s; return(updated); } An entirely analogous improvement can be made to the prod function. Radford Neal __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Speeding up matrix multiplies

is supposed to work. Radford Neal __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Speeding up matrix multiplies

On Aug 23, 2010, at 7:39 PM, Radford Neal wrote: In particular, all matrix x vector and vector x matrix products will in this version be done in the matprod routine, not the Fortran routine. And this is the right thing to do, since the time for the ISNAN check before calling the Fortan

[Rd] Speed improvement to evalList

of maybe 15% is possible, with some programs probably speeding up by a factor of two. For now, though, I'll just give the revised versions of evalList and evalListKeepMissing, below. Radford Neal - /* Used in eval and applyMethod