[Rd] --enable-BLAS-shlib conflict with --with-lapack in configure?
Dear developeRs, I'm trying to build R (2.7.1, on an x86_64 running Linux, but I believe I have observed this on x86 and with earlier versions of R as well) using external BLAS and LAPACK libraries _and_ generating libR.so, libRblas.so and libRlapack.so. Without --enable-BLAS-shlib, configure is able to find and use the external LAPACK library: ./configure \ --with-blas=-L/site/intel/cmkl/8.1/lib/em64t -lmkl -lvml -lguide -lpthread \ --with-lapack=-L/site/intel/cmkl/8.1/lib/em64t -lmkl_lapack64 -lmkl \ --enable-R-shlib [...] External libraries:readline, BLAS(generic), LAPACK(generic) Additional capabilities: PNG, JPEG, iconv, MBCS, NLS Options enabled: shared R library, R profiling, Java However, if I add --enable-BLAS-shlib, the external LAPACK will not be used: ./configure \ --with-blas=-L/site/intel/cmkl/8.1/lib/em64t -lmkl -lvml -lguide -lpthread \ --with-lapack=-L/site/intel/cmkl/8.1/lib/em64t -lmkl_lapack64 -lmkl \ --enable-R-shlib --enable-BLAS-shlib [...] checking for zgeev_ in -L/site/intel/cmkl/8.1/lib/em64t -lmkl_lapack64 -lmkl... no checking for zgeev_ in -llapack... no [...] External libraries:readline, BLAS(generic) Additional capabilities: PNG, JPEG, iconv, MBCS, NLS Options enabled: shared R library, shared BLAS, R profiling, Java From config.log, it seems configure is trying to link with libRblas when testing for LAPACK -- but that has not been built yet: configure:37825: checking for zgeev_ configure:37889: gcc -std=gnu99 -o conftest -g -O2 -fpic -I/usr/local/include -L/usr/local/lib64 conftest.c -L$(R_HOME)/lib$(R_ARCH) -lRblas -lg2c -lm -ldl -lm 5 /usr/bin/ld: cannot find -lRblas [...] configure:37924: checking for zgeev_ in -L/site/intel/cmkl/8.1/lib/em64t -lmkl_lapack64 -lmkl configure:37962: gcc -std=gnu99 -o conftest -g -O2 -fpic -I/usr/local/include -L/usr/local/lib64 conftest.c -L/site/intel/cmkl/8.1/lib/em64t -lmkl_lapack64 -lmkl -L$(R_HOME)/lib$(R_ARCH) -lRblas -lg2c -lm -ldl -lm 5 /usr/bin/ld: cannot find -lRblas [...] configure:37993: checking for zgeev_ in -llapack configure:38036: gcc -std=gnu99 -o conftest -g -O2 -fpic -I/usr/local/include -L/usr/local/lib64 conftest.c -llapack -L$(R_HOME)/lib$(R_ARCH) -lRblas -lg2c -lm -ldl -lm 5 /usr/bin/ld: cannot find -lRblas (Removing the --enable-R-shlib does not make a difference in this respect.) Is this intentional or a bug? Am I doing something wrong here? -- Regards, Bjørn-Helge Mevik __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] --enable-BLAS-shlib conflict with --with-lapack in configure?
This is intentional. Please note what the R-admin manual says about --with-lapack, including 'definitely *not* recommended' If you are concerned about performance you definitely do not want to use shared R, not a shared BLAS. On Thu, 3 Jul 2008, Bjørn-Helge Mevik wrote: Dear developeRs, I'm trying to build R (2.7.1, on an x86_64 running Linux, but I believe I have observed this on x86 and with earlier versions of R as well) using external BLAS and LAPACK libraries _and_ generating libR.so, libRblas.so and libRlapack.so. Without --enable-BLAS-shlib, configure is able to find and use the external LAPACK library: ./configure \ --with-blas=-L/site/intel/cmkl/8.1/lib/em64t -lmkl -lvml -lguide -lpthread \ --with-lapack=-L/site/intel/cmkl/8.1/lib/em64t -lmkl_lapack64 -lmkl \ --enable-R-shlib [...] External libraries:readline, BLAS(generic), LAPACK(generic) Additional capabilities: PNG, JPEG, iconv, MBCS, NLS Options enabled: shared R library, R profiling, Java However, if I add --enable-BLAS-shlib, the external LAPACK will not be used: ./configure \ --with-blas=-L/site/intel/cmkl/8.1/lib/em64t -lmkl -lvml -lguide -lpthread \ --with-lapack=-L/site/intel/cmkl/8.1/lib/em64t -lmkl_lapack64 -lmkl \ --enable-R-shlib --enable-BLAS-shlib [...] checking for zgeev_ in -L/site/intel/cmkl/8.1/lib/em64t -lmkl_lapack64 -lmkl... no checking for zgeev_ in -llapack... no [...] External libraries:readline, BLAS(generic) Additional capabilities: PNG, JPEG, iconv, MBCS, NLS Options enabled: shared R library, shared BLAS, R profiling, Java From config.log, it seems configure is trying to link with libRblas when testing for LAPACK -- but that has not been built yet: configure:37825: checking for zgeev_ configure:37889: gcc -std=gnu99 -o conftest -g -O2 -fpic -I/usr/local/include -L/usr/local/lib64 conftest.c -L$(R_HOME)/lib$(R_ARCH) -lRblas -lg2c -lm -ldl -lm 5 /usr/bin/ld: cannot find -lRblas [...] configure:37924: checking for zgeev_ in -L/site/intel/cmkl/8.1/lib/em64t -lmkl_lapack64 -lmkl configure:37962: gcc -std=gnu99 -o conftest -g -O2 -fpic -I/usr/local/include -L/usr/local/lib64 conftest.c -L/site/intel/cmkl/8.1/lib/em64t -lmkl_lapack64 -lmkl -L$(R_HOME)/lib$(R_ARCH) -lRblas -lg2c -lm -ldl -lm 5 /usr/bin/ld: cannot find -lRblas [...] configure:37993: checking for zgeev_ in -llapack configure:38036: gcc -std=gnu99 -o conftest -g -O2 -fpic -I/usr/local/include -L/usr/local/lib64 conftest.c -llapack -L$(R_HOME)/lib$(R_ARCH) -lRblas -lg2c -lm -ldl -lm 5 /usr/bin/ld: cannot find -lRblas (Removing the --enable-R-shlib does not make a difference in this respect.) Is this intentional or a bug? Am I doing something wrong here? -- Regards, Bjørn-Helge Mevik __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel -- Brian D. Ripley, [EMAIL PROTECTED] Professor of Applied Statistics, http://www.stats.ox.ac.uk/~ripley/ University of Oxford, Tel: +44 1865 272861 (self) 1 South Parks Road, +44 1865 272866 (PA) Oxford OX1 3TG, UKFax: +44 1865 272595__ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] Filename bug in png
In R2.7.1, the first line of the body of png() reads: checkIntFormat(filename) This should really be: if (!checkIntFormat(file)) stop(invalid 'file') Otherwise a command such as png(foo%s.png) causes R to crash (see bug #10571). - Regards, Richie. Mathematical Sciences Unit HSL -- View this message in context: http://www.nabble.com/Filename-bug-in-png-tp18256526p18256526.html Sent from the R devel mailing list archive at Nabble.com. __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] --enable-BLAS-shlib conflict with --with-lapack in configure?
Prof Brian Ripley [EMAIL PROTECTED] writes: This is intentional. Ok. Please note what the R-admin manual says about --with-lapack, including 'definitely *not* recommended' Oh. I have always understood that as referring to --with-lapack, not --with-lapack=-L/something If you are concerned about performance you definitely do not want to use shared R, not a shared BLAS. I'm sorry, I don't quite follow you here. Do you mean not create a shared R and not create a shared BLAS, or was there a not too many? -- Bjørn-Helge Mevik __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] --enable-BLAS-shlib conflict with --with-lapack in configure?
On Thu, 3 Jul 2008, Bjørn-Helge Mevik wrote: Prof Brian Ripley [EMAIL PROTECTED] writes: This is intentional. Ok. Please note what the R-admin manual says about --with-lapack, including 'definitely *not* recommended' Oh. I have always understood that as referring to --with-lapack, not --with-lapack=-L/something It applies to both. If you are concerned about performance you definitely do not want to use shared R, not a shared BLAS. I'm sorry, I don't quite follow you here. Do you mean not create a shared R and not create a shared BLAS, or was there a not too many? Yes, create neither. Using shared objects has a noticeable performance overhead. -- Bjørn-Helge Mevik __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel -- Brian D. Ripley, [EMAIL PROTECTED] Professor of Applied Statistics, http://www.stats.ox.ac.uk/~ripley/ University of Oxford, Tel: +44 1865 272861 (self) 1 South Parks Road, +44 1865 272866 (PA) Oxford OX1 3TG, UKFax: +44 1865 272595__ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] --enable-BLAS-shlib conflict with --with-lapack in configure?
Thank you for the advice! -- Bjørn-Helge Mevik __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] --enable-BLAS-shlib conflict with --with-lapack in configure?
Bjørn-Helge Mevik wrote: If you are concerned about performance you definitely do not want to use shared R, not a shared BLAS. I'm sorry, I don't quite follow you here. Do you mean not create a shared R and not create a shared BLAS, or was there a not too many? Nor, I think. (One of a set of confusing typos. Others include not instead of now or note. Note that I would ever make such silly mistakes.) As for the subject matter, you _can_ actually create such at thing as an optimized, shared BLAS/Lapack, and it is probably faster than an unoptimized, shared one (well, at least you could create them when I played with this some years ago). It's just that people think it is a bit silly, so they are not doing it for you, and as I recall it, you also need to tinker a bit more than usual with the build process of the libraries. -- O__ Peter Dalgaard Øster Farimagsgade 5, Entr.B c/ /'_ --- Dept. of Biostatistics PO Box 2099, 1014 Cph. K (*) \(*) -- University of Copenhagen Denmark Ph: (+45) 35327918 ~~ - ([EMAIL PROTECTED]) FAX: (+45) 35327907 __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] Bug repository down Friday
There needs to be a physical relocation of servers tomorrow (email and home directories), so things are likely to be dysfunctional for the day (hopefully not more). I'll be out of email reach myself, for the same reason. -- O__ Peter Dalgaard Øster Farimagsgade 5, Entr.B c/ /'_ --- Dept. of Biostatistics PO Box 2099, 1014 Cph. K (*) \(*) -- University of Copenhagen Denmark Ph: (+45) 35327918 ~~ - ([EMAIL PROTECTED]) FAX: (+45) 35327907 __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] GetRNGstate and PutRNGstate
Hi, I've got a simulation function, written in C and called from R, that uses the R random number functions. It's not a very complicated simulation - 280 lines total, with the main function (the one called with .C) repeatedly calling another function, with multiple calls to unif_rand() in both functions. At the beginning of the main function I call GetRNGstate(), and the last thing I do before returning to R is PutRNGstate(). On rereading Chapter 5 of the R Extensions manual, I'm not sure I understand the GetRNGstate/PutRNGstate calls. Should I also include them in the helper function? Do I need to wrap every call to unif_rand with these calls, or is it ok that I've just used them at the beginning and end of the C code? I'm getting different results from different simulations, so what I've done appears ok, but I'm not sure I would be able to detect if every call to the helper function was returning the same result. Thanks, Tyler -- __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] GetRNGstate and PutRNGstate
On Thu, 3 Jul 2008, tyler wrote: Hi, I've got a simulation function, written in C and called from R, that uses the R random number functions. It's not a very complicated simulation - 280 lines total, with the main function (the one called with .C) repeatedly calling another function, with multiple calls to unif_rand() in both functions. At the beginning of the main function I call GetRNGstate(), and the last thing I do before returning to R is PutRNGstate(). On rereading Chapter 5 of the R Extensions manual, I'm not sure I understand the GetRNGstate/PutRNGstate calls. Should I also include them in the helper function? Do I need to wrap every call to unif_rand with these calls, or is it ok that I've just used them at the beginning and end of the C code? I find the manual very much clearer than your email: However, before these are used, the user must call GetRNGstate(); and after all the required variates have been generated, call PutRNGstate(); So in so far as I understand your prose, you followed that advice by calling *one of* them 'at the beginning and end of the C code'. I'm getting different results from different simulations, so what I've done appears ok, but I'm not sure I would be able to detect if every call to the helper function was returning the same result. Thanks, Tyler -- __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel -- Brian D. Ripley, [EMAIL PROTECTED] Professor of Applied Statistics, http://www.stats.ox.ac.uk/~ripley/ University of Oxford, Tel: +44 1865 272861 (self) 1 South Parks Road, +44 1865 272866 (PA) Oxford OX1 3TG, UKFax: +44 1865 272595 __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] Rgui.exe(R.dll) crashes when set the locale to Korean (PR#11903)
Full_Name: Sang Hyun Koh Version: 2.7.0, 2.7.1 OS: MS Windows XP, Server 2003 Submission from: (NULL) (202.30.14.68) Since R 2.7.0 recent 2.7.1, Rgui.exe crashes on Korean Windows XP and 2003. After change the locale language to Japanese, Chinese, English, and etc on control panel, I had no problem in running 2.7.x on Korean Windows. And I re-install Windows XP, English version, and I could reproduce the bug by changing locale language to Korean on control panel. of course, no other language setting made the problem. So, It must be a problem between Korean language localization on windows and Rgui.exe. Please, help to remove the bug. __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] [.data.frame speedup
TH == Tim Hesterberg [EMAIL PROTECTED] on Tue, 1 Jul 2008 15:23:53 -0700 writes: TH There is a bug in the standard version of [.data.frame; TH it mixes up handling duplicates and NAs when subscripting rows. TH x - data.frame(x=1:3, y=2:4, row.names=c(a,b,NA)) TH y - x[c(2:3, NA),] TH y TH It creates a data frame with duplicate rows, but won't print. and that's a bug, indeed (introduced to R version 2.5.0, when the [.data.frame code was much optimized for speed, with quite some care), and I have commited a fix (and a regression test) to both R-devel and R-patched. Thanks a lot for the bug report, Tim! Now about your newly proposed code: I'm sorry to say that it looks so much different from the source code in https://svn.r-project.org/R/trunk/src/library/base/R/dataframe.R that I don't think we would accept it as a substitute, easily. Could you try to provide a minimal patch against the source code and also a selfcontained example that exhibits the speed gain you are aiming for ? Best regards, Martin Maechler, ETH Zurich [.] TH On Tue, Jul 1, 2008 at 11:20 AM, Tim Hesterberg [EMAIL PROTECTED] TH wrote: Below is a version of [.data.frame that is faster for subscripting rows of large data frames; it avoids calling duplicated(rows) if there is no need to check for duplicate row names, when: i is logical attr(x, dup.row.names) is not NULL (S+ compatibility) i is numeric and negative i is strictly increasing TH [[alternative HTML version deleted]] TH __ TH R-devel@r-project.org mailing list TH https://stat.ethz.ch/mailman/listinfo/r-devel __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] [.data.frame speedup
I made a couple of a changes from the previous version:
- don't use functions anyMissing or notSorted (which aren't in base R)
- don't check for dup.row.names attribute (need to modify other functions
before that is useful)
I have not tested this with a wide variety of inputs; I'm assuming that
you have some regression tests.
Here are the file differences. Let me know if you'd like a different
format.
$ diff -c dataframe.R dataframe2.R
*** dataframe.RThu Jul 3 15:48:12 2008
--- dataframe2.RThu Jul 3 16:36:46 2008
***
*** 530,535
--- 530,541
x - .Call(R_copyDFattr, xx, x, PACKAGE=base)
oldClass(x) - attr(x, row.names) - NULL
+ # Do not want to check for duplicates if don't need to
+ noDuplicateRowNames - (is.logical(i) ||
+ length(i) 2 ||
+ (is.numeric(i) min(i, 0, na.rm=TRUE) 0)
||
+ (!any(is.na(i)) all(i[-length(i)]i[-1])))
+
if(!missing(j)) { # df[i, j]
x - x[j]
cols - names(x) # needed for 'drop'
***
*** 579,592
## row names might have NAs.
if(is.null(rows)) rows - attr(xx, row.names)
rows - rows[i]
! if((ina - any(is.na(rows))) | (dup - any(duplicated(rows {
! ## both will coerce integer 'rows' to character:
! if (!dup is.character(rows)) dup - NA %in% rows
! if(ina)
! rows[is.na(rows)] - NA
! if(dup)
! rows - make.unique(as.character(rows))
! }
## new in 1.8.0 -- might have duplicate columns
if(any(duplicated(nm - names(x names(x) - make.unique(nm)
if(is.null(rows)) rows - attr(xx, row.names)[i]
--- 585,594
## row names might have NAs.
if(is.null(rows)) rows - attr(xx, row.names)
rows - rows[i]
! if(any(is.na(rows)))
! rows[is.na(rows)] - NA # coerces to integer
! if(!noDuplicateRowNames any(duplicated(rows)))
! rows - make.unique(as.character(rows)) # coerces to integer
## new in 1.8.0 -- might have duplicate columns
if(any(duplicated(nm - names(x names(x) - make.unique(nm)
if(is.null(rows)) rows - attr(xx, row.names)[i]
Here's some code for testing, and timings
# Use:
# R --no-init-file --no-site-file
x - data.frame(a=1:4, b=2:5)
# Run these commands with the default and new versions of [.data.frame
trace(duplicated)
trace(make.unique)
x[2:1]
x[1]
x[1:2]
x[1:3, ]# save one call to duplicated(rows)
x[c(T,F,F,T), ] # save one call to duplicated(rows)
x[-1,] # save one call to duplicated(rows)
x[-(1:2),] # save one call to duplicated(rows)
x[3:1, ]
x[c(1,3,2,4,3), ]
untrace(duplicated)
untrace(make.unique)
# Timings
# Run one of these lines, then everything afterward
n - 10^5
n - 10^6
n - 10^7
y - data.frame(a=1:n, b=1:n)
i - 1:n
system.time(temp - y[i, ])
# n old new
# 10^5.128.052
# 10^6.237.591
# 10^73.102.882
i - rep(TRUE, n)
system.time(temp - y[i, ])
# n old new
# 10^5.157.053
# 10^6.787.449
# 10^73.799 2.138
i - -1
system.time(temp - y[i, ])
# n old new
# 10^5.157.051
# 10^6.614.497
# 10^74.163 2.482
i - rep(1:(n/2), 2) # expect no speedup for this case
system.time(temp - y[i, ])
# n old new
# 10^5.559.782
# 10^66.066 6.078
# Times shown are the user times reported by system.time
# The time savings are mostly quite substantial in the
# cases I expect a savings.
# I've noticed a lot of variability in results from system.time,
# so I don't view these as very accurate, and I don't worry
# much about the cases where the time appears worse.
On Thu, Jul 3, 2008 at 1:08 PM, Martin Maechler [EMAIL PROTECTED]
wrote:
TH == Tim Hesterberg [EMAIL PROTECTED]
on Tue, 1 Jul 2008 15:23:53 -0700 writes:
TH There is a bug in the standard version of [.data.frame;
TH it mixes up handling duplicates and NAs when subscripting rows.
TH x - data.frame(x=1:3, y=2:4, row.names=c(a,b,NA))
TH y - x[c(2:3, NA),]
TH y
TH It creates a data frame with duplicate rows, but won't print.
and that's a bug, indeed
(introduced to R version 2.5.0, when the [.data.frame code was much
optimized for speed, with quite some care), and I have commited
a fix (and a regression test) to both R-devel and R-patched.
Thanks a lot for the bug report, Tim!
Now about your newly proposed code:
I'm sorry to say that it looks so much different from the source
code in
https://svn.r-project.org/R/trunk/src/library/base/R/dataframe.R
that I don't think we would accept it as a substitute, easily.
Could you try to provide a minimal patch against the source code
and also a selfcontained example that
[Rd] update on dnbinom with large size
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
~ turns out I don't need to look at the C code.
~ if one uses the mu/size parameterization of the
negative binomial, R computes size/(size+mu) to
switch parameterizations. If sizemu this
gets rounded to 1 ... should be easy enough
to test and return NA under these circumstances?
function (x, size, prob, mu, log = FALSE)
{
~if (!missing(mu)) {
~if (!missing(prob))
~stop('prob' and 'mu' both specified)
~prob - size/(size + mu)
~}
~.Internal(dnbinom(x, size, prob, log))
}
~ Ben Bolker
- --
~ Check it out:
curve(dnbinom(1,mu=0.5,size=x),log=x,from=1,to=1e18)
abline(h=dpois(1,lambda=0.5),col=2,lty=2)
text(1,dpois(1,lambda=0.5)+0.02,Poisson,col=2,pos=4)
~ I will take a look in the C code when I get
a chance to see if I can offer
a patch, but in the meantime wanted to alert people to
this feature ... (it looks like I will have to go
through dnbinom and dbinom_raw to see where the problem
is ...)
~ Ben Bolker
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
iD8DBQFIbYyRc5UpGjwzenMRAh57AJ49SUuWs1++D7sacUnFZud2MAROhACfThzD
OmM9WuuFZrk4kO1s9MafLlk=
=wzaH
-END PGP SIGNATURE-
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
