Re: [R] Timings of function execution in R [was Re: R in Industry]
Ravi == Ravi Varadhan [EMAIL PROTECTED]
on Thu, 8 Feb 2007 18:41:38 -0500 writes:
Ravi Hi,
Ravi greaterOf is indeed an interesting function. It is much faster
than the
Ravi equivalent R function, pmax, because pmax does a lot of checking for
Ravi missing data and for recycling. Tom Lumley suggested a simple
function to
Ravi replace pmax, without these checks, that is analogous to greaterOf,
which I
Ravi call fast.pmax.
Ravi fast.pmax - function(x,y) {i- xy; x[i]-y[i]; x}
Ravi Interestingly, greaterOf is even faster than fast.pmax, although you
have to
Ravi be dealing with very large vectors (O(10^6)) to see any real
difference.
Yes. Indeed, I have a file, first version dated from 1992
where I explore the slowness of pmin() and pmax() (in S-plus
3.2 then). I had since added quite a few experiments and versions to that
file in the past.
As consequence, in the robustbase CRAN package (which is only a bit
more than a year old though), there's a file, available as
https://svn.r-project.org/R-packages/robustbase/R/Auxiliaries.R
with the very simple content {note line 3 !}:
-
### Fast versions of pmin() and pmax() for 2 arguments only:
### FIXME: should rather add these to R
pmin2 - function(k,x) (x+k - abs(x-k))/2
pmax2 - function(k,x) (x+k + abs(x-k))/2
-
{the funny argument name 'k' comes from the use of these to
compute Huber's psi() fast :
psiHuber - function(x,k) pmin2(k, pmax2(- k, x))
curve(psiHuber(x, 1.35), -3,3, asp = 1)
}
One point *is* that I think proper function names would be pmin2() and
pmax2() since they work with exactly 2 arguments,
whereas IIRC the feature to work with '...' is exactly the
reason that pmax() and pmin() are so much slower.
I've haven't checked if Gabor's
pmax2.G - function(x,y) {z - x y; z * (x-y) + y}
is even faster than the abs() using one.
It may have the advantage of giving *identical* results (to the
last bit!) to pmax() which my version does not --- IIRC the
only reason I did not follow my own 'FIXME' above.
I had then planned to implement pmin2() and pmax2() in C code, trivially,
and and hence get identical (to the last bit!) behavior as
pmin()/pmax(); but I now tend to think that the proper approach is to
code pmin() and pmax() via .Internal() and hence C code ...
[Not before DSC and my vacations though!!]
Martin Maechler, ETH Zurich
__
R-help@stat.math.ethz.ch mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Re: [R] Timings of function execution in R [was Re: R in Industry]
The other reason why pmin/pmax are preferable to your functions is that
they are fully generic. It is not easy to write C code which takes into
account that , [, [- and is.na are all generic. That is not to say that
it is not worth having faster restricted alternatives, as indeed we do
with rep.int and seq.int.
Anything that uses arithmetic is making strong assumptions about the
inputs. It ought to be possible to write a fast C version that worked for
atomic vectors (logical, integer, real and character), but is there
any evidence of profiled real problems where speed is an issue?
On Fri, 9 Feb 2007, Martin Maechler wrote:
Ravi == Ravi Varadhan [EMAIL PROTECTED]
on Thu, 8 Feb 2007 18:41:38 -0500 writes:
Ravi Hi,
Ravi greaterOf is indeed an interesting function. It is much faster
than the
Ravi equivalent R function, pmax, because pmax does a lot of checking
for
Ravi missing data and for recycling. Tom Lumley suggested a simple
function to
Ravi replace pmax, without these checks, that is analogous to greaterOf,
which I
Ravi call fast.pmax.
Ravi fast.pmax - function(x,y) {i- xy; x[i]-y[i]; x}
Ravi Interestingly, greaterOf is even faster than fast.pmax, although you
have to
Ravi be dealing with very large vectors (O(10^6)) to see any real
difference.
Yes. Indeed, I have a file, first version dated from 1992
where I explore the slowness of pmin() and pmax() (in S-plus
3.2 then). I had since added quite a few experiments and versions to that
file in the past.
As consequence, in the robustbase CRAN package (which is only a bit
more than a year old though), there's a file, available as
https://svn.r-project.org/R-packages/robustbase/R/Auxiliaries.R
with the very simple content {note line 3 !}:
-
### Fast versions of pmin() and pmax() for 2 arguments only:
### FIXME: should rather add these to R
pmin2 - function(k,x) (x+k - abs(x-k))/2
pmax2 - function(k,x) (x+k + abs(x-k))/2
-
{the funny argument name 'k' comes from the use of these to
compute Huber's psi() fast :
psiHuber - function(x,k) pmin2(k, pmax2(- k, x))
curve(psiHuber(x, 1.35), -3,3, asp = 1)
}
One point *is* that I think proper function names would be pmin2() and
pmax2() since they work with exactly 2 arguments,
whereas IIRC the feature to work with '...' is exactly the
reason that pmax() and pmin() are so much slower.
I've haven't checked if Gabor's
pmax2.G - function(x,y) {z - x y; z * (x-y) + y}
is even faster than the abs() using one.
It may have the advantage of giving *identical* results (to the
last bit!) to pmax() which my version does not --- IIRC the
only reason I did not follow my own 'FIXME' above.
I had then planned to implement pmin2() and pmax2() in C code, trivially,
and and hence get identical (to the last bit!) behavior as
pmin()/pmax(); but I now tend to think that the proper approach is to
code pmin() and pmax() via .Internal() and hence C code ...
[Not before DSC and my vacations though!!]
Martin Maechler, ETH Zurich
__
R-help@stat.math.ethz.ch mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
--
Brian D. Ripley, [EMAIL PROTECTED]
Professor of Applied Statistics, http://www.stats.ox.ac.uk/~ripley/
University of Oxford, Tel: +44 1865 272861 (self)
1 South Parks Road, +44 1865 272866 (PA)
Oxford OX1 3TG, UKFax: +44 1865 272595
__
R-help@stat.math.ethz.ch mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Re: [R] Timings of function execution in R [was Re: R in Industry]
On 2/9/07, Prof Brian Ripley [EMAIL PROTECTED] wrote:
The other reason why pmin/pmax are preferable to your functions is that
they are fully generic. It is not easy to write C code which takes into
account that , [, [- and is.na are all generic. That is not to say that
it is not worth having faster restricted alternatives, as indeed we do
with rep.int and seq.int.
Anything that uses arithmetic is making strong assumptions about the
inputs. It ought to be possible to write a fast C version that worked for
atomic vectors (logical, integer, real and character), but is there
any evidence of profiled real problems where speed is an issue?
Yes. I don't have the profiled timings available now and one would
need to go back to earlier versions of R to reproduce them but I did
encounter a situation where the bottleneck in a practical computation
was pmin/pmax. The binomial and poisson families for generalized
linear models used pmin and pmax to avoid boundary conditions when
evaluating the inverse link and other functions. When I profiled the
execution of some generalized linear model and, more importantly for
me, generalized linear mixed model fits, these calls to pmin and pmax
were the bottleneck. That is why I moved some of the calculations for
the binomial and poisson families in the stats package to compiled
code.
In that case I didn't rewrite the general form of pmin and pmax, I
replaced specific calls in the compiled code.
On Fri, 9 Feb 2007, Martin Maechler wrote:
Ravi == Ravi Varadhan [EMAIL PROTECTED]
on Thu, 8 Feb 2007 18:41:38 -0500 writes:
Ravi Hi,
Ravi greaterOf is indeed an interesting function. It is much faster
than the
Ravi equivalent R function, pmax, because pmax does a lot of checking
for
Ravi missing data and for recycling. Tom Lumley suggested a simple
function to
Ravi replace pmax, without these checks, that is analogous to
greaterOf, which I
Ravi call fast.pmax.
Ravi fast.pmax - function(x,y) {i- xy; x[i]-y[i]; x}
Ravi Interestingly, greaterOf is even faster than fast.pmax, although
you have to
Ravi be dealing with very large vectors (O(10^6)) to see any real
difference.
Yes. Indeed, I have a file, first version dated from 1992
where I explore the slowness of pmin() and pmax() (in S-plus
3.2 then). I had since added quite a few experiments and versions to that
file in the past.
As consequence, in the robustbase CRAN package (which is only a bit
more than a year old though), there's a file, available as
https://svn.r-project.org/R-packages/robustbase/R/Auxiliaries.R
with the very simple content {note line 3 !}:
-
### Fast versions of pmin() and pmax() for 2 arguments only:
### FIXME: should rather add these to R
pmin2 - function(k,x) (x+k - abs(x-k))/2
pmax2 - function(k,x) (x+k + abs(x-k))/2
-
{the funny argument name 'k' comes from the use of these to
compute Huber's psi() fast :
psiHuber - function(x,k) pmin2(k, pmax2(- k, x))
curve(psiHuber(x, 1.35), -3,3, asp = 1)
}
One point *is* that I think proper function names would be pmin2() and
pmax2() since they work with exactly 2 arguments,
whereas IIRC the feature to work with '...' is exactly the
reason that pmax() and pmin() are so much slower.
I've haven't checked if Gabor's
pmax2.G - function(x,y) {z - x y; z * (x-y) + y}
is even faster than the abs() using one.
It may have the advantage of giving *identical* results (to the
last bit!) to pmax() which my version does not --- IIRC the
only reason I did not follow my own 'FIXME' above.
I had then planned to implement pmin2() and pmax2() in C code, trivially,
and and hence get identical (to the last bit!) behavior as
pmin()/pmax(); but I now tend to think that the proper approach is to
code pmin() and pmax() via .Internal() and hence C code ...
[Not before DSC and my vacations though!!]
Martin Maechler, ETH Zurich
__
R-help@stat.math.ethz.ch mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
--
Brian D. Ripley, [EMAIL PROTECTED]
Professor of Applied Statistics, http://www.stats.ox.ac.uk/~ripley/
University of Oxford, Tel: +44 1865 272861 (self)
1 South Parks Road, +44 1865 272866 (PA)
Oxford OX1 3TG, UKFax: +44 1865 272595
__
R-help@stat.math.ethz.ch mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal,
Re: [R] Timings of function execution in R [was Re: R in Industry]
On 2/9/07, Prof Brian Ripley [EMAIL PROTECTED] wrote:
The other reason why pmin/pmax are preferable to your functions is that
they are fully generic. It is not easy to write C code which takes into
account that , [, [- and is.na are all generic. That is not to say that
it is not worth having faster restricted alternatives, as indeed we do
with rep.int and seq.int.
Anything that uses arithmetic is making strong assumptions about the
inputs. It ought to be possible to write a fast C version that worked for
atomic vectors (logical, integer, real and character), but is there
any evidence of profiled real problems where speed is an issue?
I had an example just last month of an MCMC calculation where profiling showed
that pmax(x,0) was taking about 30% of the total time. I used
function(x) {z - x0; x[z] - 0; x}
which was significantly faster. I didn't try the arithmetic solution. Also, I
didn't check if a solution like this would still be faster when both arguments
are vectors (but there was a recent mailing list thread where someone else did).
-thomas
Thomas Lumley Assoc. Professor, Biostatistics
[EMAIL PROTECTED] University of Washington, Seattle
__
R-help@stat.math.ethz.ch mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Re: [R] Timings of function execution in R [was Re: R in Industry]
Hi Profs. Ripley and Bates,
I also recollect from a Tom Lumley email that when he profiled an MCMC
computation, he found that pmin/pmax was the bottleneck. That is why he
suggested the function that I called fast.pmax. I think that it would be
nice to have restricted alternative functions dealing exclusively with
numeric mode.
Best,
Ravi.
---
Ravi Varadhan, Ph.D.
Assistant Professor, The Center on Aging and Health
Division of Geriatric Medicine and Gerontology
Johns Hopkins University
Ph: (410) 502-2619
Fax: (410) 614-9625
Email: [EMAIL PROTECTED]
Webpage: http://www.jhsph.edu/agingandhealth/People/Faculty/Varadhan.html
-Original Message-
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On Behalf Of Douglas Bates
Sent: Friday, February 09, 2007 10:05 AM
To: Prof Brian Ripley
Cc: R-Help
Subject: Re: [R] Timings of function execution in R [was Re: R in Industry]
On 2/9/07, Prof Brian Ripley [EMAIL PROTECTED] wrote:
The other reason why pmin/pmax are preferable to your functions is that
they are fully generic. It is not easy to write C code which takes into
account that , [, [- and is.na are all generic. That is not to say that
it is not worth having faster restricted alternatives, as indeed we do
with rep.int and seq.int.
Anything that uses arithmetic is making strong assumptions about the
inputs. It ought to be possible to write a fast C version that worked for
atomic vectors (logical, integer, real and character), but is there
any evidence of profiled real problems where speed is an issue?
Yes. I don't have the profiled timings available now and one would
need to go back to earlier versions of R to reproduce them but I did
encounter a situation where the bottleneck in a practical computation
was pmin/pmax. The binomial and poisson families for generalized
linear models used pmin and pmax to avoid boundary conditions when
evaluating the inverse link and other functions. When I profiled the
execution of some generalized linear model and, more importantly for
me, generalized linear mixed model fits, these calls to pmin and pmax
were the bottleneck. That is why I moved some of the calculations for
the binomial and poisson families in the stats package to compiled
code.
In that case I didn't rewrite the general form of pmin and pmax, I
replaced specific calls in the compiled code.
On Fri, 9 Feb 2007, Martin Maechler wrote:
Ravi == Ravi Varadhan [EMAIL PROTECTED]
on Thu, 8 Feb 2007 18:41:38 -0500 writes:
Ravi Hi,
Ravi greaterOf is indeed an interesting function. It is much
faster than the
Ravi equivalent R function, pmax, because pmax does a lot of
checking for
Ravi missing data and for recycling. Tom Lumley suggested a simple
function to
Ravi replace pmax, without these checks, that is analogous to
greaterOf, which I
Ravi call fast.pmax.
Ravi fast.pmax - function(x,y) {i- xy; x[i]-y[i]; x}
Ravi Interestingly, greaterOf is even faster than fast.pmax,
although you have to
Ravi be dealing with very large vectors (O(10^6)) to see any real
difference.
Yes. Indeed, I have a file, first version dated from 1992
where I explore the slowness of pmin() and pmax() (in S-plus
3.2 then). I had since added quite a few experiments and versions to
that
file in the past.
As consequence, in the robustbase CRAN package (which is only a bit
more than a year old though), there's a file, available as
https://svn.r-project.org/R-packages/robustbase/R/Auxiliaries.R
with the very simple content {note line 3 !}:
-
### Fast versions of pmin() and pmax() for 2 arguments only:
### FIXME: should rather add these to R
pmin2 - function(k,x) (x+k - abs(x-k))/2
pmax2 - function(k,x) (x+k + abs(x-k))/2
-
{the funny argument name 'k' comes from the use of these to
compute Huber's psi() fast :
psiHuber - function(x,k) pmin2(k, pmax2(- k, x))
curve(psiHuber(x, 1.35), -3,3, asp = 1)
}
One point *is* that I think proper function names would be pmin2() and
pmax2() since they work with exactly 2 arguments,
whereas IIRC the feature to work with '...' is exactly the
reason that pmax() and pmin() are so much slower.
I've haven't checked if Gabor's
pmax2.G - function(x,y) {z - x y; z * (x-y) + y}
is even faster than the abs() using one.
It may have the advantage of giving *identical* results (to the
last bit!) to pmax() which my version does not --- IIRC the
only reason I did not follow my own 'FIXME' above.
I had then planned to implement pmin2() and pmax2() in C code,
trivially,
and and hence get identical (to the last bit!) behavior as
Re: [R] Timings of function execution in R [was Re: R in Industry]
Thomas Lumley wrote:
On 2/9/07, Prof Brian Ripley [EMAIL PROTECTED] wrote:
The other reason why pmin/pmax are preferable to your functions is that
they are fully generic. It is not easy to write C code which takes into
account that , [, [- and is.na are all generic. That is not to say that
it is not worth having faster restricted alternatives, as indeed we do
with rep.int and seq.int.
Anything that uses arithmetic is making strong assumptions about the
inputs. It ought to be possible to write a fast C version that worked for
atomic vectors (logical, integer, real and character), but is there
any evidence of profiled real problems where speed is an issue?
I had an example just last month of an MCMC calculation where profiling
showed that pmax(x,0) was taking about 30% of the total time. I used
function(x) {z - x0; x[z] - 0; x}
which was significantly faster. I didn't try the arithmetic solution. Also, I
didn't check if a solution like this would still be faster when both
arguments are vectors (but there was a recent mailing list thread where
someone else did).
-thomas
I looked in all the code for the Hmisc and Design packages and didn't
find a single example where pmin or pmax did not have 2 arguments. So I
think it is important to have pmin2 and pmax2.
Frank
Thomas Lumley Assoc. Professor, Biostatistics
[EMAIL PROTECTED] University of Washington, Seattle
--
Frank E Harrell Jr Professor and Chair School of Medicine
Department of Biostatistics Vanderbilt University
__
R-help@stat.math.ethz.ch mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Re: [R] Timings of function execution in R [was Re: R in Industry]
TL == Thomas Lumley [EMAIL PROTECTED]
on Fri, 9 Feb 2007 08:13:54 -0800 (PST) writes:
TL On 2/9/07, Prof Brian Ripley [EMAIL PROTECTED] wrote:
The other reason why pmin/pmax are preferable to your functions is that
they are fully generic. It is not easy to write C code which takes into
account that , [, [- and is.na are all generic. That is not to say
that
it is not worth having faster restricted alternatives, as indeed we do
with rep.int and seq.int.
Anything that uses arithmetic is making strong assumptions about the
inputs. It ought to be possible to write a fast C version that worked
for
atomic vectors (logical, integer, real and character), but is there
any evidence of profiled real problems where speed is an issue?
TL I had an example just last month of an MCMC calculation where profiling
showed that pmax(x,0) was taking about 30% of the total time. I used
TL function(x) {z - x0; x[z] - 0; x}
TL which was significantly faster. I didn't try the
TL arithmetic solution.
I did - eons ago as mentioned in my message earlier in this
thread. I can assure you that those (also mentioned)
pmin2 - function(k,x) (x+k - abs(x-k))/2
pmax2 - function(k,x) (x+k + abs(x-k))/2
are faster still, particularly if you hardcode the special case of k=0!
{that's how I came about these: pmax(x,0) is also denoted x_+, and
x_+ := (x + |x|)/2
x_- := (x - |x|)/2
}
TL Also, I didn't check if a solution like this would still
TL be faster when both arguments are vectors (but there was
TL a recent mailing list thread where someone else did).
indeed, and they are faster.
Martin
__
R-help@stat.math.ethz.ch mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Re: [R] Timings of function execution in R [was Re: R in Industry]
On Fri, 9 Feb 2007, Frank E Harrell Jr wrote: [...] I looked in all the code for the Hmisc and Design packages and didn't find a single example where pmin or pmax did not have 2 arguments. So I think it is important to have pmin2 and pmax2. Why? Do you have any reason to suppose that these will be significantly faster than the general case of 1 or more arguments? (The current code fails obscurely for 0 args.) What I am playing with are fast internal pmin.int and pmax.int for the case of all atomic vectors and no classes. -- Brian D. Ripley, [EMAIL PROTECTED] Professor of Applied Statistics, http://www.stats.ox.ac.uk/~ripley/ University of Oxford, Tel: +44 1865 272861 (self) 1 South Parks Road, +44 1865 272866 (PA) Oxford OX1 3TG, UKFax: +44 1865 272595 __ R-help@stat.math.ethz.ch mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Timings of function execution in R [was Re: R in Industry]
x - rnorm(1)
system.time(for(i in 1:1000) pmax(x, 0))
user system elapsed
4.430.054.54
pmax2 - function(k,x) (x+k + abs(x-k))/2
system.time(for(i in 1:1000) pmax2(x, 0))
user system elapsed
0.640.030.67
pm - function(x) {z - x0; x[z] - 0; x}
system.time(for(i in 1:1000) pm(x))
user system elapsed
0.590.000.59
system.time(for(i in 1:1000) pmax.int(x, 0))
user system elapsed
0.360.000.36
So at least on one system Thomas' solution is a little faster, but a
C-level n-args solution handling NAs is quite a lot faster.
On Fri, 9 Feb 2007, Martin Maechler wrote:
TL == Thomas Lumley [EMAIL PROTECTED]
on Fri, 9 Feb 2007 08:13:54 -0800 (PST) writes:
TL On 2/9/07, Prof Brian Ripley [EMAIL PROTECTED] wrote:
The other reason why pmin/pmax are preferable to your functions is that
they are fully generic. It is not easy to write C code which takes
into
account that , [, [- and is.na are all generic. That is not to say
that
it is not worth having faster restricted alternatives, as indeed we do
with rep.int and seq.int.
Anything that uses arithmetic is making strong assumptions about the
inputs. It ought to be possible to write a fast C version that worked
for
atomic vectors (logical, integer, real and character), but is there
any evidence of profiled real problems where speed is an issue?
TL I had an example just last month of an MCMC calculation where
profiling showed that pmax(x,0) was taking about 30% of the total time. I
used
TL function(x) {z - x0; x[z] - 0; x}
TL which was significantly faster. I didn't try the
TL arithmetic solution.
I did - eons ago as mentioned in my message earlier in this
thread. I can assure you that those (also mentioned)
pmin2 - function(k,x) (x+k - abs(x-k))/2
pmax2 - function(k,x) (x+k + abs(x-k))/2
are faster still, particularly if you hardcode the special case of k=0!
{that's how I came about these: pmax(x,0) is also denoted x_+, and
x_+ := (x + |x|)/2
x_- := (x - |x|)/2
}
TL Also, I didn't check if a solution like this would still
TL be faster when both arguments are vectors (but there was
TL a recent mailing list thread where someone else did).
indeed, and they are faster.
Martin
--
Brian D. Ripley, [EMAIL PROTECTED]
Professor of Applied Statistics, http://www.stats.ox.ac.uk/~ripley/
University of Oxford, Tel: +44 1865 272861 (self)
1 South Parks Road, +44 1865 272866 (PA)
Oxford OX1 3TG, UKFax: +44 1865 272595
__
R-help@stat.math.ethz.ch mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Re: [R] Timings of function execution in R [was Re: R in Industry]
On 2/9/2007 1:33 PM, Prof Brian Ripley wrote:
x - rnorm(1)
system.time(for(i in 1:1000) pmax(x, 0))
user system elapsed
4.430.054.54
pmax2 - function(k,x) (x+k + abs(x-k))/2
system.time(for(i in 1:1000) pmax2(x, 0))
user system elapsed
0.640.030.67
pm - function(x) {z - x0; x[z] - 0; x}
system.time(for(i in 1:1000) pm(x))
user system elapsed
0.590.000.59
system.time(for(i in 1:1000) pmax.int(x, 0))
user system elapsed
0.360.000.36
So at least on one system Thomas' solution is a little faster, but a
C-level n-args solution handling NAs is quite a lot faster.
For this special case we can do a lot better using
pospart - function(x) (x + abs(x))/2
The less specialized function
pmax2 - function(x,y) {
diff - x - y
y + (diff + abs(diff))/2
}
is faster on my system than pm, but not as fast as pospart:
system.time(for(i in 1:1000) pm(x))
[1] 0.77 0.01 0.78 NA NA
system.time(for(i in 1:1000) pospart(x))
[1] 0.27 0.02 0.28 NA NA
system.time(for(i in 1:1000) pmax2(x,0))
[1] 0.47 0.00 0.47 NA NA
Duncan Murdoch
On Fri, 9 Feb 2007, Martin Maechler wrote:
TL == Thomas Lumley [EMAIL PROTECTED]
on Fri, 9 Feb 2007 08:13:54 -0800 (PST) writes:
TL On 2/9/07, Prof Brian Ripley [EMAIL PROTECTED] wrote:
The other reason why pmin/pmax are preferable to your functions is
that
they are fully generic. It is not easy to write C code which takes
into
account that , [, [- and is.na are all generic. That is not to say
that
it is not worth having faster restricted alternatives, as indeed we do
with rep.int and seq.int.
Anything that uses arithmetic is making strong assumptions about the
inputs. It ought to be possible to write a fast C version that
worked for
atomic vectors (logical, integer, real and character), but is there
any evidence of profiled real problems where speed is an issue?
TL I had an example just last month of an MCMC calculation where
profiling showed that pmax(x,0) was taking about 30% of the total time. I
used
TL function(x) {z - x0; x[z] - 0; x}
TL which was significantly faster. I didn't try the
TL arithmetic solution.
I did - eons ago as mentioned in my message earlier in this
thread. I can assure you that those (also mentioned)
pmin2 - function(k,x) (x+k - abs(x-k))/2
pmax2 - function(k,x) (x+k + abs(x-k))/2
are faster still, particularly if you hardcode the special case of k=0!
{that's how I came about these: pmax(x,0) is also denoted x_+, and
x_+ := (x + |x|)/2
x_- := (x - |x|)/2
}
TL Also, I didn't check if a solution like this would still
TL be faster when both arguments are vectors (but there was
TL a recent mailing list thread where someone else did).
indeed, and they are faster.
Martin
__
R-help@stat.math.ethz.ch mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Re: [R] Timings of function execution in R [was Re: R in Industry]
Duncan == Duncan Murdoch [EMAIL PROTECTED]
on Fri, 09 Feb 2007 13:52:25 -0500 writes:
Duncan On 2/9/2007 1:33 PM, Prof Brian Ripley wrote:
x - rnorm(1)
system.time(for(i in 1:1000) pmax(x, 0))
user system elapsed
4.430.054.54
pmax2 - function(k,x) (x+k + abs(x-k))/2
system.time(for(i in 1:1000) pmax2(x, 0))
user system elapsed
0.640.030.67
pm - function(x) {z - x0; x[z] - 0; x}
system.time(for(i in 1:1000) pm(x))
user system elapsed
0.590.000.59
system.time(for(i in 1:1000) pmax.int(x, 0))
user system elapsed
0.360.000.36
So at least on one system Thomas' solution is a little faster, but a
C-level n-args solution handling NAs is quite a lot faster.
Duncan For this special case we can do a lot better using
Duncan pospart - function(x) (x + abs(x))/2
Indeed, that's what I meant when I talked about doing the
special case 'k = 0' explicitly -- and also what my timings
where based on.
Thank you Duncan -- and Brian for looking into providing an even
faster and more general C-internal version!
Martin
Duncan The less specialized function
Duncan pmax2 - function(x,y) {
Duncan diff - x - y
Duncan y + (diff + abs(diff))/2
Duncan }
Duncan is faster on my system than pm, but not as fast as pospart:
system.time(for(i in 1:1000) pm(x))
Duncan [1] 0.77 0.01 0.78 NA NA
system.time(for(i in 1:1000) pospart(x))
Duncan [1] 0.27 0.02 0.28 NA NA
system.time(for(i in 1:1000) pmax2(x,0))
Duncan [1] 0.47 0.00 0.47 NA NA
Duncan Duncan Murdoch
On Fri, 9 Feb 2007, Martin Maechler wrote:
TL == Thomas Lumley [EMAIL PROTECTED]
on Fri, 9 Feb 2007 08:13:54 -0800 (PST) writes:
TL On 2/9/07, Prof Brian Ripley [EMAIL PROTECTED] wrote:
The other reason why pmin/pmax are preferable to your functions is
that
they are fully generic. It is not easy to write C code which takes
into
account that , [, [- and is.na are all generic. That is not to
say that
it is not worth having faster restricted alternatives, as indeed we
do
with rep.int and seq.int.
Anything that uses arithmetic is making strong assumptions about the
inputs. It ought to be possible to write a fast C version that
worked for
atomic vectors (logical, integer, real and character), but is there
any evidence of profiled real problems where speed is an issue?
TL I had an example just last month of an MCMC calculation where profiling
showed that pmax(x,0) was taking about 30% of the total time. I used
TL function(x) {z - x0; x[z] - 0; x}
TL which was significantly faster. I didn't try the
TL arithmetic solution.
I did - eons ago as mentioned in my message earlier in this
thread. I can assure you that those (also mentioned)
pmin2 - function(k,x) (x+k - abs(x-k))/2
pmax2 - function(k,x) (x+k + abs(x-k))/2
are faster still, particularly if you hardcode the special case of k=0!
{that's how I came about these: pmax(x,0) is also denoted x_+, and
x_+ := (x + |x|)/2
x_- := (x - |x|)/2
}
TL Also, I didn't check if a solution like this would still
TL be faster when both arguments are vectors (but there was
TL a recent mailing list thread where someone else did).
indeed, and they are faster.
Martin
__
R-help@stat.math.ethz.ch mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
[R] Timings of function execution in R [was Re: R in Industry]
On 2/8/07, Albrecht, Dr. Stefan (AZ Private Equity Partner)
[EMAIL PROTECTED] wrote:
Dear all,
Thanks a lot for your comments.
I very well agree with you that writing efficient code is about optimisation.
The most important rules I know would be:
- vectorization
- pre-definition of vectors, etc.
- use matrix instead of data.frame
- do not use named objects
- use pure matrix instead of involved S4 (perhaps also S3) objects (can have
enormous effects)
- use function instead of expression
- use compiled code
- I guess indexing with numbers (better variables) is also much faster than
with text (names) (see also above)
- I even made, for example, my own min, max, since they are slow, e.g.,
greaterOf - function(x, y){
# Returns for each element of x and y (numeric)
# x or y may be a multiple of the other
z - x y
z*x + (!z)*y
That's an interesting function. I initially was tempted to respond
that you have managed to reinvent a specialized form of the ifelse
function but then I decided to do the timings just to check (always a
good idea). The enclosed timings show that your function is indeed
faster than a call to ifelse. A couple of comments:
- I needed to make the number of components in the vectors x and y
quite large before I could get reliable timings on the system I am
using.
- The recommended way of doing timings is with system.time function,
which makes an effort to minimize the effects of garbage collection on
the timings.
- Even when using system.time there is often a big difference in
timing between the first execution of a function call that generates a
large object and subsequent executions of the same function call.
[additional parts of the original message not relevant to this
discussion have been removed]
x - rnorm(100)
y - rnorm(100)
system.time(r1 - greaterOf(x, y))
user system elapsed
0.255 0.023 0.278
system.time(r1 - greaterOf(x, y))
user system elapsed
0.054 0.029 0.084
system.time(r1 - greaterOf(x, y))
user system elapsed
0.057 0.028 0.086
system.time(r1 - greaterOf(x, y))
user system elapsed
0.083 0.040 0.124
system.time(r1 - greaterOf(x, y))
user system elapsed
0.099 0.026 0.124
system.time(r2 - ifelse(x y, x, y))
user system elapsed
0.805 0.109 0.913
system.time(r2 - ifelse(x y, x, y))
user system elapsed
0.723 0.113 0.835
system.time(r2 - ifelse(x y, x, y))
user system elapsed
0.641 0.116 0.757
system.time(r2 - ifelse(x y, x, y))
user system elapsed
0.647 0.111 0.757
all.equal(r1,r2)
[1] TRUE
__
R-help@stat.math.ethz.ch mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Re: [R] Timings of function execution in R [was Re: R in Industry]
This may not be exactly the same to the last decimal but is nearly
twice as fast again:
set.seed(1)
n - 100
x - rnorm(n)
y - rnorm(n)
system.time({z - x y; z*x+(!z)*y})
user system elapsed
0.640.080.72
system.time({z - x y; z * (x-y) + y})
user system elapsed
0.350.040.39
On 2/8/07, Douglas Bates [EMAIL PROTECTED] wrote:
On 2/8/07, Albrecht, Dr. Stefan (AZ Private Equity Partner)
[EMAIL PROTECTED] wrote:
Dear all,
Thanks a lot for your comments.
I very well agree with you that writing efficient code is about
optimisation. The most important rules I know would be:
- vectorization
- pre-definition of vectors, etc.
- use matrix instead of data.frame
- do not use named objects
- use pure matrix instead of involved S4 (perhaps also S3) objects (can
have enormous effects)
- use function instead of expression
- use compiled code
- I guess indexing with numbers (better variables) is also much faster than
with text (names) (see also above)
- I even made, for example, my own min, max, since they are slow, e.g.,
greaterOf - function(x, y){
# Returns for each element of x and y (numeric)
# x or y may be a multiple of the other
z - x y
z*x + (!z)*y
That's an interesting function. I initially was tempted to respond
that you have managed to reinvent a specialized form of the ifelse
function but then I decided to do the timings just to check (always a
good idea). The enclosed timings show that your function is indeed
faster than a call to ifelse. A couple of comments:
- I needed to make the number of components in the vectors x and y
quite large before I could get reliable timings on the system I am
using.
- The recommended way of doing timings is with system.time function,
which makes an effort to minimize the effects of garbage collection on
the timings.
- Even when using system.time there is often a big difference in
timing between the first execution of a function call that generates a
large object and subsequent executions of the same function call.
[additional parts of the original message not relevant to this
discussion have been removed]
__
R-help@stat.math.ethz.ch mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
__
R-help@stat.math.ethz.ch mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Re: [R] Timings of function execution in R [was Re: R in Industry]
Hi,
greaterOf is indeed an interesting function. It is much faster than the
equivalent R function, pmax, because pmax does a lot of checking for
missing data and for recycling. Tom Lumley suggested a simple function to
replace pmax, without these checks, that is analogous to greaterOf, which I
call fast.pmax.
fast.pmax - function(x,y) {i- xy; x[i]-y[i]; x}
Interestingly, greaterOf is even faster than fast.pmax, although you have to
be dealing with very large vectors (O(10^6)) to see any real difference.
n - 200
x1 - runif(n)
x2 - rnorm(n)
system.time( ans1 - greaterOf(x1,x2) )
[1] 0.17 0.06 0.23 NA NA
system.time( ans2 - pmax(x1,x2) )
[1] 0.72 0.19 0.94 NA NA
system.time( ans3 - fast.pmax(x1,x2) )
[1] 0.29 0.05 0.35 NA NA
all.equal(ans1,ans2,ans3)
[1] TRUE
Ravi.
---
Ravi Varadhan, Ph.D.
Assistant Professor, The Center on Aging and Health
Division of Geriatric Medicine and Gerontology
Johns Hopkins University
Ph: (410) 502-2619
Fax: (410) 614-9625
Email: [EMAIL PROTECTED]
Webpage: http://www.jhsph.edu/agingandhealth/People/Faculty/Varadhan.html
-Original Message-
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On Behalf Of Douglas Bates
Sent: Thursday, February 08, 2007 6:00 PM
To: R-Help
Subject: [R] Timings of function execution in R [was Re: R in Industry]
On 2/8/07, Albrecht, Dr. Stefan (AZ Private Equity Partner)
[EMAIL PROTECTED] wrote:
Dear all,
Thanks a lot for your comments.
I very well agree with you that writing efficient code is about
optimisation. The most important rules I know would be:
- vectorization
- pre-definition of vectors, etc.
- use matrix instead of data.frame
- do not use named objects
- use pure matrix instead of involved S4 (perhaps also S3) objects (can
have enormous effects)
- use function instead of expression
- use compiled code
- I guess indexing with numbers (better variables) is also much faster
than with text (names) (see also above)
- I even made, for example, my own min, max, since they are slow, e.g.,
greaterOf - function(x, y){
# Returns for each element of x and y (numeric)
# x or y may be a multiple of the other
z - x y
z*x + (!z)*y
That's an interesting function. I initially was tempted to respond
that you have managed to reinvent a specialized form of the ifelse
function but then I decided to do the timings just to check (always a
good idea). The enclosed timings show that your function is indeed
faster than a call to ifelse. A couple of comments:
- I needed to make the number of components in the vectors x and y
quite large before I could get reliable timings on the system I am
using.
- The recommended way of doing timings is with system.time function,
which makes an effort to minimize the effects of garbage collection on
the timings.
- Even when using system.time there is often a big difference in
timing between the first execution of a function call that generates a
large object and subsequent executions of the same function call.
[additional parts of the original message not relevant to this
discussion have been removed]
__
R-help@stat.math.ethz.ch mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
