Re: [Rd] Automatic Differentiation for R
[MM stumbling over on old thread ... he'd be interested]
GaGr == Gabor Grothendieck ggrothendi...@gmail.com
on Wed, 15 Apr 2009 09:53:18 -0400 writes:
GaGr Not sure if this is sufficient for your needs but R does include
symbolic
GaGr differentiation, see ?D, and the Ryacas and rSymPy
GaGr packages interface R to the yacas and sympy computer algebra
GaGr systems (CAS) and those system include symbolic differentiation.
No, symbolic differentiation is not enough.
Automatic Differentiation (AD) is something much more general (in one
way) and much less mathematical from a classical view point:
But then, AD is much more generally useful for minimization as, basically,
the input is an R function
f(x) {with x multidimensional}
or f(x1,x2, ..., xp) {with scalar x1, x2, ..}
and the output is again an R function
which computes f() and all {or just selected} partial
derivatives d f / d{xi}.
Now consider that the function f() can contain if() and while()
clauses and conceptually ever language feature of R.
In practice, I'm pretty sure the list of features would have to
be restricted, similarly as they'd have to for an R compiler to
be feasible.
I agree that AD for R would be very nice and could be very
useful.
I'd also be interested to help AD people learn the S4 classes
and methods (hoping that it's close enough to what they call
operator overloading something I'd presume to be less general
than the powerful S4 class/methods system).
Martin Maechler, ETH Zurich
GaGr http://ryacas.googlecode.com
GaGr http://rsympy.googlecode.com
[.]
GaGr On Wed, Apr 15, 2009 at 9:30 AM, John C Nash nas...@uottawa.ca
wrote:
In efforts to improve optimization tools for R, one of my
interests has been getting automatic differentiation capabilities
so that analytic rather than numerical derivatives can be used. They
would be helpful in several other areas besides optimization, My timings
show
factors of the order of 1000s in time improvements by avoiding
numerical derivatives in some cases.
There has been some work in this e.g.,
http://code.google.com/p/pbs-software/
is an R interface to ADMB (Automatic Differentiation Model Builder).
However,
as far as I can see, this is directed essentially to nonlinear least
squares
modelling,
an important but not general problem.
Tom Coleman of Waterloo responded favourably with some advice, but the
most
enthusiastic answer came from Shaun Forth, which I have included below. I
read
this as an opportunity to develop what could be a profitable
collaboration
with
the AD community. Unfortunately, I cannot take up the invitation to join
the
AD
folk in Oxford due to a pre-existing obligation. Nor am I more than a
complete
novice with S3 and S4 classes etc. I am, nevertheless, willing to help
organize
the effort e.g., do some of the communications, chasing grant money,
getting
Google Summer of Code applications filled in etc.
Can the R community come up with a few people who can provide the AD
workers with appropriate information? If so, is there a reasonable
chance to
generate sufficient funding for a student? I suspect the answer in both
cases
is yes, but that we need some form of booster cables to get things
going.
(In Canada, booster cables are used to get cars started in winter by
connecting
a running vehicle's battery to that of a dead one.)
I suggest communications off-list until there is progress to report.
Possibly
there is a better forum for this -- suggestions welcome.
John Nash
included msg from Shaun Forth ---
Hi John,
My computational statistics colleague Trevor Ringrose has asked me to
consider AD in R in the past. As you may or may not be aware AD is
implemented in one of two ways: overloading using OO features of the
target language, or source transformation using compiler tools (after
several man years of development) to read in the target code and spit
out the differentiated code. Last time I looked I didn't think the
object oriented features of R were up to overloading but on checking
today I can see that this might now be possible (I can see overloading
of arithmetic operators and functions for example now which I didn't see
last time).
I'd certainly be interested in following this up particularly on the
overloading side but would need to get funding for a PhD student to do
the graft. It would be particularly interesting doing this in an open
source language because we could then perhaps tweak some of the core
language features if they didn't seamlessly support the AD (we can't do
this in Matlab and that is a pain!).
My immediate suggestion is that you,
Re: [Rd] Automatic Differentiation for R
Martin (see below) gives a good explanation of the difference between AD and
symbolic
differentiations. I'm of the opinion we can use both. However, the real issue
as far
as I'm concerned (from an optimizer's point of view, which may also be that of
ODE and
PDE folk) is that right now none of the offerings that we have are easy to use.
Indeed,
usability is one of the key issues in my current efforts to improve
optimization tools.
There are several people in UK from both AD and R sides who are communicating,
an initiative that Shaun F. helped get going. I'm copying the folk I think are
involved -- forgive omissions and pass things along please. The main purpose of
this
message is to ensure Martin's offer is noted, as in my opinion his knowledge of
the
R internals is very valuable.
Cheers, JN
[MM stumbling over on old thread ... he'd be interested]
GaGr == Gabor Grothendieck ggrothendi...@gmail.com
on Wed, 15 Apr 2009 09:53:18 -0400 writes:
GaGr Not sure if this is sufficient for your needs but R does include
symbolic
GaGr differentiation, see ?D, and the Ryacas and rSymPy
GaGr packages interface R to the yacas and sympy computer algebra
GaGr systems (CAS) and those system include symbolic differentiation.
No, symbolic differentiation is not enough.
Automatic Differentiation (AD) is something much more general (in one
way) and much less mathematical from a classical view point:
But then, AD is much more generally useful for minimization as, basically,
the input is an R function
f(x) {with x multidimensional}
or f(x1,x2, ..., xp) {with scalar x1, x2, ..}
and the output is again an R function
which computes f() and all {or just selected} partial
derivatives d f / d{xi}.
Now consider that the function f() can contain if() and while()
clauses and conceptually ever language feature of R.
In practice, I'm pretty sure the list of features would have to
be restricted, similarly as they'd have to for an R compiler to
be feasible.
I agree that AD for R would be very nice and could be very
useful.
I'd also be interested to help AD people learn the S4 classes
and methods (hoping that it's close enough to what they call
operator overloading something I'd presume to be less general
than the powerful S4 class/methods system).
Martin Maechler, ETH Zurich
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Automatic Differentiation for R
On Tue, May 19, 2009 at 9:08 AM, Martin Maechler
maech...@stat.math.ethz.ch wrote:
[MM stumbling over on old thread ... he'd be interested]
GaGr == Gabor Grothendieck ggrothendi...@gmail.com
on Wed, 15 Apr 2009 09:53:18 -0400 writes:
GaGr Not sure if this is sufficient for your needs but R does include
symbolic
GaGr differentiation, see ?D, and the Ryacas and rSymPy
GaGr packages interface R to the yacas and sympy computer algebra
GaGr systems (CAS) and those system include symbolic differentiation.
No, symbolic differentiation is not enough.
Automatic Differentiation (AD) is something much more general (in one
way) and much less mathematical from a classical view point:
But then, AD is much more generally useful for minimization as, basically,
the input is an R function
f(x) {with x multidimensional}
or f(x1,x2, ..., xp) {with scalar x1, x2, ..}
and the output is again an R function
which computes f() and all {or just selected} partial
derivatives d f / d{xi}.
Now consider that the function f() can contain if() and while()
clauses and conceptually ever language feature of R.
In practice, I'm pretty sure the list of features would have to
be restricted, similarly as they'd have to for an R compiler to
be feasible.
I agree that AD for R would be very nice and could be very
useful.
I'd also be interested to help AD people learn the S4 classes
and methods (hoping that it's close enough to what they call
operator overloading something I'd presume to be less general
than the powerful S4 class/methods system).
The overloading facilities present have already been discussed in
this thread including a complete illustration of using them for the
problem at hand.
rSymPy and Ryacas both support overloading.
Ryacas also supports automatic differentiation
of one line R functions but its not fully developed and very limited.
See demo(Ryacas-Function) which shows differentiation of the
Burr CDF to get the PDF.
Here are a few more simpler examples to illustrate overloading in
these packages.
library(Ryacas)
Loading required package: XML
x - Sym(x)
x+x
[1] Starting Yacas!
expression(2 * x)
library(rSymPy)
Loading required package: rJava
source(http://rsympy.googlecode.com/svn/trunk/R/Sym.R;)
y - Sym(sympy(var('y')))
y+y
[1] 2*y
Check the home pages of the packages for more info.
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Automatic Differentiation for R
Thanks for your interest in this topic.
In the AD community we are used to dealing with essentially arbitrarily complex
computer code. Some Fortran and C activities have looked at applications such
as global climate models and computational fluid dynamics codes for aerospace
design. Fortran and C have relatively small intrinsic operations and function
set. I've done quite of lot or work with Matlab and had to consider
intrinsics such as linear solves and determinants; this is probably more like
the R approach. Matlab software is used by industry and commerce for involved
computations with loops, branches and some collaborators have considered OO
programs.
I look forward to hearing about R at the Oxford AD meeting and perhaps seeing
if we can start to explore some efficient AD implementations.
Shaun
#
Dr Shaun Forth
Applied Mathematics Scientific Computation
Cranfield Defence and Security
Cranfield University, Shrivenham Campus
Swindon SN6 8LA, England
tel: +44 (0)1793 785311
fax: +44 (0)1793 784196
email: s.a.fo...@cranfield.ac.uk
http://www.amorg.co.uk
#
-Original Message-
From: Gabor Grothendieck [mailto:ggrothendi...@gmail.com]
Sent: Tuesday, May 19, 2009 2:58 PM
To: Martin Maechler
Cc: John C Nash; r-devel@r-project.org; Forth, Shaun
Subject: Re: [Rd] Automatic Differentiation for R
On Tue, May 19, 2009 at 9:08 AM, Martin Maechler
maech...@stat.math.ethz.ch wrote:
[MM stumbling over on old thread ... he'd be interested]
GaGr == Gabor Grothendieck ggrothendi...@gmail.com
on Wed, 15 Apr 2009 09:53:18 -0400 writes:
GaGr Not sure if this is sufficient for your needs but R
does include symbolic
GaGr differentiation, see ?D, and the Ryacas and rSymPy
GaGr packages interface R to the yacas and sympy computer algebra
GaGr systems (CAS) and those system include symbolic
differentiation.
No, symbolic differentiation is not enough.
Automatic Differentiation (AD) is something much more general (in one
way) and much less mathematical from a classical view point:
But then, AD is much more generally useful for minimization
as, basically,
the input is an R function
f(x) {with x multidimensional}
or f(x1,x2, ..., xp) {with scalar x1, x2, ..}
and the output is again an R function
which computes f() and all {or just selected} partial
derivatives d f / d{xi}.
Now consider that the function f() can contain if() and while()
clauses and conceptually ever language feature of R.
In practice, I'm pretty sure the list of features would have to
be restricted, similarly as they'd have to for an R compiler to
be feasible.
I agree that AD for R would be very nice and could be very
useful.
I'd also be interested to help AD people learn the S4 classes
and methods (hoping that it's close enough to what they call
operator overloading something I'd presume to be less general
than the powerful S4 class/methods system).
The overloading facilities present have already been discussed in
this thread including a complete illustration of using them for the
problem at hand.
rSymPy and Ryacas both support overloading.
Ryacas also supports automatic differentiation
of one line R functions but its not fully developed and very limited.
See demo(Ryacas-Function) which shows differentiation of the
Burr CDF to get the PDF.
Here are a few more simpler examples to illustrate overloading in
these packages.
library(Ryacas)
Loading required package: XML
x - Sym(x)
x+x
[1] Starting Yacas!
expression(2 * x)
library(rSymPy)
Loading required package: rJava
source(http://rsympy.googlecode.com/svn/trunk/R/Sym.R;)
y - Sym(sympy(var('y')))
y+y
[1] 2*y
Check the home pages of the packages for more info.
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Automatic Differentiation for R
In efforts to improve optimization tools for R, one of my interests has been getting automatic differentiation capabilities so that analytic rather than numerical derivatives can be used. They would be helpful in several other areas besides optimization, My timings show factors of the order of 1000s in time improvements by avoiding numerical derivatives in some cases. There has been some work in this e.g., http://code.google.com/p/pbs-software/ is an R interface to ADMB (Automatic Differentiation Model Builder). However, as far as I can see, this is directed essentially to nonlinear least squares modelling, an important but not general problem. Tom Coleman of Waterloo responded favourably with some advice, but the most enthusiastic answer came from Shaun Forth, which I have included below. I read this as an opportunity to develop what could be a profitable collaboration with the AD community. Unfortunately, I cannot take up the invitation to join the AD folk in Oxford due to a pre-existing obligation. Nor am I more than a complete novice with S3 and S4 classes etc. I am, nevertheless, willing to help organize the effort e.g., do some of the communications, chasing grant money, getting Google Summer of Code applications filled in etc. Can the R community come up with a few people who can provide the AD workers with appropriate information? If so, is there a reasonable chance to generate sufficient funding for a student? I suspect the answer in both cases is yes, but that we need some form of booster cables to get things going. (In Canada, booster cables are used to get cars started in winter by connecting a running vehicle's battery to that of a dead one.) I suggest communications off-list until there is progress to report. Possibly there is a better forum for this -- suggestions welcome. John Nash included msg from Shaun Forth --- Hi John, My computational statistics colleague Trevor Ringrose has asked me to consider AD in R in the past. As you may or may not be aware AD is implemented in one of two ways: overloading using OO features of the target language, or source transformation using compiler tools (after several man years of development) to read in the target code and spit out the differentiated code. Last time I looked I didn't think the object oriented features of R were up to overloading but on checking today I can see that this might now be possible (I can see overloading of arithmetic operators and functions for example now which I didn't see last time). I'd certainly be interested in following this up particularly on the overloading side but would need to get funding for a PhD student to do the graft. It would be particularly interesting doing this in an open source language because we could then perhaps tweak some of the core language features if they didn't seamlessly support the AD (we can't do this in Matlab and that is a pain!). My immediate suggestion is that you, or some other more local (to UK) R expert talks at the next European AD workshop in Oxford http://www.autodiff.org/?module=Workshopssubmenu=EuroAD/8/main We're a very friendly group and I'm sure there are others who might like to tackle R or perhaps we could put together a multigroup project. If someone could give a talk on R, its language features including the OO aspects, and some optimisation examples with associated code, the group there would be able to give you the best feedback on the planet on the possibilities. Please do treat this as a positive response and let's keep in touch on this. Regards Shaun Dr Shaun Forth Applied Mathematics Scientific Computation Cranfield Defence and Security Cranfield University, Shrivenham Campus Swindon SN6 8LA, England tel: +44 (0)1793 785311 fax: +44 (0)1793 784196 email: s.a.fo...@cranfield.ac.uk http://www.amorg.co.uk # __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Automatic Differentiation for R
Not sure if this is sufficient for your needs but R does include symbolic differentiation, see ?D, and the Ryacas and rSymPy packages interface R to the yacas and sympy computer algebra systems (CAS) and those system include symbolic differentiation. http://ryacas.googlecode.com http://rsympy.googlecode.com Note that Ryacas communicates with yacas via XML but recent versions of the XML package changed in a way that breaks Ryacas so you will likely have to use an old version of XML and Ryacas if you want to try that one -- see home page. The rsympy interface is early stage but its functional and is easier to install since it includes the entire CAS right in the R package. On Wed, Apr 15, 2009 at 9:30 AM, John C Nash nas...@uottawa.ca wrote: In efforts to improve optimization tools for R, one of my interests has been getting automatic differentiation capabilities so that analytic rather than numerical derivatives can be used. They would be helpful in several other areas besides optimization, My timings show factors of the order of 1000s in time improvements by avoiding numerical derivatives in some cases. There has been some work in this e.g., http://code.google.com/p/pbs-software/ is an R interface to ADMB (Automatic Differentiation Model Builder). However, as far as I can see, this is directed essentially to nonlinear least squares modelling, an important but not general problem. Tom Coleman of Waterloo responded favourably with some advice, but the most enthusiastic answer came from Shaun Forth, which I have included below. I read this as an opportunity to develop what could be a profitable collaboration with the AD community. Unfortunately, I cannot take up the invitation to join the AD folk in Oxford due to a pre-existing obligation. Nor am I more than a complete novice with S3 and S4 classes etc. I am, nevertheless, willing to help organize the effort e.g., do some of the communications, chasing grant money, getting Google Summer of Code applications filled in etc. Can the R community come up with a few people who can provide the AD workers with appropriate information? If so, is there a reasonable chance to generate sufficient funding for a student? I suspect the answer in both cases is yes, but that we need some form of booster cables to get things going. (In Canada, booster cables are used to get cars started in winter by connecting a running vehicle's battery to that of a dead one.) I suggest communications off-list until there is progress to report. Possibly there is a better forum for this -- suggestions welcome. John Nash included msg from Shaun Forth --- Hi John, My computational statistics colleague Trevor Ringrose has asked me to consider AD in R in the past. As you may or may not be aware AD is implemented in one of two ways: overloading using OO features of the target language, or source transformation using compiler tools (after several man years of development) to read in the target code and spit out the differentiated code. Last time I looked I didn't think the object oriented features of R were up to overloading but on checking today I can see that this might now be possible (I can see overloading of arithmetic operators and functions for example now which I didn't see last time). I'd certainly be interested in following this up particularly on the overloading side but would need to get funding for a PhD student to do the graft. It would be particularly interesting doing this in an open source language because we could then perhaps tweak some of the core language features if they didn't seamlessly support the AD (we can't do this in Matlab and that is a pain!). My immediate suggestion is that you, or some other more local (to UK) R expert talks at the next European AD workshop in Oxford http://www.autodiff.org/?module=Workshopssubmenu=EuroAD/8/main We're a very friendly group and I'm sure there are others who might like to tackle R or perhaps we could put together a multigroup project. If someone could give a talk on R, its language features including the OO aspects, and some optimisation examples with associated code, the group there would be able to give you the best feedback on the planet on the possibilities. Please do treat this as a positive response and let's keep in touch on this. Regards Shaun Dr Shaun Forth Applied Mathematics Scientific Computation Cranfield Defence and Security Cranfield University, Shrivenham Campus Swindon SN6 8LA, England tel: +44 (0)1793 785311 fax: +44 (0)1793 784196 email: s.a.fo...@cranfield.ac.uk http://www.amorg.co.uk # __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel __
