Thanks Jason, that helps a lot. Also for the pointer to the reverse mode AD. And you are quite right, number types in Julia is fun and a good learning step in my case. As were your suggestions and looking at the different approach taken by PowerSeries and TaylorSeries as far as type definitions go! I'll certainly continue to follow the PowerSeries & TaylorSeries discussion.
My interest in HyperDual numbers was triggered by the accuracy argument in the HyperNumber paper (in fact I only need the gradient). Time to go back to focus on MCMC and a full Julia version of a Stan like tool. Regards, Rob J. Goedman goed...@icloud.com On Apr 7, 2014, at 10:34 AM, Jason Merrill <jwmerr...@gmail.com> wrote: > Also check out the work that Miles Lubin and others are doing on reverse mode > automatic differentiation. Some really cool stuff in the pipeline. > > On Monday, April 7, 2014 8:35:10 AM UTC-7, Jason Merrill wrote: > I partly meant to suggest that PowerSeries.jl > https://github.com/jwmerrill/PowerSeries.jl might also meet your needs for > 2nd order forward automatic differentiation, and it also already works to > higher orders. > > PowerSeries.jl works by computing truncated power series of functions. You > can read derivatives off the series coefficients because > > f(x + e) = f(x) + f'(x) e + f''(x)/2 e^2 + f'''(x)/3! e^3 + ... > > You can choose what order to compute to for any given application. > > To borrow your README example: > > julia> using PowerSeries > > julia> t0 = series(1.5, 1.0, 0.0) > Series2{Float64}(1.5,1.0,0.0) > > julia> f(x) = e^x / (sqrt(sin(x)^3 + cos(x)^3)) > f (generic function with 1 method) > > # f(1.5 + e) = 4.50 + 4.05e + 4.73e^2 + ... > julia> f(t0) > Series2{Float64}(4.497780053946162,4.05342789389862,4.731536840798301) > > # First derivative > julia> polyder(f(t0)) > Series1{Float64}(4.05342789389862,9.463073681596603) > > # Second derivative > julia> polyder(polyder(f(t0))) > 9.463073681596603 > > # If you start from the beginning with a higher order series, > # then you'll be able to take higher order derivatives at the end > julia> polyder(polyder(polyder(f(series(1.5, 1.0, 0.0, 0.0))))) > 32.16790451368894 > > As far as I can tell (and I could well be wrong!), if you're interested in > differentiating programs to higher orders, truncated power series are a more > fit-to-purpose extension of Dual numbers than HyperDual numbers are. > > This whole space is pretty lively right now. I think everyone is realizing > how easy it is to write new Number types in Julia, and there are *a lot* of > useful notions of number. > > If PowerSeries.jl does end up fitting your purpose, there's plenty of room > for contribution/improvement. There's a good discussion going on in an issue > right now on figuring out how to combine the best aspects of PowerSeries.jl > and a new package called TaylorSeriesl.jl > https://github.com/jwmerrill/PowerSeries.jl/issues/7 > > Or if I've missed something and HyperDual numbers have some important > advantage, that would be good to know! > > > On Sunday, April 6, 2014 4:21:33 PM UTC-7, Rob J Goedman wrote: > Hi John, > > Jeff has updated his source files with the MIT license and I've pasted those > into the LICENSE file of the Julia package. > > Jason Merrill has also given good feedback that I'm still looking into. My > interpretation of his feedback (a single package covering different hyper > number types and orders) is substantial more work and will definitely take > longer. So maybe we should publish the current version? > > Regards, > Rob J. Goedman > goe...@icloud.com > > > > On Apr 6, 2014, at 12:14 PM, Jeffrey Fike <jf...@alumni.stanford.edu> wrote: > >> Rob, >> >> Thanks for your interest. I have been meaning to look into an actual open >> source license. I went with the MIT license. I have updated the code on >> the website to reflect this. Please let me know if you need any additional >> information. >> >> Jeff Fike > > On Mar 29, 2014, at 5:55 PM, John Myles White <johnmyl...@gmail.com> wrote: > >> Thanks for looking into it, Rob. In the absence of a license, the code is >> technically not free to use. But I imagine the authors would like to share >> their code, so it should be easy to convince them to use something formal >> like the MIT or BSD licenses. >> >> — John >> >> On Mar 29, 2014, at 5:52 PM, Robert J Goedman <goe...@icloud.com> wrote: >> >>> John, >>> >>> No license is mentioned on the c++ code nor on the matlab versions as far >>> as I can see. >>> >>> I'll send the authors an email. >>> >>> Regards, >>> Rob J. Goedman >>> goe...@icloud.com >>> > >>> On Mar 29, 2014, at 5:47 PM, John Myles White <johnmyl...@gmail.com> wrote: >>> >>>> This looks really cool. Any idea what the license was on the original file? >>>> >>>> — John >>>> >>>> On Mar 29, 2014, at 5:43 PM, Robert J Goedman <goe...@icloud.com> wrote: >>>> >>>>> Hi, >>>>> >>>>> As a first 'jump into the fray' exercise I've attempted to translate >>>>> Jeffrey Fike's hyper-dual numbers code from c++ to Julia, more or less >>>>> following the DualNumbers package. >>>>> >>>>> The c++ code can be found at >>>>> http://adl.stanford.edu/hyperdual/hyperdual.h . The paper itself at >>>>> http://adl.stanford.edu/hyperdual/Fike_AIAA-2011-886.pdf . >>>>> >>>>> The Julia package can be found at: >>>>> https://github.com/goedman/HyperDualNumbers.jl.git . >>>>> >>>>> Of course, I'm pretty new at this so I'm sure there will be errors and >>>>> poor practices. So any feedback is appreciated. >>>>> >>>>> Also, I'm wondering if the type should be called Hyper or a better name >>>>> would be HyperDual. >>>>> >>>>> This work was triggered by the interesting threads around openPP, >>>>> TaylorSeries.jl, Calculus2, PowerSeries.jl (and at some time I hope >>>>> MCMC.jl). >>>>> >>>>> Rob J. Goedman >>>>> goe...@icloud.com > >
