There is also
https://github.com/mlubin/ReverseDiffSparse.jl I've never used it myself, but I thought i'd throw it out there. On Saturday, July 9, 2016 at 12:27:33 AM UTC-7, Gabriel Goh wrote: > > Forward differentiation has a bad complexity for functions of the form R^n > -> R. try using ReverseDiffSource.jl instead > > This blog posts describes positive results using ReverseDiffSource.jl on > an autoencoder > > > http://int8.io/automatic-differentiation-machine-learning-julia/#Training_autoencoder_8211_results > > since back-propagation is reverse differentiation, this should in theory > be equivalent to tensor flow's automatic differentiation. > > On Friday, July 8, 2016 at 5:02:55 PM UTC-7, Andrei Zh wrote: >> >> In Python, libraries like TensorFlow or Theano provide possibility to >> perform automatic differentiation over their computational graphs. E.g. in >> TensorFlow (example from SO >> <http://stackoverflow.com/questions/35226428/how-do-i-get-the-gradient-of-the-loss-at-a-tensorflow-variable> >> ): >> >> data = tf.placeholder(tf.float32) >> var = tf.Variable(...) >> loss = some_function_of(var, data) >> >> var_grad = tf.gradients(loss, [var])[0] >> >> What is the closest thing in Julia at the moment? >> >> Here's what I've checked so far: >> >> * ForwardDiff.jl <https://github.com/JuliaDiff/ForwardDiff.jl> - it >> computes derivatives using forward mode automatic differentiation (AD). >> Although AD has particular advantages, I found this package quite slow. >> E.g. for a vector of 1000 elements gradient takes ~100x times longer then >> the function itself. Another potential issues is that ForwardDiff.jl >> doesn't output symbolic version of gradient and thus is hardly usable for >> computation on GPU, for example. >> * *Calculus.jl* <https://github.com/johnmyleswhite/Calculus.jl> - among >> other things, this package provided symbolic differentiation. However, it >> seems to consider all symbols to be numbers and doesn't support matrices or >> vectors. >> >> I have pretty shallow knowledge of both these packages, so please correct >> me if I'm wrong somewhere in my conclusions. And if not, is there any other >> package or project that I should consider? >> >
