MarisaKirisame commented on issue #1996: [RFC][WIP] Tensor Expression level automatic differentiation URL: https://github.com/apache/incubator-tvm/issues/1996#issuecomment-595980942 @yzhliu let me explain it another way. suppose we work only at the scalar level. people had proved that reverse mode only take 3x more computation. This does not need any optimization - the gradient of f will only be a (small, typicallty less then 10) constant factor times slower then the computation of f itself. In this PR, the gradient of f might be MANY times more expensive. This is because it is calculating the jacobian, rather then the product of a vector/matrix/tensor with that jacobian, which can be fused, so the computation can be expressed in a way simpler form.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
