aidan-plenert-macdonald commented on issue #10002: General support of OPs for 
second-order gradient
URL: 
https://github.com/apache/incubator-mxnet/issues/10002#issuecomment-370929274
 
 
   I looked at the code and it seems like the problem is that the current 
implementation is kind of only doing symbolic math down one level. For example, 
[sin](https://github.com/apache/incubator-mxnet/blob/master/src/operator/tensor/elemwise_unary_op_trig.cc#L32),
   
   ```c++
   // sin                                                                       
                                                                                
                                               
   MXNET_OPERATOR_REGISTER_UNARY_WITH_RSP(sin, cpu, mshadow_op::sin)
   MXNET_ADD_SPARSE_OP_ALIAS(sin)
   .describe(R"code(Computes the element-wise sine of the input array.          
                                                                                
                                               
                                                                                
                                                                                
                                               
   The input should be in radians (:math:`2\pi` rad equals 360 degrees).        
                                                                                
                                               
                                                                                
                                                                                
                                               
   .. math::                                                                    
                                                                                
                                               
      sin([0, \pi/4, \pi/2]) = [0, 0.707, 1]                                    
                                                                                
                                               
                                                                                
                                                                                
                                               
   The storage type of ``sin`` output depends upon the input storage type:      
                                                                                
                                               
                                                                                
                                                                                
                                               
      - sin(default) = default                                                  
                                                                                
                                               
      - sin(row_sparse) = row_sparse                                            
                                                                                
                                               
                                                                                
                                                                                
                                               
   )code" ADD_FILELINE)
   .set_attr<nnvm::FGradient>("FGradient", ElemwiseGradUseIn{ "_backward_sin" 
});
   
   MXNET_OPERATOR_REGISTER_BINARY_WITH_SPARSE_CPU_DR(_backward_sin, 
unary_bwd<mshadow_op::sin_grad>);
   ```
   
   The fact that it names it `_backward_sin` instead of using `cos` is probably 
why this can't chain the differentiation down.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

Reply via email to