JohnCalhoun commented on issue #10002: General support of OPs for second-order gradient URL: https://github.com/apache/incubator-mxnet/issues/10002#issuecomment-429583110 more updates: I tried out the sin(x)->cos(x) so code looked like this: ```cpp // sin MXNET_OPERATOR_REGISTER_UNARY_WITH_RSP(sin, cpu, mshadow_op::sin) MXNET_ADD_SPARSE_OP_ALIAS(sin) .describe(R"code(Computes the element-wise sine of the input array. The input should be in radians (:math:`2\pi` rad equals 360 degrees). .. math:: sin([0, \pi/4, \pi/2]) = [0, 0.707, 1] The storage type of ``sin`` output depends upon the input storage type: - sin(default) = default - sin(row_sparse) = row_sparse )code" ADD_FILELINE) .set_attr<nnvm::FGradient>("FGradient", ElemwiseGradUseIn{ "cos" }); MXNET_OPERATOR_REGISTER_BINARY_WITH_SPARSE_CPU_DR(_backward_sin, unary_bwd<mshadow_op::sin_grad>); ``` but it does not work. this is because ElemwiseGradUseIn{ "cos"} != _backward_sin the math is similar but the function signatures dont match. here is the error > C++ exception with description "[19:00:10] ../src/io/../operator/elemwise_op_common.h:176: Check failed: in_attrs->size() == static_cast<size_t>(n_in) (2 vs. 1) in operator _backward_sin is actualy a binary function (takes in input and gradient) while cos is a unary operation. notice the line ```cpp MXNET_OPERATOR_REGISTER_BINARY_WITH_SPARSE_CPU_DR(_backward_sin, unary_bwd<mshadow_op::sin_grad>); ``` is registering _backward_sin as a BINARY operator. Digging through the code, that macro does some magic that allows mshadow_op::sin_grad to take in two arguments but ignore the second. it does not seem feasible to try and support arbitrary order gradients. but supporting second order gradients should still be good to go.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
