altanh commented on a change in pull request #8318:
URL: https://github.com/apache/tvm/pull/8318#discussion_r657526529
##########
File path: src/relay/transforms/first_order_gradient.cc
##########
@@ -174,11 +174,14 @@ struct FirstOrderReverseAD : ExprFunctor<ADValue(const
Expr&)> {
}
ADValue VisitExpr_(const TupleGetItemNode* op) final {
- Expr e = GetRef<Expr>(op);
ADValue tup = VisitExpr(op->tuple);
- auto tt = op->tuple->checked_type().as<TupleTypeNode>();
+ TupleType tt = Downcast<TupleType>(op->tuple->checked_type());
size_t idx = op->index;
- auto ret = std::make_shared<ADTensor>(ll, e, diag_ctx);
+ // reconstruct projection using let-bound variable to avoid duplicating
input tuple
+ TupleGetItem orig = TupleGetItem(tup->get<ADTensor>().forward, idx);
+ orig->checked_type_ = op->checked_type();
+ auto ret = std::make_shared<ADTensor>(ll, orig, diag_ctx);
+ // for orig = pi(tup, i), pi_grad(tup, i, g) = G where pi(G, i) = g and
pi(G, j) = 0 for j != i
Review comment:
sure, this is just describing how the gradient for a projection is
propagated back to the original tuple
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
