Wallart opened a new issue #16926: autograd.is_training() does not work on hybridized networks. URL: https://github.com/apache/incubator-mxnet/issues/16926 Hello everyone, I might have found an issue about autograd.is_training(). I've implemented a transformer. At training time my network is returning predictions and attention weights for plotting. During inference it's only returning predictions. After the training phase I'm doing validation on a new dataset. Here is a basic description of the problem Training loop : ` # training part with autograd.record(): output, attn_w_1, attn_w_2 = transformerNet(someInputs) # loss compute, backward pass, trainer step, etc. # validation part output = transformerNet(someInputs) # compute some score ` Transformer hybrid_forward end : ` # some stuff if autograd.is_training(): return logits, attn_w_1, attn_w_2 return logits ` Everything is working fine except when I hybridize the network. During validation autograd.is_training() is always true and the method returns 3 variables instead of 1.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
