fhieber commented on issue #7949: account for batch padding when updating metrics URL: https://github.com/apache/incubator-mxnet/pull/7949#issuecomment-331821052 @piiswrong I think this change introduces a regression for symbols w/ SoftmaxOutput losses and perplexity metrics: Before, one could apply SoftmaxOutput to 'flattened data': (batch_size * seq_len, classes) and compute mx.metric.Perplexity on these flattened predictions with a flattened one-hot label array (the metric also flattened the label shape). After this change ( and #8003 ), the update_metric call requires the first axis to be the batch_size only (no collapsing between batch and seq_len allowed anymore). If this is changed accordingly in the symbol (namely using SoftmaxOutput with preserve_shape=True), returning (batch_size, seq_len, vocab_size) predictions, and using non-flattened one-hot labels, the Perplexity metric fails in update(), as labels are flattened internally, but not pred (in line metric.py::652). So it seems that either all metrics should support multi-dimensional input, or the slicing from #8003 should be made less strict. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
With regards, Apache Git Services
