feevos opened a new issue #11949: Confusion in documentation/implementation of F1, MCC metrics URL: https://github.com/apache/incubator-mxnet/issues/11949 Dear all, I was looking at the ```mxnet.metric``` source [code](https://mxnet.incubator.apache.org/_modules/mxnet/metric.html#_BinaryClassificationMetrics) and [documentation](https://mxnet.incubator.apache.org/api/python/metric/metric.html) and I had a bit of confusion on the F1 and MCC metrics. In particular, if one needs to provide a 1D or a 2D NDArray predictions. I think, that since for the binary case either a 1D probability of True or a 2D probability of (False,True) predictions are equivalent, this should be unified in the code/documentation. At present one needs to provide a 2D NDArray of predictions and a 1D NDArray of labels. Yes, they are equivalent, however, in the 2D case, it **is not stated clearly which dimension represents the probability of ```True``` value**. I think this can produce confusion. So if I have ```labels = np.array([1,0,0])``` and predictions, ```preds = np.array([0.8, 0.25, 0.15])```. In order to make any of the above metrics to work, I need to provide a 2D version of ```preds_2D = np.array(list(zip (1-preds, preds)))```, where the first element of the tuple corresponds to the ```False``` prediction, while the latter in the ```True``` . ```Python labels = np.array([1,0,0]) preds = np.array([0.8, 0.25, 0.15]) f1 = mxnet.metric.F1() f1.update(labels = nd.array(labels),preds = nd.array(preds)) f1.get() # Output ('f1',1.0) ``` I think it should be relatively easy to unify both approaches (1D and 2D) under the same operator. I've tracked down the problem in the class ```_BinaryClassificationMetrics ```, line [534](https://github.com/apache/incubator-mxnet/blob/master/python/mxnet/metric.py#L534). There one calls ```np.argmax(preds, axis=1)``` to get the label index. I think issue #9865 and this [discussion](https://discuss.mxnet.io/t/a-confusion-on-evaluation-metrics/662/3) are related. Regards.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
