HuichuanLiu commented on issue #11149: Unreasonable performance of resnext models provided in model_zoo, evaluated by score.py URL: https://github.com/apache/incubator-mxnet/issues/11149#issuecomment-396461022 Thanks @lanking520 And here're some updates: 1. My experiments shows resnet-152 restored from gluon model_zoo and from the module symbol files require different preprocess. I didn't find any clear description about this in mxnet docs and it will be nice if you can add it, it's quite confusing for the green hands like me. 2. I got a higher accuracy from gluon model, comparing to [these statistics](https://github.com/apache/incubator-mxnet/blob/master/example/image-classification/README.md). Is it another inconsistence between the module and the gluon model? Details: I replaced resnext-101 with resnet-152 in score.py and received acc=~0.765, exactly the same as the [doc shows](https://github.com/apache/incubator-mxnet/blob/master/example/image-classification/README.md) Then I repeated the same procedure, i.e. the same data and the same mx.io.RecordIter setting, but loaded the resnet-152 model with gluon API(), instead of the default module symbol files. ``` from mxnet.gluon.model_zoo.vision.resnet import get_resnet net = get_resnet(version=2, num_layers=152, pretrained=True, root='./', ctx=ctx[1]) ``` This leaded to broken predictions, it gives 916 after argmax for all samples, because of unnormalized input. Next I added a standard preprocess according to the [gluon model](http://mxnet.incubator.apache.org/versions/1.2.0/api/python/gluon/model_zoo.html) > All pre-trained models expect input images normalized in the same way, i.e. mini-batches of 3-channel RGB images of shape (N x 3 x H x W), where N is the batch size, and H and W are expected to be at least 224. The images have to be loaded in to a range of [0, 1] and then normalized using mean = [0.485, 0.456, 0.406] and std = [0.229, 0.224, 0.225]. The transformation should preferrably happen at preprocessing It takes the model to acc=0.773，about 0.012 higher than the [doc claims](https://github.com/apache/incubator-mxnet/blob/master/tools/im2rec.py)
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
