canteen-man opened a new issue #14822: When using the imglist file to load the data,If the number of data set label can't divisible by batch size,the loss become nan URL: https://github.com/apache/incubator-mxnet/issues/14822 ## Description I regresion the label and the label width is just a float number. I use the imglist to load my image and label.When the total number of label can't to be divisible by the batch size ，the label become the odd value,I don’t sure if it is a random number.I print the label in the data iter,some of them are so big，others are so small,I‘m sure these number are't my label in lst file.And due to these number,the loss become nan. I don't shuffle the label and the image.I find when it iter to the last of the data set, at the first part of the last batch in this epoch are the correct number，and the second half label should be the head of the label of all data set,but they are become the odd value. When I set the batch size which the total number of label can divisie it ，the train loss iteration turn normal. I research the code，maybe the question is in the detection.py，the next_sample function and the next function are about load the imglist. It seem like declare a batch label which size is same as the batch size length.Then go through the lable of imglist file to assignment the batch label,erery step is the batch size length.So the last of label don’t get correct number. I don't sure whether this question is because of this，this mistake is too simple to feel possible.But this question real occur. ## Environment info (Required) mxnet 1.2.1 CUDA 9.1 UBUNTU Package used (Python/R/Scala/Julia): python 3.7 For Scala user, please provide: 1. Java version: (`java -version`) 2. Maven version: (`mvn -version`) 3. Scala runtime if applicable: (`scala -version`)
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
