safrooze commented on issue #9705: Added unittest for benchmarking metric performance URL: https://github.com/apache/incubator-mxnet/pull/9705#issuecomment-363977854 OK I think I addressed all the feedback: * random is seeded * nd.wait_all() used before starting timing and before ending timing * Added batch-size values of 16, 64, 256, and 1024 * Datasize varies by number of output channels to keep total runtime down to a few minutes The modified code output looks like this: ``` Metric Data-Ctx Label-Ctx Data Size Batch Size Output Dim Elapsed Time ------------------------------------------------------------------------------------------ acc cpu(0) cpu(0) 131072 16 128 1.0015 acc cpu(0) gpu(0) 131072 16 128 1.682 acc gpu(0) cpu(0) 131072 16 128 2.6263 acc gpu(0) gpu(0) 131072 16 128 3.3028 ------------------------------------------------------------------------------------------ acc cpu(0) cpu(0) 131072 64 128 0.42843 acc cpu(0) gpu(0) 131072 64 128 0.568 acc gpu(0) cpu(0) 131072 64 128 0.78586 acc gpu(0) gpu(0) 131072 64 128 0.94317 ------------------------------------------------------------------------------------------ acc cpu(0) cpu(0) 131072 256 128 0.19074 acc cpu(0) gpu(0) 131072 256 128 0.24228 acc gpu(0) cpu(0) 131072 256 128 0.21548 acc gpu(0) gpu(0) 131072 256 128 0.25075 ------------------------------------------------------------------------------------------ acc cpu(0) cpu(0) 131072 1024 128 0.1303 acc cpu(0) gpu(0) 131072 1024 128 0.14127 acc gpu(0) cpu(0) 131072 1024 128 0.055079 acc gpu(0) gpu(0) 131072 1024 128 0.065515 ------------------------------------------------------------------------------------------ ```
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
