[GitHub] safrooze commented on issue #9705: Added unittest for benchmarking metric performance

2018-02-28 Thread GitBox
safrooze commented on issue #9705: Added unittest for benchmarking metric 
performance
URL: https://github.com/apache/incubator-mxnet/pull/9705#issuecomment-369331546
 
 
   I'll move this to nightly tests then.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] safrooze commented on issue #9705: Added unittest for benchmarking metric performance

2018-02-09 Thread GitBox
safrooze commented on issue #9705: Added unittest for benchmarking metric 
performance
URL: https://github.com/apache/incubator-mxnet/pull/9705#issuecomment-364517332
 
 
   @eric-haibin-lin You're correct that nightly would be a more suitable place. 
One concern with nightly was that community wouldn't be able to see the results 
of the benchmark.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] safrooze commented on issue #9705: Added unittest for benchmarking metric performance

2018-02-07 Thread GitBox
safrooze commented on issue #9705: Added unittest for benchmarking metric 
performance
URL: https://github.com/apache/incubator-mxnet/pull/9705#issuecomment-363977854
 
 
   OK I think I addressed all the feedback:
   * random is seeded
   * nd.wait_all() used before starting timing and before ending timing
   * Added batch-size values of 16, 64, 256, and 1024
   * Datasize varies by number of output channels to keep total runtime down to 
a few minutes
   
   The modified code output looks like this:
   ```
   Metric Data-Ctx  Label-Ctx   Data Size   Batch Size Output Dim   
  Elapsed Time
   
--
   acccpu(0)cpu(0)  131072  16 128  
  1.0015
   acccpu(0)gpu(0)  131072  16 128  
  1.682
   accgpu(0)cpu(0)  131072  16 128  
  2.6263
   accgpu(0)gpu(0)  131072  16 128  
  3.3028
   
--
   acccpu(0)cpu(0)  131072  64 128  
  0.42843
   acccpu(0)gpu(0)  131072  64 128  
  0.568
   accgpu(0)cpu(0)  131072  64 128  
  0.78586
   accgpu(0)gpu(0)  131072  64 128  
  0.94317
   
--
   acccpu(0)cpu(0)  131072  256128  
  0.19074
   acccpu(0)gpu(0)  131072  256128  
  0.24228
   accgpu(0)cpu(0)  131072  256128  
  0.21548
   accgpu(0)gpu(0)  131072  256128  
  0.25075
   
--
   acccpu(0)cpu(0)  131072  1024   128  
  0.1303
   acccpu(0)gpu(0)  131072  1024   128  
  0.14127
   accgpu(0)cpu(0)  131072  1024   128  
  0.055079
   accgpu(0)gpu(0)  131072  1024   128  
  0.065515
   
--
   ```


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] safrooze commented on issue #9705: Added unittest for benchmarking metric performance

2018-02-07 Thread GitBox
safrooze commented on issue #9705: Added unittest for benchmarking metric 
performance
URL: https://github.com/apache/incubator-mxnet/pull/9705#issuecomment-363977854
 
 
   OK I think I addressed all the feedback:
   * random is seeded
   * nd.wait_all() used before starting timing and before ending timing
   * Added batch-size values of 16, 64, 256, and 1024
   * Datasize varies by number of output channels to keep total runtime down to 
a few minutes
   
   The modified code output looks like this:
   ```
   Metric Data-Ctx  Label-Ctx   Data Size   Batch Size Output Dim   
  Elapsed Time
   
--
   acccpu(0)cpu(0)  131072  16 128  
  1.0015
   acccpu(0)gpu(0)  131072  16 128  
  1.682
   accgpu(0)cpu(0)  131072  16 128  
  2.6263
   accgpu(0)gpu(0)  131072  16 128  
  3.3028
   
--
   acccpu(0)cpu(0)  131072  64 128  
  0.42843
   acccpu(0)gpu(0)  131072  64 128  
  0.568
   accgpu(0)cpu(0)  131072  64 128  
  0.78586
   accgpu(0)gpu(0)  131072  64 128  
  0.94317
   
--
   ```


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] safrooze commented on issue #9705: Added unittest for benchmarking metric performance

2018-02-05 Thread GitBox
safrooze commented on issue #9705: Added unittest for benchmarking metric 
performance
URL: https://github.com/apache/incubator-mxnet/pull/9705#issuecomment-363328924
 
 
   The intention is to observe a measurable elapsed time (hence large data 
size) and amplify the difference between CPU and GPU processing (hence 
processing all the data in one batch). A valid alternative is to use small 
batch size and iterate over multiple batches. 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] safrooze commented on issue #9705: Added unittest for benchmarking metric performance

2018-02-05 Thread GitBox
safrooze commented on issue #9705: Added unittest for benchmarking metric 
performance
URL: https://github.com/apache/incubator-mxnet/pull/9705#issuecomment-363302529
 
 
   @szha Please review.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services