sandeep-krishnamurthy commented on issue #14977: Add an utility for operator benchmarks URL: https://github.com/apache/incubator-mxnet/pull/14977#issuecomment-495435263 Updates: 1. Add functionality to query MXNet registry for operator. 2. Add an example of running 20 binary_broadcast operators without explicit operator calling. See https://github.com/apache/incubator-mxnet/pull/14977/files#diff-bbbeb0f4b497dc86cf7f3e2e9713dc64 and https://github.com/apache/incubator-mxnet/pull/14977/files#diff-a5217faf4046473dec2565d1ee07236d Below are the observations: 1. I have added the required utilities, to query MXNet registry for all registered operators. 2. With 20 operators, we see that whatever we have in this PR as of today can evolve to suggested ideal automated way. 3. Automated way to query the operator and running benchmarks cannot work for all the operators. Operators will have different inputs and input criteria. Example: Input shapes for broadcast_add cannot work for elementwise_add and so on. - This is different than just auto-generating Python API docs, because here, we need to execute so we need to know semantics of the parameters. Ex: should param1 match shape of param2. - At minimum, we need to set rules to generate inputs. 4. We will have a few operators like elementwise, broadcast - unary/binary operators almost covered with automated way, few complex operators like convolution, pooling, where rules for inputs are explicit. Having, seen how this utility can be evolved. This PR having all the basic utilities and >20 operator benchmarks in automated way and few more in explicit way. @szha - Do you see any other blocking concerns?
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
