sandeep-krishnamurthy commented on issue #14977: Add an utility for operator benchmarks URL: https://github.com/apache/incubator-mxnet/pull/14977#issuecomment-494956431 Thanks @szha for the discussion and putting the summary here. I wanted to add few more points we discussed in the context of this PR: 1. We agreed that the current functionality in this PR is helpful for existing operators. For purpose like PR builds, nightly performance test CI, quick comparison tools for developers like working on Int64 etc. For which one of your suggestion was to maintain a separate repository / move it to mxnet jenkins or CI repository. 2. Things not favorable in this PR is that it will be an additional burden for developers adding new operators or doing breaking changes in existing operators(ex: parameters change, namespace change) to come and make modifications in this utility which we all agree is a maintenance overhead. 3. We also discussed with automated solution, we cannot handle every operator and every new use case in a fully automated way. So it would still be a combination of automated (TODO) + explicit tests (currently in this PR).
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
