Great work Zhu, thanks a lot! sandeep krishnamurthy <[email protected]> schrieb am Do., 4. Juli 2019, 06:41:
> Thank you Zhaoqi for all your contributions to MXNet community. > > This feature of capturing detailed information for a custom operator in > MXNet profiler output will be very helpful for lot of MXNet users who has > custom operators and want to know more about the bottlenecks. > > Best, > Sandeep > > On Wed, Jul 3, 2019, 3:01 PM Zhu Zhaoqi <[email protected]> wrote: > >> Sorry about the image. I re-uploaded it: >> [image: Screen Shot 2019-06-11 at 1.15.48 PM.png] >> >> Zhu Zhaoqi <[email protected]> 于2019年7月3日周三 下午2:58写道： >> >>> Dear MXNet Community, >>> >>> >>> >>> A couple of weeks ago I sent out my custom operator profiling >>> enhancement project RFC ( >>> https://github.com/apache/incubator-mxnet/issues/15241) to the >>> community and got positive feedbacks, and now, the PR ( >>> https://github.com/apache/incubator-mxnet/pull/15210) has been merged. >>> A new section on profiling custom operators has also been added to the >>> profiler tutorial here: >>> https://mxnet.incubator.apache.org/versions/master/tutorials/python/profiler.html#profiling-custom-operators. >>> With that said, here I will have a brief introduction to the changes: >>> >>> >>> >>> Before, MXNet allowed user-defined custom operators, but their profiling >>> was not well supported. More specifically, the custom operator events as we >>> see in the dump file were way longer than the actual execution time of the >>> operators. Also, custom operators consist of pure python code and NDArray >>> operators (“sub-operators”), but at a particular point in a custom operator >>> event, we were not able to tell which ones were executing. >>> >>> >>> >>> To allow for a more informative custom operator profiling, my project >>> helps break a single custom operator call into several events: now, we have >>> individual events for both the “pure python” code, namely the forward() and >>> backward() functions, and the “sub-operators” , which are the NDArray >>> operators we call within forward() and backward() of the custom operator. >>> >>> >>> >>> All the new, fine-grained events will fall under a new domain/category >>> called “Custom Operator”. Also, each event there will have a prefix in its >>> name, which is the name we give to the custom operator. Below is a sample >>> dump file (opened in chome://tracing): >>> >>> >>> >>> [image: >>> /var/folders/fb/wknbw2r95zs5vkwrz9r80bzjsgk_ng/T/com.microsoft.Outlook/WebArchiveCopyPasteTempFiles/[email protected]] >>> >>> >>> >>> Note that: we need to make sure “profile_imperative” is set to True even >>> when we are using custom operators in symbolic mode. The reason is that >>> within custom operators, even in symbolic mode, sub-operators (and pure >>> python code) are still called imperatively. >>> >>> >>> >>> Should there be any issue using this new feature, please feel free to >>> tag me. Additionally, any feedback is welcome! >>> >>> >>> >>> Thanks, >>> >>> Zhaoqi >>> >>> >>> >>
