Re: PR Merged: Custom Operator Profiling Enhancement
Great work Zhu, thanks a lot! sandeep krishnamurthy schrieb am Do., 4. Juli 2019, 06:41: > Thank you Zhaoqi for all your contributions to MXNet community. > > This feature of capturing detailed information for a custom operator in > MXNet profiler output will be very helpful for lot of MXNet users who has > custom operators and want to know more about the bottlenecks. > > Best, > Sandeep > > On Wed, Jul 3, 2019, 3:01 PM Zhu Zhaoqi wrote: > >> Sorry about the image. I re-uploaded it: >> [image: Screen Shot 2019-06-11 at 1.15.48 PM.png] >> >> Zhu Zhaoqi 于2019年7月3日周三 下午2:58写道： >> >>> Dear MXNet Community, >>> >>> >>> >>> A couple of weeks ago I sent out my custom operator profiling >>> enhancement project RFC ( >>> https://github.com/apache/incubator-mxnet/issues/15241) to the >>> community and got positive feedbacks, and now, the PR ( >>> https://github.com/apache/incubator-mxnet/pull/15210) has been merged. >>> A new section on profiling custom operators has also been added to the >>> profiler tutorial here: >>> https://mxnet.incubator.apache.org/versions/master/tutorials/python/profiler.html#profiling-custom-operators. >>> With that said, here I will have a brief introduction to the changes: >>> >>> >>> >>> Before, MXNet allowed user-defined custom operators, but their profiling >>> was not well supported. More specifically, the custom operator events as we >>> see in the dump file were way longer than the actual execution time of the >>> operators. Also, custom operators consist of pure python code and NDArray >>> operators (“sub-operators”), but at a particular point in a custom operator >>> event, we were not able to tell which ones were executing. >>> >>> >>> >>> To allow for a more informative custom operator profiling, my project >>> helps break a single custom operator call into several events: now, we have >>> individual events for both the “pure python” code, namely the forward() and >>> backward() functions, and the “sub-operators” , which are the NDArray >>> operators we call within forward() and backward() of the custom operator. >>> >>> >>> >>> All the new, fine-grained events will fall under a new domain/category >>> called “Custom Operator”. Also, each event there will have a prefix in its >>> name, which is the name we give to the custom operator. Below is a sample >>> dump file (opened in chome://tracing): >>> >>> >>> >>> [image: >>> /var/folders/fb/wknbw2r95zs5vkwrz9r80bzjsgk_ng/T/com.microsoft.Outlook/WebArchiveCopyPasteTempFiles/cidimage001.png@01D52DA3.0A2F7860] >>> >>> >>> >>> Note that: we need to make sure “profile_imperative” is set to True even >>> when we are using custom operators in symbolic mode. The reason is that >>> within custom operators, even in symbolic mode, sub-operators (and pure >>> python code) are still called imperatively. >>> >>> >>> >>> Should there be any issue using this new feature, please feel free to >>> tag me. Additionally, any feedback is welcome! >>> >>> >>> >>> Thanks, >>> >>> Zhaoqi >>> >>> >>> >>
Re: PR Merged: Custom Operator Profiling Enhancement
Thank you Zhaoqi for all your contributions to MXNet community. This feature of capturing detailed information for a custom operator in MXNet profiler output will be very helpful for lot of MXNet users who has custom operators and want to know more about the bottlenecks. Best, Sandeep On Wed, Jul 3, 2019, 3:01 PM Zhu Zhaoqi wrote: > Sorry about the image. I re-uploaded it: > [image: Screen Shot 2019-06-11 at 1.15.48 PM.png] > > Zhu Zhaoqi 于2019年7月3日周三 下午2:58写道： > >> Dear MXNet Community, >> >> >> >> A couple of weeks ago I sent out my custom operator profiling enhancement >> project RFC (https://github.com/apache/incubator-mxnet/issues/15241) to >> the community and got positive feedbacks, and now, the PR ( >> https://github.com/apache/incubator-mxnet/pull/15210) has been merged. A >> new section on profiling custom operators has also been added to the >> profiler tutorial here: >> https://mxnet.incubator.apache.org/versions/master/tutorials/python/profiler.html#profiling-custom-operators. >> With that said, here I will have a brief introduction to the changes: >> >> >> >> Before, MXNet allowed user-defined custom operators, but their profiling >> was not well supported. More specifically, the custom operator events as we >> see in the dump file were way longer than the actual execution time of the >> operators. Also, custom operators consist of pure python code and NDArray >> operators (“sub-operators”), but at a particular point in a custom operator >> event, we were not able to tell which ones were executing. >> >> >> >> To allow for a more informative custom operator profiling, my project >> helps break a single custom operator call into several events: now, we have >> individual events for both the “pure python” code, namely the forward() and >> backward() functions, and the “sub-operators” , which are the NDArray >> operators we call within forward() and backward() of the custom operator. >> >> >> >> All the new, fine-grained events will fall under a new domain/category >> called “Custom Operator”. Also, each event there will have a prefix in its >> name, which is the name we give to the custom operator. Below is a sample >> dump file (opened in chome://tracing): >> >> >> >> [image: >> /var/folders/fb/wknbw2r95zs5vkwrz9r80bzjsgk_ng/T/com.microsoft.Outlook/WebArchiveCopyPasteTempFiles/cidimage001.png@01D52DA3.0A2F7860] >> >> >> >> Note that: we need to make sure “profile_imperative” is set to True even >> when we are using custom operators in symbolic mode. The reason is that >> within custom operators, even in symbolic mode, sub-operators (and pure >> python code) are still called imperatively. >> >> >> >> Should there be any issue using this new feature, please feel free to tag >> me. Additionally, any feedback is welcome! >> >> >> >> Thanks, >> >> Zhaoqi >> >> >> >
PR Merged: Custom Operator Profiling Enhancement
Dear MXNet Community, A couple of weeks ago I sent out my custom operator profiling enhancement project RFC (https://github.com/apache/incubator-mxnet/issues/15241) to the community and got positive feedbacks, and now, the PR ( https://github.com/apache/incubator-mxnet/pull/15210) has been merged. A new section on profiling custom operators has also been added to the profiler tutorial here: https://mxnet.incubator.apache.org/versions/master/tutorials/python/profiler.html#profiling-custom-operators. With that said, here I will have a brief introduction to the changes: Before, MXNet allowed user-defined custom operators, but their profiling was not well supported. More specifically, the custom operator events as we see in the dump file were way longer than the actual execution time of the operators. Also, custom operators consist of pure python code and NDArray operators (“sub-operators”), but at a particular point in a custom operator event, we were not able to tell which ones were executing. To allow for a more informative custom operator profiling, my project helps break a single custom operator call into several events: now, we have individual events for both the “pure python” code, namely the forward() and backward() functions, and the “sub-operators” , which are the NDArray operators we call within forward() and backward() of the custom operator. All the new, fine-grained events will fall under a new domain/category called “Custom Operator”. Also, each event there will have a prefix in its name, which is the name we give to the custom operator. Below is a sample dump file (opened in chome://tracing): [image: /var/folders/fb/wknbw2r95zs5vkwrz9r80bzjsgk_ng/T/com.microsoft.Outlook/WebArchiveCopyPasteTempFiles/cidimage001.png@01D52DA3.0A2F7860] Note that: we need to make sure “profile_imperative” is set to True even when we are using custom operators in symbolic mode. The reason is that within custom operators, even in symbolic mode, sub-operators (and pure python code) are still called imperatively. Should there be any issue using this new feature, please feel free to tag me. Additionally, any feedback is welcome! Thanks, Zhaoqi
