Re: [apache/incubator-mxnet] [RFC] Custom Operator Part 2 (#17006)
Speed. All those `std::string` and `std::unordered_map` objects don't come cheaply. I compared an integrated fork with a custom operator. https://github.com/kpuatamazon/incubator-mxnet/tree/intgemm integrated version end-to-end Sockeye performance (based on 1.6.0): ``` real2m57.962s user7m3.986s sys 0m6.724s ``` Custom operator version (based on 1.7.x. because it had to be for custom operators): ``` real3m16.879s user7m43.727s sys 0m8.273s ``` Conditions: `unset MXNET_ENGINE_TYPE; export OMP_NUM_THREADS=2; numactl -C 0-7 translate.sh` Both were compiled with the MKL backend hack for the remaining fp32 operations. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/apache/incubator-mxnet/issues/17006#issuecomment-636870342
Re: [apache/incubator-mxnet] [RFC] Custom Operator Part 2 (#17006)
Custom ops should be able to set the inplace property. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/apache/incubator-mxnet/issues/17006#issuecomment-616807857
Re: [apache/incubator-mxnet] [RFC] Custom Operator Part 2 (#17006)
@larroy Users may need matrix operators and DNN Op(e.g. ReLU, Conv) when writing a custom Op. Although they can implement it by third-party libraries, it is more convenient to use the built-in functions in MXNet. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/apache/incubator-mxnet/issues/17006#issuecomment-569467244
Re: [apache/incubator-mxnet] [RFC] Custom Operator Part 2 (#17006)
We should create a namespace for the stuff in the lib_api.h file as suggested by @larroy: https://github.com/apache/incubator-mxnet/pull/15760/files#r311756416 -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/apache/incubator-mxnet/issues/17006#issuecomment-569422074
Re: [apache/incubator-mxnet] [RFC] Custom Operator Part 2 (#17006)
@wkcn could you explain your suggestion? calling gemm back into the framework which gets dispatched to GPU or CPU? -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/apache/incubator-mxnet/issues/17006#issuecomment-569131388
Re: [apache/incubator-mxnet] [RFC] Custom Operator Part 2 (#17006)
Need to include a fix for the test error https://github.com/apache/incubator-mxnet/pull/15921#pullrequestreview-328686634 -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/apache/incubator-mxnet/issues/17006#issuecomment-563085461
Re: [apache/incubator-mxnet] [RFC] Custom Operator Part 2 (#17006)
Hi @samskalicky , thank you for the contribution! I have several suggestions. - custom GPU operators 1. Provide CUDA stream in `OpResource`. 2. Share the same function on CPU and GPU. Users can discriminate the context by `MXTensor::dltensor::ctx` - Call framework specific math helper It is important for a custom operator. Users may call gemm, even convolution op in custom op. Thanks. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/apache/incubator-mxnet/issues/17006#issuecomment-562898682
[apache/incubator-mxnet] [RFC] Custom Operator Part 2 (#17006)
## Description Request for comments on the next PR for enhancing custom operator support Heres some suggestions from the initial PR (Part 1): - custom GPU operators - Random number generator resource request - sparse data types - migrate lambda functions in MXLoadLib in src/c_api/c_api.cc to classes defined elsewhere - Documentation, add the "library" python package to the namespace to the doc: https://mxnet.apache.org/api/python/docs/api/ ? ## References - initial PR (Part 1): #15921 -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/apache/incubator-mxnet/issues/17006
