anirudh2290 commented on issue #13598: More fine-grained operator implementation dispatch & memory planning flow URL: https://github.com/apache/incubator-mxnet/issues/13598#issuecomment-448139994 This is a good proposal for separator of operator implementations for different accelerators. One thing that can be done to lessen the overhead of cross acc implementation in FInferStorageEx is registration of supports functions for different accelerators. Something like the following: ``` .set_attr<SupportsAccl>("MKLDNN", IsMKLDNNSupported) .set_attr<SupportAccl>("CUDNN", IsCUDNNSupported) ``` The default implementation for FInferStorageEx will then be: ``` void FInferStorageTypeEx(const std::vector<TShape>& in_shapes, const std::vector<int>& in_types, const std::vector<int>& in_stype, const std::vector<TShape>& out_shape, const std::vector<int>& out_type, std::vector<int>* out_stype, int dev_mask, NodeAttrs* attrs, // mutable DispatchMode* dispatch_mode // mutable) { // GPU if (dev_mask == kGPU) { out_stype[0] = kDefaultStorage; dispatch_mode = kFCompute; #if MXNET_USE_CUDNN if (SupportsCUDNN(...)) { attrs.exec_func = CovolutionComputeImplCUDNN; } else { attrs.exec_func = CovolutionComputeImplGPU; } #else attrs.exec_func = CovolutionComputeImplGPU; #endif // CPU } else { #if MXNET_USE_MKLDNN if (SupportsMKLDNN(...)) { attrs.exec_func = CovolutionComputeMKL out_stype[0] = kDefaultStorage; dispatch_mode = kFComputeEx; } else { attrs.exec_func = ConvolutionComputeCPU } #else attrs.exec_func = CovolutionComputeCPU ... #endif } ``` This way the devs working on different accelerators for most cases, will only have to worry about registering FCompute and Supports implementations.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
