cjolivier01 commented on a change in pull request #9931: Add axes support to 
Dropout for variational dropout in NLP
URL: https://github.com/apache/incubator-mxnet/pull/9931#discussion_r171930460

 File path: src/operator/nn/dropout-inl.h
 @@ -178,30 +184,17 @@ class DropoutOp {
    * \brief Dropout kernel, compute dropout tensor
-  struct DropoutKernel {
-    /*!
-     * \brief Dropout kernel function
-     * \param id Thread number (0-based representing count)
-     * \param gen Random number generator
-     * \param N Total number of items in the output
-     * \param step Step between items, related to parallelism
-     * \param dropout_out Output dropout values
-     * \param mask_out  Output mask (is multiplied to create dropout output, 
may be 0)
-     * \param input_data Input data to perform the dropout on
-     * \param pkeep Dropout rate (keep when the generated random number is 
less than this value)
-     */
+  struct BernoulliKernel {
+    /*! \brief Bernoulli kernel for generating mask */
     MSHADOW_XINLINE static void Map(int id,
                                     RandGenerator<xpu, DType> gen,
                                     const int N,
                                     const int step,
-                                    DType *dropout_out,
                                     DType *mask_out,
-                                    const DType *input_data,
                                     const real_t pkeep) {
       RNG_KERNEL_LOOP(xpu, DType, id, gen, N, step, {
         const real_t rand_num = static_cast<real_t>(genImpl.uniform());
         mask_out[i] = mshadow_op::threshold::Map<real_t>(rand_num, pkeep) * 
(1.0f / pkeep);
-        dropout_out[i] = input_data[i] * mask_out[i];
 Review comment:
   I am not saying it needs to be done, but have you considered merging this 
operation with the other kernel, perhaps by deriving from broadcast_kernel or 
passing a modified version of the mul OP to broadcast_kernel?
   Making two full passes across the memory is going to cause a performance hit 
due to both OMP overhead as well as CPU cache.

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:

With regards,
Apache Git Services

Reply via email to