zhanghang1989 commented on a change in pull request #9931: Add axes support to
Dropout for variational dropout in NLP
URL: https://github.com/apache/incubator-mxnet/pull/9931#discussion_r171931760
##########
File path: src/operator/nn/dropout-inl.h
##########
@@ -178,30 +184,17 @@ class DropoutOp {
/*!
* \brief Dropout kernel, compute dropout tensor
*/
- struct DropoutKernel {
- /*!
- * \brief Dropout kernel function
- * \param id Thread number (0-based representing count)
- * \param gen Random number generator
- * \param N Total number of items in the output
- * \param step Step between items, related to parallelism
- * \param dropout_out Output dropout values
- * \param mask_out Output mask (is multiplied to create dropout output,
may be 0)
- * \param input_data Input data to perform the dropout on
- * \param pkeep Dropout rate (keep when the generated random number is
less than this value)
- */
+ struct BernoulliKernel {
+ /*! \brief Bernoulli kernel for generating mask */
MSHADOW_XINLINE static void Map(int id,
RandGenerator<xpu, DType> gen,
const int N,
const int step,
- DType *dropout_out,
DType *mask_out,
- const DType *input_data,
const real_t pkeep) {
RNG_KERNEL_LOOP(xpu, DType, id, gen, N, step, {
const real_t rand_num = static_cast<real_t>(genImpl.uniform());
mask_out[i] = mshadow_op::threshold::Map<real_t>(rand_num, pkeep) *
(1.0f / pkeep);
- dropout_out[i] = input_data[i] * mask_out[i];
Review comment:
I am not sure I understand you clearly.
I separate the original dropout kernel into two parts: 1) BernoulliKernel 2)
broad_cast Mul
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
