apeforest commented on issue #16735: Use single-bit for mask in dropout operator URL: https://github.com/apache/incubator-mxnet/pull/16735#issuecomment-586737774 > > Why does it increase memory load? > > If there are N elements, per the Bernoulli distribution generation in VSL, we still need to allocate memory and write `N*4` bytes to it. To generate bit mask, we need load the `N*4` bytes back and write `N/8` bytes with bits. The memory for bit-mask is not extra memory. `N*sizeof(DType)` was used in the master branch: https://github.com/apache/incubator-mxnet/blob/master/src/operator/nn/dropout.cc#L124 So for the MKL dropout case, master branch uses memory `N*4` + `N*sizeof(DType)` vs. this PR `N*4` + `N/8`. This memory reduction is verified through the MXNet profiler results reported in the PR description section.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
