[GitHub] asitstands commented on issue #10575: [MXNET-145] Remove global PRNGs of numpy and python used in mx.image package

[GitBox]

asitstands commented on issue #10575: [MXNET-145] Remove global PRNGs of numpy 
and python used in mx.image package
URL: https://github.com/apache/incubator-mxnet/pull/10575#issuecomment-382063974
 
 
   I experimented with the following two functions. The first one takes 37s and 
the second one takes 28s to run the above `ImageIter` test. So the second one 
has the same performance with the implementation in the master using python 
RNG. The difference is from the serialization by the engine. I didn't expect 
that it causes such a large difference. To avoid the serialization, we need an 
independent RNG that is not intended for sharing. But I think that it should 
not be the global python/numpy RNG. I'll make a separate PR for this.
   ```c++
   // Faster than sampling operator but still slow
   int rand(mx_float low, mx_float high, mx_float* out) {
     API_BEGIN();
     Context ctx = Context::CPU();
     Engine::VarHandle var = Engine::Get()->NewVariable();
     mxnet::Resource resource = mxnet::ResourceManager::Get()->Request(
       ctx, ResourceRequest::kRandom);
     Engine::Get()->PushSync([low, high, out, resource](RunContext rctx) {
       mshadow::Random<mxnet::cpu, float> *prnd = 
         resource.get_random<mxnet::cpu, float>(rctx.get_stream<mxnet::cpu>());
       mshadow::Tensor<mxnet::cpu, 1, float> tmp(out, mshadow::Shape1(1));
       prnd->SampleUniform(&tmp, low, high);
     }, ctx, {}, {var, resource.var},
     FnProperty::kNormal, 0, PROFILER_MESSAGE_FUNCNAME);
     Engine::Get()->WaitForVar(var);
     Engine::Get()->DeleteVariable([](mxnet::RunContext) {}, mxnet::Context(), 
var);
     API_END();
   }
   
   // Fast but not safe as `prnd` is shared
   int rand(mx_float low, mx_float high, mx_float* out) {
     API_BEGIN();
     Context ctx = Context::CPU();
     mxnet::Resource resource = mxnet::ResourceManager::Get()->Request(
       ctx, ResourceRequest::kRandom);
     mshadow::Random<mxnet::cpu, float> *prnd = 
         resource.get_random<mxnet::cpu, float>(nullptr);
     mshadow::Tensor<mxnet::cpu, 1, float> tmp(out, mshadow::Shape1(1));
     prnd->SampleUniform(&tmp, low, high);
     API_END();
   }
   ```

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services