Zha0q1 opened a new issue #20296: URL: https://github.com/apache/incubator-mxnet/issues/20296
I was trying to run a c++ bert example with multi-threading and naive engine with `MXPredCreateMultiThread` (currently this api can only run with naive engine). However the lazy initialization is set to `true` in `MXPredCreateMultiThread` https://github.com/apache/incubator-mxnet/blob/v1.x/src/c_api/c_predict_api.cc#L430. This means that executor binding will not happen when we create the predictors in https://github.com/apache/incubator-mxnet/blob/v1.x/src/c_api/c_predict_api.cc#L293, but rather when we run inference for the first time with `MXPredForward` https://github.com/apache/incubator-mxnet/blob/v1.x/src/c_api/c_predict_api.cc#L75. This behavior is not thread safe since two `Executor::Bind()` calls can overlap from two different threads. I have observed that in about every 5 runs one would error out (4 threads to run inference in total). The issue happened right in `Executor::Bind()`, and more specifically, in `exec::BuildSubgraph()` https://github.com/apache/incubator-mxnet/blob/v1.x/src/executor/graph_executor.cc#L2074 for MKLDNN. After I change the lazy initialization to `false` https://github.com/apache/incubator-mxnet/blob/v1 .x/src/c_api/c_predict_api.cc#L430 this got resolved. Any ideas on why lazy initialization is set to true originally? I can open a pr to toggle it to false. @samskalicky @szha @josephevans -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
