mkolod edited a comment on issue #11325: [MXNET-703] TensorRT runtime integration URL: https://github.com/apache/incubator-mxnet/pull/11325#issuecomment-408247485 @piiswrong I'm adding a check to the SimpleBind call in the backend. The forward-pass-only workloads such as with TensorRT work fine if `grad_req` is set to `null` in Python. If grad_req is not set or is set to `write` or `add`, `SimpleBind` fails without any useful message. So, now I do the following: 1) If `MXNET_USE_TENSORRT` is set, a check is made that all elements of `grad_req_type_vec` have a value of kNullOp. If they don't then an exception is thrown with a helpful message, which suggests 2 courses of action: - if this is an inference-only graph, the user is told to set `grad_req` to `null` - if the graph is a training graph, then the user is told to unset MXNET_USE_TENSORRT Here is the message that will be printed: [17:50:46] src/executor/graph_executor.cc:1163: MXNET_USE_TENSORRT set but graph is not inference-only. If it is an inference graph, set grad_req to null during simple_bind call. If it is a training graph, unset the MXNET_USE_TENSORRT env variable 2) If `MXNET_USE_TENSORRT` is set and `grad_req` is `null`, then calling `backward` on the symbol will have no effect. Also, if `grad_req` is `null`, the `forward()` call will work whether `is_train` is `True` or `False`. This ensures that the user isn't burdened with configuration, as long as the binding is explicitly set for an inference-only graph (`grad_req = null`). I hope this is what you're looking for in terms of guarding that the user only run an inference graph, and that there is a helpful message as to what to do if the user tries running a training graph. The code doing the check is [here](https://github.com/mkolod/incubator-mxnet/commit/eebd373f0a8c863b96e7211311e50f6aa2ce9f13).
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
