jcf94 commented on a change in pull request #8808:
URL: https://github.com/apache/tvm/pull/8808#discussion_r696218247
##########
File path: src/runtime/contrib/tensorrt/tensorrt_runtime.cc
##########
@@ -125,17 +147,26 @@ class TensorRTRuntime : public JSONRuntimeBase {
/*! \brief Run inference using built engine. */
void Run() override {
+
Review comment:
> We have to build the engine first to know the input binding order that
tensorrt assigns to the inputs. It might not match TVM input signature directly.
> This is the same strategy used by TF-TRT
Yeah, I mean we can also just put all of the input infromation check here,
use something like `unordered_map<string, void*>`, `unordered_map<string,
size>` to create the calibrator and set_batch here. The trt `get_batch`
function has a parameter `const char* names[]`, that can be used when getting
data out.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
