jsheng-jian opened a new pull request, #11614: URL: https://github.com/apache/tvm/pull/11614
Before the fix, after `GetCachedEnginesFromDisk()`, `max_batch_size_` is -1 which results in the [batch_size <= max_batch_size_](https://github.com/apache/tvm/blob/main/src/runtime/contrib/tensorrt/tensorrt_runtime.cc#L258) to be `False`. Engine will be rebuilt because [find_engine_flag](https://github.com/apache/tvm/blob/main/src/runtime/contrib/tensorrt/tensorrt_runtime.cc#L276) is `False`. Log before the fix: ``` [19:41:41] /home/ubuntu/neo-ai/apache_tvm/tvm/src/runtime/contrib/tensorrt/tensorrt_runtime.cc:356: Loading cached TensorRT engine from ./tvmgen_default_tensorrt_main_0_fp32.plan [19:41:43] /home/ubuntu/neo-ai/apache_tvm/tvm/src/runtime/contrib/tensorrt/tensorrt_runtime.cc:379: finished saving engine and context ... [19:41:43] /home/ubuntu/neo-ai/apache_tvm/tvm/src/runtime/contrib/tensorrt/tensorrt_runtime.cc:356: Loading cached TensorRT engine from ./tvmgen_default_tensorrt_main_1_fp32.plan [19:41:43] /home/ubuntu/neo-ai/apache_tvm/tvm/src/runtime/contrib/tensorrt/tensorrt_runtime.cc:379: finished saving engine and context ... [19:41:43] /home/ubuntu/neo-ai/apache_tvm/tvm/src/runtime/contrib/tensorrt/tensorrt_runtime.cc:287: Building new TensorRT engine for subgraph tvmgen_default_tensorrt_main_0 with batch size 1 ``` Log after the fix: ``` [19:42:46] /home/ubuntu/neo-ai/apache_tvm/tvm/src/runtime/contrib/tensorrt/tensorrt_runtime.cc:356: Loading cached TensorRT engine from ./tvmgen_default_tensorrt_main_0_fp32.plan [19:42:48] /home/ubuntu/neo-ai/apache_tvm/tvm/src/runtime/contrib/tensorrt/tensorrt_runtime.cc:380: finished loading engine and context ... [19:42:48] /home/ubuntu/neo-ai/apache_tvm/tvm/src/runtime/contrib/tensorrt/tensorrt_runtime.cc:356: Loading cached TensorRT engine from ./tvmgen_default_tensorrt_main_1_fp32.plan [19:42:48] /home/ubuntu/neo-ai/apache_tvm/tvm/src/runtime/contrib/tensorrt/tensorrt_runtime.cc:380: finished loading engine and context ... ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
