guest2180 opened a new issue, #20040:
URL: https://github.com/apache/tvm/issues/20040

   ### Problem
   
   Relax TensorRT BYOC builds the TensorRT engine during the first runtime 
initialization or inference. For YOLO-style models, I observed a cold-start 
delay of approximately 20–80 seconds, depending on the model complexity and the 
number of BYOC subgraphs. The model currently produces a single TensorRT 
subgraph, but other models may be partitioned into multiple subgraphs.
   
   This is consistent with TensorRT documentation, which states that engine 
building can take significant time and is normally treated as an offline step:
   
   
https://docs.nvidia.com/deeplearning/tensorrt/latest/inference-library/c-api-docs.html#the-build-phase
   
   `TVM_TENSORRT_CACHE_DIR` avoids rebuilding on subsequent runs, but it is not 
very convenient for deployment:
   
   - The first process still pays the complete build cost.
   - A writable persistent cache directory is required.
   - `.plan` and `.meta` files must be managed separately.
   - Cache configuration is process-wide rather than model-local.
   
   For comparison, I tested the same YOLO model with TVM native CUDA. 
Compilation took about 24.2 seconds and export took 3.0 seconds, but these 
costs occurred offline. In a new process, loading the `.so` took about 319 ms 
and VM initialization took about 143 ms. There was no additional runtime build 
delay comparable to TensorRT BYOC.
   
   Is there a supported way to perform the expensive TensorRT engine-building 
work during compilation/export instead of the first runtime initialization?
   
   TensorRT plans are device- and version-dependent, but even a target-machine 
export workflow would be easier to manage than an implicit build during 
application startup.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to