eleflea opened a new issue #7782:
URL: https://github.com/apache/tvm/issues/7782
Hi.
I try to deploy yolov5 model with C++ API. I have export a `.so` file with
`lib.export_library`. Below is a part of the C++ code.
```C++
class YoloDetection
{
public:
YoloDetection() {}
~YoloDetection();
YoloDetection(string, string);
vector<Det> predict(const cv::Mat &);
void load(string, string);
float conf_thresh = 0.4;
float nms_thresh = 0.6;
int max_dets = 300;
int max_wh = 4096;
protected:
string lib_path = "";
DLContext device;
DLTensor *input_tensor;
tvm::runtime::Module handler;
};
YoloDetection::YoloDetection(string lib_path, string device):
lib_path(lib_path)
{
this->device = device == "gpu" ? DLContext{kDLGPU, 0} :
DLContext{kDLCPU, 0};
tvm::runtime::Module lib = tvm::runtime::Module::LoadFromFile(lib_path);
int64_t in_shape[4] = {1, 3, IN_SIZE_H, IN_SIZE_W};
TVMArrayAlloc(in_shape, 4, kDLFloat, 32, 1, kDLGPU, 0,
&this->input_tensor);
this->handler = lib.GetFunction("default")(this->device);
}
vector<Det> YoloDetection::predict(const cv::Mat &image) {
auto pimg = pre_process(image);
auto set_input = handler.GetFunction("set_input");
auto get_output = handler.GetFunction("get_output");
auto run = handler.GetFunction("run");
TVMArrayCopyFromBytes(this->input_tensor, (void *)pimg.data(),
IN_NUMEL*sizeof(float));
set_input("input", this->input_tensor);
run();
tvm::runtime::NDArray out = get_output(0);
size_t s = 1;
for (auto i = 0; i < out->ndim; ++i)
{
s *= out->shape[i];
}
if (this->device.device_type != kDLCPU) {
out = out.CopyTo(DLContext{kDLCPU, 0});
}
auto dets = post_process(static_cast<float *>(out->data), s,
this->conf_thresh, this->nms_thresh, image.size());
return dets;
}
```
But when I run this code, it shows:
```
Floating point exception (core dumped)
```
I found that error was in `run();` line.
I moved `lib.GetFunction("default")(this->device)` to `predict` method. The
program is fine, but got a time overhead due to that function call (for ~26.5ms
I believe).
So how to avoid GetFunction of default in predict, or there is other best
practices?
Thank you very much!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
