Coder-nlper opened a new issue #7052: URL: https://github.com/apache/tvm/issues/7052
I use the following example, and modify it to load my model. https://github.com/apache/tvm/blob/main/apps/howto_deploy/cpp_deploy.cc I count the inference time of python and C++. c++ code: clock_t startTime, endTime; startTime=clock(); for (int i = 0; i < 36; ++i) { static_cast<int*>(x->data)[i] = array[i]; } // set the right input set_input("input_ids", x); // run the code run(); // get the output get_output(0, y); for (int i = 0; i < 36; ++i) { cout << static_cast<float*>(y->data)[i] << " "; } endTime = clock(); cout<<(double)(endTime - startTime)/CLOCKS_PER_SEC<<endl; python code: start = time() m.set_input("input_ids", x) m.run() tvm_output= m.get_output(0) predictions = np.squeeze(tvm_output.asnumpy()) end = time() print("cost {}".format(end - start)) the result: python cost : 0.00779s c++ cost: 0.033162s c++ is much slower than python, I can not figure out why???? my platform info: cpu: Intel(R) Core(TM) i7-6700 CPU @ 3.40GHz os: Linux x-np 5.4.0-56-generic #62~18.04.1-Ubuntu x86_64 tvm: 0.8 llvm: 6.0 gcc: 7.5.0 ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
