ayeganov commented on issue #9939:
URL: https://github.com/apache/tvm/issues/9939#issuecomment-1014867710
@masahi thank you so much for your answers and time. If you are on patreon
or some open source donations platform, drop the link here so I can donate
toward your projects. That said, after looking into this for a little while I
was able to achieve what I need to do in Python, as my testing ground. Here is
roughly what I ended up doing:
```
onnx_model = onnx.load(model_path)
input_shape = (1, 4, 256, 144) # I don't care about dynamic batches, since
my case is pure inference of camera frames
shape_dict = {input_name: input_shape}
mod, params = relay.frontend.from_onnx(onnx_model, shape_dict)
target = "metal"
with tvm.transform.PassContext(opt_level=1):
executor = relay.build_module.create_executor(
"graph", mod, tvm.metal(1), target, params
).evaluate()
while True:
res = executor(tvm.nd.array(final_img, tvm.metal(1)))
tvm_output = res
time.sleep(0.016)
```
Since generally VMs come with a performance hit I like that I am able to
avoid it in this case. What I essentially want is to repeat these steps in C++
using `tvm` as a 3rd party library with the minimal number of headers. As my
starting point I used `apps/howto_deploy/cpp_deploy.cc`, but modifying the code
to build for Metal `howto_deploy/prepare_test_libs.py:47` and then running on
Metal device `apps/howto_deploy/cpp_deploy.cc:89` yielded an error that Metal
loader is not registered. Looking in `apps/howto_deploy/tvm_runtime_pack.cc`
leads to lines 79 and 80 for enabling Metal, which are ObjC files, at which
point the example fails to build due to ObjC syntax in
`/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/System/Library/Frameworks/Foundation.framework/Headers/NSObjCRuntime.h:523`.
So my question at this point is - can Metal be done in C++ at all?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
