areusch commented on a change in pull request #7785:
URL: https://github.com/apache/tvm/pull/7785#discussion_r608902101
##########
File path: python/tvm/micro/model_library_format.py
##########
@@ -86,10 +86,13 @@ def _build_memory_map(graph_json):
list :
A list with one entry per storage id describing that memory.
"""
- graph = json.loads(graph_json)
+ memory_map = []
+ if graph_str.startswith("primfn"):
Review comment:
Okay yeah I understand what you're doing here, but I agree with
@manupa-arm that we should not overload graph_json. `tvm.relay.build` is
currently specific to building artifacts to be launched by GraphExecutor. I
don't think it should remain specific to that, but that's our API surface now.
Currently, although you can unpack `graph_json, lib, params`, this is
actually deprecated behavior. The documented return value is a `Module`
implementing the `GraphExecutorFactoryModule` interface, which is wrapped by a
Python-side `GraphExecutorFactoryModule` class. I think we should create a
corresponding `AotExecutorFactoryModule` and update
`tvm.micro.export_model_library_format` to recognize that and consume
appropriately. Though this is not proper duck-typing, I think we eventually
intend to promote `export_library_model_format` to a member function of
`*ExecutorFactoryModule` once it's completely solidified and gains traction
outside of the C runtime (e.g. is supported in the c++ cases for
`apps/bundle_deploy` and perhaps with GPUs).
With regards to the memory map: I'd like to separate the tensor pinning in
this PR from the AOT change and instead (in the short term) continue to rely on
TVMBackendAllocWorkspace for AOT-allocated tensors. I agree we need to do
something _like_ tensor pinning, but I prefer if we can resolve that in a
separate RFC/PR inside GraphPlanMemory. Such an RFC should also broaden
GraphPlanMemory and introduce concepts e.g. `storage_id` as a user-facing
identifier so that developers understand how to communicate tensor placement to
the TVM runtime.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
