areusch commented on a change in pull request #7785:
URL: https://github.com/apache/tvm/pull/7785#discussion_r610796796
##########
File path: include/tvm/runtime/crt/aot_executor.h
##########
@@ -68,6 +67,9 @@ extern "C" {
typedef struct {
} tvm_context_t;
+typedef int32_t(tvm_function_t)(void* args, void* arg_type_ids, int32_t
num_args,
Review comment:
can you use `TVMBackendPackedCFunc`?
##########
File path: include/tvm/target/target_kind.h
##########
@@ -140,6 +140,12 @@ static constexpr const char* kTvmRuntimeCpp = "c++";
/*! \brief Value used with --runtime in target specs to indicate the C
runtime. */
static constexpr const char* kTvmRuntimeCrt = "c";
+/*! \brief Value used with --executor in target specs to indicate the graph
executor. */
+static constexpr const char* kTvmExecutorGraph = "graph";
Review comment:
@jroesch @tqchen can you look at this interface and see if you agree?
##########
File path: python/tvm/relay/backend/executor_factory.py
##########
@@ -0,0 +1,216 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Graph executor factory."""
+import warnings
+from ..._ffi.base import string_types
+from ..._ffi.registry import get_global_func
+from ...runtime import ndarray
+from tvm import tir
+
+
+class ExecutorFactoryModule:
+ """Graph executor factory module.
+ This is a module of graph executor factory
+
+ Parameters
+ ----------
+ graph_str : str
+ Depending on executor:
+ * Graph executor: the graph to be deployed in json format output by
graph compiler.
+ The graph can contain operator(tvm_op) that points to the name of
+ PackedFunc in the libmod.
+ * AOT executor: the string representation of the TIR executor
PrimFunction
+ target : tvm.Target
+ The Target used to build this module.
+ libmod : tvm.Module
+ The module of the corresponding function
+ libmod_name: str
+ The name of module
+ params : dict of str to NDArray
+ The parameters of module
+ """
+
+ def get_internal_repr(self):
Review comment:
can you document these since it is now an interface and raise
NotImplementedError? If these are required to be implemented, can you make this
class use abc.ABCMeta and decorate these functions as abc.abstractmethod?
##########
File path: python/tvm/relay/build_module.py
##########
@@ -197,6 +201,9 @@ def get_params(self):
ret[key] = value.data
return ret
+ def get_executor(self):
Review comment:
add a docstring
##########
File path: src/runtime/crt/common/aot_backend_api.c
##########
@@ -0,0 +1,59 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+// LINT_C_FILE
+
+#include <assert.h>
+#include <inttypes.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <tvm/runtime/c_backend_api.h>
+#include <tvm/runtime/crt/error_codes.h>
+#include <tvm/runtime/crt/logging.h>
+#include <tvm/runtime/crt/platform.h>
+
+#include "crt_config.h"
+
+void* TVMBackendAllocWorkspace(int device_type, int device_id, uint64_t
nbytes, int dtype_code_hint,
Review comment:
I feel like these implementations are basically identical to those in
`src/runtime/common/crt_backend_api.c`. I prefer not to introduce a copy of
these functions and rather split the library `src/runtime/common` into e.g.
`src/runtime/crt/backend` and `src/runtime/crt/runtime`.
##########
File path: python/tvm/relay/build_module.py
##########
@@ -245,8 +252,10 @@ def build(ir_mod, target=None, target_host=None,
params=None, mod_name="default"
Returns
-------
- graph_json : str
- The json string that can be accepted by graph executor.
+ graph : str
Review comment:
let's keep graph_json
##########
File path: python/tvm/micro/model_library_format.py
##########
@@ -132,14 +135,25 @@ def export_model_library_format(mod:
graph_executor_factory.GraphExecutorFactory
Path to the .tar archive to generate.
"""
tempdir = utils.tempdir()
+ is_aot = False
+ for v in mod.target.values():
+ if v.attrs.get("executor", "graph_runtime") == "aot":
+ is_aot = True
+ break
+
+ runtime = ["graph"]
+ if is_aot:
+ runtime = ["aot"]
+
metadata = {
"version": 1,
"model_name": mod.libmod_name,
"export_datetime": datetime.datetime.now().strftime("%Y-%m-%d
%H:%M:%SZ"),
- "memory": _build_memory_map(mod.graph_json),
+ "memory": _build_memory_map(mod.graph),
"target": {int(k): str(v) for k, v in mod.target.items()},
- "runtimes": ["graph"],
+ "runtimes": runtime,
Review comment:
yeah it looks like we missed a rename--it should be executors. Let's do
this in another PR, I think we may need to rev the MLF version
##########
File path: python/tvm/relay/backend/executor_factory.py
##########
@@ -0,0 +1,216 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Graph executor factory."""
+import warnings
+from ..._ffi.base import string_types
+from ..._ffi.registry import get_global_func
+from ...runtime import ndarray
+from tvm import tir
+
+
+class ExecutorFactoryModule:
+ """Graph executor factory module.
Review comment:
clarify the comment
##########
File path: python/tvm/relay/backend/executor_factory.py
##########
@@ -0,0 +1,216 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Graph executor factory."""
Review comment:
can you clarify the comment?
##########
File path: src/runtime/crt/memory/stack_memory.c
##########
@@ -0,0 +1,47 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+// LINT_C_FILE
+
+#include <tvm/runtime/crt/stack_memory.h>
+
+void* MemoryManager_Allocate(tvm_workspace_t *tvm_runtime_workspace, int32_t
nbytes) {
Review comment:
Should this be StackMemoryManager_Allocate?
##########
File path: python/tvm/relay/backend/executor_factory.py
##########
@@ -0,0 +1,216 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Graph executor factory."""
+import warnings
+from ..._ffi.base import string_types
+from ..._ffi.registry import get_global_func
+from ...runtime import ndarray
+from tvm import tir
+
+
+class ExecutorFactoryModule:
+ """Graph executor factory module.
+ This is a module of graph executor factory
+
+ Parameters
+ ----------
+ graph_str : str
+ Depending on executor:
+ * Graph executor: the graph to be deployed in json format output by
graph compiler.
+ The graph can contain operator(tvm_op) that points to the name of
+ PackedFunc in the libmod.
+ * AOT executor: the string representation of the TIR executor
PrimFunction
+ target : tvm.Target
+ The Target used to build this module.
+ libmod : tvm.Module
+ The module of the corresponding function
+ libmod_name: str
+ The name of module
+ params : dict of str to NDArray
+ The parameters of module
+ """
+
+ def get_internal_repr(self):
+ return self.internal_repr
+
+ def get_params(self):
+ return None
+
+ def get_lib(self):
+ return None
+
+ def __getitem__(self, item):
+ return None
+
+ def __iter__(self):
+ warnings.warn(
+ "legacy graph executor behavior of producing json / lib / params
will be "
+ "removed in the next release."
+ " Please see documents of tvm.contrib.graph_executor.GraphModule
for the "
+ " new recommended usage.",
+ DeprecationWarning,
+ 2,
+ )
+ return self
+
+ def save_config(self, config_path):
+ pass
+
+ def __next__(self):
+ if self.iter_cnt > 2:
+ raise StopIteration
+
+ objs = [self.internal_repr, self.lib, self.params]
+ obj = objs[self.iter_cnt]
+ self.iter_cnt += 1
+ return obj
+
+
+class AOTExecutorFactoryModule(ExecutorFactoryModule):
+ """Graph executor factory module.
+ This is a module of graph executor factory
+
+ Parameters
+ ----------
+ graph_str : str
+ Depending on executor:
+ * Graph executor: the graph to be deployed in json format output by
graph compiler.
+ The graph can contain operator(tvm_op) that points to the name of
+ PackedFunc in the libmod.
+ * AOT executor: the string representation of the TIR executor
PrimFunction
+ target : tvm.Target
+ The Target used to build this module.
+ libmod : tvm.Module
+ The module of the corresponding function
+ libmod_name: str
+ The name of module
+ params : dict of str to NDArray
+ The parameters of module
+ """
+
+ def __init__(self, ir_mod, target, runner_function, libmod, libmod_name,
params):
+ assert isinstance(runner_function, tir.PrimFunc)
+ args = []
+ for k, v in params.items():
+ args.append(k)
+ args.append(ndarray.array(v))
+
+ self.ir_mod = ir_mod
+ self.target = target
+ self.internal_repr = runner_function
+ self.lib = libmod
+ self.libmod_name = libmod_name
+ self.params = params
+ self.iter_cnt = 0
+
+ # Sometimes we want to get params explicitly.
+ # For example, we want to save its params value to
+ # an independent file.
+ def get_params(self):
+ return self.params
+
+ def get_runner_function(self):
+ return self.internal_repr
+
+ def get_lib(self):
+ return self.lib
+
+ def __getitem__(self, item):
+ return self.module.__getitem__(item)
+
+ def __iter__(self):
Review comment:
seems like this and `__next__` will fall over in this case? can you add
some tests or delete?
##########
File path: tests/python/relay/aot/infra.py
##########
@@ -83,7 +84,23 @@ def create_main(test_name, input_list, output_list,
output_path):
main_file.write('#include "output_data%i.h"\n' % i)
main_file.write("extern tvm_model_t network;\n")
- main_file.write("extern tvm_workspace_t *tvm_runtime_workspace;\n")
+ main_file.write("tvm_workspace_t app_workspace;\n")
+ main_file.write(
+ """
+tvm_crt_error_t TVMPlatformMemoryAllocate(size_t num_bytes, DLDevice dev,
void** out_ptr){
Review comment:
nit: space between `) {` here and on 94
##########
File path: python/tvm/relay/backend/executor_factory.py
##########
@@ -0,0 +1,216 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Graph executor factory."""
+import warnings
+from ..._ffi.base import string_types
+from ..._ffi.registry import get_global_func
+from ...runtime import ndarray
+from tvm import tir
+
+
+class ExecutorFactoryModule:
+ """Graph executor factory module.
+ This is a module of graph executor factory
+
+ Parameters
+ ----------
+ graph_str : str
+ Depending on executor:
+ * Graph executor: the graph to be deployed in json format output by
graph compiler.
+ The graph can contain operator(tvm_op) that points to the name of
+ PackedFunc in the libmod.
+ * AOT executor: the string representation of the TIR executor
PrimFunction
+ target : tvm.Target
+ The Target used to build this module.
+ libmod : tvm.Module
+ The module of the corresponding function
+ libmod_name: str
+ The name of module
+ params : dict of str to NDArray
+ The parameters of module
+ """
+
+ def get_internal_repr(self):
+ return self.internal_repr
+
+ def get_params(self):
+ return None
+
+ def get_lib(self):
+ return None
+
+ def __getitem__(self, item):
+ return None
+
+ def __iter__(self):
+ warnings.warn(
+ "legacy graph executor behavior of producing json / lib / params
will be "
+ "removed in the next release."
+ " Please see documents of tvm.contrib.graph_executor.GraphModule
for the "
+ " new recommended usage.",
+ DeprecationWarning,
+ 2,
+ )
+ return self
+
+ def save_config(self, config_path):
+ pass
+
+ def __next__(self):
+ if self.iter_cnt > 2:
+ raise StopIteration
+
+ objs = [self.internal_repr, self.lib, self.params]
+ obj = objs[self.iter_cnt]
+ self.iter_cnt += 1
+ return obj
+
+
+class AOTExecutorFactoryModule(ExecutorFactoryModule):
+ """Graph executor factory module.
Review comment:
also clarify the comment
##########
File path: python/tvm/micro/model_library_format.py
##########
@@ -73,7 +73,7 @@ def _populate_codegen_dir(mod, codegen_dir: str):
dso_mod.save(file_name)
-def _build_memory_map(graph_json):
+def _build_memory_map(graph_str):
Review comment:
can you revert these changes?
##########
File path: include/tvm/runtime/crt/stack_memory.h
##########
@@ -0,0 +1,55 @@
+/*
Review comment:
let's rename the other `include/tvm/runtime/crt/memory.h` to qualify it.
`include/tvm/runtime/crt/page_memory.h`? or,
`include/tvm/runtime/crt/memory/page_allocator.h` and equivalent
`stack_allocator.h`?
##########
File path: python/tvm/relay/build_module.py
##########
@@ -289,11 +296,17 @@ def build(ir_mod, target=None, target_host=None,
params=None, mod_name="default"
with tophub_context:
bld_mod = BuildModule()
+ runtime_repr, runtime_mod, params = bld_mod.build(mod=ir_mod,
target=target, params=params)
+
+ if bld_mod.get_executor() == "aot":
+ executor_factory = _executor_factory.AOTExecutorFactoryModule(
+ ir_mod, target, runtime_repr, runtime_mod, mod_name, params
+ )
+ else:
Review comment:
add an `else if ... == "graph":` and an `else: assert False, <error
message>`
##########
File path: src/relay/backend/build_module.cc
##########
@@ -59,17 +62,50 @@ struct BuildOutput {
*/
struct GraphCodegen {
public:
- GraphCodegen() {
- auto pf = GetPackedFunc("relay.build_module._GraphExecutorCodegen");
- mod = (*pf)();
+ explicit GraphCodegen(Target target_host) : target_host_(target_host) {
+ const String executor_str =
+ target_host->GetAttr<String>("executor").value_or(kTvmExecutorGraph);
+ if (executor_str == kTvmExecutorGraph) {
+ executor_ = Executor::Graph;
+ auto pf = GetPackedFunc("relay.build_module._GraphExecutorCodegen");
+ mod = (*pf)();
+ } else if (executor_str == kTvmExecutorAot) {
+ executor_ = Executor::Aot;
+ auto pf = GetPackedFunc("relay.build_module._GraphAOTCodegen");
+ mod = (*pf)();
+ } else {
+ LOG(FATAL) << "Executor not supported";
Review comment:
can you print the executor `<< "Executor " << executor_str << " not
supported" here and below
##########
File path: include/tvm/runtime/crt/stack_memory.h
##########
@@ -0,0 +1,55 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+// LINT_C_FILE
+#ifndef TVM_RUNTIME_CRT_STACK_MEMORY_H_
+#define TVM_RUNTIME_CRT_STACK_MEMORY_H_
+#include <stddef.h>
+#include <stdint.h>
+
+#include "error_codes.h"
+
+/*! Memory alignment for allocator */
+
+#ifndef TVM_RUNTIME_ALLOC_ALIGNMENT
+#define TVM_RUNTIME_ALLOC_ALIGNMENT 16
+#endif
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+typedef struct {
+ uint8_t* next_alloc; /** Pointer to the next block of bytes to allocate */
Review comment:
is it a block of bytes because of alignment?
##########
File path: src/relay/backend/build_module.cc
##########
@@ -43,12 +43,15 @@ namespace backend {
using TargetsMap = Map<tvm::Integer, tvm::Target>;
using namespace tvm::relay::transform;
+enum class Executor { Graph, Aot };
Review comment:
can you add a docstring? Also, let's name this ExecutorType or
something, since Executor sounds like an interface name.
##########
File path: src/relay/backend/build_module.cc
##########
@@ -43,12 +43,15 @@ namespace backend {
using TargetsMap = Map<tvm::Integer, tvm::Target>;
using namespace tvm::relay::transform;
+enum class Executor { Graph, Aot };
+
/*!
* \brief Output of building module
*
*/
struct BuildOutput {
- std::string graph_json;
+ std::string graph;
Review comment:
keep with graph_json
##########
File path: include/tvm/runtime/crt/aot_executor.h
##########
@@ -0,0 +1,98 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file include/tvm/runtime/crt/aot/tvm_executor.h
+ * \brief TVM Executor for the Ahead-of-Time Runtime
+ *
+ * AOT models are described by the TVM model descriptor format
+ * which can be passed to tvm_runtime_run. These descriptors will be
+ * generated by the AOT compilation process. This can optionally be
+ * augmented with platform specific context to be passed to the TVM
+ * operators.
+ *
+ * Example:
+ * extern tvm_model_t my_network;
+ * int main() {
+ * void* data = get_data();
+ * void* output[4] = {0, 0, 0, 0};
+ * void* inputs = {data};
+ * void* outputs = {output};
+ * tvm_context_t my_context = {
+ * .driver = ...;
+ * };
+ * tvm_runtime_run(
+ * &my_network,
+ * inputs,
+ * outputs
+ * &my_context
+ * );
+ * return 0;
+ * }
+ */
+
+#ifndef TVM_RUNTIME_CRT_AOT_TVM_EXECUTOR_H_
+#define TVM_RUNTIME_CRT_AOT_TVM_EXECUTOR_H_
+
+#include <stdint.h>
+
+#include "error_codes.h"
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+/*!
+ * \brief Context information for future integrations
+ * which is passed through to the operators.
+ *
+ * \note Can be used for drivers and platform specific information.
+ */
+typedef struct {
+} tvm_context_t;
+
+typedef int32_t(tvm_function_t)(void* args, void* arg_type_ids, int32_t
num_args,
+ void* out_ret_value, void* out_ret_tcode,
void* resource_handle);
+
+/*!
+ * \brief TVM Model descriptor to describe the
+ * model to the runtime.
+ */
+typedef struct {
+ uint32_t num_input_tensors; /** Number of expected input tensors */
+ uint32_t num_output_tensors; /** Number of expected output tensors */
+ tvm_function_t* run_func; /** Generated model function, called through
tvm_runtime_run */
+} tvm_model_t;
+
+/*!
+ * \brief Main entry point for
Review comment:
can you complete the docstring?
##########
File path: python/tvm/relay/build_module.py
##########
@@ -181,9 +185,9 @@ def optimize(self, mod, target=None, params=None):
def _set_params(self, params):
self._set_params_func(_convert_param_map(params))
- def get_json(self):
+ def get_graph(self):
Review comment:
can you revert this change?
##########
File path: include/tvm/runtime/crt/stack_memory.h
##########
@@ -0,0 +1,55 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+// LINT_C_FILE
+#ifndef TVM_RUNTIME_CRT_STACK_MEMORY_H_
+#define TVM_RUNTIME_CRT_STACK_MEMORY_H_
+#include <stddef.h>
+#include <stdint.h>
+
+#include "error_codes.h"
Review comment:
I think this should be #include <tvm/runtime/crt/error_codes.h>
##########
File path: python/tvm/driver/tvmc/compiler.py
##########
@@ -239,7 +239,7 @@ def compile_model(
# TODO we need to update this return to use the updated graph module APIs
# as these getter functions will be deprecated in the next release
(@leandron)
- return graph_module.get_json(), graph_module.get_lib(),
graph_module.get_params(), dumps
+ return graph_module.get_graph(), graph_module.get_lib(),
graph_module.get_params(), dumps
Review comment:
can you revert this change?
##########
File path: tests/python/unittest/test_runtime_module_based_interface.py
##########
@@ -526,11 +526,11 @@ def test_debug_graph_executor():
out = get_output(0).asnumpy()
tvm.testing.assert_allclose(out, verify(data), atol=1e-5)
- # debug graph executor wrapper
- debug_g_mod = debug_executor.GraphModuleDebug(
- complied_graph_lib["debug_create"]("default", dev),
- [dev],
- complied_graph_lib.get_json(),
+ # debug graph runtime wrapper
+ debug_g_mod = debug_runtime.GraphModuleDebug(
Review comment:
revert this change
##########
File path: python/tvm/relay/backend/executor_factory.py
##########
@@ -0,0 +1,216 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Graph executor factory."""
+import warnings
+from ..._ffi.base import string_types
+from ..._ffi.registry import get_global_func
+from ...runtime import ndarray
+from tvm import tir
+
+
+class ExecutorFactoryModule:
+ """Graph executor factory module.
+ This is a module of graph executor factory
+
+ Parameters
+ ----------
+ graph_str : str
+ Depending on executor:
+ * Graph executor: the graph to be deployed in json format output by
graph compiler.
+ The graph can contain operator(tvm_op) that points to the name of
+ PackedFunc in the libmod.
+ * AOT executor: the string representation of the TIR executor
PrimFunction
+ target : tvm.Target
+ The Target used to build this module.
+ libmod : tvm.Module
+ The module of the corresponding function
+ libmod_name: str
+ The name of module
+ params : dict of str to NDArray
+ The parameters of module
+ """
+
+ def get_internal_repr(self):
+ return self.internal_repr
+
+ def get_params(self):
+ return None
+
+ def get_lib(self):
+ return None
+
+ def __getitem__(self, item):
+ return None
+
+ def __iter__(self):
+ warnings.warn(
+ "legacy graph executor behavior of producing json / lib / params
will be "
+ "removed in the next release."
+ " Please see documents of tvm.contrib.graph_executor.GraphModule
for the "
+ " new recommended usage.",
+ DeprecationWarning,
+ 2,
+ )
+ return self
+
+ def save_config(self, config_path):
Review comment:
i'd vote for us to qualify config: `save_executor_config`. Also, let's
have this function either return a string containing the config or accept an
io.BytesIO so that it's not required to write to a file.
##########
File path: include/tvm/runtime/crt/aot_executor.h
##########
@@ -0,0 +1,98 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file include/tvm/runtime/crt/aot/tvm_executor.h
+ * \brief TVM Executor for the Ahead-of-Time Runtime
+ *
+ * AOT models are described by the TVM model descriptor format
+ * which can be passed to tvm_runtime_run. These descriptors will be
+ * generated by the AOT compilation process. This can optionally be
+ * augmented with platform specific context to be passed to the TVM
+ * operators.
+ *
+ * Example:
+ * extern tvm_model_t my_network;
+ * int main() {
+ * void* data = get_data();
+ * void* output[4] = {0, 0, 0, 0};
+ * void* inputs = {data};
+ * void* outputs = {output};
+ * tvm_context_t my_context = {
+ * .driver = ...;
+ * };
+ * tvm_runtime_run(
+ * &my_network,
+ * inputs,
+ * outputs
+ * &my_context
+ * );
+ * return 0;
+ * }
+ */
+
+#ifndef TVM_RUNTIME_CRT_AOT_TVM_EXECUTOR_H_
+#define TVM_RUNTIME_CRT_AOT_TVM_EXECUTOR_H_
+
+#include <stdint.h>
+
+#include "error_codes.h"
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+/*!
+ * \brief Context information for future integrations
+ * which is passed through to the operators.
+ *
+ * \note Can be used for drivers and platform specific information.
+ */
+typedef struct {
Review comment:
GH has decided the past comment is out of date :(
I would prefer if we can merge this structure in a follow-on PR where it's
used, if that's ok with you.
##########
File path: src/relay/backend/build_module.cc
##########
@@ -166,6 +216,14 @@ class RelayBuildModule : public runtime::ModuleNode {
ICHECK_EQ(args.num_args, 2);
*rv = this->Optimize(args[0], args[1], this->params_);
});
+ } else if (name == "get_executor") {
Review comment:
let's update this function name to match the name we choose for Executor
enum above
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
