[GitHub] [incubator-tvm] liangfu commented on a change in pull request #3934: [Runtime] MISRA-C compliant TVM runtime
liangfu commented on a change in pull request #3934: [Runtime] MISRA-C compliant TVM runtime URL: https://github.com/apache/incubator-tvm/pull/3934#discussion_r388107817 ## File path: apps/bundle_deploy_c/runtime.c ## @@ -0,0 +1,42 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +/* Explicitly declare posix_memalign function */ +#if _POSIX_C_SOURCE < 200112L +#undef _POSIX_C_SOURCE +#define _POSIX_C_SOURCE 200809L +#endif + +#define TVM_CRT_DEBUG 0 + +#define TVM_CRT_MAX_NDIM 6 Review comment: added comments for these macros This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-tvm] liangfu commented on a change in pull request #3934: [Runtime] MISRA-C compliant TVM runtime
liangfu commented on a change in pull request #3934: [Runtime] MISRA-C compliant TVM runtime URL: https://github.com/apache/incubator-tvm/pull/3934#discussion_r388107841 ## File path: src/runtime/crt/load_json.c ## @@ -0,0 +1,359 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +/*! + * \file saveload_json.cc Review comment: fixed This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-tvm] liangfu commented on a change in pull request #3934: [Runtime] MISRA-C compliant TVM runtime
liangfu commented on a change in pull request #3934: [Runtime] MISRA-C compliant TVM runtime URL: https://github.com/apache/incubator-tvm/pull/3934#discussion_r388107012 ## File path: apps/bundle_deploy_c/README.md ## @@ -0,0 +1,53 @@ + + + + + + + + + + + + + + + + + + +How to Bundle TVM Modules += + +This folder contains an example on how to bundle a TVM module (with the required +interpreter runtime modules such as `runtime::GraphRuntime`, the graph JSON, and +the params) into a single, self-contained shared object (`bundle.so`) which +exposes a C API wrapping the appropriate `runtime::GraphRuntime` instance. + +This is useful for cases where we'd like to avoid deploying the TVM runtime +components to the target host in advance - instead, we simply deploy the bundled +shared-object to the host, which embeds both the model and the runtime +components. The bundle should only depend on libc/libc++. + +It also contains an example code (`demo.cc`) to load this shared object and +invoke the packaged TVM model instance. This is a dependency-free binary that +uses the functionality packaged in `bundle.so` (which means that `bundle.so` can +be deployed lazily at runtime, instead of at compile time) to invoke TVM +functionality. + +Type the following command to run the sample code under the current folder, +after building TVM first. + +```bash +make demo +``` + +This will: + +- Download the mobilenet0.25 model from the MXNet Gluon Model Zoo +- Compile the model with NNVM Review comment: fixed This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-tvm] liangfu commented on a change in pull request #3934: [Runtime] MISRA-C compliant TVM runtime
liangfu commented on a change in pull request #3934: [Runtime] MISRA-C
compliant TVM runtime
URL: https://github.com/apache/incubator-tvm/pull/3934#discussion_r384347340
##
File path: src/runtime/crt/bundle.c
##
@@ -0,0 +1,66 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+#include "graph_runtime.h"
+
+extern unsigned char build_graph_json[];
+extern unsigned int build_graph_json_len;
+extern unsigned char build_params_bin[];
+extern unsigned int build_params_bin_len;
+
+#define TVM_BUNDLE_FUNCTION __attribute__((visibility("default")))
+
+TVM_BUNDLE_FUNCTION GraphRuntime * tvm_runtime_create() {
Review comment:
I have improved this a bit, but generating multiple GraphRuntime via a
factory is not yet implemented. I think the user should be able to easily
create multiple runtimes with the improved interface.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] liangfu commented on a change in pull request #3934: [Runtime] MISRA-C compliant TVM runtime
liangfu commented on a change in pull request #3934: [Runtime] MISRA-C
compliant TVM runtime
URL: https://github.com/apache/incubator-tvm/pull/3934#discussion_r384315312
##
File path: src/runtime/crt/graph_runtime.h
##
@@ -0,0 +1,517 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file graph_runtime.h
+ * \brief Tiny graph runtime that can run graph containing only tvm PackedFunc.
+ */
+#ifndef TVM_RUNTIME_CRT_GRAPH_RUNTIME_H_
+#define TVM_RUNTIME_CRT_GRAPH_RUNTIME_H_
+
+#include
+
+#include "load_json.h"
+#include "ndarray.h"
+#include "packed_func.h"
+#include "module.h"
+
+/*! \brief macro to do C API call */
+#define TVM_CCALL(func)\
+ {\
+int ret = (func); \
+CHECK_EQ(ret, 0) \
+<< TVMGetLastError(); \
+ }
+
+/*! \brief operator attributes about tvm op */
+typedef struct tvm_op_param_t {
+ char func_name[120];
+ uint32_t num_inputs;
+ uint32_t num_outputs;
+ uint32_t flatten_data;
+} TVMOpParam;
+
+// Memory pool entry.
+typedef struct graph_runtime_pool_entry_t {
+ size_t size;
+ int device_type;
+} PoolEntry;
+
+// Node entry
+typedef struct graph_runtime_node_entry_t {
+ uint32_t node_id;
+ uint32_t index;
+ uint32_t version;
+ // JSON Loader
+ void (*Load)(JSONReader *reader);
+} NodeEntry;
+
+static inline int NodeEntry_Load(NodeEntry * entry, JSONReader * reader) {
+ int status = 0;
+ reader->BeginArray(reader);
+ if (!(reader->NextArrayItem(reader))) {
+fprintf(stderr, "invalid json format: failed to parse `node_id`\n");
+ }
+ reader->ReadUnsignedInteger(reader, &(entry->node_id));
+ if (!(reader->NextArrayItem(reader))) {
+fprintf(stderr, "invalid json format: failed to parse `index`\n");
+ }
+ reader->ReadUnsignedInteger(reader, &(entry->index));
+ if (reader->NextArrayItem(reader)) {
+reader->ReadUnsignedInteger(reader, &(entry->version));
+if (reader->NextArrayItem(reader)) {
+ fprintf(stderr, "invalid json format: failed to parse `version`\n");
+}
+ } else {
+entry->version = 0;
+ }
+ return status;
+}
+
+// Node
+typedef struct graph_runtime_node_t {
Review comment:
fixed
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] liangfu commented on a change in pull request #3934: [Runtime] MISRA-C compliant TVM runtime
liangfu commented on a change in pull request #3934: [Runtime] MISRA-C
compliant TVM runtime
URL: https://github.com/apache/incubator-tvm/pull/3934#discussion_r384315210
##
File path: src/runtime/crt/graph_runtime.c
##
@@ -0,0 +1,375 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file graph_runtime.c
+ * \brief implement graph runtime in pure C
+ */
+#include
+
+#include "graph_runtime.h"
+
+#ifndef MAX
+#define MAX(a, b) (((a) > (b)) ? (a) : (b))
+#endif // MAX
+
+static inline uint32_t Shape_Accumulate(int64_t * shape, uint32_t ndim) {
+ int64_t accum = 1;
+ uint32_t idx;
+ for (idx = 0; idx < ndim; idx++) {
+if (shape[idx] == 0) { break; }
+accum *= shape[idx];
+ }
+ return accum;
+}
+
+/*!
+ * \brief Get the input index given the name of input.
+ * \param name The name of the input.
+ * \return The index of input.
+ */
+int GraphRuntime_GetInputIndex(GraphRuntime * runtime, const char * name) {
+ uint32_t i;
+ int32_t rv = -1;
+ for (i = 0; i< runtime->input_nodes_count; ++i) {
+uint32_t nid = runtime->input_nodes[i];
+if (!strcmp(runtime->nodes[nid].name, name)) {
+ rv = i;
+ break;
+}
+ }
+ if (rv < 0) {
+fprintf(stderr, "cannot find \"%s\" among input", name);
+ }
+ return rv;
+}
+
+/*!
+ * \brief set index-th input to the graph.
+ * \param index The input index.
+ * \param data_in The input data.
+ */
+void GraphRuntime_SetInput(struct graph_runtime_t * runtime, const char *
name, DLTensor* data_in) {
+ uint32_t index = runtime->GetInputIndex(runtime, name);
+ if (index >= runtime->input_nodes_count) {
+fprintf(stderr, "given index is greater than num of input nodes.\n");
+ }
+ uint32_t eid = runtime->GetEntryId(runtime, runtime->input_nodes[index], 0);
+ runtime->data_entry[eid].dl_tensor = *data_in;
+}
+
+int GraphRuntime_LoadParams(struct graph_runtime_t * runtime, const char *
param_blob,
+const uint32_t param_size) {
+ int status = 0;
+ const char * bptr = param_blob;
+ uint64_t header, reserved;
+ header = ((uint64_t*)bptr)[0]; // NOLINT(*)
+ bptr += sizeof(header);
+ if (header != kTVMNDArrayListMagic) {
+fprintf(stderr, "Invalid parameters file format");
+ }
+ reserved = ((uint64_t*)bptr)[0]; // NOLINT(*)
+ bptr += sizeof(reserved);
+
+ // read names
+ char names[GRAPH_RUNTIME_MAX_NODES][80];
+ memset(names, 0, sizeof(names));
+ uint64_t names_count;
+ int idx;
+ names_count = ((uint64_t*)bptr)[0]; // NOLINT(*)
+ bptr += sizeof(names_count);
+ for (idx = 0; idx < names_count; idx++) {
+uint64_t name_length;
+name_length = ((uint64_t*)bptr)[0]; // NOLINT(*)
+bptr += sizeof(name_length);
+if (name_length >= 80) {
+ fprintf(stderr, "Error: function name longer than expected.\n");
+}
+memcpy(names[idx], bptr, name_length);
+bptr += name_length;
+ }
+
+ // read sizes
+ uint64_t sz;
+ sz = ((uint64_t*)bptr)[0]; // NOLINT(*)
+ bptr += sizeof(sz);
+ uint32_t size = sz;
+ if (size != names_count) {
+fprintf(stderr, "Invalid parameters file format\n");
+status = -1;
+ }
+
+ for (idx = 0; idx < size; idx++) {
+int32_t in_idx = runtime->GetInputIndex(runtime, names[idx]);
+if (!(in_idx >= 0)) {
+ fprintf(stderr, "Found param for non-existent input: %s\n", names[idx]);
+ status = -1;
+}
+uint32_t eid = runtime->GetEntryId(runtime, runtime->input_nodes[in_idx],
0);
+if (!(eid < runtime->data_entry_count)) {
+ fprintf(stderr, "`entry_id`=%d is greater than expected(%d).\n",
+ eid, runtime->data_entry_count);
+ status = -1;
+}
+
+status |= NDArray_Load(&(runtime->data_entry[eid]), );
+#if TVM_CRT_DEBUG
+NDArray * entry = &(runtime->data_entry[eid]);
+printf("param %s loaded, in_idx=%d, eid=%d, ndim=%d, data[0]=%f\n",
+ names[idx], in_idx, eid, entry->dl_tensor.ndim,
+ ((float*)entry->dl_tensor.data)[0]); // NOLINT(*)
+#endif // TVM_CRT_DEBUG
+ }
+
+ return status;
+}
+
+/*!
+ * \brief Run all the operations one by one.
+ */
+void GraphRuntime_Run(GraphRuntime * runtime) {
+ // setup the array and requirements.
+ uint32_t idx;
+ for (idx = 0; idx
[GitHub] [incubator-tvm] liangfu commented on a change in pull request #3934: [Runtime] MISRA-C compliant TVM runtime
liangfu commented on a change in pull request #3934: [Runtime] MISRA-C
compliant TVM runtime
URL: https://github.com/apache/incubator-tvm/pull/3934#discussion_r384315132
##
File path: src/runtime/crt/c_backend_api.c
##
@@ -0,0 +1,63 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+#include
+
+#include
+#include
+#include
+#include
+
+void* TVMBackendAllocWorkspace(int device_type, int device_id, uint64_t
nbytes, int dtype_code_hint,
+ int dtype_bits_hint) {
+ void* ptr = 0;
+ assert(nbytes > 0);
+ unsigned int dtype_bytes = dtype_bits_hint / 8;
+#ifdef __ANDROID__
+ ptr = memalign(64, nbytes * dtype_bytes);
Review comment:
This is a great idea! We may need a virtual memory system implemented in C
for returning the address in the stack. I think we might need a separate PR for
this feature.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
