[incubator-tvm] branch master updated (639358e -> 16d0d56)
This is an automated email from the ASF dual-hosted git repository. masahi pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/incubator-tvm.git. from 639358e [relay][topi] Add operation relay.nn.dilate() which calls topi.nn.dilate() (#5331) add 16d0d56 [Pytorch] fix translation of transpose when axis argument is as a list (#5451) No new revisions were added by this update. Summary of changes: python/tvm/relay/frontend/pytorch.py | 2 +- tests/python/frontend/pytorch/test_forward.py | 5 + 2 files changed, 6 insertions(+), 1 deletion(-)
[GitHub] [incubator-tvm] SXM-inspur opened a new pull request #5450: Optimizations of global_ave_pool for NHWC layout
SXM-inspur opened a new pull request #5450: URL: https://github.com/apache/incubator-tvm/pull/5450 The runtime of global_ave_pool took about 14.8% in Resnet50_v2 with batchsize of 32, when Tensor Core is enabled on Tesla T4 GPU. The runtime decreased to 0.134%, after optimizations in this PR were made for NHWC layout. The results of unit tests are listed below, and the latency is reported with unit of ms. As we can see from the table, great performance improvements have been achieved. batch original After optimization speedup 16 1.16 0.03 38.67 32 1.17 0.06 19.5 256 1.65 0.52 3.17 Table 1. Shape of input feature maps is batchx7x7x2048. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-tvm] maheshambule commented on a change in pull request #5452: [Frontend][TFLite] L2_POOL_2D operator
maheshambule commented on a change in pull request #5452: URL: https://github.com/apache/incubator-tvm/pull/5452#discussion_r415688419 ## File path: python/tvm/relay/frontend/tflite.py ## @@ -1770,6 +1775,12 @@ def convert_pool2d(self, op, pool_type): assert self.has_same_qnn_params(input_tensor, output_tensor), \ "qnn.op.max_pool2d requires input and output qnn params to be same" out = _op.nn.max_pool2d(in_expr, **params) +elif pool_type == "l2": +# l2_pool_2d is equivalent to sqrt(avg_pool(sqr(in_data))) Review comment: corrected This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-tvm] n-nez commented on pull request #5451: [Fontend][Pytorch] Fix translation of transpose when axis argument is as a list
n-nez commented on pull request #5451: URL: https://github.com/apache/incubator-tvm/pull/5451#issuecomment-619808194 cc @masahi This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-tvm] n-nez opened a new pull request #5451: [Fontend][Pytorch] Fix translation of transpose when axis argument is as a list
n-nez opened a new pull request #5451: URL: https://github.com/apache/incubator-tvm/pull/5451 Symptoms ``` Traceback (most recent call last): File "test_forward.py", line 1646, in test_forward_transpose() File "test_forward.py", line 755, in test_forward_transpose verify_model(Transpose3().float().eval(), input_data=input_data) File "test_forward.py", line 175, in verify_model custom_convert_map) File "/home/nnez/devel/public-tvm/python/tvm/relay/frontend/pytorch.py", line 2117, in from_pytorch outputs, ret_name, convert_map, prelude) File "/home/nnez/devel/public-tvm/python/tvm/relay/frontend/pytorch.py", line 2031, in convert_operators relay_out = relay_op(inputs, _get_input_types(op_node)) File "/home/nnez/devel/public-tvm/python/tvm/relay/frontend/pytorch.py", line 780, in _impl return _op.transform.transpose(data, axes) File "/home/nnez/devel/public-tvm/python/tvm/relay/op/transform.py", line 125, in transpose axes = list(axes) ``` This happens because of the current implementation of `prim::ListConstruct` translates a list of integer into `relay.Var` with the list encoded as the shape of it. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-tvm] maheshambule opened a new pull request #5452: [Frontend][TFLite] L2_POOL_2D operator
maheshambule opened a new pull request #5452: URL: https://github.com/apache/incubator-tvm/pull/5452 Adds support for L2_POOL_2D operator in TFLITE frontened. Ref: https://www.tensorflow.org/lite/guide/ops_compatibility This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[incubator-tvm] branch master updated (a60de36 -> 639358e)
This is an automated email from the ASF dual-hosted git repository. kazum pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/incubator-tvm.git. from a60de36 [ONNX]GatherNd, Round, IsNaN, IsInf (#5445) add 639358e [relay][topi] Add operation relay.nn.dilate() which calls topi.nn.dilate() (#5331) No new revisions were added by this update. Summary of changes: include/tvm/relay/attrs/nn.h| 10 + python/tvm/relay/op/nn/_nn.py | 24 + python/tvm/relay/op/nn/nn.py| 19 python/tvm/relay/op/op_attrs.py | 5 + src/relay/op/nn/nn.cc | 48 + tests/python/relay/test_any.py | 28 6 files changed, 134 insertions(+)
[GitHub] [incubator-tvm] kazum commented on pull request #5331: [relay][topi] Add operation relay.nn.dilate() which calls topi.nn.dilate()
kazum commented on pull request #5331: URL: https://github.com/apache/incubator-tvm/pull/5331#issuecomment-619792700 Thanks @notoraptor @siju-samuel ! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-tvm] FrozenGene commented on a change in pull request #5452: [Frontend][TFLite] L2_POOL_2D operator
FrozenGene commented on a change in pull request #5452: URL: https://github.com/apache/incubator-tvm/pull/5452#discussion_r415679526 ## File path: python/tvm/relay/frontend/tflite.py ## @@ -1770,6 +1775,12 @@ def convert_pool2d(self, op, pool_type): assert self.has_same_qnn_params(input_tensor, output_tensor), \ "qnn.op.max_pool2d requires input and output qnn params to be same" out = _op.nn.max_pool2d(in_expr, **params) +elif pool_type == "l2": +# l2_pool_2d is equivalent to sqrt(avg_pool(sqr(in_data))) Review comment: Better name `sqrt` to `square_root`, `sqr` to `square`. `sqrt` is too similar with `sqr` so that code readers confuse easily. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-tvm] masahi commented on pull request #5451: [Fontend][Pytorch] Fix translation of transpose when axis argument is as a list
masahi commented on pull request #5451: URL: https://github.com/apache/incubator-tvm/pull/5451#issuecomment-619925257 Thanks @n-nez This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-tvm] maheshambule commented on a change in pull request #5452: [Frontend][TFLite] L2_POOL_2D operator
maheshambule commented on a change in pull request #5452: URL: https://github.com/apache/incubator-tvm/pull/5452#discussion_r415749383 ## File path: tests/python/frontend/tflite/test_forward.py ## @@ -487,6 +487,31 @@ def test_forward_pooling(): strides=[2, 1]) +def _test_l2_pool2d(input_shape, ksize, strides, padding, data_format, fused_func_name=None): +x = np.arange(np.prod(input_shape), dtype=np.float32).reshape(input_shape) - 1 + +with tf.Graph().as_default(): +in_data = tf.placeholder( +dtype=tf.float32, name="input", shape=input_shape) +out = tf.sqrt(tf.nn.avg_pool( +tf.square(in_data), ksize=ksize, strides=strides, +padding=padding, data_format=data_format)) +out = with_fused_activation_function(out, fused_func_name) + +compare_tflite_with_tvm(x, 'input', [in_data], [out]) Review comment: TFLite doesn't have support for quantized L2_POOL_2D yet. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-tvm] u99127 commented on a change in pull request #5452: [Frontend][TFLite] L2_POOL_2D operator
u99127 commented on a change in pull request #5452: URL: https://github.com/apache/incubator-tvm/pull/5452#discussion_r415699554 ## File path: tests/python/frontend/tflite/test_forward.py ## @@ -487,6 +487,31 @@ def test_forward_pooling(): strides=[2, 1]) +def _test_l2_pool2d(input_shape, ksize, strides, padding, data_format, fused_func_name=None): +x = np.arange(np.prod(input_shape), dtype=np.float32).reshape(input_shape) - 1 + +with tf.Graph().as_default(): +in_data = tf.placeholder( +dtype=tf.float32, name="input", shape=input_shape) +out = tf.sqrt(tf.nn.avg_pool( +tf.square(in_data), ksize=ksize, strides=strides, +padding=padding, data_format=data_format)) +out = with_fused_activation_function(out, fused_func_name) + +compare_tflite_with_tvm(x, 'input', [in_data], [out]) Review comment: Very quick drive-by review : Don't we need a qnn test here ? I haven't managed to read up L2_POOL_2D in the tflite documentation yet. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[incubator-tvm] branch master updated (16d0d56 -> 17cd27d)
This is an automated email from the ASF dual-hosted git repository. tqchen pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/incubator-tvm.git. from 16d0d56 [Pytorch] fix translation of transpose when axis argument is as a list (#5451) add 17cd27d [TFLite Runtime] Add TFLite Runtime dependencies to CI CPU docker build (#5437) No new revisions were added by this update. Summary of changes: docker/Dockerfile.ci_cpu| 8 docker/install/ubuntu_install_tflite.sh | 12 +--- 2 files changed, 17 insertions(+), 3 deletions(-)
[GitHub] [incubator-tvm] mbaret commented on pull request #5409: [BYOC] Don't annotate constants
mbaret commented on pull request #5409: URL: https://github.com/apache/incubator-tvm/pull/5409#issuecomment-620051291 I don't think we'd need to treat constant nodes differently everywhere else. At least, in the previous version of AnnotatedRegionSet using the recursive pattern, the constants ended up in the correct region without explicit annotation. I think with the new traversal order though this no longer happens. The main reason I'm not in favour of explicitly annotating a constant with begin/end is that it implies there is a data flow path in through the constant. We're then violating a different assumption which is that all begin/end annotations cut across data flow. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-tvm] siju-samuel opened a new pull request #5453: [RELAY][ONNX]ReduceLogSumExp Operator support
siju-samuel opened a new pull request #5453: URL: https://github.com/apache/incubator-tvm/pull/5453 `ReduceLogSumExp` op support added in Relay and ONNX Frontend. @masahi @FrozenGene Could you please help me to review this PR. Thanks This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-tvm] areusch commented on a change in pull request #5417: [RUNTIME][uTVM] AutoTVM + uTVM for Cortex-M7
areusch commented on a change in pull request #5417:
URL: https://github.com/apache/incubator-tvm/pull/5417#discussion_r415981441
##
File path: src/runtime/micro/host_driven/utvm_runtime.c
##
@@ -34,89 +34,148 @@ extern "C" {
#include "utvm_runtime.h"
-// Task pointers must be patched before calling a function.
-UTVMTask utvm_task = {
-.func = NULL,
-.arg_values = NULL,
-.arg_type_codes = NULL,
-.num_args = 0,
-};
-
-size_t utvm_word_size = 0; // NOLINT(*)
+// TODO(areusch): move defines into header
+#define TASK_QUEUE_SIZE 20
+volatile UTVMTask utvm_tasks[TASK_QUEUE_SIZE] = { };
+volatile uint32_t utvm_num_tasks = 0;
+volatile uint32_t utvm_task_times[TASK_QUEUE_SIZE] = { };
Review comment:
done. we can look at runtime configuration more after we switch to
building full binaries all the time.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [incubator-tvm] areusch commented on a change in pull request #5417: [RUNTIME][uTVM] AutoTVM + uTVM for Cortex-M7
areusch commented on a change in pull request #5417:
URL: https://github.com/apache/incubator-tvm/pull/5417#discussion_r415980487
##
File path: src/runtime/micro/host_driven/utvm_device_dylib_redirect.c
##
@@ -32,10 +32,10 @@ extern "C" {
#include
#include
-void *(*TVMBackendAllocWorkspace_)(int, int, uint64_t, int, int) =
-(void *(*)(int, int, uint64_t, int, int)) NULL;
-int (*TVMBackendFreeWorkspace_)(int, int, void*) = (int (*)(int, int, void*))
NULL;
-void (*TVMAPISetLastError_)(const char*) = (void (*)(const char*)) NULL;
+// TODO(areusch): compiler errors say volatile qualifier is discarded. should
we just get rid of em?
Review comment:
done
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [incubator-tvm] kevinthesun commented on a change in pull request #5429: [RELAY][TF] Support symbolic newshape for Reshape
kevinthesun commented on a change in pull request #5429:
URL: https://github.com/apache/incubator-tvm/pull/5429#discussion_r416046045
##
File path: src/relay/op/tensor/transform.cc
##
@@ -650,13 +670,59 @@ Array ReshapeCompute(const Attrs& attrs,
return { topi::reshape(inputs[0], newshape) };
}
+double ToScalar(const runtime::NDArray& array, int i = 0) {
+ if (array->dtype.code == kDLInt) {
+if (array->dtype.bits == 8) {
+ return reinterpret_cast(array->data)[i];
+} else if (array->dtype.bits == 16) {
+ return reinterpret_cast(array->data)[i];
+} else if (array->dtype.bits == 32) {
+ return reinterpret_cast(array->data)[i];
+} else if (array->dtype.bits == 64) {
+ return reinterpret_cast(array->data)[i];
+}
+ } else if (array->dtype.code == kDLUInt) {
+if (array->dtype.bits == 8) {
+ return reinterpret_cast(array->data)[i];
+} else if (array->dtype.bits == 16) {
+ return reinterpret_cast(array->data)[i];
+} else if (array->dtype.bits == 32) {
+ return reinterpret_cast(array->data)[i];
+} else if (array->dtype.bits == 64) {
+ return reinterpret_cast(array->data)[i];
+}
+ } else if (array->dtype.code == kDLFloat) {
+#if (__ARM_FP16_FORMAT_IEEE == 1)
+if (array->dtype.bits == 16) {
+ return reinterpret_cast<__fp16*>(array->data)[i];
+}
+#endif
+if (array->dtype.bits == 32) {
+ return reinterpret_cast(array->data)[i];
+} else if (array->dtype.bits == 64) {
+ return reinterpret_cast(array->data)[i];
+}
+ }
+ LOG(FATAL) << "Unknown data type: " <<
tvm::runtime::DLDataType2String(array->dtype);
+ // make compiler happy
+ return -std::numeric_limits::infinity();
+}
+
Expr MakeReshape(Expr data,
- Array newshape) {
+ Expr newshape) {
auto attrs = make_object();
- attrs->newshape = std::move(newshape);
+
+ if (const ConstantNode *c = newshape.as()) {
Review comment:
Also set attrs->newshape when newshape is symbolic?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [incubator-tvm] areusch commented on a change in pull request #5417: [RUNTIME][uTVM] AutoTVM + uTVM for Cortex-M7
areusch commented on a change in pull request #5417: URL: https://github.com/apache/incubator-tvm/pull/5417#discussion_r415970450 ## File path: python/tvm/target/arm_isa.py ## @@ -0,0 +1,34 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +"""Defines functions to analyze availble opcodes in the ARM ISA.""" Review comment: done This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-tvm] areusch commented on a change in pull request #5417: [RUNTIME][uTVM] AutoTVM + uTVM for Cortex-M7
areusch commented on a change in pull request #5417: URL: https://github.com/apache/incubator-tvm/pull/5417#discussion_r415990923 ## File path: src/target/source/codegen_c_host.cc ## @@ -20,10 +20,10 @@ /*! * \file codegen_c_host.cc */ -#include +#include "codegen_c_host.h" Review comment: done, though, can you explain why tvm_header and local_header are distinct things in this codebase? does tvm_header mean headers in `include`, since they are installed? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-tvm] areusch commented on a change in pull request #5417: [RUNTIME][uTVM] AutoTVM + uTVM for Cortex-M7
areusch commented on a change in pull request #5417: URL: https://github.com/apache/incubator-tvm/pull/5417#discussion_r415942734 ## File path: Makefile ## @@ -73,7 +73,10 @@ build/libtvm_web_runtime.js: build/libtvm_web_runtime.bc cpplint: python3 3rdparty/dmlc-core/scripts/lint.py vta cpp vta/include vta/src python3 3rdparty/dmlc-core/scripts/lint.py topi cpp topi/include; - python3 3rdparty/dmlc-core/scripts/lint.py tvm cpp include src \ + # Note: exclude src/runtime/micro/host_driven becuase it contains C99 files. + python3 3rdparty/dmlc-core/scripts/lint.py tvm cpp \ +--exclude_path=src/runtime/micro/host_driven \ Review comment: done. in the long term, we need to figure out whether we extend the dmlc lint script to not recognize .c files as c++ (this is why I originally excluded it). for now, the errors are only related to casting so we can leave it on. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-tvm] areusch commented on a change in pull request #5417: [RUNTIME][uTVM] AutoTVM + uTVM for Cortex-M7
areusch commented on a change in pull request #5417: URL: https://github.com/apache/incubator-tvm/pull/5417#discussion_r415959267 ## File path: python/tvm/micro/device/host.py ## @@ -17,12 +17,26 @@ """Compilation and config definitions for the host emulated device""" import sys -from . import create_micro_lib_base, register_device +from . import create_micro_lib_base, register_device, gen_mem_layout, MemConstraint -DEVICE_ID = "host" -TOOLCHAIN_PREFIX = "" +DEVICE_ID = 'host' Review comment: done This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-tvm] areusch commented on a change in pull request #5417: [RUNTIME][uTVM] AutoTVM + uTVM for Cortex-M7
areusch commented on a change in pull request #5417:
URL: https://github.com/apache/incubator-tvm/pull/5417#discussion_r415958560
##
File path: python/tvm/micro/device/base.py
##
@@ -92,71 +95,142 @@ def create_micro_lib_base(
options : List[str]
additional options to pass to GCC
+
+lib_src_paths : Optional[List[str]]
+paths to additional source files to be compiled into the library
"""
+# look at these (specifically `strip`):
+#
https://stackoverflow.com/questions/15314581/g-compiler-flag-to-minimize-binary-size
base_compile_cmd = [
-f"{toolchain_prefix}gcc",
-"-std=c11",
-"-Wall",
-"-Wextra",
-"--pedantic",
-"-c",
-"-O0",
-"-g",
-"-nostartfiles",
-"-nodefaultlibs",
-"-nostdlib",
-"-fdata-sections",
-"-ffunction-sections",
+f'{toolchain_prefix}gcc',
Review comment:
undid all the quote conversions, except those that made sense (i.e. to
avoid escapes, as in f'unknown variable "{var}"'). can you point out any other
changes you want me to revert? I didn't originally author most of this code so
I don't have all the context in my head.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [incubator-tvm] areusch commented on a change in pull request #5417: [RUNTIME][uTVM] AutoTVM + uTVM for Cortex-M7
areusch commented on a change in pull request #5417:
URL: https://github.com/apache/incubator-tvm/pull/5417#discussion_r415958851
##
File path: python/tvm/micro/device/base.py
##
@@ -92,71 +95,142 @@ def create_micro_lib_base(
options : List[str]
additional options to pass to GCC
+
+lib_src_paths : Optional[List[str]]
+paths to additional source files to be compiled into the library
"""
+# look at these (specifically `strip`):
+#
https://stackoverflow.com/questions/15314581/g-compiler-flag-to-minimize-binary-size
base_compile_cmd = [
-f"{toolchain_prefix}gcc",
-"-std=c11",
-"-Wall",
-"-Wextra",
-"--pedantic",
-"-c",
-"-O0",
-"-g",
-"-nostartfiles",
-"-nodefaultlibs",
-"-nostdlib",
-"-fdata-sections",
-"-ffunction-sections",
+f'{toolchain_prefix}gcc',
+'-std=c11',
+'-Wall',
+'-Wextra',
+'--pedantic',
+'-c',
+'-g',
+'-nostartfiles',
+'-nodefaultlibs',
+'-nostdlib',
+'-fdata-sections',
+'-ffunction-sections',
]
if options is not None:
base_compile_cmd += options
src_paths = []
include_paths = find_include_path() + [get_micro_host_driven_dir()]
tmp_dir = _util.tempdir()
-# we might transform the src path in one of the branches below
+# we need to create a new src file in the operator branch
new_in_src_path = in_src_path
if lib_type == LibType.RUNTIME:
dev_dir = _get_device_source_dir(device_id)
-dev_src_paths = glob.glob(f"{dev_dir}/*.[csS]")
+
+dev_src_paths = glob.glob(f'{dev_dir}/*.[csS]')
# there needs to at least be a utvm_timer.c file
assert dev_src_paths
-assert "utvm_timer.c" in map(os.path.basename, dev_src_paths)
+assert 'utvm_timer.c' in map(os.path.basename, dev_src_paths)
+
src_paths += dev_src_paths
elif lib_type == LibType.OPERATOR:
-# create a temporary copy of the source, so we can inject the dev lib
+# create a temporary copy of the operator source, so we can inject the
dev lib
# header without modifying the original.
-temp_src_path = tmp_dir.relpath("temp.c")
-with open(in_src_path, "r") as f:
+temp_src_path = tmp_dir.relpath('temp.c')
+with open(in_src_path, 'r') as f:
src_lines = f.read().splitlines()
-src_lines.insert(0, "#include \"utvm_device_dylib_redirect.c\"")
-with open(temp_src_path, "w") as f:
-f.write("\n".join(src_lines))
+src_lines.insert(0, '#include "utvm_device_dylib_redirect.c"')
+with open(temp_src_path, 'w') as f:
+f.write('\n'.join(src_lines))
new_in_src_path = temp_src_path
-base_compile_cmd += ["-c"]
else:
-raise RuntimeError("unknown lib type")
+raise RuntimeError('unknown lib type')
src_paths += [new_in_src_path]
+# add any src paths required by the operator
+if lib_src_paths is not None:
+src_paths += lib_src_paths
+
+# print(f'include paths: {include_paths}')
for path in include_paths:
-base_compile_cmd += ["-I", path]
+base_compile_cmd += ['-I', path]
Review comment:
done
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [incubator-tvm] wpan11nv commented on a change in pull request #5428: [CODEGEN][CUDA] Fix a bug when vectorized load was involved for…
wpan11nv commented on a change in pull request #5428:
URL: https://github.com/apache/incubator-tvm/pull/5428#discussion_r416007695
##
File path: src/target/source/codegen_cuda.cc
##
@@ -274,9 +274,21 @@ void CodeGenCUDA::PrintVecElemLoad(
static const char access[] = {'x', 'y', 'z', 'w'};
CHECK(i >= 0 && i < (t.is_float16() ? 8 : 4));
if ((t.is_int()) && t.bits() == 8) {
-os << "((char)(" << vec << " >> " << i * 8 << "))";
+if (t.lanes() == 1) {
+ os << vec;
+} else if (t.lanes() == 2) {
+ os << vec << "." << access[i % 2];
+} else {
+ os << "((char)(" << vec << " >> " << i * 8 << "))";
+}
} else if ((t.is_uint()) && t.bits() == 8) {
-os << "((unsigned char)(" << vec << " >> " << i * 8 << "))";
+if (t.lanes() == 1) {
Review comment:
This "lane = 1" logic could be promoted for all cases.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [incubator-tvm] maheshambule commented on a change in pull request #5452: [Frontend][TFLite] L2_POOL_2D operator
maheshambule commented on a change in pull request #5452: URL: https://github.com/apache/incubator-tvm/pull/5452#discussion_r416041366 ## File path: python/tvm/relay/frontend/tflite.py ## @@ -1770,6 +1775,12 @@ def convert_pool2d(self, op, pool_type): assert self.has_same_qnn_params(input_tensor, output_tensor), \ "qnn.op.max_pool2d requires input and output qnn params to be same" out = _op.nn.max_pool2d(in_expr, **params) +elif pool_type == "l2": +# l2_pool_2d is equivalent to square_root(avg_pool(square(in_data))) Review comment: TFLite does not support quantized L2_POOL_2D. That's why the check is not added. Now I have added a comment for the same. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-tvm] areusch commented on a change in pull request #5417: [RUNTIME][uTVM] AutoTVM + uTVM for Cortex-M7
areusch commented on a change in pull request #5417: URL: https://github.com/apache/incubator-tvm/pull/5417#discussion_r415947706 ## File path: python/tvm/micro/device/arm/stm32f746xx.py ## @@ -36,23 +55,41 @@ def create_micro_lib(obj_path, src_path, lib_type, options=None): options : Optional[List[str]] additional options to pass to GCC + +lib_src_paths : Optional[List[str]] +TODO """ if options is None: options = [] +else: +options = list(options) + options += [ -"-mcpu=cortex-m7", -"-mlittle-endian", -"-mfloat-abi=hard", -"-mfpu=fpv5-sp-d16", -"-mthumb", -"-gdwarf-5", +# TODO(weberlo): make a debug flag +'-O2', +'-march=armv7e-m', +'-mcpu=cortex-m7', +'-mlittle-endian', +'-mfloat-abi=hard', +'-mfpu=fpv5-sp-d16', +'-mthumb', +'-ffast-math', +'-gdwarf-5', +'-DARM_MATH_CM7', +'-D__FPU_PRESENT=1U', +'-DARM_MATH_DSP', +'-Wno-unused-variable', Review comment: these are in the generated GEMM code. for example: `/var/folders/9y/3j808g591ln3kys4qpyl3qmcgn/T/tmpb4sk1ylf/temp.c:104:11: error: unused variable \'arg1_code\' [-Werror=unused-variable]` it would be nice if we did not need to include this, but i'm hoping we can merge this PR as-is and incrementally improve things like this (especially since I didn't author most of the code generation stuff and will need a bit of time to understand how to improve it to remove errors like this). the next PR will look at the generated code more in detail, so if it's a quick fix, I can fix it then. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-tvm] areusch commented on a change in pull request #5417: [RUNTIME][uTVM] AutoTVM + uTVM for Cortex-M7
areusch commented on a change in pull request #5417:
URL: https://github.com/apache/incubator-tvm/pull/5417#discussion_r415961064
##
File path: python/tvm/relay/_parser.py
##
@@ -340,7 +342,10 @@ def visitLocalVar(self, ctx):
return local_var
def visitGraphVar(self, ctx):
-return self.graph_expr[int(ctx.NAT().getText())]
+graph_var_idx = int(ctx.NAT().getText())
+if graph_var_idx >= len(self.graph_expr):
+raise ParseError(f"graph var `%{graph_var_idx}` is unbound")
+return self.graph_expr[graph_var_idx]
Review comment:
ok, what about the changes on line 154 as well? I don't know what they
do yet.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [incubator-tvm] areusch commented on a change in pull request #5417: [RUNTIME][uTVM] AutoTVM + uTVM for Cortex-M7
areusch commented on a change in pull request #5417:
URL: https://github.com/apache/incubator-tvm/pull/5417#discussion_r415967381
##
File path: python/tvm/relay/op/strategy/arm_cpu.py
##
@@ -267,7 +281,7 @@ def bitserial_conv2d_strategy_arm_cpu(attrs, inputs,
out_type, target):
raise ValueError("Data layout {} not supported.".format(layout))
return strategy
-@bitserial_dense_strategy.register("arm_cpu")
+@bitserial_dense_strategy.register(["arm_cpu", "micro_dev"])
Review comment:
oh good catch, reverted. yes we are punting on that for a bit.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [incubator-tvm] areusch commented on a change in pull request #5417: [RUNTIME][uTVM] AutoTVM + uTVM for Cortex-M7
areusch commented on a change in pull request #5417: URL: https://github.com/apache/incubator-tvm/pull/5417#discussion_r415991766 ## File path: topi/python/topi/arm_cpu/cortex_m7/conv2d/direct.py ## @@ -0,0 +1,177 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +# pylint: disable=invalid-name Review comment: this placement seems to be pretty conventional for `topi`, since most of the variable names are too short to be considered valid? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-tvm] areusch commented on a change in pull request #5417: [RUNTIME][uTVM] AutoTVM + uTVM for Cortex-M7
areusch commented on a change in pull request #5417:
URL: https://github.com/apache/incubator-tvm/pull/5417#discussion_r415983620
##
File path: src/runtime/micro/host_driven/utvm_runtime.c
##
@@ -34,89 +34,148 @@ extern "C" {
#include "utvm_runtime.h"
-// Task pointers must be patched before calling a function.
-UTVMTask utvm_task = {
-.func = NULL,
-.arg_values = NULL,
-.arg_type_codes = NULL,
-.num_args = 0,
-};
-
-size_t utvm_word_size = 0; // NOLINT(*)
+// TODO(areusch): move defines into header
+#define TASK_QUEUE_SIZE 20
+volatile UTVMTask utvm_tasks[TASK_QUEUE_SIZE] = { };
+volatile uint32_t utvm_num_tasks = 0;
+volatile uint32_t utvm_task_times[TASK_QUEUE_SIZE] = { };
// These pointers are patched at load time to point to the workspace section.
-char* utvm_workspace_start = NULL; // NOLINT(*)
-char* utvm_workspace_end = NULL;// NOLINT(*)
-char* utvm_workspace_curr = NULL; // NOLINT(*)
+volatile char* utvm_workspace_start = NULL; // NOLINT(*)
+volatile char* utvm_workspace_end = NULL;// NOLINT(*)
+volatile char* utvm_workspace_curr = NULL; // NOLINT(*)
+#define MAX_WS_ALLOCS 10
+volatile char* utvm_alloc_ends[MAX_WS_ALLOCS] = {}; // NOLINT(*)
+volatile uint32_t utvm_alloc_idx = 0;
// Keep track of how many active allocations there are on the workspace.
-size_t utvm_num_active_allocs = 0;
+volatile uint32_t utvm_num_active_allocs = 0;
+
+volatile uint32_t utvm_word_size = 0;
-const char* utvm_last_error = NULL; // NOLINT(*)
-int32_t utvm_return_code = 0;// NOLINT(*)
+volatile int32_t utvm_last_error = 0; // NOLINT(*)
-uint32_t utvm_task_time = 0;
+volatile uint32_t utvm_done = 0;
// Gets called by UTVMInit, after device-specific initialization is finished.
void UTVMMain() {
+ utvm_done = 0;
+ // loss of precision should be fine here, since we only care about the lower
bits
+ if (((uint32_t) utvm_workspace_start) % utvm_word_size) {
Review comment:
downcasting to a smaller integral type is well-defined, but deciding
what to do when the cast is missing can be less well-defined (i believe this
problem is fixed in c++ and potentially newer C). it's fine here. another
embedded way to fix this is to assume utvm_word_size is a power of two # of
bytes, and do
`if (utvm_workspace_start & (utvm_word_size - 1))`
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [incubator-tvm] areusch commented on a change in pull request #5417: [RUNTIME][uTVM] AutoTVM + uTVM for Cortex-M7
areusch commented on a change in pull request #5417:
URL: https://github.com/apache/incubator-tvm/pull/5417#discussion_r415984080
##
File path: src/runtime/micro/host_driven/utvm_runtime.c
##
@@ -34,89 +34,148 @@ extern "C" {
#include "utvm_runtime.h"
-// Task pointers must be patched before calling a function.
-UTVMTask utvm_task = {
-.func = NULL,
-.arg_values = NULL,
-.arg_type_codes = NULL,
-.num_args = 0,
-};
-
-size_t utvm_word_size = 0; // NOLINT(*)
+// TODO(areusch): move defines into header
+#define TASK_QUEUE_SIZE 20
+volatile UTVMTask utvm_tasks[TASK_QUEUE_SIZE] = { };
+volatile uint32_t utvm_num_tasks = 0;
+volatile uint32_t utvm_task_times[TASK_QUEUE_SIZE] = { };
// These pointers are patched at load time to point to the workspace section.
-char* utvm_workspace_start = NULL; // NOLINT(*)
-char* utvm_workspace_end = NULL;// NOLINT(*)
-char* utvm_workspace_curr = NULL; // NOLINT(*)
+volatile char* utvm_workspace_start = NULL; // NOLINT(*)
+volatile char* utvm_workspace_end = NULL;// NOLINT(*)
+volatile char* utvm_workspace_curr = NULL; // NOLINT(*)
+#define MAX_WS_ALLOCS 10
+volatile char* utvm_alloc_ends[MAX_WS_ALLOCS] = {}; // NOLINT(*)
+volatile uint32_t utvm_alloc_idx = 0;
// Keep track of how many active allocations there are on the workspace.
-size_t utvm_num_active_allocs = 0;
+volatile uint32_t utvm_num_active_allocs = 0;
+
+volatile uint32_t utvm_word_size = 0;
-const char* utvm_last_error = NULL; // NOLINT(*)
-int32_t utvm_return_code = 0;// NOLINT(*)
+volatile int32_t utvm_last_error = 0; // NOLINT(*)
-uint32_t utvm_task_time = 0;
+volatile uint32_t utvm_done = 0;
// Gets called by UTVMInit, after device-specific initialization is finished.
void UTVMMain() {
+ utvm_done = 0;
+ // loss of precision should be fine here, since we only care about the lower
bits
+ if (((uint32_t) utvm_workspace_start) % utvm_word_size) {
+utvm_last_error = UTVM_ERR_WS_UNALIGNED_START;
+UTVMDone();
+return;
+ }
utvm_workspace_curr = utvm_workspace_start;
utvm_num_active_allocs = 0;
- utvm_last_error = NULL; // NOLINT(*)
- utvm_return_code = 0;
- utvm_task_time = 0;
- UTVMTimerReset();
- int32_t err = UTVMTimerStart();
- if (err < 0) {
-utvm_return_code = err;
-UTVMDone();
+ utvm_alloc_idx = 0;
+ utvm_last_error = UTVM_ERR_NOT_FINISHED;
+ for (uint32_t i = 0; i < utvm_num_tasks; i++) {
+int32_t err = UTVM_ERR_OK;
+utvm_task_times[i] = 0;
+err = UTVMTimerStart();
+if (err < 0) {
+ utvm_last_error = err;
+ UTVMDone();
+ return;
+}
+err = utvm_tasks[i].func(
+(void*) utvm_tasks[i].arg_values, // NOLINT(*)
+(void*) utvm_tasks[i].arg_type_codes, // NOLINT(*)
+utvm_tasks[i].num_args);
+if (err < 0) {
+ UTVMDone();
+ return;
+}
+utvm_task_times[i] = UTVMTimerStop();
+if (err < 0) {
+ utvm_last_error = err;
+ UTVMDone();
+ return;
+}
+ }
+ if (utvm_last_error == UTVM_ERR_NOT_FINISHED) {
+utvm_last_error = UTVM_ERR_OK;
}
- utvm_return_code = utvm_task.func(
- (void*) utvm_task.arg_values, // NOLINT(*)
- (void*) utvm_task.arg_type_codes, // NOLINT(*)
- utvm_task.num_args);
- UTVMTimerStop();
- utvm_task_time = UTVMTimerRead();
UTVMDone();
}
// We use a dummy function to signal execution is finished for device
// backends which require breakpoints.
-void UTVMDone() { }
+void __attribute__((noinline)) UTVMDone() {
+ utvm_done = 1;
+}
+
+#define ALIGNED_UP(x, word_size) \
+ word_size) - (((uintptr_t) (x)) % (word_size))) % (word_size)) + (x))
void* TVMBackendAllocWorkspace(int device_type, int device_id, uint64_t size,
int dtype_code_hint, int dtype_bits_hint) {
- // Align up to 8 bytes.
- utvm_workspace_curr +=
-(utvm_word_size - ((uintptr_t) utvm_workspace_curr % utvm_word_size)) %
utvm_word_size; // NOLINT(*)
- if (utvm_workspace_curr + size > utvm_workspace_end) {
+ if (size == 0) {
+utvm_last_error = UTVM_ERR_WS_ZERO_SIZE_ALLOC;
+return NULL;
+ }
+ size_t alloc_requested_bytes = size;
+ size_t alloc_size_words = (alloc_requested_bytes + utvm_word_size - 1) /
utvm_word_size;
+ size_t alloc_size_bytes = alloc_size_words * utvm_word_size;
+
+ // Align up to the target word size.
+ if (utvm_workspace_curr + alloc_size_bytes > utvm_workspace_end) {
// Out of space in workspace.
+utvm_last_error = UTVM_ERR_WS_OUT_OF_SPACE;
+return NULL;
+ }
+ if (utvm_alloc_idx == MAX_WS_ALLOCS - 1) {
+// Exceeded number of allocs we can keep track of.
+utvm_last_error = UTVM_ERR_WS_TOO_MANY_ALLOCS;
return NULL;
}
void* ret_ptr = (void*) utvm_workspace_curr; // NOLINT(*)
- utvm_workspace_curr += size;
+ utvm_workspace_curr = utvm_workspace_curr + alloc_size_bytes;
+ // store the *end* of the alloc, so we can restore the WS pointer when
freeing
+
[GitHub] [incubator-tvm] siju-samuel commented on a change in pull request #5447: [TOPI,RELAY][TFLITE] Sparse to dense operator
siju-samuel commented on a change in pull request #5447:
URL: https://github.com/apache/incubator-tvm/pull/5447#discussion_r415997917
##
File path: python/tvm/relay/frontend/tflite.py
##
@@ -2075,6 +2077,29 @@ def convert_space_to_depth(self, op):
return out
+def convert_sparse_to_dense(self, op):
+"""Convert TFLite SPARSE_TO_DENSE"""
+try:
+from tflite.Operator import Operator
+except ImportError:
+raise ImportError("The tflite package must be installed")
+
+assert isinstance(op, Operator)
Review comment:
This code can be removed., its already checked in `convert_op_to_relay`
##
File path: python/tvm/relay/op/transform.py
##
@@ -884,3 +884,29 @@ def unravel_index(indices, shape):
"""
return _make.unravel_index(indices, shape)
+
+def sparse_to_dense(sparse_indices, sparse_values, default_value,
output_shape):
+"""Converts a sparse representation into a dense tensor.
+ Example::
+ - sparse_to_dense([[0, 0], [1, 1]], [3, 3], 0, [2, 2]) = [[3, 0], [0,
3]]
+
+ Parameters
+ --
+ sparse_indices : tvm.te.Tensor
+ A 0-D, 1-D, or 2-D tensor of type `int32` or `int64`.
+ `sparse_indices[i]` contains the complete index where
`sparse_values[i]` will be placed.
+ sparse_values : tvm.te.Tensor
+ A 0-D or 1-D tensor.
+ Values corresponding to each row of sparse_indices,
+ or a scalar value to be used for all sparse indices.
+ default_value : tvm.te.Tensor
Review comment:
type shoube be `relay.Expr` here
##
File path: tests/python/relay/test_op_level3.py
##
@@ -780,4 +826,5 @@ def verify_unravel_index(indices, shape, dtype):
test_gather_nd()
test_isfinite()
test_isinf()
-test_unravel_index()
\ No newline at end of file
+test_unravel_index()
+test_sparse_to_dense()
Review comment:
Final newline missing
##
File path: src/relay/op/tensor/transform.cc
##
@@ -2732,5 +2732,87 @@ Example::
.set_attr("FTVMCompute", UnRavelIndexCompute)
.set_attr("TOpPattern", kInjective);
+// sparse_to_dense
+TVM_REGISTER_NODE_TYPE(SparseToDenseAttrs);
+
+bool SparseToDenseRel(const Array& types,
+int num_inputs,
+const Attrs& attrs,
+const TypeReporter& reporter) {
Review comment:
Not aligned
##
File path: src/relay/op/tensor/transform.cc
##
@@ -2732,5 +2732,87 @@ Example::
.set_attr("FTVMCompute", UnRavelIndexCompute)
.set_attr("TOpPattern", kInjective);
+// sparse_to_dense
+TVM_REGISTER_NODE_TYPE(SparseToDenseAttrs);
+
+bool SparseToDenseRel(const Array& types,
+int num_inputs,
+const Attrs& attrs,
+const TypeReporter& reporter) {
+ CHECK_EQ(num_inputs, 3);
+ auto sparse_indices = types[0].as();
+ auto sparse_values = types[1].as();
+ auto default_value = types[2].as();
+ CHECK(sparse_indices != nullptr && sparse_values != nullptr && default_value
!= nullptr);
+
+ CHECK(sparse_indices->dtype.is_int())
+ << "sparse_indices must be tensor of integers";
+
+ CHECK_LE(sparse_indices->shape.size(), 3)
+ << "sparse_indices must be a tensor of either 0D, 1D or 2D";
+
+ CHECK_LE(sparse_values->shape.size(), 2)
+ << "sparse_values must be a tensor of either 0D, 1D";
+
+ CHECK_EQ(default_value->shape.size(), 0)
+<< "default_value should be a scalar";
+
+ const auto* param = attrs.as();
+ CHECK(param != nullptr);
+
+ Array oshape;
+ for (auto i : param->output_shape) {
+oshape.push_back(i);
+ }
+ reporter->Assign(types[3], TensorType(oshape, sparse_values->dtype));
+ return true;
+}
+
+Array SparseToDenseCompute(const Attrs& attrs,
+ const Array& inputs,
+ const Type& out_type) {
+ CHECK_EQ(inputs.size(), 3);
+ const auto* param = attrs.as();
+ CHECK(param != nullptr);
+ return {topi::sparse_to_dense(inputs[0], inputs[1], inputs[2],
param->output_shape)};
+}
+
+TVM_REGISTER_GLOBAL("relay.op._make.sparse_to_dense")
+.set_body_typed([](Expr indices, Expr values, Expr default_value,
Array output_shape) {
+ auto attrs = make_object();
+ attrs->output_shape = std::move(output_shape);
+ static const Op& op = Op::Get("sparse_to_dense");
+ return Call(op, {indices, values, default_value}, Attrs(attrs));
+});
+
+RELAY_REGISTER_OP("sparse_to_dense")
+.describe(R"code(A dense tensor from a sparse representation.
+
+- **sparse_indices**: A 0-D, 1-D, or 2-D tensor of integers containing
location of sparse values
+
+- **sparse_values**: A 0-D or 1-D tensor containing the sparse values for the
sparse indices.
+
+- **default_value**: A 0-D tensor containing the default value for the
remaining locations. Defaults to 0.
+
+- **output_shape**: A list of integers. Shape of the dense output tensor.
+
+ Example::
+-
[GitHub] [incubator-tvm] wpan11nv commented on pull request #5382: [TE] Fix MakeLoopNest for warp memory
wpan11nv commented on pull request #5382: URL: https://github.com/apache/incubator-tvm/pull/5382#issuecomment-620140371 > Do you mean requiring the users to tag the iter_var or we do it in InferBound? If the former, maybe we can merge this PR first and then start an RFC for the API change. If the latter, can you point out where to tag the iter_var? I am not familiar with InferBound. OK. I do not have a good idea on this. Probably we could just use this convention. I think there is no issue for CUDA. For OpenCL, a warp or subgroup is not necessarily dispatched along threadidx.x FWIK. Probably we care less on OpenCL part? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-tvm] areusch commented on a change in pull request #5417: [RUNTIME][uTVM] AutoTVM + uTVM for Cortex-M7
areusch commented on a change in pull request #5417:
URL: https://github.com/apache/incubator-tvm/pull/5417#discussion_r415949272
##
File path: python/tvm/autotvm/tuner/tuner.py
##
@@ -150,7 +150,15 @@ def tune(self, n_trial, measure_option,
early_stopping=None, callbacks=(), si_pr
i + k + 1, si_prefix, format_si_prefix(flops,
si_prefix),
format_si_prefix(self.best_flops, si_prefix),
res, config)
-i += len(results)
+num_successes = 0
+for result in results:
+if isinstance(result.costs[0], float):
+num_successes += 1
+if num_successes != len(results):
+logger.debug('not counting %d failures towards trial count',
+ len(results) - num_successes)
+i += num_successes
+
Review comment:
hmm okay. we do need to improve AutoTVM error handling. @tqchen what is
best to do here?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [incubator-tvm] areusch commented on a change in pull request #5417: [RUNTIME][uTVM] AutoTVM + uTVM for Cortex-M7
areusch commented on a change in pull request #5417:
URL: https://github.com/apache/incubator-tvm/pull/5417#discussion_r415981837
##
File path: src/runtime/micro/host_driven/utvm_runtime.c
##
@@ -34,89 +34,148 @@ extern "C" {
#include "utvm_runtime.h"
-// Task pointers must be patched before calling a function.
-UTVMTask utvm_task = {
-.func = NULL,
-.arg_values = NULL,
-.arg_type_codes = NULL,
-.num_args = 0,
-};
-
-size_t utvm_word_size = 0; // NOLINT(*)
+// TODO(areusch): move defines into header
+#define TASK_QUEUE_SIZE 20
+volatile UTVMTask utvm_tasks[TASK_QUEUE_SIZE] = { };
+volatile uint32_t utvm_num_tasks = 0;
+volatile uint32_t utvm_task_times[TASK_QUEUE_SIZE] = { };
// These pointers are patched at load time to point to the workspace section.
-char* utvm_workspace_start = NULL; // NOLINT(*)
-char* utvm_workspace_end = NULL;// NOLINT(*)
-char* utvm_workspace_curr = NULL; // NOLINT(*)
+volatile char* utvm_workspace_start = NULL; // NOLINT(*)
+volatile char* utvm_workspace_end = NULL;// NOLINT(*)
+volatile char* utvm_workspace_curr = NULL; // NOLINT(*)
+#define MAX_WS_ALLOCS 10
+volatile char* utvm_alloc_ends[MAX_WS_ALLOCS] = {}; // NOLINT(*)
Review comment:
let's leave it as is, since building binaries will likely change how we
configure most of this
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [incubator-tvm] areusch commented on a change in pull request #5417: [RUNTIME][uTVM] AutoTVM + uTVM for Cortex-M7
areusch commented on a change in pull request #5417:
URL: https://github.com/apache/incubator-tvm/pull/5417#discussion_r415989331
##
File path: src/runtime/micro/micro_common.h
##
@@ -52,28 +53,115 @@ enum class SectionKind : size_t {
kNumKinds,
};
-/*! \brief union for storing values on varying target word sizes */
-union TargetVal {
- /*! \brief 32-bit pointer */
- uint32_t val32;
- /*! \brief 64-bit pointer */
- uint64_t val64;
+/*! \brief data type for word sizes */
+class TargetWordSize {
+ private:
+ size_t word_size_bits_;
+
+ public:
+ explicit TargetWordSize(size_t word_size_bits) :
word_size_bits_{word_size_bits} {
+CHECK(word_size_bits == 32 || word_size_bits == 64)
+ << "only 32-bit and 64-bit are supported now";
+ }
+
+ size_t bytes() const {
+return word_size_bits_ / 8;
+ }
+
+ size_t bits() const {
+return word_size_bits_;
+ }
};
-/*! \brief absolute device address */
-class DevPtr {
+
+/*! \brief class for storing values on varying target word sizes */
+class TargetVal {
+ private:
+ size_t width_bits_;
+ uint64_t value_;
+
public:
- /*! \brief construct a device address with value `value` */
- explicit DevPtr(std::uintptr_t value) : value_(TargetVal { .val64 = value })
{}
+ /*! \brief construct a TargetVal matching the size of the given integral
argument */
+ template::value, T>::type>
+ explicit constexpr TargetVal(T value) : TargetVal(sizeof(T) * 8, value) {}
+
+ /*! \brief construct an uninitialized value */
+ TargetVal() : width_bits_{0}, value_{0} {}
+
+ /*! \brief construct a TargetVal with explicit size and value */
+ TargetVal(size_t width_bits, uint64_t value) : width_bits_{width_bits} {
+CHECK(width_bits >= 8 &&
+ width_bits <= 64 &&
+ (width_bits & (width_bits - 1)) == 0)
+ << "width_bits must be a power of 2 in [8, 64], got " << width_bits;
+value_ = value & bitmask();
+ }
+
+ bool is_initialized() const { return width_bits_ != 0; }
Review comment:
done
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [incubator-tvm] areusch commented on a change in pull request #5417: [RUNTIME][uTVM] AutoTVM + uTVM for Cortex-M7
areusch commented on a change in pull request #5417:
URL: https://github.com/apache/incubator-tvm/pull/5417#discussion_r415988618
##
File path: src/runtime/micro/micro_common.h
##
@@ -52,28 +53,115 @@ enum class SectionKind : size_t {
kNumKinds,
};
-/*! \brief union for storing values on varying target word sizes */
-union TargetVal {
- /*! \brief 32-bit pointer */
- uint32_t val32;
- /*! \brief 64-bit pointer */
- uint64_t val64;
+/*! \brief data type for word sizes */
+class TargetWordSize {
+ private:
+ size_t word_size_bits_;
Review comment:
done
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [incubator-tvm] areusch commented on a change in pull request #5417: [RUNTIME][uTVM] AutoTVM + uTVM for Cortex-M7
areusch commented on a change in pull request #5417:
URL: https://github.com/apache/incubator-tvm/pull/5417#discussion_r415988262
##
File path: src/runtime/micro/host_driven/utvm_runtime.h
##
@@ -46,20 +62,38 @@ typedef struct {
int32_t num_args;
} UTVMTask;
+/*!
+ * \brief TODO
+ */
extern void UTVMInit();
-extern void UTVMTimerReset();
-
+/*!
+ * \brief TODO
+ */
extern int32_t UTVMTimerStart();
-extern void UTVMTimerStop();
-
-extern uint32_t UTVMTimerRead();
+/*!
+ * \brief TODO
+ */
+extern uint32_t UTVMTimerStop(int32_t* err);
+/*!
+ * \brief TODO
+ */
void UTVMMain();
+/*!
+ * \brief TODO
Review comment:
done
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [incubator-tvm] siju-samuel commented on a change in pull request #5452: [Frontend][TFLite] L2_POOL_2D operator
siju-samuel commented on a change in pull request #5452: URL: https://github.com/apache/incubator-tvm/pull/5452#discussion_r415987001 ## File path: tests/python/frontend/tflite/test_forward.py ## @@ -561,6 +561,31 @@ def test_forward_pooling(): strides=[2, 1]) +def _test_l2_pool2d(input_shape, ksize, strides, padding, data_format, fused_func_name=None): +x = np.arange(np.prod(input_shape), dtype=np.float32).reshape(input_shape) - 1 + +with tf.Graph().as_default(): +in_data = tf.placeholder( +dtype=tf.float32, name="input", shape=input_shape) +out = tf.sqrt(tf.nn.avg_pool( +tf.square(in_data), ksize=ksize, strides=strides, +padding=padding, data_format=data_format)) +out = with_fused_activation_function(out, fused_func_name) + +compare_tflite_with_tvm(x, 'input', [in_data], [out]) + + +def test_l2_pool2d(): Review comment: add `test_l2_pool2d` to __main__ ## File path: python/tvm/relay/frontend/tflite.py ## @@ -1770,6 +1775,12 @@ def convert_pool2d(self, op, pool_type): assert self.has_same_qnn_params(input_tensor, output_tensor), \ "qnn.op.max_pool2d requires input and output qnn params to be same" out = _op.nn.max_pool2d(in_expr, **params) +elif pool_type == "l2": +# l2_pool_2d is equivalent to square_root(avg_pool(square(in_data))) Review comment: Add a protection check to throw error if you receive quantized inputs. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-tvm] tmoreau89 commented on a change in pull request #5417: [RUNTIME][uTVM] AutoTVM + uTVM for Cortex-M7
tmoreau89 commented on a change in pull request #5417:
URL: https://github.com/apache/incubator-tvm/pull/5417#discussion_r416116610
##
File path: python/tvm/autotvm/tuner/tuner.py
##
@@ -150,7 +150,15 @@ def tune(self, n_trial, measure_option,
early_stopping=None, callbacks=(), si_pr
i + k + 1, si_prefix, format_si_prefix(flops,
si_prefix),
format_si_prefix(self.best_flops, si_prefix),
res, config)
-i += len(results)
+num_successes = 0
+for result in results:
+if isinstance(result.costs[0], float):
+num_successes += 1
+if num_successes != len(results):
+logger.debug('not counting %d failures towards trial count',
+ len(results) - num_successes)
+i += num_successes
+
Review comment:
Yeah I would be in favor of cleaning that up; it's implied that trial
count includes both successful and unsuccessful runs. If we wanted to change
that definition, I think we'd need an RFC to isolate that change.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [incubator-tvm] u99127 commented on a change in pull request #5447: [TOPI,RELAY][TFLITE] Sparse to dense operator
u99127 commented on a change in pull request #5447:
URL: https://github.com/apache/incubator-tvm/pull/5447#discussion_r416152206
##
File path: python/tvm/relay/frontend/tflite.py
##
@@ -2075,6 +2077,29 @@ def convert_space_to_depth(self, op):
return out
+def convert_sparse_to_dense(self, op):
+"""Convert TFLite SPARSE_TO_DENSE"""
+try:
+from tflite.Operator import Operator
+except ImportError:
+raise ImportError("The tflite package must be installed")
+
+assert isinstance(op, Operator)
+input_tensors = self.get_input_tensors(op)
+assert len(input_tensors) == 4, "input tensors length should be 4"
+indices, values = input_tensors[0], input_tensors[2]
+default_value = input_tensors[3]
+output_shape = input_tensors[1]
+
Review comment:
Do we need to have some additional data type checking for int32 and
int64 ?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [incubator-tvm] tmoreau89 commented on a change in pull request #5417: [RUNTIME][uTVM] AutoTVM + uTVM for Cortex-M7
tmoreau89 commented on a change in pull request #5417:
URL: https://github.com/apache/incubator-tvm/pull/5417#discussion_r416120588
##
File path: src/runtime/micro/micro_session.cc
##
@@ -209,59 +246,124 @@ MicroSession::~MicroSession() {
low_level_device_ = nullptr;
}
-double MicroSession::PushToExecQueue(DevPtr func_ptr, const TVMArgs& args) {
+void MicroSession::PushToTaskQueue(TargetPtr func_ptr, const TVMArgs& args) {
if (thumb_mode_) {
+// TODO(areusch): should be |=
func_ptr += 1;
}
+ TargetVal func_dev_addr = func_ptr.value();
+
+ std::tuple arg_field_addrs =
EncoderAppend(_args_encoder_, args);
+ TargetVal arg_values_dev_addr{std::get<0>(arg_field_addrs).value()};
+ TargetVal arg_type_codes_dev_addr{std::get<1>(arg_field_addrs).value()};
+
+ task_queue_.push_back(
+ DevTask {
+.func = func_dev_addr,
+.arg_values = arg_values_dev_addr,
+.arg_type_codes = arg_type_codes_dev_addr,
+.num_args = args.num_args
+ });
+
+ if (task_queue_.size() == MicroSession::kTaskQueueCapacity) {
+FlushTaskQueue();
+ }
+}
- // Create an allocator stream for the memory region after the most recent
- // allocation in the args section.
- DevPtr args_addr = GetAllocator(SectionKind::kArgs)->curr_end_addr();
- TargetDataLayoutEncoder encoder(args_addr, word_size_);
-
- std::tuple arg_field_addrs = EncoderAppend(, args);
-
- // Flush `stream` to device memory.
- DevPtr stream_dev_addr =
- GetAllocator(SectionKind::kArgs)->Allocate(encoder.buf_size());
- low_level_device()->Write(stream_dev_addr,
-reinterpret_cast(encoder.data()),
-encoder.buf_size());
-
- TargetVal arg_values_dev_addr = std::get<0>(arg_field_addrs).value();
- TargetVal arg_type_codes_dev_addr = std::get<1>(arg_field_addrs).value();
- if (word_size_ == 4) {
-UTVMTask32 task = {
- .func = func_ptr.value().val32,
- .arg_values = arg_values_dev_addr.val32,
- .arg_type_codes = arg_type_codes_dev_addr.val32,
- .num_args = args.num_args,
-};
-// Write the task.
-DevSymbolWrite(runtime_symbol_map_, "utvm_task", task);
- } else if (word_size_ == 8) {
-UTVMTask64 task = {
- .func = func_ptr.value().val64,
- .arg_values = arg_values_dev_addr.val64,
- .arg_type_codes = arg_type_codes_dev_addr.val64,
- .num_args = args.num_args,
-};
-// Write the task.
-DevSymbolWrite(runtime_symbol_map_, "utvm_task", task);
+void MicroSession::FlushTaskQueue() {
+ if (task_queue_.size() == 0) {
+// nothing to run
+return;
+ }
+ if (word_size_.bytes() == 4) {
+FlushTaskQueuePriv();
+ } else if (word_size_.bytes() == 8) {
+FlushTaskQueuePriv();
}
+}
- DevPtr utvm_init_addr = runtime_symbol_map_["UTVMInit"];
- DevPtr utvm_done_addr = runtime_symbol_map_["UTVMDone"];
+template
+void MicroSession::FlushTaskQueuePriv() {
+ // std::cout << "[MicroSession::FlushTaskQueue]" << std::endl;
+ std::vector prepped_tasks;
+ for (const auto& task : task_queue_) {
+prepped_tasks.push_back(T(task));
+ }
+
+ // Flush `args` to device memory.
+ low_level_device()->Write(
+ batch_args_encoder_.start_addr(),
+ reinterpret_cast(batch_args_encoder_.data()),
+ batch_args_encoder_.buf_size());
+
+ // Flush `tasks` to device memory.
+// runtime_symbol_map_.Dump(std::cout);
+ TargetPtr dev_tasks_addr = runtime_symbol_map_["utvm_tasks"];
+ low_level_device()->Write(
+ dev_tasks_addr,
+ reinterpret_cast(prepped_tasks.data()),
+ prepped_tasks.size() * sizeof(T));
+ DevSymbolWrite(runtime_symbol_map_, "utvm_num_tasks",
prepped_tasks.size());
+
+ TargetPtr utvm_init_addr = runtime_symbol_map_["UTVMInit"];
+ TargetPtr utvm_done_addr = runtime_symbol_map_["UTVMDone"];
if (thumb_mode_) {
+// TODO(areusch): should be |=
utvm_init_addr += 1;
}
+ std::chrono::time_point<
+std::chrono::high_resolution_clock, std::chrono::nanoseconds> tbegin, tend;
+ tbegin = std::chrono::high_resolution_clock::now();
+ // std::string tmp;
Review comment:
Do we want to clean up those comments
##
File path: src/runtime/micro/micro_session.cc
##
@@ -209,59 +246,124 @@ MicroSession::~MicroSession() {
low_level_device_ = nullptr;
}
-double MicroSession::PushToExecQueue(DevPtr func_ptr, const TVMArgs& args) {
+void MicroSession::PushToTaskQueue(TargetPtr func_ptr, const TVMArgs& args) {
if (thumb_mode_) {
+// TODO(areusch): should be |=
func_ptr += 1;
}
+ TargetVal func_dev_addr = func_ptr.value();
+
+ std::tuple arg_field_addrs =
EncoderAppend(_args_encoder_, args);
+ TargetVal arg_values_dev_addr{std::get<0>(arg_field_addrs).value()};
+ TargetVal arg_type_codes_dev_addr{std::get<1>(arg_field_addrs).value()};
+
+ task_queue_.push_back(
+ DevTask {
+.func = func_dev_addr,
+.arg_values = arg_values_dev_addr,
+.arg_type_codes =
[GitHub] [incubator-tvm] tmoreau89 commented on a change in pull request #5417: [RUNTIME][uTVM] AutoTVM + uTVM for Cortex-M7
tmoreau89 commented on a change in pull request #5417: URL: https://github.com/apache/incubator-tvm/pull/5417#discussion_r416121088 ## File path: src/runtime/micro/micro_session.cc ## @@ -404,39 +509,74 @@ DevPtr MicroSession::EncoderAppend(TargetDataLayoutEncoder* encoder, const DLTen return tvm_arr_slot.start_addr(); } +// TODO(weberlo): switch over entirely to error codes that expand to error +// messages on the host side. Review comment: I suggest it gets cleaned up then This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-tvm] tmoreau89 commented on a change in pull request #5417: [RUNTIME][uTVM] AutoTVM + uTVM for Cortex-M7
tmoreau89 commented on a change in pull request #5417: URL: https://github.com/apache/incubator-tvm/pull/5417#discussion_r416121933 ## File path: tests/python/unittest/test_runtime_micro.py ## @@ -25,8 +25,25 @@ from tvm.micro import create_micro_mod from tvm.relay.testing import resnet -# Use the host emulated micro device. -DEV_CONFIG = micro.device.host.default_config() +# # Use the host emulated micro device. +DEV_CONFIG_A = micro.device.host.generate_config() +DEV_CONFIG_B = micro.device.host.generate_config() +TARGET = 'c -device=micro_dev' + +# # TODO why do spike examples have memory that starts at 0x1000, but you Review comment: Do we need to keep these lines commented out? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-tvm] tqchen commented on issue #5455: [CI] QNN Compilation Error During Bionic Docker Update
tqchen commented on issue #5455: URL: https://github.com/apache/incubator-tvm/issues/5455#issuecomment-620223340 also cc @yongfeng-nv @hzfan since both of you have touched/reviewed the const int bound recently This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-tvm] areusch commented on a change in pull request #5417: [RUNTIME][uTVM] AutoTVM + uTVM for Cortex-M7
areusch commented on a change in pull request #5417:
URL: https://github.com/apache/incubator-tvm/pull/5417#discussion_r416140211
##
File path: python/tvm/autotvm/tuner/callback.py
##
@@ -144,12 +144,12 @@ def __del__(self):
def _callback(tuner, inputs, results):
ctx.ct += len(inputs)
-flops = 0
+flops = float("inf")
Review comment:
reverted, this was cruft that snuck in
##
File path: python/tvm/autotvm/tuner/callback.py
##
@@ -144,12 +144,12 @@ def __del__(self):
def _callback(tuner, inputs, results):
ctx.ct += len(inputs)
-flops = 0
+flops = float("inf")
for inp, res in zip(inputs, results):
if res.error_no == 0:
-flops = inp.task.flop / np.mean(res.costs)
+flops = min(inp.task.flop / np.mean(res.costs), flops)
Review comment:
reverted
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [incubator-tvm] areusch commented on a change in pull request #5417: [RUNTIME][uTVM] AutoTVM + uTVM for Cortex-M7
areusch commented on a change in pull request #5417:
URL: https://github.com/apache/incubator-tvm/pull/5417#discussion_r416140782
##
File path: python/tvm/autotvm/tuner/tuner.py
##
@@ -150,7 +150,15 @@ def tune(self, n_trial, measure_option,
early_stopping=None, callbacks=(), si_pr
i + k + 1, si_prefix, format_si_prefix(flops,
si_prefix),
format_si_prefix(self.best_flops, si_prefix),
res, config)
-i += len(results)
+num_successes = 0
+for result in results:
+if isinstance(result.costs[0], float):
+num_successes += 1
+if num_successes != len(results):
+logger.debug('not counting %d failures towards trial count',
+ len(results) - num_successes)
+i += num_successes
+
Review comment:
reverted
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [incubator-tvm] mbrookhart opened a new pull request #5454: Add RoiAlign to Onnx frontend
mbrookhart opened a new pull request #5454: URL: https://github.com/apache/incubator-tvm/pull/5454 @jwfromm @masahi @soiferj Thank you! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-tvm] tmoreau89 commented on a change in pull request #5417: [RUNTIME][uTVM] AutoTVM + uTVM for Cortex-M7
tmoreau89 commented on a change in pull request #5417:
URL: https://github.com/apache/incubator-tvm/pull/5417#discussion_r416122541
##
File path: topi/python/topi/arm_cpu/cortex_m7/micro_kernel/gemm.py
##
@@ -0,0 +1,221 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+# pylint: disable=invalid-name, no-value-for-parameter
+"""Defines gemm intrinsics for SIMD matrix multiplication."""
+
+import random
+import string
+
+import tvm
+from tvm import te
+
+##
+# MxKxN MatMul Intrinsic #
+##
+
+# NOTE this is transposed matmul (A * B^T)
+def intrin_gemm_MxKxN(M, K, N, in_dtype, out_dtype):
+"""Defines a SIMD-accelerated transposed matmul."""
+# we generate a unique ID for every intrinsic definition, to prevent name
+# collisions in the generated source (e.g., if there are multiple operators
+# in the same module that use the same intrinsic)
+#
+# TODO(weberlo, areusch): to cut down on memory usage, we should cache
each intrinsic
+# instantiation and include it only once, eliminating the need for unique
+# IDs
+UNIQ_ID_LEN = 8
+uniq_id = ''.join(random.choices(string.ascii_uppercase, k=UNIQ_ID_LEN))
+
+if isinstance(M, tvm.tir.IntImm):
+M = M.value
+if isinstance(K, tvm.tir.IntImm):
+K = K.value
+if isinstance(N, tvm.tir.IntImm):
+N = N.value
+assert K % 4 == 0
+# TODO(weberlo, areusch): support more dtypes?
+assert in_dtype == 'int8'
+assert out_dtype == 'int32'
+A = te.placeholder((M, K), name='a', dtype=in_dtype)
+B = te.placeholder((N, K), name='b', dtype=in_dtype)
+k = te.reduce_axis((0, K), name='k')
+C = te.compute(
+(M, N),
+lambda i, j: te.sum(A[i, k].astype(out_dtype) * B[j,
k].astype(out_dtype), axis=k),
+name='c')
+A_buf = tvm.tir.decl_buffer(
+A.shape, A.dtype,
+name="A",
+offset_factor=1,
+strides=[te.var("A_s"), 1])
+B_buf = tvm.tir.decl_buffer(
+B.shape, B.dtype,
+name="B",
+offset_factor=1,
+strides=[te.var("B_s"), 1])
+C_buf = tvm.tir.decl_buffer(
+C.shape, C.dtype,
+name="C",
+offset_factor=1,
+strides=[te.var("C_s"), 1])
+def intrin_func(ins, outs):
+aa, bb = ins
+cc = outs[0]
+def _reduce_update():
+ib = tvm.tir.ir_builder.create()
+ib.emit(tvm.tir.call_extern("int32",
f"gemm_{M}x{K}x{N}_update_{uniq_id}",
+aa.access_ptr("r"),
+bb.access_ptr("r"),
+cc.access_ptr("w"),
+aa.strides[0],
+bb.strides[0],
+cc.strides[0]))
+return ib.get()
+def _reduce_reset():
+ib = tvm.tir.ir_builder.create()
+ib.emit(tvm.tir.call_extern("int32",
f"gemm_{M}x{K}x{N}_reset_{uniq_id}",
+cc.access_ptr("w"),
+cc.strides[0]))
+return ib.get()
+def _body():
+ib = tvm.tir.ir_builder.create()
+# # NOTE we need the reset in the body for cases where the buffer
Review comment:
cleanup?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [incubator-tvm] tmoreau89 commented on a change in pull request #5417: [RUNTIME][uTVM] AutoTVM + uTVM for Cortex-M7
tmoreau89 commented on a change in pull request #5417:
URL: https://github.com/apache/incubator-tvm/pull/5417#discussion_r416122882
##
File path: python/tvm/autotvm/tuner/callback.py
##
@@ -144,12 +144,12 @@ def __del__(self):
def _callback(tuner, inputs, results):
ctx.ct += len(inputs)
-flops = 0
+flops = float("inf")
Review comment:
keep it set to 0 by default
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [incubator-tvm] tmoreau89 commented on a change in pull request #5417: [RUNTIME][uTVM] AutoTVM + uTVM for Cortex-M7
tmoreau89 commented on a change in pull request #5417:
URL: https://github.com/apache/incubator-tvm/pull/5417#discussion_r416122964
##
File path: python/tvm/autotvm/tuner/callback.py
##
@@ -144,12 +144,12 @@ def __del__(self):
def _callback(tuner, inputs, results):
ctx.ct += len(inputs)
-flops = 0
+flops = float("inf")
for inp, res in zip(inputs, results):
if res.error_no == 0:
-flops = inp.task.flop / np.mean(res.costs)
+flops = min(inp.task.flop / np.mean(res.costs), flops)
Review comment:
max flops?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [incubator-tvm] tqchen commented on issue #5449: Windows OSError: "exception: access violation reading 0x0000000000000000"
tqchen commented on issue #5449: URL: https://github.com/apache/incubator-tvm/issues/5449#issuecomment-620220730 Please open a new trouble shooting thread on https://discuss.tvm.ai/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-tvm] areusch commented on a change in pull request #5417: [RUNTIME][uTVM] AutoTVM + uTVM for Cortex-M7
areusch commented on a change in pull request #5417:
URL: https://github.com/apache/incubator-tvm/pull/5417#discussion_r416138054
##
File path: src/runtime/micro/micro_session.cc
##
@@ -209,59 +246,124 @@ MicroSession::~MicroSession() {
low_level_device_ = nullptr;
}
-double MicroSession::PushToExecQueue(DevPtr func_ptr, const TVMArgs& args) {
+void MicroSession::PushToTaskQueue(TargetPtr func_ptr, const TVMArgs& args) {
if (thumb_mode_) {
+// TODO(areusch): should be |=
func_ptr += 1;
}
+ TargetVal func_dev_addr = func_ptr.value();
+
+ std::tuple arg_field_addrs =
EncoderAppend(_args_encoder_, args);
+ TargetVal arg_values_dev_addr{std::get<0>(arg_field_addrs).value()};
+ TargetVal arg_type_codes_dev_addr{std::get<1>(arg_field_addrs).value()};
+
+ task_queue_.push_back(
+ DevTask {
+.func = func_dev_addr,
+.arg_values = arg_values_dev_addr,
+.arg_type_codes = arg_type_codes_dev_addr,
+.num_args = args.num_args
+ });
+
+ if (task_queue_.size() == MicroSession::kTaskQueueCapacity) {
+FlushTaskQueue();
+ }
+}
- // Create an allocator stream for the memory region after the most recent
- // allocation in the args section.
- DevPtr args_addr = GetAllocator(SectionKind::kArgs)->curr_end_addr();
- TargetDataLayoutEncoder encoder(args_addr, word_size_);
-
- std::tuple arg_field_addrs = EncoderAppend(, args);
-
- // Flush `stream` to device memory.
- DevPtr stream_dev_addr =
- GetAllocator(SectionKind::kArgs)->Allocate(encoder.buf_size());
- low_level_device()->Write(stream_dev_addr,
-reinterpret_cast(encoder.data()),
-encoder.buf_size());
-
- TargetVal arg_values_dev_addr = std::get<0>(arg_field_addrs).value();
- TargetVal arg_type_codes_dev_addr = std::get<1>(arg_field_addrs).value();
- if (word_size_ == 4) {
-UTVMTask32 task = {
- .func = func_ptr.value().val32,
- .arg_values = arg_values_dev_addr.val32,
- .arg_type_codes = arg_type_codes_dev_addr.val32,
- .num_args = args.num_args,
-};
-// Write the task.
-DevSymbolWrite(runtime_symbol_map_, "utvm_task", task);
- } else if (word_size_ == 8) {
-UTVMTask64 task = {
- .func = func_ptr.value().val64,
- .arg_values = arg_values_dev_addr.val64,
- .arg_type_codes = arg_type_codes_dev_addr.val64,
- .num_args = args.num_args,
-};
-// Write the task.
-DevSymbolWrite(runtime_symbol_map_, "utvm_task", task);
+void MicroSession::FlushTaskQueue() {
+ if (task_queue_.size() == 0) {
+// nothing to run
+return;
+ }
+ if (word_size_.bytes() == 4) {
+FlushTaskQueuePriv();
+ } else if (word_size_.bytes() == 8) {
+FlushTaskQueuePriv();
}
+}
- DevPtr utvm_init_addr = runtime_symbol_map_["UTVMInit"];
- DevPtr utvm_done_addr = runtime_symbol_map_["UTVMDone"];
+template
+void MicroSession::FlushTaskQueuePriv() {
+ // std::cout << "[MicroSession::FlushTaskQueue]" << std::endl;
+ std::vector prepped_tasks;
+ for (const auto& task : task_queue_) {
+prepped_tasks.push_back(T(task));
+ }
+
+ // Flush `args` to device memory.
+ low_level_device()->Write(
+ batch_args_encoder_.start_addr(),
+ reinterpret_cast(batch_args_encoder_.data()),
+ batch_args_encoder_.buf_size());
+
+ // Flush `tasks` to device memory.
+// runtime_symbol_map_.Dump(std::cout);
+ TargetPtr dev_tasks_addr = runtime_symbol_map_["utvm_tasks"];
+ low_level_device()->Write(
+ dev_tasks_addr,
+ reinterpret_cast(prepped_tasks.data()),
+ prepped_tasks.size() * sizeof(T));
+ DevSymbolWrite(runtime_symbol_map_, "utvm_num_tasks",
prepped_tasks.size());
+
+ TargetPtr utvm_init_addr = runtime_symbol_map_["UTVMInit"];
+ TargetPtr utvm_done_addr = runtime_symbol_map_["UTVMDone"];
if (thumb_mode_) {
+// TODO(areusch): should be |=
utvm_init_addr += 1;
}
+ std::chrono::time_point<
+std::chrono::high_resolution_clock, std::chrono::nanoseconds> tbegin, tend;
+ tbegin = std::chrono::high_resolution_clock::now();
+ // std::string tmp;
Review comment:
these ones in particular might be okay to leave in for now since they
are helpful if you want to debug. after this, since we want to change the way
we load code, i'd suggest we improve the debugging story here. but, it will
change as we move to flashing binaries.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [incubator-tvm] u99127 commented on a change in pull request #5417: [RUNTIME][uTVM] AutoTVM + uTVM for Cortex-M7
u99127 commented on a change in pull request #5417: URL: https://github.com/apache/incubator-tvm/pull/5417#discussion_r413829506 ## File path: python/tvm/autotvm/measure/local_executor.py ## @@ -145,6 +145,7 @@ def submit(self, func, *args, **kwargs): if not self.do_fork: return LocalFutureNoFork(func(*args, **kwargs)) +# TODO why they choose a queue size of 2? add a comment Review comment: Why is this 2 ? ## File path: python/tvm/micro/device/arm/stm32f746xx.py ## @@ -36,23 +55,41 @@ def create_micro_lib(obj_path, src_path, lib_type, options=None): options : Optional[List[str]] additional options to pass to GCC + +lib_src_paths : Optional[List[str]] +TODO """ if options is None: options = [] +else: +options = list(options) + options += [ -"-mcpu=cortex-m7", -"-mlittle-endian", -"-mfloat-abi=hard", -"-mfpu=fpv5-sp-d16", -"-mthumb", -"-gdwarf-5", +# TODO(weberlo): make a debug flag +'-O2', +'-march=armv7e-m', Review comment: This is superfluous. -mcpu is a specific manifestation of -march for a particular implementation. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-tvm] tqchen commented on issue #5455: [CI] QNN Compilation Error During Bionic Docker Update
tqchen commented on issue #5455:
URL: https://github.com/apache/incubator-tvm/issues/5455#issuecomment-620223143
```
self =
args = (IRModuleNode( {GlobalVar(main): FunctionNode([Var(x,
ty=TensorType([1, 1, 64, 64], uint8))], TensorType([1, 16, 64, 6...Type([1, 1,
64, 64], uint8), TensorType([16, 1, 3, 3], int8)]), [], (nullptr))}), {1: llvm
-mcpu=skylake-avx512}, None)
temp_args = [{1: llvm -mcpu=skylake-avx512}]
values =
tcodes =
def __call__(self, *args):
"""Call the function with positional arguments
args : list
The positional arguments to the function call.
"""
temp_args = []
values, tcodes, num_args = _make_tvm_args(args, temp_args)
ret_val = TVMValue()
ret_tcode = ctypes.c_int()
if _LIB.TVMFuncCall(
self.handle, values, tcodes, ctypes.c_int(num_args),
ctypes.byref(ret_val), ctypes.byref(ret_tcode)) != 0:
> raise get_last_ffi_error()
E tvm._ffi.base.TVMError: Traceback (most recent call last):
E [bt] (8) /workspace/build/libtvm.so(+0x95e727) [0x7fe426761727]
E [bt] (7) /workspace/build/libtvm.so(+0x96e3b4) [0x7fe4267713b4]
E [bt] (6) /workspace/build/libtvm.so(+0x969334) [0x7fe42676c334]
E [bt] (5) /workspace/build/libtvm.so(+0x96fd39) [0x7fe426772d39]
E [bt] (4) /workspace/build/libtvm.so(+0x95e727) [0x7fe426761727]
E [bt] (3) /workspace/build/libtvm.so(+0x96e1e9) [0x7fe4267711e9]
E [bt] (2) /workspace/build/libtvm.so(+0x940f54) [0x7fe426743f54]
E [bt] (1) /workspace/build/libtvm.so(+0x94c913) [0x7fe42674f913]
E [bt] (0) /workspace/build/libtvm.so(+0xa9f52b) [0x7fe4268a252b]
E File "/workspace/python/tvm/relay/backend/_backend.py", line
49, in lower
E f = tvm.driver.lower(sch, inputs, name=func_name)
E File "/workspace/python/tvm/driver/build_module.py", line 215,
in lower
E mod = optimize(mod)
E File "/workspace/python/tvm/ir/transform.py", line 141, in
__call__
E return _ffi_transform_api.RunPass(self, mod)
E File "/workspace/python/tvm/_ffi/_ctypes/packed_func.py", line
219, in __call__
E raise get_last_ffi_error()
E [bt] (8)
/workspace/build/libtvm.so(tvm::tir::ExprVisitor::VisitExpr_(tvm::tir::LoadNode
const*)+0x16) [0x7fe4261e0cf6]
E [bt] (7) /workspace/build/libtvm.so(+0x4bf027) [0x7fe4262c2027]
E [bt] (6)
/workspace/build/libtvm.so(tvm::arith::ConstIntBoundAnalyzer::operator()(tvm::PrimExpr
const&, std::unordered_map,
std::equal_to,
std::allocator > >*)+0x1b) [0x7fe42600882b]
E [bt] (5) /workspace/build/libtvm.so(+0x20da92) [0x7fe426010a92]
E [bt] (4) /workspace/build/libtvm.so(+0x20f1a9) [0x7fe4260121a9]
E [bt] (3) /workspace/build/libtvm.so(+0x20da92) [0x7fe426010a92]
E [bt] (2) /workspace/build/libtvm.so(+0x20e27b) [0x7fe42601127b]
E [bt] (1) /workspace/build/libtvm.so(+0x20dd2f) [0x7fe426010d2f]
E [bt] (0) /workspace/build/libtvm.so(+0x12a65c) [0x7fe425f2d65c]
E File "/workspace/src/arith/const_int_bound.cc", line 157
E File "/workspace/python/tvm/_ffi/_ctypes/packed_func.py", line
78, in cfun
E rv = local_pyfunc(*pyargs)
E File "/workspace/python/tvm/relay/backend/_backend.py", line
57, in lower
E raise RuntimeError(msg)
E File "/workspace/python/tvm/relay/backend/_backend.py", line
49, in lower
E f = tvm.driver.lower(sch, inputs, name=func_name)
E File "/workspace/python/tvm/driver/build_module.py", line 215,
in lower
E mod = optimize(mod)
E File "/workspace/python/tvm/ir/transform.py", line 141, in
__call__
E return _ffi_transform_api.RunPass(self, mod)
E File "/workspace/python/tvm/_ffi/_ctypes/packed_func.py", line
219, in __call__
E raise get_last_ffi_error()
E [bt] (8)
/workspace/build/libtvm.so(tvm::tir::ExprVisitor::VisitExpr_(tvm::tir::LoadNode
const*)+0x16) [0x7fe4261e0cf6]
E [bt] (7) /workspace/build/libtvm.so(+0x4bf027) [0x7fe4262c2027]
E [bt] (6)
/workspace/build/libtvm.so(tvm::arith::ConstIntBoundAnalyzer::operator()(tvm::PrimExpr
const&, std::unordered_map,
std::equal_to,
std::allocator > >*)+0x1b) [0x7fe42600882b]
E [bt] (5) /workspace/build/libtvm.so(+0x20da92) [0x7fe426010a92]
E [bt] (4) /workspace/build/libtvm.so(+0x20f1a9) [0x7fe4260121a9]
E [bt] (3) /workspace/build/libtvm.so(+0x20da92) [0x7fe426010a92]
E [bt] (2)
[GitHub] [incubator-tvm] tqchen opened a new issue #5455: [CI] QNN Compilation Error During Bionic Docker Update
tqchen opened a new issue #5455: URL: https://github.com/apache/incubator-tvm/issues/5455 In the past week I attempted to upgrade the docker image CPU to bionic(ubuntu 18.04), during the time, a new unittest error occurs(note that the master CI was fine) in the int8 intrinsic test, it would be great it we can look into it. To reproduce, use the docker image `tvmai/ci-cpu:v0.62-t0` http://ci.tvm.ai:8080/job/temp-ci-docker-staging/job/ci-stage/30/execution/node/131/log/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-tvm] tqchen commented on issue #5455: [CI] QNN Compilation Error During Bionic Docker Update
tqchen commented on issue #5455: URL: https://github.com/apache/incubator-tvm/issues/5455#issuecomment-620222412 cc @anijain2305 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-tvm] areusch commented on a change in pull request #5417: [RUNTIME][uTVM] AutoTVM + uTVM for Cortex-M7
areusch commented on a change in pull request #5417: URL: https://github.com/apache/incubator-tvm/pull/5417#discussion_r416138909 ## File path: tests/python/unittest/test_runtime_micro.py ## @@ -25,8 +25,25 @@ from tvm.micro import create_micro_mod from tvm.relay.testing import resnet -# Use the host emulated micro device. -DEV_CONFIG = micro.device.host.default_config() +# # Use the host emulated micro device. +DEV_CONFIG_A = micro.device.host.generate_config() +DEV_CONFIG_B = micro.device.host.generate_config() +TARGET = 'c -device=micro_dev' + +# # TODO why do spike examples have memory that starts at 0x1000, but you Review comment: i'm not sure what they were there for. it seemed like a thing we would hit again, but we can always pull them from the demo tag. deleted. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-tvm] areusch commented on a change in pull request #5417: [RUNTIME][uTVM] AutoTVM + uTVM for Cortex-M7
areusch commented on a change in pull request #5417:
URL: https://github.com/apache/incubator-tvm/pull/5417#discussion_r416139282
##
File path: topi/python/topi/arm_cpu/cortex_m7/micro_kernel/gemm.py
##
@@ -0,0 +1,221 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+# pylint: disable=invalid-name, no-value-for-parameter
+"""Defines gemm intrinsics for SIMD matrix multiplication."""
+
+import random
+import string
+
+import tvm
+from tvm import te
+
+##
+# MxKxN MatMul Intrinsic #
+##
+
+# NOTE this is transposed matmul (A * B^T)
+def intrin_gemm_MxKxN(M, K, N, in_dtype, out_dtype):
+"""Defines a SIMD-accelerated transposed matmul."""
+# we generate a unique ID for every intrinsic definition, to prevent name
+# collisions in the generated source (e.g., if there are multiple operators
+# in the same module that use the same intrinsic)
+#
+# TODO(weberlo, areusch): to cut down on memory usage, we should cache
each intrinsic
+# instantiation and include it only once, eliminating the need for unique
+# IDs
+UNIQ_ID_LEN = 8
+uniq_id = ''.join(random.choices(string.ascii_uppercase, k=UNIQ_ID_LEN))
+
+if isinstance(M, tvm.tir.IntImm):
+M = M.value
+if isinstance(K, tvm.tir.IntImm):
+K = K.value
+if isinstance(N, tvm.tir.IntImm):
+N = N.value
+assert K % 4 == 0
+# TODO(weberlo, areusch): support more dtypes?
+assert in_dtype == 'int8'
+assert out_dtype == 'int32'
+A = te.placeholder((M, K), name='a', dtype=in_dtype)
+B = te.placeholder((N, K), name='b', dtype=in_dtype)
+k = te.reduce_axis((0, K), name='k')
+C = te.compute(
+(M, N),
+lambda i, j: te.sum(A[i, k].astype(out_dtype) * B[j,
k].astype(out_dtype), axis=k),
+name='c')
+A_buf = tvm.tir.decl_buffer(
+A.shape, A.dtype,
+name="A",
+offset_factor=1,
+strides=[te.var("A_s"), 1])
+B_buf = tvm.tir.decl_buffer(
+B.shape, B.dtype,
+name="B",
+offset_factor=1,
+strides=[te.var("B_s"), 1])
+C_buf = tvm.tir.decl_buffer(
+C.shape, C.dtype,
+name="C",
+offset_factor=1,
+strides=[te.var("C_s"), 1])
+def intrin_func(ins, outs):
+aa, bb = ins
+cc = outs[0]
+def _reduce_update():
+ib = tvm.tir.ir_builder.create()
+ib.emit(tvm.tir.call_extern("int32",
f"gemm_{M}x{K}x{N}_update_{uniq_id}",
+aa.access_ptr("r"),
+bb.access_ptr("r"),
+cc.access_ptr("w"),
+aa.strides[0],
+bb.strides[0],
+cc.strides[0]))
+return ib.get()
+def _reduce_reset():
+ib = tvm.tir.ir_builder.create()
+ib.emit(tvm.tir.call_extern("int32",
f"gemm_{M}x{K}x{N}_reset_{uniq_id}",
+cc.access_ptr("w"),
+cc.strides[0]))
+return ib.get()
+def _body():
+ib = tvm.tir.ir_builder.create()
+# # NOTE we need the reset in the body for cases where the buffer
Review comment:
done
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [incubator-tvm] u99127 commented on a change in pull request #5452: [Frontend][TFLite] L2_POOL_2D operator
u99127 commented on a change in pull request #5452: URL: https://github.com/apache/incubator-tvm/pull/5452#discussion_r416144493 ## File path: tests/python/frontend/tflite/test_forward.py ## @@ -487,6 +487,31 @@ def test_forward_pooling(): strides=[2, 1]) +def _test_l2_pool2d(input_shape, ksize, strides, padding, data_format, fused_func_name=None): +x = np.arange(np.prod(input_shape), dtype=np.float32).reshape(input_shape) - 1 + +with tf.Graph().as_default(): +in_data = tf.placeholder( +dtype=tf.float32, name="input", shape=input_shape) +out = tf.sqrt(tf.nn.avg_pool( +tf.square(in_data), ksize=ksize, strides=strides, +padding=padding, data_format=data_format)) +out = with_fused_activation_function(out, fused_func_name) + +compare_tflite_with_tvm(x, 'input', [in_data], [out]) Review comment: Thanks , possibly worth a belts and braces check to ensure we catch this in the frontend to ensure we think through qnn support rather than getting an unknown failure. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-tvm] masahi commented on issue #5455: [CI] QNN Compilation Error During Bionic Docker Update
masahi commented on issue #5455: URL: https://github.com/apache/incubator-tvm/issues/5455#issuecomment-620230784 I also saw this error when running `test_op_level2.py` locally. It happens randomly. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-tvm] tqchen commented on issue #5448: TypeError:'NoneType' object is not callable
tqchen commented on issue #5448: URL: https://github.com/apache/incubator-tvm/issues/5448#issuecomment-620221216 It is likely that you will also need to update your cython module, by typing `make cython3`, for future questions, please open a new trouble shooting thread on https://discuss.tvm.ai/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-tvm] areusch commented on a change in pull request #5417: [RUNTIME][uTVM] AutoTVM + uTVM for Cortex-M7
areusch commented on a change in pull request #5417:
URL: https://github.com/apache/incubator-tvm/pull/5417#discussion_r416140904
##
File path: src/runtime/micro/micro_session.cc
##
@@ -209,59 +246,124 @@ MicroSession::~MicroSession() {
low_level_device_ = nullptr;
}
-double MicroSession::PushToExecQueue(DevPtr func_ptr, const TVMArgs& args) {
+void MicroSession::PushToTaskQueue(TargetPtr func_ptr, const TVMArgs& args) {
if (thumb_mode_) {
+// TODO(areusch): should be |=
func_ptr += 1;
}
+ TargetVal func_dev_addr = func_ptr.value();
+
+ std::tuple arg_field_addrs =
EncoderAppend(_args_encoder_, args);
+ TargetVal arg_values_dev_addr{std::get<0>(arg_field_addrs).value()};
+ TargetVal arg_type_codes_dev_addr{std::get<1>(arg_field_addrs).value()};
+
+ task_queue_.push_back(
+ DevTask {
+.func = func_dev_addr,
+.arg_values = arg_values_dev_addr,
+.arg_type_codes = arg_type_codes_dev_addr,
+.num_args = args.num_args
+ });
+
+ if (task_queue_.size() == MicroSession::kTaskQueueCapacity) {
+FlushTaskQueue();
+ }
+}
- // Create an allocator stream for the memory region after the most recent
- // allocation in the args section.
- DevPtr args_addr = GetAllocator(SectionKind::kArgs)->curr_end_addr();
- TargetDataLayoutEncoder encoder(args_addr, word_size_);
-
- std::tuple arg_field_addrs = EncoderAppend(, args);
-
- // Flush `stream` to device memory.
- DevPtr stream_dev_addr =
- GetAllocator(SectionKind::kArgs)->Allocate(encoder.buf_size());
- low_level_device()->Write(stream_dev_addr,
-reinterpret_cast(encoder.data()),
-encoder.buf_size());
-
- TargetVal arg_values_dev_addr = std::get<0>(arg_field_addrs).value();
- TargetVal arg_type_codes_dev_addr = std::get<1>(arg_field_addrs).value();
- if (word_size_ == 4) {
-UTVMTask32 task = {
- .func = func_ptr.value().val32,
- .arg_values = arg_values_dev_addr.val32,
- .arg_type_codes = arg_type_codes_dev_addr.val32,
- .num_args = args.num_args,
-};
-// Write the task.
-DevSymbolWrite(runtime_symbol_map_, "utvm_task", task);
- } else if (word_size_ == 8) {
-UTVMTask64 task = {
- .func = func_ptr.value().val64,
- .arg_values = arg_values_dev_addr.val64,
- .arg_type_codes = arg_type_codes_dev_addr.val64,
- .num_args = args.num_args,
-};
-// Write the task.
-DevSymbolWrite(runtime_symbol_map_, "utvm_task", task);
+void MicroSession::FlushTaskQueue() {
+ if (task_queue_.size() == 0) {
+// nothing to run
+return;
+ }
+ if (word_size_.bytes() == 4) {
+FlushTaskQueuePriv();
+ } else if (word_size_.bytes() == 8) {
+FlushTaskQueuePriv();
}
+}
- DevPtr utvm_init_addr = runtime_symbol_map_["UTVMInit"];
- DevPtr utvm_done_addr = runtime_symbol_map_["UTVMDone"];
+template
+void MicroSession::FlushTaskQueuePriv() {
+ // std::cout << "[MicroSession::FlushTaskQueue]" << std::endl;
+ std::vector prepped_tasks;
+ for (const auto& task : task_queue_) {
+prepped_tasks.push_back(T(task));
+ }
+
+ // Flush `args` to device memory.
+ low_level_device()->Write(
+ batch_args_encoder_.start_addr(),
+ reinterpret_cast(batch_args_encoder_.data()),
+ batch_args_encoder_.buf_size());
+
+ // Flush `tasks` to device memory.
+// runtime_symbol_map_.Dump(std::cout);
Review comment:
done
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [incubator-tvm] u99127 commented on a change in pull request #5452: [Frontend][TFLite] L2_POOL_2D operator
u99127 commented on a change in pull request #5452: URL: https://github.com/apache/incubator-tvm/pull/5452#discussion_r416146885 ## File path: python/tvm/relay/frontend/tflite.py ## @@ -1770,6 +1775,12 @@ def convert_pool2d(self, op, pool_type): assert self.has_same_qnn_params(input_tensor, output_tensor), \ "qnn.op.max_pool2d requires input and output qnn params to be same" out = _op.nn.max_pool2d(in_expr, **params) +elif pool_type == "l2": +# l2_pool_2d is equivalent to square_root(avg_pool(square(in_data))) Review comment: Putting in the check has the following benefits: 1. Better user experience as the rest of the stack doesn't get basically undefined behaviour because of random input.. 2. A signal to us as a community in the form of a bug report that this is appearing. Ramana This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-tvm] kevinthesun commented on a change in pull request #5429: [RELAY][TF] Support symbolic newshape for Reshape
kevinthesun commented on a change in pull request #5429:
URL: https://github.com/apache/incubator-tvm/pull/5429#discussion_r416207982
##
File path: src/relay/transforms/pattern_util.h
##
@@ -554,7 +554,10 @@ static inline Expr Reshape(Expr data, Array
newshape) {
attrs->newshape = std::move(newshape);
attrs->reverse = false;
static const Op& op = Op::Get("reshape");
- return Call(op, {data}, Attrs(attrs), {});
+ std::vector value{1};
+ auto dummy_newshape_tensor = MakeConstantTensor(DataType::Int(32), {1},
value);
Review comment:
I think we should change the Reshape API here as well to make it
consistent. After that we can change the type of newshape attr to be Expr as
well.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [incubator-tvm] zhiics opened a new pull request #5457: [Fix] Add ConstantNode to IsAtomic
zhiics opened a new pull request #5457: URL: https://github.com/apache/incubator-tvm/pull/5457 The problem is described here: https://discuss.tvm.ai/t/relay-pass-feature-check-fail-during-fold-constant-pass/6498 @MarisaKirisame Can you take a look and see if the change makes sense? cc @kevinthesun This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-tvm] kevinthesun commented on a change in pull request #4312: [TOPI][Relay][OP] Dynamic NMS and strided_slice
kevinthesun commented on a change in pull request #4312: URL: https://github.com/apache/incubator-tvm/pull/4312#discussion_r416214792 ## File path: python/tvm/relay/op/transform.py ## @@ -613,13 +613,13 @@ def strided_slice(data, begin, end, strides=None): data : relay.Expr The source array to be sliced. -begin: list of int +begin: relay.Expr Review comment: Should we allow begin, end and strides to be list/tuple as well? And do convertsion to const before calling backend API. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-tvm] kevinthesun commented on a change in pull request #5429: [RELAY][TF] Support symbolic newshape for Reshape
kevinthesun commented on a change in pull request #5429:
URL: https://github.com/apache/incubator-tvm/pull/5429#discussion_r416228918
##
File path: src/relay/transforms/pattern_util.h
##
@@ -554,7 +554,10 @@ static inline Expr Reshape(Expr data, Array
newshape) {
attrs->newshape = std::move(newshape);
attrs->reverse = false;
static const Op& op = Op::Get("reshape");
- return Call(op, {data}, Attrs(attrs), {});
+ std::vector value{1};
+ auto dummy_newshape_tensor = MakeConstantTensor(DataType::Int(32), {1},
value);
Review comment:
Anther way is that we can overload Reshape in pattern_util.h to allow
newshape to be Array, but convert to a Constant inside it before calling actual
Reshape API.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [incubator-tvm] boh-inspur commented on a change in pull request #5428: [CODEGEN][CUDA] Fix a bug when vectorized load was involved for…
boh-inspur commented on a change in pull request #5428:
URL: https://github.com/apache/incubator-tvm/pull/5428#discussion_r416249062
##
File path: src/target/source/codegen_cuda.cc
##
@@ -274,9 +274,21 @@ void CodeGenCUDA::PrintVecElemLoad(
static const char access[] = {'x', 'y', 'z', 'w'};
CHECK(i >= 0 && i < (t.is_float16() ? 8 : 4));
if ((t.is_int()) && t.bits() == 8) {
-os << "((char)(" << vec << " >> " << i * 8 << "))";
+if (t.lanes() == 1) {
+ os << vec;
+} else if (t.lanes() == 2) {
+ os << vec << "." << access[i % 2];
+} else {
+ os << "((char)(" << vec << " >> " << i * 8 << "))";
+}
} else if ((t.is_uint()) && t.bits() == 8) {
-os << "((unsigned char)(" << vec << " >> " << i * 8 << "))";
+if (t.lanes() == 1) {
Review comment:
Do you mean store int8*2 as int16_t?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [incubator-tvm] liangfu commented on a change in pull request #5417: [RUNTIME][uTVM] AutoTVM + uTVM for Cortex-M7
liangfu commented on a change in pull request #5417: URL: https://github.com/apache/incubator-tvm/pull/5417#discussion_r416266168 ## File path: src/target/source/codegen_c_host.cc ## @@ -20,10 +20,10 @@ /*! * \file codegen_c_host.cc */ -#include +#include "codegen_c_host.h" Review comment: I think tvm_headers are those header files in the include dir, and they are exposed to external developers; while local headers are those used internally, and they are designed to interface multiple .cc source code files. Just my humble opinion. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-tvm] liangfu commented on a change in pull request #5417: [RUNTIME][uTVM] AutoTVM + uTVM for Cortex-M7
liangfu commented on a change in pull request #5417: URL: https://github.com/apache/incubator-tvm/pull/5417#discussion_r416272213 ## File path: Makefile ## @@ -73,7 +73,10 @@ build/libtvm_web_runtime.js: build/libtvm_web_runtime.bc cpplint: python3 3rdparty/dmlc-core/scripts/lint.py vta cpp vta/include vta/src python3 3rdparty/dmlc-core/scripts/lint.py topi cpp topi/include; - python3 3rdparty/dmlc-core/scripts/lint.py tvm cpp include src \ + # Note: exclude src/runtime/micro/host_driven becuase it contains C99 files. + python3 3rdparty/dmlc-core/scripts/lint.py tvm cpp \ +--exclude_path=src/runtime/micro/host_driven \ Review comment: agree This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-tvm] wpan11nv commented on a change in pull request #5428: [CODEGEN][CUDA] Fix a bug when vectorized load was involved for…
wpan11nv commented on a change in pull request #5428:
URL: https://github.com/apache/incubator-tvm/pull/5428#discussion_r416240568
##
File path: src/target/source/codegen_cuda.cc
##
@@ -274,9 +274,21 @@ void CodeGenCUDA::PrintVecElemLoad(
static const char access[] = {'x', 'y', 'z', 'w'};
CHECK(i >= 0 && i < (t.is_float16() ? 8 : 4));
if ((t.is_int()) && t.bits() == 8) {
-os << "((char)(" << vec << " >> " << i * 8 << "))";
+if (t.lanes() == 1) {
+ os << vec;
+} else if (t.lanes() == 2) {
+ os << vec << "." << access[i % 2];
+} else {
+ os << "((char)(" << vec << " >> " << i * 8 << "))";
+}
} else if ((t.is_uint()) && t.bits() == 8) {
-os << "((unsigned char)(" << vec << " >> " << i * 8 << "))";
+if (t.lanes() == 1) {
Review comment:
I know that, I meant this is true for all types. not necessarily for
char. You could common up logic for both branches,
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [incubator-tvm] wpan11nv commented on a change in pull request #5428: [CODEGEN][CUDA] Fix a bug when vectorized load was involved for…
wpan11nv commented on a change in pull request #5428:
URL: https://github.com/apache/incubator-tvm/pull/5428#discussion_r416251080
##
File path: src/target/source/codegen_cuda.cc
##
@@ -274,9 +274,21 @@ void CodeGenCUDA::PrintVecElemLoad(
static const char access[] = {'x', 'y', 'z', 'w'};
CHECK(i >= 0 && i < (t.is_float16() ? 8 : 4));
if ((t.is_int()) && t.bits() == 8) {
-os << "((char)(" << vec << " >> " << i * 8 << "))";
+if (t.lanes() == 1) {
+ os << vec;
+} else if (t.lanes() == 2) {
+ os << vec << "." << access[i % 2];
+} else {
+ os << "((char)(" << vec << " >> " << i * 8 << "))";
+}
} else if ((t.is_uint()) && t.bits() == 8) {
-os << "((unsigned char)(" << vec << " >> " << i * 8 << "))";
+if (t.lanes() == 1) {
Review comment:
I meant you have two "if (t.lanes() == 1) return vec". You could just
move this logic out, as it is true for all inputs.
The other part is nice-to-have if you can naturally support char2/3/4./8.
that will help simplify the codegen logic.
e.g. char2/3/4 can be stored as uint32_t. The only difference is only lower
k bytes are useful.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [incubator-tvm] masahi commented on pull request #5454: Add RoiAlign to Onnx frontend
masahi commented on pull request #5454: URL: https://github.com/apache/incubator-tvm/pull/5454#issuecomment-620320682 Thanks @mbrookhart This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[incubator-tvm] branch master updated (17cd27d -> 702db6f)
This is an automated email from the ASF dual-hosted git repository. masahi pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/incubator-tvm.git. from 17cd27d [TFLite Runtime] Add TFLite Runtime dependencies to CI CPU docker build (#5437) add 702db6f Add RoiAlign to Onnx frontend (#5454) No new revisions were added by this update. Summary of changes: python/tvm/relay/frontend/onnx.py | 32 +++ src/relay/op/vision/rcnn_op.cc | 2 + tests/python/frontend/onnx/test_forward.py | 63 ++ 3 files changed, 97 insertions(+)
[GitHub] [incubator-tvm] tmoreau89 commented on pull request #5417: [RUNTIME][uTVM] AutoTVM + uTVM for Cortex-M7
tmoreau89 commented on pull request #5417: URL: https://github.com/apache/incubator-tvm/pull/5417#issuecomment-620258389 @u99127 the pieces that changed the behavior of autoTVM are a small fraction of the overall PR; so post-changes, the PR will remain significantly large / unchanged. Let us know if you'd like to be able to do a review before it gets merged! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-tvm] kevinthesun commented on a change in pull request #5429: [RELAY][TF] Support symbolic newshape for Reshape
kevinthesun commented on a change in pull request #5429:
URL: https://github.com/apache/incubator-tvm/pull/5429#discussion_r416201926
##
File path: src/relay/op/tensor/transform.cc
##
@@ -650,13 +670,59 @@ Array ReshapeCompute(const Attrs& attrs,
return { topi::reshape(inputs[0], newshape) };
}
+double ToScalar(const runtime::NDArray& array, int i = 0) {
+ if (array->dtype.code == kDLInt) {
+if (array->dtype.bits == 8) {
+ return reinterpret_cast(array->data)[i];
+} else if (array->dtype.bits == 16) {
+ return reinterpret_cast(array->data)[i];
+} else if (array->dtype.bits == 32) {
+ return reinterpret_cast(array->data)[i];
+} else if (array->dtype.bits == 64) {
+ return reinterpret_cast(array->data)[i];
+}
+ } else if (array->dtype.code == kDLUInt) {
+if (array->dtype.bits == 8) {
+ return reinterpret_cast(array->data)[i];
+} else if (array->dtype.bits == 16) {
+ return reinterpret_cast(array->data)[i];
+} else if (array->dtype.bits == 32) {
+ return reinterpret_cast(array->data)[i];
+} else if (array->dtype.bits == 64) {
+ return reinterpret_cast(array->data)[i];
+}
+ } else if (array->dtype.code == kDLFloat) {
+#if (__ARM_FP16_FORMAT_IEEE == 1)
+if (array->dtype.bits == 16) {
+ return reinterpret_cast<__fp16*>(array->data)[i];
+}
+#endif
+if (array->dtype.bits == 32) {
+ return reinterpret_cast(array->data)[i];
+} else if (array->dtype.bits == 64) {
+ return reinterpret_cast(array->data)[i];
+}
+ }
+ LOG(FATAL) << "Unknown data type: " <<
tvm::runtime::DLDataType2String(array->dtype);
+ // make compiler happy
+ return -std::numeric_limits::infinity();
+}
+
Expr MakeReshape(Expr data,
- Array newshape) {
+ Expr newshape) {
auto attrs = make_object();
- attrs->newshape = std::move(newshape);
+
+ if (const ConstantNode *c = newshape.as()) {
Review comment:
Since we didn't change the type of ReshapeAttrs->newshape to Expr, this
is not a problem.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [incubator-tvm] boh-inspur commented on a change in pull request #5428: [CODEGEN][CUDA] Fix a bug when vectorized load was involved for…
boh-inspur commented on a change in pull request #5428:
URL: https://github.com/apache/incubator-tvm/pull/5428#discussion_r416253058
##
File path: src/target/source/codegen_cuda.cc
##
@@ -274,9 +274,21 @@ void CodeGenCUDA::PrintVecElemLoad(
static const char access[] = {'x', 'y', 'z', 'w'};
CHECK(i >= 0 && i < (t.is_float16() ? 8 : 4));
if ((t.is_int()) && t.bits() == 8) {
-os << "((char)(" << vec << " >> " << i * 8 << "))";
+if (t.lanes() == 1) {
+ os << vec;
+} else if (t.lanes() == 2) {
+ os << vec << "." << access[i % 2];
+} else {
+ os << "((char)(" << vec << " >> " << i * 8 << "))";
+}
} else if ((t.is_uint()) && t.bits() == 8) {
-os << "((unsigned char)(" << vec << " >> " << i * 8 << "))";
+if (t.lanes() == 1) {
Review comment:
That's a good suggestion, thanks a lot, I will modify the code, and
commit it later.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [incubator-tvm] s1113950 opened a new pull request #5456: Creates a TVM wheel install
s1113950 opened a new pull request #5456: URL: https://github.com/apache/incubator-tvm/pull/5456 Thanks for contributing to TVM! Please refer to guideline https://tvm.apache.org/docs/contribute/ for useful information and tips. After the pull request is submitted, please request code reviews from [Reviewers](https://github.com/apache/incubator-tvm/blob/master/CONTRIBUTORS.md#reviewers) by @ them in the pull request thread. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-tvm] tmoreau89 commented on a change in pull request #5417: [RUNTIME][uTVM] AutoTVM + uTVM for Cortex-M7
tmoreau89 commented on a change in pull request #5417: URL: https://github.com/apache/incubator-tvm/pull/5417#discussion_r416178024 ## File path: python/tvm/autotvm/measure/local_executor.py ## @@ -145,6 +145,7 @@ def submit(self, func, *args, **kwargs): if not self.do_fork: return LocalFutureNoFork(func(*args, **kwargs)) +# TODO why they choose a queue size of 2? add a comment Review comment: A quick git blame points to this: https://github.com/apache/incubator-tvm/pull/1651 @merrymercy made the change with "The old queue size is too small. It will stall the executor due to race condition." as comment This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-tvm] wpan11nv commented on a change in pull request #5428: [CODEGEN][CUDA] Fix a bug when vectorized load was involved for…
wpan11nv commented on a change in pull request #5428:
URL: https://github.com/apache/incubator-tvm/pull/5428#discussion_r416246643
##
File path: src/target/source/codegen_cuda.cc
##
@@ -274,9 +274,21 @@ void CodeGenCUDA::PrintVecElemLoad(
static const char access[] = {'x', 'y', 'z', 'w'};
CHECK(i >= 0 && i < (t.is_float16() ? 8 : 4));
if ((t.is_int()) && t.bits() == 8) {
-os << "((char)(" << vec << " >> " << i * 8 << "))";
+if (t.lanes() == 1) {
+ os << vec;
+} else if (t.lanes() == 2) {
+ os << vec << "." << access[i % 2];
+} else {
+ os << "((char)(" << vec << " >> " << i * 8 << "))";
+}
} else if ((t.is_uint()) && t.bits() == 8) {
-os << "((unsigned char)(" << vec << " >> " << i * 8 << "))";
+if (t.lanes() == 1) {
Review comment:
It should look like:
if lane == 1:
return vec
else {
stored as int and extracted with bit ops.
}
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [incubator-tvm] tqchen opened a new issue #5458: [CI] Fix gluoncv tutorial under mxnet-mkl
tqchen opened a new issue #5458: URL: https://github.com/apache/incubator-tvm/issues/5458 This is an error message generated by the mxnet-mkl image in the current master. The error was due to the operator `amp_multicast` is not implmented in the mxnet frontend(likely due to a mkl specialization). http://ci.tvm.ai:8080/job/temp-ci-docker-staging/job/ci-stage2/6/execution/node/309/log/?consoleFull Please see if we are interested in quick a fix, since we have docker upgrades depending on it. In the meanwhile, I will revert the docker image to mxnet for now until the next upgrade. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-tvm] tqchen commented on issue #5458: [CI] Fix gluoncv tutorial under mxnet-mkl
tqchen commented on issue #5458: URL: https://github.com/apache/incubator-tvm/issues/5458#issuecomment-620312769 cc @icemelon9 @shoubhik @anijain2305 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-tvm] boh-inspur commented on a change in pull request #5428: [CODEGEN][CUDA] Fix a bug when vectorized load was involved for…
boh-inspur commented on a change in pull request #5428:
URL: https://github.com/apache/incubator-tvm/pull/5428#discussion_r416238967
##
File path: src/target/source/codegen_cuda.cc
##
@@ -274,9 +274,21 @@ void CodeGenCUDA::PrintVecElemLoad(
static const char access[] = {'x', 'y', 'z', 'w'};
CHECK(i >= 0 && i < (t.is_float16() ? 8 : 4));
if ((t.is_int()) && t.bits() == 8) {
-os << "((char)(" << vec << " >> " << i * 8 << "))";
+if (t.lanes() == 1) {
+ os << vec;
+} else if (t.lanes() == 2) {
+ os << vec << "." << access[i % 2];
+} else {
+ os << "((char)(" << vec << " >> " << i * 8 << "))";
+}
} else if ((t.is_uint()) && t.bits() == 8) {
-os << "((unsigned char)(" << vec << " >> " << i * 8 << "))";
+if (t.lanes() == 1) {
Review comment:
When the lane is 1, the type is "char", it doesn't need to do shift
operation or type cast, so deal it in a separate logic.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [incubator-tvm] boh-inspur commented on a change in pull request #5428: [CODEGEN][CUDA] Fix a bug when vectorized load was involved for…
boh-inspur commented on a change in pull request #5428:
URL: https://github.com/apache/incubator-tvm/pull/5428#discussion_r416245487
##
File path: src/target/source/codegen_cuda.cc
##
@@ -274,9 +274,21 @@ void CodeGenCUDA::PrintVecElemLoad(
static const char access[] = {'x', 'y', 'z', 'w'};
CHECK(i >= 0 && i < (t.is_float16() ? 8 : 4));
if ((t.is_int()) && t.bits() == 8) {
-os << "((char)(" << vec << " >> " << i * 8 << "))";
+if (t.lanes() == 1) {
+ os << vec;
+} else if (t.lanes() == 2) {
+ os << vec << "." << access[i % 2];
+} else {
+ os << "((char)(" << vec << " >> " << i * 8 << "))";
+}
} else if ((t.is_uint()) && t.bits() == 8) {
-os << "((unsigned char)(" << vec << " >> " << i * 8 << "))";
+if (t.lanes() == 1) {
Review comment:
Yes, we can do that. The generated code may like the following:
`char _1;`
`char _2 = ((char)((_1)<<0))`
That's correct, but the code is a little strange, and may takes more time in
runtime? If that's OK, I think we can common up logic for both branches, what
is your opinion?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [incubator-tvm] boh-inspur commented on pull request #5428: [CODEGEN][CUDA] Fix a bug when vectorized load was involved for…
boh-inspur commented on pull request #5428:
URL: https://github.com/apache/incubator-tvm/pull/5428#issuecomment-620311830
>
>
> Do you see any issue to implement in a consistent way?
>
> uint_8x2 could be also stored as uint16_t, the same as uint8_t. If I am
not mistaken, this allows us to support {u}int8_tx3 naturally. Right?
>
>
> Do you see any issue to implement in a consistent way?
>
> uint_8x2 could be also stored as uint16_t, the same as uint8_t. If I am
not mistaken, this allows us to support {u}int8_tx3 naturally. Right?
Yes, it's other way to stored as uint16_t. And I think the author of the
type logic may concern about {u}int8_tx3. In my code I've concerned about this
condition before, but I think maybe it is impossible to use {u}int8_tx3, so the
logic is not include this type int8_t*3, but if needed, little modification of
this code can support {u}int8_tx3.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [incubator-tvm] liangfu commented on a change in pull request #5417: [RUNTIME][uTVM] AutoTVM + uTVM for Cortex-M7
liangfu commented on a change in pull request #5417:
URL: https://github.com/apache/incubator-tvm/pull/5417#discussion_r416273515
##
File path: python/tvm/exec/rpc_server.py
##
@@ -100,27 +101,34 @@ def server_shutdown():
parser.add_argument('--port-end', type=int, default=9199,
help='The end search port of the RPC')
parser.add_argument('--tracker', type=str,
-help="The address of RPC tracker in host:port format. "
- "e.g. (10.77.1.234:9190)")
+help=('The address of RPC tracker in host:port format.
'
+ 'e.g. (10.77.1.234:9190)'))
parser.add_argument('--key', type=str, default="",
-help="The key used to identify the device type in
tracker.")
+help='The key used to identify the device type in
tracker.')
parser.add_argument('--silent', action='store_true',
-help="Whether run in silent mode.")
+help='Whether run in silent mode.')
parser.add_argument('--load-library', type=str,
-help="Additional library to load")
+help='Additional library to load')
Review comment:
Please undo these as well.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [incubator-tvm] liangfu commented on a change in pull request #5417: [RUNTIME][uTVM] AutoTVM + uTVM for Cortex-M7
liangfu commented on a change in pull request #5417: URL: https://github.com/apache/incubator-tvm/pull/5417#discussion_r416273092 ## File path: python/tvm/micro/device/arm/stm32f746xx.py ## @@ -36,23 +55,41 @@ def create_micro_lib(obj_path, src_path, lib_type, options=None): options : Optional[List[str]] additional options to pass to GCC + +lib_src_paths : Optional[List[str]] +TODO """ if options is None: options = [] +else: +options = list(options) + options += [ -"-mcpu=cortex-m7", -"-mlittle-endian", -"-mfloat-abi=hard", -"-mfpu=fpv5-sp-d16", -"-mthumb", -"-gdwarf-5", +# TODO(weberlo): make a debug flag +'-O2', +'-march=armv7e-m', +'-mcpu=cortex-m7', +'-mlittle-endian', +'-mfloat-abi=hard', +'-mfpu=fpv5-sp-d16', +'-mthumb', +'-ffast-math', +'-gdwarf-5', +'-DARM_MATH_CM7', +'-D__FPU_PRESENT=1U', +'-DARM_MATH_DSP', +'-Wno-unused-variable', Review comment: umm.. it seems to be a problem in codegen, please feel free to leave this line as-is. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-tvm] yongfeng-nv commented on issue #5455: [CI] [TEST] test_conv2d_int8_intrinsics
yongfeng-nv commented on issue #5455: URL: https://github.com/apache/incubator-tvm/issues/5455#issuecomment-620373532 Ran the test in tvmai/ci-cpu:v0.62-t0, but couldn't reproduce the failure. I cloned and built TVM by myself, as I I didn't find TVM in the docker image. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[incubator-tvm] branch tmoreau89-patch-1 created (now 57e5c13)
This is an automated email from the ASF dual-hosted git repository. moreau pushed a change to branch tmoreau89-patch-1 in repository https://gitbox.apache.org/repos/asf/incubator-tvm.git. at 57e5c13 [COMMUNITY] @liangfu -> committer This branch includes the following new commits: new 57e5c13 [COMMUNITY] @liangfu -> committer The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference.
[incubator-tvm] 01/01: [COMMUNITY] @liangfu -> committer
This is an automated email from the ASF dual-hosted git repository. moreau pushed a commit to branch tmoreau89-patch-1 in repository https://gitbox.apache.org/repos/asf/incubator-tvm.git commit 57e5c131992ee59a03d6ce11c97f65437d27dea6 Author: Thierry Moreau AuthorDate: Mon Apr 27 22:31:34 2020 -0700 [COMMUNITY] @liangfu -> committer --- CONTRIBUTORS.md | 1 + 1 file changed, 1 insertion(+) diff --git a/CONTRIBUTORS.md b/CONTRIBUTORS.md index 10b247a..3553e34 100644 --- a/CONTRIBUTORS.md +++ b/CONTRIBUTORS.md @@ -41,6 +41,7 @@ We do encourage everyone to work anything they are interested in. - [Aditya Atluri](https://github.com/adityaatluri): @adityaatluri - rocm - [Tianqi Chen](https://github.com/tqchen) (PPMC): @tqchen - topi, compiler, relay, docs +- [Liangfu Chen](https://github.com/liangfu): @liangfu - vta, chisel, intel FPGA, c runtime - [Wei Chen](https://github.com/wweic): @wweic - runtime, relay, vm - [Zhi Chen](https://github.com/zhiics): @zhiics - relay, quantization, pass manager - [Yuwei Hu](https://github.com/Huyuwei): @Huyuwei - topi, frontends
[GitHub] [incubator-tvm] tmoreau89 opened a new pull request #5460: [COMMUNITY] @liangfu -> committer
tmoreau89 opened a new pull request #5460: URL: https://github.com/apache/incubator-tvm/pull/5460 Please join us to welcome @liangfu as a committer. He has helped maintain much of the Chisel codebase for VTA and notably has brought support to Intel FPGAs. More recently he contributed by adding a MISRA-C compliant lightweight runtime for micro-controllers. He also built early support for sparse operations in CSR form. Commits: https://github.com/apache/incubator-tvm/commits?author=liangfu Code Review: https://github.com/apache/incubator-tvm/pulls?utf8=%E2%9C%93=reviewed-by%3Aliangfu Community: https://discuss.tvm.ai/u/liangfu/summary This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-tvm] kevinthesun opened a new pull request #5459: Support symbolic TopK, Ones, Zeros and Full
kevinthesun opened a new pull request #5459: URL: https://github.com/apache/incubator-tvm/pull/5459 Support symbolic k for TopK. Support symbolic shape for Ones, Zeros and Full. @icemelon9 @yongwww @lixiaoquan This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-tvm] yzhliu commented on a change in pull request #5357: [Relay] enable blocking format in x86 conv2d and fold scale axis
yzhliu commented on a change in pull request #5357:
URL: https://github.com/apache/incubator-tvm/pull/5357#discussion_r416315915
##
File path: src/relay/transforms/fold_scale_axis.cc
##
@@ -39,6 +39,10 @@ namespace relay {
*
* Use namespace to reduce potential naming conflict.
*/
+
+extern Expr MakeReshape(Expr data,
+ Array newshape);
Review comment:
how about src/relay/op/tensor/transform.h ?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [incubator-tvm] murdockhou opened a new issue #5448: TypeError:'NoneType' object is not callable
murdockhou opened a new issue #5448:
URL: https://github.com/apache/incubator-tvm/issues/5448
I have update the tvm to the latest(2020/04/27). After this, I run the code
that convert pyorch model to tvm. But get error in the first like this:
```
Traceback (most recent call last):
File "convert/from_pytorch.py", line 45, in
from tvm import relay
File "/media/hsw/E/work/github/tvm_github/python/tvm/relay/__init__.py",
line 27, in
from . import expr_functor
File
"/media/hsw/E/work/github/tvm_github/python/tvm/relay/expr_functor.py", line
25, in
from .op import Op
File
"/media/hsw/E/work/github/tvm_github/python/tvm/relay/op/__init__.py", line 23,
in
from . import strategy
File
"/media/hsw/E/work/github/tvm_github/python/tvm/relay/op/strategy/__init__.py",
line 22, in
from .generic import *
File
"/media/hsw/E/work/github/tvm_github/python/tvm/relay/op/strategy/generic.py",
line 22, in
import topi
File "/media/hsw/E/work/github/tvm_github/topi/python/topi/__init__.py",
line 46, in
from . import cuda
File
"/media/hsw/E/work/github/tvm_github/topi/python/topi/cuda/__init__.py", line
26, in
from .conv2d_int8 import *
File
"/media/hsw/E/work/github/tvm_github/topi/python/topi/cuda/conv2d_int8.py",
line 168, in
_dp4a = dp4a('shared', 'shared', 'local')
File
"/media/hsw/E/work/github/tvm_github/topi/python/topi/cuda/tensor_intrin.py",
line 43, in dp4a
x = te.placeholder((n,), name='x', dtype='int8')
File "/media/hsw/E/work/github/tvm_github/python/tvm/te/operation.py",
line 55, in placeholder
shape, dtype, name)
File "tvm/_ffi/_cython/./packed_func.pxi", line 312, in
tvm._ffi._cy3.core.PackedFuncBase.__call__
File "tvm/_ffi/_cython/./packed_func.pxi", line 247, in
tvm._ffi._cy3.core.FuncCall
File "tvm/_ffi/_cython/./packed_func.pxi", line 235, in
tvm._ffi._cy3.core.FuncCall3
File "tvm/_ffi/_cython/./packed_func.pxi", line 158, in
tvm._ffi._cy3.core.make_arg
TypeError: 'NoneType' object is not callable
```
I'm confused with that, I run these command line below to update tvm:
```
git pull
git submodule update
cd build
cmake ..
make -j8
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [incubator-tvm] lixiaoquan commented on a change in pull request #5429: [RELAY][TF] Support symbolic newshape for Reshape
lixiaoquan commented on a change in pull request #5429:
URL: https://github.com/apache/incubator-tvm/pull/5429#discussion_r415554589
##
File path: include/tvm/relay/op_attr_types.h
##
@@ -81,10 +81,16 @@ using TOpIsStateful = bool;
*/
using TNonComputational = bool;
+enum ShapeDependantKind {
+ kShapeDependantShape = 0,
+ kShapeDependantData = 1,
+ kShapeDependantBoth = 2,
Review comment:
Thanks, it's solved now
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [incubator-tvm] schliffen opened a new issue #5449: Windows OSError: "exception: access violation reading 0x0000000000000000"
schliffen opened a new issue #5449: URL: https://github.com/apache/incubator-tvm/issues/5449 We are trying to convert a mxnet model into tvm (v7) in windows using the following part of code: loading model (works fine): ` mx_sym, args, auxs = mx.model.load_checkpoint(root + ap.mdlrt, 0) mod, params = relay.frontend.from_mxnet(mx_sym, shape_dict, dtype, arg_params=args, aux_params=auxs) ` build tvm model (fails): `with relay.build_config(opt_level=3): graph, mod, params = relay.build(mod, target, params=params) ` in this relay.build we faced with the following error (full log): `[08:31:15] C:\Jenkins\workspace\mxnet-tag\mxnet\src\nnvm\legacy_json_util.cc:209: Loading symbol saved by previous version v1.2.0. Attempting to upgrade... [08:31:15] C:\Jenkins\workspace\mxnet-tag\mxnet\src\nnvm\legacy_json_util.cc:217: Symbol successfully upgraded! Traceback (most recent call last): File "E:/TVM/faceTransfer/mxnet2Tvm.py", line 96, in save_tvm_model graph, mod, params = relay.build(mod, target, params=params) File "C:\Python\Python37\site-packages\tvm-0.7.dev1-py3.7-win-amd64.egg\tvm\relay\build_module.py", line 251, in build graph_json, mod, params = bld_mod.build(mod, target, target_host, params) File "C:\Python\Python37\site-packages\tvm-0.7.dev1-py3.7-win-amd64.egg\tvm\relay\build_module.py", line 120, in build self._build(mod, target, target_host) File "C:\Python\Python37\site-packages\tvm-0.7.dev1-py3.7-win-amd64.egg\tvm\_ffi\_ctypes\packed_func.py", line 215, in __call__ ctypes.byref(ret_val), ctypes.byref(ret_tcode)) != 0: OSError: exception: access violation reading 0x` system details: Windows 10, Visual Studio-2015, c++14, cuda 10.1, llvm 6.0.1, tvm v7 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[incubator-tvm] branch master updated (37e5754 -> a60de36)
This is an automated email from the ASF dual-hosted git repository. masahi pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/incubator-tvm.git. from 37e5754 Improve IntervalSet's floormod (#5367) add a60de36 [ONNX]GatherNd, Round, IsNaN, IsInf (#5445) No new revisions were added by this update. Summary of changes: python/tvm/relay/frontend/onnx.py | 12 ++ tests/python/frontend/onnx/test_forward.py | 68 ++ 2 files changed, 80 insertions(+)
[GitHub] [incubator-tvm] masahi commented on issue #5133: [Torch] A list of missing op conversion in need of help
masahi commented on issue #5133: URL: https://github.com/apache/incubator-tvm/issues/5133#issuecomment-619770517 @siju-samuel @woniuasd Since relay doesn't support in-place op, you cannot use that python idiom. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
