Re: [PR] [OpenCL][Texture] Improved texture memory planning [tvm]
echuraev commented on code in PR #15058:
URL: https://github.com/apache/tvm/pull/15058#discussion_r1354122827
##
src/runtime/memory/memory_manager.cc:
##
@@ -128,6 +128,7 @@ Allocator* MemoryManager::GetOrCreateAllocator(Device dev,
AllocatorType type) {
// Look for any available, else create Naive.
if (type == AllocatorType::kAny) {
+it = m->allocators_.find(dev);
Review Comment:
Got it. Thank you!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] [OpenCL][Texture] Improved texture memory planning [tvm]
echuraev commented on code in PR #15058: URL: https://github.com/apache/tvm/pull/15058#discussion_r1354122128 ## apps/android_deploy/app/src/main/jni/tvm_runtime.h: ## @@ -49,6 +49,5 @@ #include "../src/runtime/opencl/opencl_device_api.cc" #include "../src/runtime/opencl/opencl_module.cc" #include "../src/runtime/opencl/opencl_wrapper/opencl_wrapper.cc" Review Comment: Thank you for the clarification! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[tvm] branch nightly updated (899435956a -> eb2a4bc8e2)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch nightly in repository https://gitbox.apache.org/repos/asf/tvm.git from 899435956a [Relay][Bugfix] fix axis parsing of repeat converter in the MXNet frontend (#15891) add a79f632333 [CI][ADRENO] Few updates to Adreno docker setup (#15897) add eb2a4bc8e2 [TFLite][Frontend] Fix test failures caused by div-by-zero (#15844) No new revisions were added by this update. Summary of changes: apps/cpp_clml/scripts/clml_codegen.py| 2 +- docker/Dockerfile.ci_adreno | 8 ++-- tests/python/frontend/tflite/test_forward.py | 15 ++- tests/scripts/task_config_build_adreno.sh| 1 + 4 files changed, 22 insertions(+), 4 deletions(-)
Re: [PR] [TFLite][Frontend] Support quantized ELU [tvm]
tlopex commented on PR #15821: URL: https://github.com/apache/tvm/pull/15821#issuecomment-1756797590 cc @leandron @lhutton1 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[tvm] branch unity updated: [Unity] [Bugfix] Fix MaxPool TypeError in ONNX frontend (#15908)
This is an automated email from the ASF dual-hosted git repository.
syfeng pushed a commit to branch unity
in repository https://gitbox.apache.org/repos/asf/tvm.git
The following commit(s) were added to refs/heads/unity by this push:
new 67d61935df [Unity] [Bugfix] Fix MaxPool TypeError in ONNX frontend
(#15908)
67d61935df is described below
commit 67d61935df054f8113b526464d404ebf4bfaa36f
Author: Thrsu <89128704+th...@users.noreply.github.com>
AuthorDate: Wed Oct 11 10:54:40 2023 +0800
[Unity] [Bugfix] Fix MaxPool TypeError in ONNX frontend (#15908)
* Fix MaxPool TypeError
* Add regression test case.
---
python/tvm/relax/frontend/onnx/onnx_frontend.py | 2 +-
tests/python/relax/test_frontend_onnx.py| 9 +
2 files changed, 10 insertions(+), 1 deletion(-)
diff --git a/python/tvm/relax/frontend/onnx/onnx_frontend.py
b/python/tvm/relax/frontend/onnx/onnx_frontend.py
index fbd478ee5a..5333812c05 100644
--- a/python/tvm/relax/frontend/onnx/onnx_frontend.py
+++ b/python/tvm/relax/frontend/onnx/onnx_frontend.py
@@ -1424,7 +1424,7 @@ class MaxPool(OnnxOpConverter):
dilations = attr.get("dilations", [1, 1])
kernel_shape = attr.get("kernel_shape")
pads = attr.get("pads", 0)
-strides = attr.get("strides", 1)
+strides = attr.get("strides", [1, 1])
assert len(kernel_shape) == 2, "Currently only 2D pooling is
supported."
assert auto_pad in [
diff --git a/tests/python/relax/test_frontend_onnx.py
b/tests/python/relax/test_frontend_onnx.py
index d587d70636..a896ebb0b9 100644
--- a/tests/python/relax/test_frontend_onnx.py
+++ b/tests/python/relax/test_frontend_onnx.py
@@ -1590,6 +1590,15 @@ def test_max_pool():
strides=[2, 2],
),
)
+verify_unary(
+"MaxPool",
+[1, 1, 32, 32],
+dict(
+auto_pad="SAME_UPPER",
+kernel_shape=[3, 3],
+pads=None,
+),
+)
def test_global_average_pool():
Re: [PR] [Unity] [Bugfix] Fix MaxPool TypeError in ONNX frontend [tvm]
Hzfengsy merged PR #15908: URL: https://github.com/apache/tvm/pull/15908 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [Unity] [Bugfix] Fix MaxPool TypeError in ONNX frontend [tvm]
Thrsu commented on PR #15908: URL: https://github.com/apache/tvm/pull/15908#issuecomment-1756650178 Please review and consider merging this PR. Thank you! cc @Hzfengsy @leandron -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[PR] [TIR] Fix offset_factor in cuda tensor core intrins [tvm]
vinx13 opened a new pull request, #15913: URL: https://github.com/apache/tvm/pull/15913 This PR updated the offset factor of cuda tensor intrinsic. Now the offset factor is always the leading dimension of the fragment size and fixed some failing test cases previous not caught by CI as they are skipped cc @spectrometerHBH @masahi -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[PR] [VM] modify the date pointer of tensor to point to the buffer [tvm]
yongwww opened a new pull request, #15912: URL: https://github.com/apache/tvm/pull/15912 This change is from the unity, the kv cache will need it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[I] [Bug] [TensorIR] Wrong results with reverse_compute_at [tvm]
TH3CHARLie opened a new issue, #15911:
URL: https://github.com/apache/tvm/issues/15911
### Expected behavior
program scheduled with `reverse_compute_at` should generate the same answer.
### Actual behavior
wrong result, seems like some part of array is not computed
### Environment
ubuntu 20.04, tvm main branch
### Steps to reproduce
A program to reproduce:
```python
import tvm
from tvm.ir.module import IRModule
from tvm.script import tir as T
import numpy as np
from tvm import te
import tvm.testing
# compute reference
tgt = tvm.target.Target(target="llvm", host="llvm")
dev = tvm.device(tgt.kind.name, 0)
n = te.var("n")
p = te.const(32, "int32")
A = te.placeholder((n,), dtype="float32", name="A")
B = te.placeholder((n,), dtype="float32", name="B")
C = te.compute(A.shape, lambda i: A[i] + B[i], name="C")
D = te.compute(A.shape, lambda i: C[i] + C[i], name="D")
E = te.compute(A.shape, lambda i: D[i] + A[i], name="E")
func = te.create_prim_func([A, B, E])
ir_mod = IRModule({"main": func})
mod = tvm.build(ir_mod, tgt)
n = 1024
np_a = np.arange(n).astype(A.dtype)
np_b = np.arange(n).astype(B.dtype)
ref_a = tvm.nd.array(np_a, dev)
ref_b = tvm.nd.array(np_b, dev)
ref_e = tvm.nd.array(np.zeros(n, dtype=E.dtype), dev)
mod(ref_a, ref_b, ref_e)
# apply the schedule and compute the results
schedule = tvm.tir.Schedule(ir_mod)
block_d = schedule.get_block("D")
i, = schedule.get_loops(block_d)
i_0, i_1 = schedule.split(i, factors=[None, 16])
schedule.reorder(i_1, i_0)
schedule.add_unit_loop(block_d)
block_c = schedule.get_block("C")
schedule.compute_inline(block_c)
block_e = schedule.get_block("E")
schedule.reverse_compute_at(block_e, i_1)
ir_mod = schedule.mod
mod = tvm.build(ir_mod, tgt)
tgt_e = tvm.nd.array(np.zeros(n, dtype=E.dtype), dev)
mod(ref_a, ref_b, tgt_e)
tvm.testing.assert_allclose(ref_e.numpy(), tgt_e.numpy(), rtol=1e-5)
```
Essentially the program computes A+B+A+B+A so we should expect the final
output being 5 times of A, clearly `tgt_e` is not the case.
### Triage
Please refer to the list of label tags
[here](https://github.com/apache/tvm/wiki/Issue-Triage-Labels) to find the
relevant tags and add them below in a bullet format (example below).
* needs-triage
* tir:schedule
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[PR] [Unity] Paged KV Cache as LM Support [tvm]
MasterJH5574 opened a new pull request, #15910: URL: https://github.com/apache/tvm/pull/15910 This PR introduces the PagedKVCache object to `lm_support.cc` for the KV cache value management in batching settings. One test file is included. Note that this file does not contain the test of attention function/kernel. That part will be uploaded and tested separately. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [TOPI][ADRENO] Add conv2d transpose nchw texture schedule [tvm]
srkreddy1238 commented on PR #15786: URL: https://github.com/apache/tvm/pull/15786#issuecomment-1755917006 Over all LGTM. Few comments above. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [TOPI][ADRENO] Add conv2d transpose nchw texture schedule [tvm]
srkreddy1238 commented on code in PR #15786:
URL: https://github.com/apache/tvm/pull/15786#discussion_r1353039853
##
tests/python/relay/opencl_texture/test_conv2d_transpose_nchw_texture.py:
##
@@ -0,0 +1,119 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+import re
+import tvm
+import numpy as np
+from tvm import relay
+from tvm.relay import testing
+from tvm.contrib import utils
+from utils.adreno_utils import gpu_preprocess, build_run_compare,
build_run_compare_vm
+import pytest
+
+executor_type = tvm.testing.parameter("ge")
+dtype = tvm.testing.parameter("float32")
+
+
+@tvm.testing.requires_opencl
+@tvm.testing.parametrize_targets("opencl -device=adreno")
+def test_conv2d_transpose_adreno(remote, target, executor_type, dtype):
+# Conv2d transpose test cases lists
+trials = [
+[4, 4, (1, 1), (2, 2), (1, 1), 64, (256, 100, 100), (False, False)],
+[4, 4, (0, 0), (2, 2), (1, 1), 256, (32, 64, 64), (False, False)],
+[3, 3, (0, 0), (2, 2), (1, 1), 64, (256, 12, 12), (True, True)],
+[4, 4, (1, 1), (1, 1), (1, 1), 512, (16, 100, 100), (False, False)],
+[5, 5, (2, 2), (2, 2), (1, 1), 4, (16, 100, 100), (True, False)],
+[7, 7, (3, 3), (2, 2), (1, 1), 8, (4, 100, 100), (False, True)],
+]
+
+for (
+kernel_h,
+kernel_w,
+pad,
+stride,
+dilation,
+out_channels,
+shape,
+composite,
+) in trials:
+shape = (1, *shape)
+has_bias = composite[0]
+has_activation = composite[1]
+input_shape = shape
+filter_shape = (shape[1], out_channels, kernel_w, kernel_h)
+x = relay.var("data", shape=input_shape, dtype=dtype)
+w = relay.var("weight", shape=filter_shape, dtype=dtype)
+inputs = [x, w]
+y = relay.nn.conv2d_transpose(
+x,
+w,
+channels=out_channels,
+kernel_size=(kernel_w, kernel_h),
+strides=stride,
+padding=pad,
+kernel_layout="IOHW",
+data_layout="NCHW",
+dilation=dilation,
+)
+np.random.seed(0)
+initializer = relay.testing.init.Xavier()
+filter_data = np.zeros(filter_shape).astype(dtype)
+initializer("weight", filter_data)
+params1 = {
+"weight": tvm.nd.array(filter_data),
+}
+
+if has_bias:
+b = relay.var("bias", shape=(out_channels,), dtype=dtype)
+y = relay.nn.bias_add(y, b, axis=1)
+inputs.append(b)
+bias_data = np.zeros((out_channels,)).astype(dtype)
+initializer("bias", bias_data)
+params1["bias"] = tvm.nd.array(bias_data)
+
+if has_activation:
+y = relay.nn.relu(y)
+
+mod = relay.Function(inputs, y)
+
+if executor_type == "ge":
+build_run_compare(
+remote,
+mod,
+params1,
+{"data": input_shape},
+{"data": dtype},
+target,
+[],
Review Comment:
Also pls add few cases to check the expected memory scope attribute in
generated graph json.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] [TOPI][ADRENO] Add conv2d transpose nchw texture schedule [tvm]
srkreddy1238 commented on code in PR #15786:
URL: https://github.com/apache/tvm/pull/15786#discussion_r1353034383
##
tests/python/relay/opencl_texture/test_conv2d_transpose_nchw_texture.py:
##
@@ -0,0 +1,119 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+import re
+import tvm
+import numpy as np
+from tvm import relay
+from tvm.relay import testing
+from tvm.contrib import utils
+from utils.adreno_utils import gpu_preprocess, build_run_compare,
build_run_compare_vm
+import pytest
+
+executor_type = tvm.testing.parameter("ge")
Review Comment:
Probably we could enabled VM executor too.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] [TOPI][ADRENO] Add conv2d transpose nchw texture schedule [tvm]
krishnaraj36 commented on PR #15786: URL: https://github.com/apache/tvm/pull/15786#issuecomment-1755879197 @srkreddy1238 : Please review -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [RFC] Scalable vectors in TIR [tvm-rfcs]
kparzysz-quic commented on PR #104: URL: https://github.com/apache/tvm-rfcs/pull/104#issuecomment-1755730601 > Is there any technical reason blocking us from extending DLDataType to have a `is_scalable` vector field, allowing us to maintain the meaning of the lanes field to represent the number of lanes? DLDataType comes from dlpack not TVM. Changing it may affect the ABI of any function accepting or returning a value of that type, and will affect the memory layout of a DLTensor (and likely more). As a consequence, code build with older TVM will not be compatible with that built with a newer TVM, plus it will have an impact on any other project using dlpack. Changing it is not impossible, but we should be careful about it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [TFLite][Frontend] Fix test failures caused by div-by-zero [tvm]
Aleksei-grovety commented on code in PR #15844:
URL: https://github.com/apache/tvm/pull/15844#discussion_r1352800533
##
tests/python/frontend/tflite/test_forward.py:
##
@@ -2480,6 +2481,16 @@ def __test_elemwise(in_data):
inq0_min, inq0_max = (out_min, out_max)
inq1_min, inq1_max = (out_min, out_max)
+if exclude_zero_point:
+if inq1_max == inq1_min:
+raise ZeroDivisionError("Input range is 0.")
+
+# only compute for rhs.
+quant_scale = 255 / (inq1_max - inq1_min)
Review Comment:
Shouldn't there be a dependency on the data type for the value 255? Since
the data type can be int8, int16.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] [IR] Make sure Range types match when binding to Var [tvm]
kparzysz-quic commented on PR #15909: URL: https://github.com/apache/tvm/pull/15909#issuecomment-1755694020 This is a replacement PR for https://github.com/apache/tvm/pull/15909. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [RFC] Scalable vectors in TIR [tvm-rfcs]
neildhickey commented on PR #104: URL: https://github.com/apache/tvm-rfcs/pull/104#issuecomment-1755692891 > Another idea to handle this would be to add a new field to DLDataType, e.g. bool is_scalable, but I'm not sure how feasible changing that standard is. I feel extending DLDataType to represent scalable vectors explicitly would be a more robust design than depending on interpreting -1 in a special way for the lane parameter. Is there any technical reason blocking us from extending DLDataType to have a `is_scalable` vector field, allowing us to maintain the meaning of the lanes field to represent the number of lanes? `` can then be encoded by setting the `is_scalable` field and setting the `lane` field to 8 and we do not need to introduce any special handling. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] [Bug] [Unity] TypeError when converting TorchFX layer_norm model to TVM using from_fx [tvm]
Thrsu closed issue #15892: [Bug] [Unity] TypeError when converting TorchFX layer_norm model to TVM using from_fx URL: https://github.com/apache/tvm/issues/15892 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [TIR] Autocorrect Range min and extent types when possible [tvm]
kparzysz-quic closed pull request #15872: [TIR] Autocorrect Range min and extent types when possible URL: https://github.com/apache/tvm/pull/15872 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [TIR] Autocorrect Range min and extent types when possible [tvm]
kparzysz-quic commented on PR #15872: URL: https://github.com/apache/tvm/pull/15872#issuecomment-1755669269 This cannot be done correctly in `Range` alone. This is because there are cases like `Bind(var, Range(a, b))`. If `a` and `b` are both immediates of different types, then there is no way to infer the correct common type, because the correct type is that of the `var`, which is not known to the range. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [TFLite][Frontend] Fix test failures caused by div-by-zero [tvm]
p3achyjr commented on PR #15844: URL: https://github.com/apache/tvm/pull/15844#issuecomment-1755646894 @Lunderberg thanks for getting back!! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[tvm] branch main updated: [TFLite][Frontend] Fix test failures caused by div-by-zero (#15844)
This is an automated email from the ASF dual-hosted git repository.
lunderberg pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/tvm.git
The following commit(s) were added to refs/heads/main by this push:
new eb2a4bc8e2 [TFLite][Frontend] Fix test failures caused by div-by-zero
(#15844)
eb2a4bc8e2 is described below
commit eb2a4bc8e2ca982427983f999dc61d7c0861088f
Author: Anatol Liu
AuthorDate: Tue Oct 10 10:35:54 2023 -0400
[TFLite][Frontend] Fix test failures caused by div-by-zero (#15844)
* [TFLite][Frontend] Support quantized floor_mod
* [TVM][Frontend] Fix zero-point issues in quantized div/floor_div
* [TVM][Frontend] Fix zero-point issues in quantized div/floor_div
---
tests/python/frontend/tflite/test_forward.py | 15 ++-
1 file changed, 14 insertions(+), 1 deletion(-)
diff --git a/tests/python/frontend/tflite/test_forward.py
b/tests/python/frontend/tflite/test_forward.py
index d9a53fd517..f601667024 100644
--- a/tests/python/frontend/tflite/test_forward.py
+++ b/tests/python/frontend/tflite/test_forward.py
@@ -2452,6 +2452,7 @@ def _test_elemwise(
qnn_op=None,
same_qnn_params=False,
comparison_op=False,
+exclude_zero_point=False,
):
"""One iteration of elemwise"""
@@ -2480,6 +2481,16 @@ def _test_elemwise(
inq0_min, inq0_max = (out_min, out_max)
inq1_min, inq1_max = (out_min, out_max)
+if exclude_zero_point:
+if inq1_max == inq1_min:
+raise ZeroDivisionError("Input range is 0.")
+
+# only compute for rhs.
+quant_scale = 255 / (inq1_max - inq1_min)
+zero_point = int(round(-inq1_min * quant_scale))
+data[1][data[1] == zero_point] += 1
+data[1][data[1] == 0] += 1
+
# fake_quant will keep the tensors in float32 until the conversion
in the session
inq_data = [
tf.quantization.fake_quant_with_min_max_args(
@@ -2619,6 +2630,7 @@ def _test_div(data, fused_activation_function=None,
quantized=False, qnn_op=None
quantized,
qnn_op,
same_qnn_params=True,
+exclude_zero_point=True,
)
@@ -2802,6 +2814,7 @@ def _test_floor_divide(data,
fused_activation_function=None, quantized=False, qn
quantized,
qnn_op,
same_qnn_params=True,
+exclude_zero_point=True,
)
@@ -2882,7 +2895,7 @@ def _test_elemwise_qnn_out_range(qnn_op):
def test_all_elemwise():
-"""All_elewise"""
+"""All_elemwise"""
_test_forward_elemwise(_test_add)
_test_forward_elemwise_quantized(_test_add)
_test_forward_elemwise(partial(_test_add,
fused_activation_function="RELU"))
Re: [PR] [TFLite][Frontend] Fix test failures caused by div-by-zero [tvm]
Lunderberg merged PR #15844: URL: https://github.com/apache/tvm/pull/15844 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [TFLite][Frontend] Fix test failures caused by div-by-zero [tvm]
Lunderberg commented on PR #15844: URL: https://github.com/apache/tvm/pull/15844#issuecomment-1755560175 @p3achyjr Thank you for the reminder, and it would still be good to get this PR in, even if re-enabling the flaky `floor_mod` test is saved for a follow-up PR. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [RFC] Scalable vectors in TIR [tvm-rfcs]
kparzysz-quic commented on PR #104: URL: https://github.com/apache/tvm-rfcs/pull/104#issuecomment-1755456586 I guess we could pass an argument to the vectorizer whether to generate SVE-friendly code. If this is limited to emitting additional TIR builtins, then I'm ok with that. I just want to be able to reuse as much of the vectorization code as possible between SVE and non-SVE targets. As far as predication goes, you're right---it's somewhat independent from SVE. To take full advantage of SVE we'd need to be able to vectorize loops with iteration count that is not known at compile time, which is the part I'm interested in. Are you planning to implement that in the near future, or is this a longer-term goal? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[PR] [Unity] [Bugfix] Fix MaxPool TypeError in ONNX frontend [tvm]
Thrsu opened a new pull request, #15908: URL: https://github.com/apache/tvm/pull/15908 As the ONNX documentation description [here](https://onnx.ai/onnx/operators/onnx__MaxPool.html) The parameter strides in MaxPool2d is an optional attribute using [1, 1] as default. This PR fixed it and added a corresponding regression test. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [OpenCL][Texture] Improved texture memory planning [tvm]
srkreddy1238 commented on code in PR #15058:
URL: https://github.com/apache/tvm/pull/15058#discussion_r1352506642
##
src/runtime/opencl/opencl_device_api.cc:
##
@@ -224,8 +241,57 @@ void*
OpenCLWorkspace::CreateHostPtrIfEnabled(cl::BufferDescriptor* desc, Device
void* OpenCLWorkspace::AllocDataSpace(Device dev, size_t size, size_t
alignment,
DLDataType type_hint) {
this->Init();
+ return AllocCLBuffer(dev, size, alignment, type_hint);
+}
+
+void* OpenCLWorkspace::AllocDataSpace(Device dev, size_t width, size_t height,
DLDataType type_hint,
+ Optional mem_scope) {
+ // Texture allocation given width and height
+ cl_uint row_align = GetImageAlignment(dev.device_id);
+ size_t pixel_size = (type_hint.bits * type_hint.lanes + 7) / 8;
+ size_t row_pitch = ALIGN_UP(width * pixel_size * 4, row_align); // CL_RGBA
= 4
+ size_t mem_size = row_pitch * height;
+
+ // Alloc back buffer from pool
+ cl::BufferDescriptor* back_buffer = nullptr;
+ if (IsBufferToImageSupported(dev.device_id)) {
+auto buf = MemoryManager::GetOrCreateAllocator(dev, AllocatorType::kAny)
+ ->Alloc(mem_size, kTempAllocaAlignment, type_hint);
+back_buffer = static_cast(buf.data);
+back_buffer->mbuf = buf;
+ }
+
+ if (!mem_scope.defined()) {
Review Comment:
This fallback setting is coming from the below code migration.
https://github.com/apache/tvm/blob/a79f632333d0f319f938ad58575bbd5ea85bd0d3/src/runtime/opencl/texture_pool.cc#L90
Getting global is definitely an error here, I will assert. In fact
```defined``` and ```empty``` are handled later too in
```cl::BufferDescriptor::MemoryLayoutFromScope```.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] [OpenCL][Texture] Improved texture memory planning [tvm]
srkreddy1238 commented on code in PR #15058:
URL: https://github.com/apache/tvm/pull/15058#discussion_r1352416290
##
src/runtime/opencl/opencl_device_api.cc:
##
@@ -269,36 +430,32 @@ void OpenCLWorkspace::FreeDataSpace(Device dev, void*
ptr) {
OPENCL_CALL(clFinish(this->GetQueue(dev)));
cl::BufferDescriptor* desc = static_cast(ptr);
- if (desc->host_ptr) {
-clEnqueueUnmapMemObject(this->GetQueue(dev), desc->buffer,
-reinterpret_cast(desc->host_ptr), 0,
nullptr, nullptr);
- }
- OPENCL_CALL(clReleaseMemObject(desc->buffer));
- delete desc;
-}
-cl_mem OpenCLWorkspace::AllocTexture(Device dev, size_t width, size_t height,
- DLDataType type_hint) {
- this->Init();
- cl_device_id device_id = GetCLDeviceID(dev.device_id);
- auto platform = device_to_platform[device_id];
- cl_int err_code;
- cl_channel_type cl_type = DTypeToOpenCLChannelType(type_hint);
- cl_image_format format = {CL_RGBA, cl_type};
- cl_image_desc descriptor = {CL_MEM_OBJECT_IMAGE2D, width, height, 0, 0, 0,
0, 0, 0};
- cl_mem mptr = clCreateImage(this->contexts[platform], CL_MEM_CREATE_FLAGS,
, ,
- nullptr, _code);
- OPENCL_CHECK_ERROR(err_code);
- return mptr;
-}
-
-void* OpenCLWorkspace::AllocTextureWorkspace(Device dev, size_t width, size_t
height,
- DLDataType type_hint) {
- return GetThreadEntry()->texture_pool.AllocTexture(dev, width, height,
type_hint);
-}
-
-void OpenCLWorkspace::FreeTextureWorkspace(Device dev, void* ptr) {
- GetThreadEntry()->texture_pool.FreeTexture(dev, ptr);
+ if (desc->back_buffer) {
+// 2D Image w/ back buffer allocated from pool
+OPENCL_CALL(clReleaseMemObject(desc->buffer));
+// GetThreadEntry()->mpool.FreeMemory(dev, desc->back_buffer);
Review Comment:
Missed it while rebasing. Will remove.
##
src/runtime/opencl/opencl_device_api.cc:
##
@@ -269,36 +430,32 @@ void OpenCLWorkspace::FreeDataSpace(Device dev, void*
ptr) {
OPENCL_CALL(clFinish(this->GetQueue(dev)));
cl::BufferDescriptor* desc = static_cast(ptr);
- if (desc->host_ptr) {
-clEnqueueUnmapMemObject(this->GetQueue(dev), desc->buffer,
-reinterpret_cast(desc->host_ptr), 0,
nullptr, nullptr);
- }
- OPENCL_CALL(clReleaseMemObject(desc->buffer));
- delete desc;
-}
-cl_mem OpenCLWorkspace::AllocTexture(Device dev, size_t width, size_t height,
- DLDataType type_hint) {
- this->Init();
- cl_device_id device_id = GetCLDeviceID(dev.device_id);
- auto platform = device_to_platform[device_id];
- cl_int err_code;
- cl_channel_type cl_type = DTypeToOpenCLChannelType(type_hint);
- cl_image_format format = {CL_RGBA, cl_type};
- cl_image_desc descriptor = {CL_MEM_OBJECT_IMAGE2D, width, height, 0, 0, 0,
0, 0, 0};
- cl_mem mptr = clCreateImage(this->contexts[platform], CL_MEM_CREATE_FLAGS,
, ,
- nullptr, _code);
- OPENCL_CHECK_ERROR(err_code);
- return mptr;
-}
-
-void* OpenCLWorkspace::AllocTextureWorkspace(Device dev, size_t width, size_t
height,
- DLDataType type_hint) {
- return GetThreadEntry()->texture_pool.AllocTexture(dev, width, height,
type_hint);
-}
-
-void OpenCLWorkspace::FreeTextureWorkspace(Device dev, void* ptr) {
- GetThreadEntry()->texture_pool.FreeTexture(dev, ptr);
+ if (desc->back_buffer) {
+// 2D Image w/ back buffer allocated from pool
+OPENCL_CALL(clReleaseMemObject(desc->buffer));
+// GetThreadEntry()->mpool.FreeMemory(dev, desc->back_buffer);
+MemoryManager::GetAllocator(dev, desc->back_buffer->mbuf.alloc_type)
+->Free(desc->back_buffer->mbuf);
+delete desc;
+ } else {
+if (desc->layout == cl::BufferDescriptor::MemoryLayout::kBuffer1D) {
+ // 1D buffer allocated from pool
+ // GetThreadEntry()->mpool.FreeMemory(dev, desc);
Review Comment:
I will remove
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] [OpenCL][Texture] Improved texture memory planning [tvm]
srkreddy1238 commented on code in PR #15058: URL: https://github.com/apache/tvm/pull/15058#discussion_r1352403642 ## apps/android_deploy/app/src/main/jni/tvm_runtime.h: ## @@ -49,6 +49,5 @@ #include "../src/runtime/opencl/opencl_device_api.cc" #include "../src/runtime/opencl/opencl_module.cc" #include "../src/runtime/opencl/opencl_wrapper/opencl_wrapper.cc" Review Comment: No device specific memory pools now. All pooling is managed by common MemoryManager now. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [OpenCL][Texture] Improved texture memory planning [tvm]
srkreddy1238 commented on code in PR #15058:
URL: https://github.com/apache/tvm/pull/15058#discussion_r1352374602
##
src/runtime/memory/memory_manager.cc:
##
@@ -128,6 +128,7 @@ Allocator* MemoryManager::GetOrCreateAllocator(Device dev,
AllocatorType type) {
// Look for any available, else create Naive.
if (type == AllocatorType::kAny) {
+it = m->allocators_.find(dev);
Review Comment:
First time the list would be empty and at line #126 we add the device key.
Here we need to refer to the newly added entry. This happens when we just
create an NDArray with out any executor creation.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[I] [Bug] inconsistent ouputs produced by the same model [tvm]
ntcmp2u opened a new issue, #15907:
URL: https://github.com/apache/tvm/issues/15907
I use TVM to optimize an ONNX model but the output of the optmized model is
inconsistent.
The poc code as the follow:
```python
import tvm
import onnx
from tvm import relay
import numpy as np
from numpy import testing
import pickle
model_input =
np.array([[6.71610641,3.85466981,6.40658808,5.93880939,4.22816277,4.05935574,4.05884647,6.85629654,4.35045576,4.98825026,4.79773617,5.90505457,4.07920122,3.91599631,5.48346233,4.68261051,5.96164513,4.04757214,6.40736198,5.78212643,3.97077084,5.38297272,5.98323917,3.33484483,5.0777626],[5.78360128,3.50137734,6.22678471,6.58975887,5.86054516,6.28650713,4.02936316,6.55729437,4.83005142,5.48753119,6.6756916,3.1908493,5.902318,3.63500333,6.6633153,4.03696537,4.59259987,5.29771805,5.47530413,6.98861551,5.21775055,6.27956057,6.06063652,6.99890566,6.74760342],[5.15759087,5.79957485,3.87717772,3.98443675,4.08592176,3.68012309,3.56660628,6.5352087,6.32946777,5.21131229,4.83698702,3.24459314,5.40390682,3.10241055,6.76744366,4.29461193,5.75841236,5.95449638,3.74979687,4.88354635,6.29391479,5.12002468,4.3089,5.12105751,3.2507937],[3.6196816,4.79808092,4.01847553,3.04140043,6.1275444,6.30466843,3.18326187,4.40191364,6.87912273,5.95551109,6.82588673,5.95711136,5.95687199,6.3148
0217,3.16748881,4.01974392,4.79580688,4.65139341,4.1572628,6.83029556,6.77638483,4.56482315,6.59405708,6.15137196,3.67091894],[6.05837917,6.34490871,6.82791471,6.16849041,3.10806918,5.42449474,5.17070627,3.94981074,4.28552246,4.36995983,6.1662178,4.59585524,3.32610178,6.27614117,4.7198143,4.82491684,5.11868238,5.589221,5.05218983,4.47221899,6.56419563,3.33831954,4.76141453,5.47758198,6.76508904],[5.27446032,4.80305195,6.86809921,5.71008205,4.43950748,4.55266809,6.0204134,4.15397835,5.40533447,4.21241283,5.2563076,5.31718588,5.03535795,5.61827183,5.727705,5.56590462,3.99580646,3.07781434,6.91844368,4.20229912,6.46835232,6.02725315,6.19255781,5.68293953,3.91823912],[5.54070759,5.55740356,4.43334723,5.79145813,4.09821415,4.99966526,6.07888222,5.35138416,5.53214931,5.38911247,6.39173222,3.72211218,4.38897705,6.03391933,5.74040985,5.49542809,6.62584782,4.33855057,6.8596487,6.41575336,6.35653496,3.90258241,6.14905071,4.76633644,5.97037888],[6.64982224,6.40295553,4.78704929,3.03489995,5.26
092148,3.17754674,3.61741209,5.90558434,3.44067335,4.10587311,5.2109766,4.80613804,6.83341599,5.98525429,6.70596409,4.64003181,4.31005573,3.14922762,6.44953346,6.32083702,4.5173068,4.73698139,5.2820859,5.57041645,3.72178435],[6.80869818,3.77339244,3.81124735,5.25667477,6.97375488,5.42400169,3.45591283,4.84353495,3.8287816,5.41924429,3.18944407,4.95114517,3.18213964,3.20189476,5.98147392,4.49553967,6.49250174,5.02359581,5.37200642,4.57207108,5.9689579,4.29011774,6.34126377,4.39409637,5.17850113],[4.48860836,3.66642022,4.86899662,3.43860793,6.94423056,4.19769287,3.77650118,4.41444111,6.10652637,6.75893021,6.0855093,5.83897305,3.97421741,4.10853386,3.62891579,5.17287302,6.16482878,5.45052338,5.91892958,3.24301076,6.47304344,5.24120235,6.28845692,3.56531596,5.53275776],[3.08167267,5.02443886,4.67914724,4.21279573,6.79280615,6.10375261,6.02019453,6.13239574,6.71213055,4.50908089,4.03038025,6.02951813,5.82834053,6.15195274,3.17172861,5.51577044,3.96025443,5.51473904,6.00128365,6.34118938,
4.94945621,3.03591824,5.76955414,6.09682941,5.2738781],[4.9579711,4.0002923,3.25455737,3.34851503,6.37354565,6.01042414,4.52113342,5.26745939,6.26920605,6.16422987,3.1255343,6.43074322,3.16986299,4.44038916,4.80228901,3.59629822,4.95137262,6.40821123,4.88515186,4.09138393,5.46744061,5.57049751,4.30876446,6.48064327,4.49197578],[5.54204559,3.35072994,5.81306267,5.0076189,6.09887314,4.83277607,3.76282811,5.04093981,6.85498047,6.33627892,4.55758762,6.29734421,3.90469551,6.35388994,4.73173046,5.31449032,3.06976533,3.54300165,3.2381289,3.11252737,3.15451765,6.34363556,6.97934818,6.39008713,6.97971106]])
onnx_model_0 = onnx.load('./model_0.onnx')
input_dict_0 = {"v5_0": model_input}
shape_dict_0 = {key: val.shape for key, val in input_dict_0.items()}
mod_0, params_0 = relay.frontend.from_onnx(onnx_model_0, shape_dict_0,
freeze_params=True)
with tvm.transform.PassContext(opt_level=4):
executor_0 = relay.build_module.create_executor("graph", mod_0,
tvm.cpu(), tvm.target.Target("llvm"), params_0).evaluate()
output_0 = executor_0(**input_dict_0)
with tvm.transform.PassContext(opt_level=4):
executor_0 = relay.build_module.create_executor("graph", mod_0,
tvm.cpu(), tvm.target.Target("llvm"), params_0).evaluate()
output_1 = executor_0(**input_dict_0)
print("")
try:
testing.assert_allclose(output_0.numpy(), output_1.numpy(),
equal_nan=True)
except BaseException as e:
print("tvm triggers the assertion failure")
print(e)
print("")
import onnxruntime as ort
sess_options = ort.SessionOptions()
[tvm] branch unity updated: [Unity] [Bugfix] Fix TypeError in TVM PyTorch frontend for LayerNorm operator (#15902)
This is an automated email from the ASF dual-hosted git repository.
syfeng pushed a commit to branch unity
in repository https://gitbox.apache.org/repos/asf/tvm.git
The following commit(s) were added to refs/heads/unity by this push:
new b1380059f0 [Unity] [Bugfix] Fix TypeError in TVM PyTorch frontend for
LayerNorm operator (#15902)
b1380059f0 is described below
commit b1380059f0eeab78f36425193f33506abb9b576b
Author: Thrsu <89128704+th...@users.noreply.github.com>
AuthorDate: Tue Oct 10 18:42:49 2023 +0800
[Unity] [Bugfix] Fix TypeError in TVM PyTorch frontend for LayerNorm
operator (#15902)
---
python/tvm/relax/frontend/torch/fx_translator.py | 12 +--
tests/python/relax/test_frontend_from_fx.py | 40
2 files changed, 49 insertions(+), 3 deletions(-)
diff --git a/python/tvm/relax/frontend/torch/fx_translator.py
b/python/tvm/relax/frontend/torch/fx_translator.py
index a52b73f3ab..3d150e3eed 100644
--- a/python/tvm/relax/frontend/torch/fx_translator.py
+++ b/python/tvm/relax/frontend/torch/fx_translator.py
@@ -975,9 +975,15 @@ class TorchFXImporter:
# functional.layer_norm
if node.target not in self.named_modules:
# static or symbolic
-normalized_shape = (
-node.args[1] if type(node.args[1]) == tuple else
self.env[node.args[1]]
-)
+arg = node.args[1]
+if isinstance(arg, tuple):
+value = arg
+else:
+try:
+value = self.env[arg]
+except TypeError:
+value = tuple(arg)
+normalized_shape = value
dim_num = len(normalized_shape)
axes = list(range(-dim_num, 0))
diff --git a/tests/python/relax/test_frontend_from_fx.py
b/tests/python/relax/test_frontend_from_fx.py
index c815051076..a1acff4974 100644
--- a/tests/python/relax/test_frontend_from_fx.py
+++ b/tests/python/relax/test_frontend_from_fx.py
@@ -1152,6 +1152,46 @@ def test_functional_layernorm():
binding = {}
verify_model(model, input_info, binding, expected2)
+class LayerNorm3(Module):
+def __init__(self, shape):
+super().__init__()
+self.shape = shape
+self.weight = torch.nn.Parameter(torch.ones(shape))
+self.bias = torch.nn.Parameter(torch.zeros(shape))
+
+def forward(self, input):
+return torch.nn.functional.layer_norm(input, self.shape,
self.weight, self.bias, 1e-5)
+
+@tvm.script.ir_module
+class expected3:
+@R.function
+def main(
+input_1: R.Tensor((1, 3, 10, 10), dtype="float32"),
+w1: R.Tensor([10, 10], dtype="float32"),
+w2: R.Tensor([10, 10], dtype="float32"),
+) -> R.Tensor((1, 3, 10, 10), dtype="float32"):
+# block 0
+with R.dataflow():
+lv: R.Tensor((1, 3, 10, 10), dtype="float32") =
R.nn.layer_norm(
+input_1,
+w1,
+w2,
+axes=[-2, -1],
+epsilon=1e-05,
+center=True,
+scale=True,
+)
+gv: R.Tensor((1, 3, 10, 10), dtype="float32") = lv
+R.output(gv)
+return gv
+
+model = LayerNorm3([10, 10])
+binding = {
+"w1": model.weight.detach().numpy(),
+"w2": model.bias.detach().numpy(),
+}
+verify_model(model, input_info, binding, expected3)
+
def test_cross_entropy():
input_info = [([3, 2], "float32"), ([3], "int32")]
Re: [PR] [Unity] [Bugfix] Fix TypeError in TVM PyTorch frontend for LayerNorm operator [tvm]
Hzfengsy merged PR #15902: URL: https://github.com/apache/tvm/pull/15902 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] [Bug] [Relay][MXNet] wrong conversion logic in Repeat operator about axis [tvm]
jikechao closed issue #15890: [Bug] [Relay][MXNet] wrong conversion logic in Repeat operator about axis URL: https://github.com/apache/tvm/issues/15890 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] [Bug] [Relay][MXNet] wrong conversion logic in Repeat operator about axis [tvm]
jikechao commented on issue #15890: URL: https://github.com/apache/tvm/issues/15890#issuecomment-1754916457 This bug has been fixed by #15891 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] [Bug] [Relay][MXNet] wrong conversion logic in Repeat operator about axis [tvm]
jikechao commented on issue #15890: URL: https://github.com/apache/tvm/issues/15890#issuecomment-1754913364 This bug has been fixed by #15891 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [Unity] [Bugfix] Fix TypeError in TVM PyTorch frontend for LayerNorm operator [tvm]
jikechao commented on PR #15902: URL: https://github.com/apache/tvm/pull/15902#issuecomment-1754903214 @Hzfengsy @echuraev @vvchernov. Could you help me review it again? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [OpenCL][Texture] Improved texture memory planning [tvm]
echuraev commented on code in PR #15058:
URL: https://github.com/apache/tvm/pull/15058#discussion_r1349867420
##
apps/android_deploy/app/src/main/jni/tvm_runtime.h:
##
@@ -49,6 +49,5 @@
#include "../src/runtime/opencl/opencl_device_api.cc"
#include "../src/runtime/opencl/opencl_module.cc"
#include "../src/runtime/opencl/opencl_wrapper/opencl_wrapper.cc"
Review Comment:
Probably we should add `opencl/memory_pool.cc` to the include list? The same
question for `android_rpc`.
##
src/runtime/memory/memory_manager.cc:
##
@@ -128,6 +128,7 @@ Allocator* MemoryManager::GetOrCreateAllocator(Device dev,
AllocatorType type) {
// Look for any available, else create Naive.
if (type == AllocatorType::kAny) {
+it = m->allocators_.find(dev);
Review Comment:
Do you really need this line? I see on the line#124 the same iterator was
found.
##
src/runtime/opencl/opencl_device_api.cc:
##
@@ -269,36 +430,32 @@ void OpenCLWorkspace::FreeDataSpace(Device dev, void*
ptr) {
OPENCL_CALL(clFinish(this->GetQueue(dev)));
cl::BufferDescriptor* desc = static_cast(ptr);
- if (desc->host_ptr) {
-clEnqueueUnmapMemObject(this->GetQueue(dev), desc->buffer,
-reinterpret_cast(desc->host_ptr), 0,
nullptr, nullptr);
- }
- OPENCL_CALL(clReleaseMemObject(desc->buffer));
- delete desc;
-}
-cl_mem OpenCLWorkspace::AllocTexture(Device dev, size_t width, size_t height,
- DLDataType type_hint) {
- this->Init();
- cl_device_id device_id = GetCLDeviceID(dev.device_id);
- auto platform = device_to_platform[device_id];
- cl_int err_code;
- cl_channel_type cl_type = DTypeToOpenCLChannelType(type_hint);
- cl_image_format format = {CL_RGBA, cl_type};
- cl_image_desc descriptor = {CL_MEM_OBJECT_IMAGE2D, width, height, 0, 0, 0,
0, 0, 0};
- cl_mem mptr = clCreateImage(this->contexts[platform], CL_MEM_CREATE_FLAGS,
, ,
- nullptr, _code);
- OPENCL_CHECK_ERROR(err_code);
- return mptr;
-}
-
-void* OpenCLWorkspace::AllocTextureWorkspace(Device dev, size_t width, size_t
height,
- DLDataType type_hint) {
- return GetThreadEntry()->texture_pool.AllocTexture(dev, width, height,
type_hint);
-}
-
-void OpenCLWorkspace::FreeTextureWorkspace(Device dev, void* ptr) {
- GetThreadEntry()->texture_pool.FreeTexture(dev, ptr);
+ if (desc->back_buffer) {
+// 2D Image w/ back buffer allocated from pool
+OPENCL_CALL(clReleaseMemObject(desc->buffer));
+// GetThreadEntry()->mpool.FreeMemory(dev, desc->back_buffer);
+MemoryManager::GetAllocator(dev, desc->back_buffer->mbuf.alloc_type)
+->Free(desc->back_buffer->mbuf);
+delete desc;
+ } else {
+if (desc->layout == cl::BufferDescriptor::MemoryLayout::kBuffer1D) {
+ // 1D buffer allocated from pool
+ // GetThreadEntry()->mpool.FreeMemory(dev, desc);
Review Comment:
Do we need this comment?
##
src/runtime/opencl/opencl_device_api.cc:
##
@@ -224,8 +241,57 @@ void*
OpenCLWorkspace::CreateHostPtrIfEnabled(cl::BufferDescriptor* desc, Device
void* OpenCLWorkspace::AllocDataSpace(Device dev, size_t size, size_t
alignment,
DLDataType type_hint) {
this->Init();
+ return AllocCLBuffer(dev, size, alignment, type_hint);
+}
+
+void* OpenCLWorkspace::AllocDataSpace(Device dev, size_t width, size_t height,
DLDataType type_hint,
+ Optional mem_scope) {
+ // Texture allocation given width and height
+ cl_uint row_align = GetImageAlignment(dev.device_id);
+ size_t pixel_size = (type_hint.bits * type_hint.lanes + 7) / 8;
+ size_t row_pitch = ALIGN_UP(width * pixel_size * 4, row_align); // CL_RGBA
= 4
+ size_t mem_size = row_pitch * height;
+
+ // Alloc back buffer from pool
+ cl::BufferDescriptor* back_buffer = nullptr;
+ if (IsBufferToImageSupported(dev.device_id)) {
+auto buf = MemoryManager::GetOrCreateAllocator(dev, AllocatorType::kAny)
+ ->Alloc(mem_size, kTempAllocaAlignment, type_hint);
+back_buffer = static_cast(buf.data);
+back_buffer->mbuf = buf;
+ }
+
+ if (!mem_scope.defined()) {
Review Comment:
Should we also add check on `mem_scope.empty() || mem_scope == "global"`? Or
`ICHECK` can be added to validate that `mem_scope` has a valid value.
##
src/runtime/opencl/opencl_device_api.cc:
##
@@ -269,36 +430,32 @@ void OpenCLWorkspace::FreeDataSpace(Device dev, void*
ptr) {
OPENCL_CALL(clFinish(this->GetQueue(dev)));
cl::BufferDescriptor* desc = static_cast(ptr);
- if (desc->host_ptr) {
-clEnqueueUnmapMemObject(this->GetQueue(dev), desc->buffer,
-reinterpret_cast(desc->host_ptr), 0,
nullptr, nullptr);
- }
- OPENCL_CALL(clReleaseMemObject(desc->buffer));
- delete desc;
-}
-cl_mem OpenCLWorkspace::AllocTexture(Device dev, size_t width, size_t
[tvm] branch main updated: [CI][ADRENO] Few updates to Adreno docker setup (#15897)
This is an automated email from the ASF dual-hosted git repository.
srk pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/tvm.git
The following commit(s) were added to refs/heads/main by this push:
new a79f632333 [CI][ADRENO] Few updates to Adreno docker setup (#15897)
a79f632333 is described below
commit a79f632333d0f319f938ad58575bbd5ea85bd0d3
Author: Siva
AuthorDate: Tue Oct 10 13:51:02 2023 +0530
[CI][ADRENO] Few updates to Adreno docker setup (#15897)
Enabling google tests and clang-format version update.
---
apps/cpp_clml/scripts/clml_codegen.py | 2 +-
docker/Dockerfile.ci_adreno | 8 ++--
tests/scripts/task_config_build_adreno.sh | 1 +
3 files changed, 8 insertions(+), 3 deletions(-)
diff --git a/apps/cpp_clml/scripts/clml_codegen.py
b/apps/cpp_clml/scripts/clml_codegen.py
index bf19c0e4b9..7540812ed5 100644
--- a/apps/cpp_clml/scripts/clml_codegen.py
+++ b/apps/cpp_clml/scripts/clml_codegen.py
@@ -57,7 +57,7 @@ def main():
f_src = open("../clml_models.cc", "w")
f_src.write("\n".join(gen_src))
f_src.close()
-os.popen("clang-format-10 -i ../clml_models.cc")
+os.popen("clang-format-15 -i ../clml_models.cc")
if __name__ == "__main__":
diff --git a/docker/Dockerfile.ci_adreno b/docker/Dockerfile.ci_adreno
index 11be0a8baa..961977c542 100644
--- a/docker/Dockerfile.ci_adreno
+++ b/docker/Dockerfile.ci_adreno
@@ -16,7 +16,7 @@
# under the License.
# CI docker GPU env
-FROM tlcpack/ci-gpu:20220908-060034-62bdc91b1
+FROM tlcpack/ci-gpu
COPY utils/apt-install-and-clear.sh /usr/local/bin/apt-install-and-clear
@@ -26,4 +26,8 @@ RUN bash /install/ubuntu_install_androidsdk.sh 25.2.9519653
3.22.1 33.0.2 33
ENV PATH /opt/android-sdk-linux/platform-tools:$PATH
# Clang tool for CLML source codegen
-RUN apt-get update && apt-install-and-clear -y clang-format-10
+RUN apt-get update && apt-install-and-clear -y clang-format-15
+
+#Google Test
+COPY install/ubuntu_install_googletest.sh /install/ubuntu_install_googletest.sh
+RUN bash install/ubuntu_install_googletest.sh
diff --git a/tests/scripts/task_config_build_adreno.sh
b/tests/scripts/task_config_build_adreno.sh
index 1b6750f165..afe6407cba 100755
--- a/tests/scripts/task_config_build_adreno.sh
+++ b/tests/scripts/task_config_build_adreno.sh
@@ -23,6 +23,7 @@ mkdir -p "$BUILD_DIR"
cd "$BUILD_DIR"
cp ../cmake/config.cmake .
+echo set\(USE_OPENCL_GTEST /googletest\) >> config.cmake
if [ -f "${ADRENO_OPENCL}/CL/cl_qcom_ml_ops.h" ] ; then
echo set\(USE_CLML ${ADRENO_OPENCL}\) >> config.cmake
else
Re: [PR] [CI][ADRENO] Few updates to Adreno docker setup [tvm]
srkreddy1238 merged PR #15897: URL: https://github.com/apache/tvm/pull/15897 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
