[GitHub] [incubator-tvm] icemelon9 commented on a change in pull request #5955: Register Shape Func for Some Operators to Handle Dynamic Shapes
icemelon9 commented on a change in pull request #5955:
URL: https://github.com/apache/incubator-tvm/pull/5955#discussion_r456753279
##
File path: python/tvm/relay/op/image/_image.py
##
@@ -64,6 +67,22 @@ def compute_crop_and_resize(attrs, inputs, out_type):
reg.register_injective_schedule("image.crop_and_resize")
+@script
+def _crop_and_resize_func(image_shape, boxes_shape, crop_size):
+out = output_tensor((4,), "int64")
+out[0] = boxes_shape[0]
+out[1] = int64(crop_size[0])
+out[2] = int64(crop_size[1])
+out[3] = image_shape[3]
+
+return out
+
+@reg.register_shape_func("image.crop_and_resize", False)
+def crop_and_resize_func(attrs, inputs, _):
+crop_size = get_const_tuple(attrs.crop_size)
Review comment:
@lsy643 Could you update on my comment?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [incubator-tvm] icemelon9 commented on pull request #6040: [TOPI] Improve schedule for injective on x86
icemelon9 commented on pull request #6040: URL: https://github.com/apache/incubator-tvm/pull/6040#issuecomment-660431091 Probably also worth checking with @kevinthesun. I remember last time he saw some performance regression due to the injective schedule. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-tvm] icemelon9 commented on pull request #6040: [TOPI] Improve schedule for injective on x86
icemelon9 commented on pull request #6040: URL: https://github.com/apache/incubator-tvm/pull/6040#issuecomment-660430944 @merrymercy Yes, I agree with @tqchen. If c++ version works well, we could just remove the python implementation and use c++ one instead. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-tvm] anijain2305 closed pull request #5354: [Tutorial] AutoTVM for TFLite model on ARM CPUs.
anijain2305 closed pull request #5354: URL: https://github.com/apache/incubator-tvm/pull/5354 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-tvm] zhiics commented on pull request #6089: [Relay][VM] Add ReshapeTensor instruction in the VM to replace the reshape op
zhiics commented on pull request #6089: URL: https://github.com/apache/incubator-tvm/pull/6089#issuecomment-660424453 @mbrookhart Yeah, I meant there is gpu context in the test. That's why the ci fails. We can probably just disable check_result for this test ATM. Heterogeneous execution needs this PR. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-tvm] zhanghaohit commented on a change in pull request #5842: [VTA][OpenCL] Cloud FPGA support
zhanghaohit commented on a change in pull request #5842: URL: https://github.com/apache/incubator-tvm/pull/5842#discussion_r456748692 ## File path: python/tvm/relay/op/strategy/generic.py ## @@ -323,8 +331,10 @@ def compute_conv2d_transpose(attrs, inputs, out_dtype): out = topi_compute( inputs[0], inputs[1], strides, padding, out_dtype) output_padding = get_const_tuple(attrs.output_padding) -out = topi.nn.pad(out, [0, 0, 0, 0], - [0, 0, output_padding[0], output_padding[1]]) +if output_padding[0] != 0 or output_padding[1] != 0: Review comment: This will cause errors when running dcgan on vta. There are two issues here: 1. if output_padding are all 0, no need to do pad 2. if the size of out.shape is not 4, it will cause problems. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-tvm] comaniac commented on issue #6090: [BUG] PartitionGraph and ccompiler too simplistic, too limited
comaniac commented on issue #6090: URL: https://github.com/apache/incubator-tvm/issues/6090#issuecomment-660422421 Could you provide more details for the crashing you encountered? DNNL codegen is mainly for BYOC demonstration purpose and it's not in production for now. On the other hand, BYOC does have the mechanism to offload supported ops to the external codegen. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-tvm] ZihengJiang commented on pull request #6079: [DSL/TE] Scalar support for `te.extern`
ZihengJiang commented on pull request #6079:
URL: https://github.com/apache/incubator-tvm/pull/6079#issuecomment-660420778
Hey Haibin , thanks for your help!
Could you try to enhance the test case with mixed `Tensor` and `Scalar`
inputs like:
```
@tvm.register_func("tvm.test_tensor_scalar_scale")
def my_scale(tensor, scalar, out):
out_np = tensor.asnumpy() * scalar
tvm.nd.array(out_np).copyto(out)
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [incubator-tvm] zhanghaohit commented on a change in pull request #5842: [VTA][OpenCL] Cloud FPGA support
zhanghaohit commented on a change in pull request #5842:
URL: https://github.com/apache/incubator-tvm/pull/5842#discussion_r456745434
##
File path: python/tvm/autotvm/measure/measure_methods.py
##
@@ -186,6 +189,16 @@ def __init__(self,
timeout=10, n_parallel=None,
number=4, repeat=3, min_repeat_ms=0, cooldown_interval=0.1,
check_correctness=False):
+static_tune = os.getenv("TVM_STATIC_TUNE_EXPERIMENTAL")
Review comment:
`TVM_STATIC_TUNE_EXPERIMENTAL` is used for static auto tune. I think as
you advised, we can split the PR into separate PRs. For static auto tune, the
interface is not well nicely done. I think maybe we remove this part first.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [incubator-tvm] zhanghaohit edited a comment on pull request #5842: [VTA][OpenCL] Cloud FPGA support
zhanghaohit edited a comment on pull request #5842: URL: https://github.com/apache/incubator-tvm/pull/5842#issuecomment-659790884 > Note that I've also uploaded the Pynq bitstream given the ISA changes here: https://github.com/uwsampl/vta-distro/tree/master/bitstreams/pynq/0.0.2 > > It would be great @liangfu if you had the bandwidth to synthesize the DE10 bistream as well and submit a pull request. Let me know if you are tight on time and may need some help. > > Finally, I will add an ultra96 variant in there as well. It would be great @zhanghaohit if you could also pre-generate the Arria10 bitstream and add it to the `vta-distro` repo. Thanks @tmoreau89 for the help. For the Arria10 bitstream, because the fpga device we are currently using is from other vendors (not Intel), it is not easy to be used by others. We're trying to adapt to the fpga devices on AWS, which are more standard. After that, we can upload the aocx/bitstream. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-tvm] yzhliu opened a new pull request #6091: [Docs] improve the doc of release
yzhliu opened a new pull request #6091: URL: https://github.com/apache/incubator-tvm/pull/6091 @tqchen This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-tvm-vta] remotego commented on a change in pull request #9: [Hardware][OpenCL] Intelfocl support
remotego commented on a change in pull request #9:
URL: https://github.com/apache/incubator-tvm-vta/pull/9#discussion_r456738642
##
File path: config/intelfocl_sample.json
##
@@ -0,0 +1,13 @@
+{
+ "TARGET" : "intelfocl",
Review comment:
I believe "Arria 10" is too restrictive here. The code should work for
all devices supported by Intel OpenCL for FPGA, namely Arria 10, Stratix V/10
and Cyclone V/10. That's the reason we name it as "intelfocl".
So far we have tested it on both Arria 10 and Stratix 10 boards, and it
worked for both boards.
The correct bitstream will be generated automatically according to the
card/BSP/toolchain installed on the system. For example, if you have a Stratix
10 card and toolchain(BSP) installed on your system, the bitstream generated
will work on your Stratix 10 card.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [incubator-tvm-vta] remotego edited a comment on pull request #9: [Hardware][OpenCL] Intelfocl support
remotego edited a comment on pull request #9: URL: https://github.com/apache/incubator-tvm-vta/pull/9#issuecomment-660411735 > Thanks for the changes. Please apply the 0.0.2 and rename the vta target to something more specific, e.g. "arria10". Also there are some CI errors related to linting that could be addressed. Thanks! Thank you very much! Sure. We will apply the 0.0.2 and address the linting errors. However. I believe "arria10" is too restrictive here. The code should work for all devices supported by Intel OpenCL for FPGA, namely Intel Arria 10, Stratix V/10 and Cyclone V/10. So far we have tested it on both Arria 10 and Stratix 10 boards, and it worked. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-tvm-vta] remotego commented on pull request #9: [Hardware][OpenCL] Intelfocl support
remotego commented on pull request #9: URL: https://github.com/apache/incubator-tvm-vta/pull/9#issuecomment-660411735 > Thanks for the changes. Please apply the 0.0.2 and rename the vta target to something more specific, e.g. "arria10". Also there are some CI errors related to linting that could be addressed. Thanks! Sure. We will apply the 0.0.2 and address the linting errors. However. I believe "arria10" is too restrictive here. The code should work for all devices supported by Intel OpenCL for FPGA, namely Intel Arria 10, Stratix V/10 and Cyclone V/10. So far we have tested it on both Arria 10 and Stratix 10 boards, and it worked. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-tvm] mikeseven opened a new issue #6090: [BUG] PartitionGraph and ccompiler too simplistic, too limited
mikeseven opened a new issue #6090: URL: https://github.com/apache/incubator-tvm/issues/6090 Using the test_pass_partition_graph.py and whitelisting nn.conv2d to be built with ccompiler crashes in codegen.cc because it seems to support only the simple operations in this test! Seems like ccompiler codegen is just a test itself??? Bummer, it would have been great to test separate compilation of real NN operators as a fallback mechanism on CPU if external compiler like DNNL are not available or not supporting an operation. Thanks. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-tvm-vta] remotego commented on a change in pull request #9: [Hardware][OpenCL] Intelfocl support
remotego commented on a change in pull request #9:
URL: https://github.com/apache/incubator-tvm-vta/pull/9#discussion_r456738642
##
File path: config/intelfocl_sample.json
##
@@ -0,0 +1,13 @@
+{
+ "TARGET" : "intelfocl",
Review comment:
I believe "Arria 10" is too restrictive here. The code should work for
all devices supported by Intel OpenCL for FPGA, namely Arria 10, Stratix V/10
and Cyclone V/10. That's the reason we name it as "intelfocl".
So far we have tested it on both Arria 10 and Stratix 10 boards, and it
worked for both boards.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [incubator-tvm] fernchen commented on pull request #6023: [CI][Caffe Frontend] add caffe environment
fernchen commented on pull request #6023: URL: https://github.com/apache/incubator-tvm/pull/6023#issuecomment-660408499 > > 1. We add caffe frontend based on SSD distribution, then this ci pr is good enough to test; > > 2. We add caffe frontend based on our maintaining tvm caffe, then I will support the layers of BVLC distribution firstly. > > I vote for second option. BVLC caffe models will have more users compared to forked versions. > Another concern is lot of dependancies are getting added for running the ci with custom distribution, i think bvlc have support for precompiled caffe installation. Yes, bvlc distribution has supported for precompiled caffe installation, and It's really convenient to use. If other people agree, we can support the official version first, and then gradually support other distribution. We can even maintain our own Caffe branch to support other layers, of course it is another topic. @tqchen This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-tvm] mbrookhart commented on pull request #6089: [Relay][VM] Add ReshapeTensor instruction in the VM to replace the reshape op
mbrookhart commented on pull request #6089: URL: https://github.com/apache/incubator-tvm/pull/6089#issuecomment-660406275 The GPU error seems to come from the check I added to fail early on GPU with dynamic models. I believe dynamic shapes wont work on cuda until we get heterogenous execution working for shape funcs, per our discussion in https://discuss.tvm.ai/t/issue-with-vm-execution-when-testing-gpu-immediately-after-cpu/7010/7 @zhiics, any update on that work? Or should we limit this test to CPU? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-tvm] zhiics edited a comment on pull request #6089: [Relay][VM] Add ReshapeTensor instruction in the VM to replace the reshape op
zhiics edited a comment on pull request #6089: URL: https://github.com/apache/incubator-tvm/pull/6089#issuecomment-660402821 Please check the error. We have gpu context in the check_result. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-tvm] zhiics commented on pull request #6089: [Relay][VM] Add ReshapeTensor instruction in the VM to replace the reshape op
zhiics commented on pull request #6089: URL: https://github.com/apache/incubator-tvm/pull/6089#issuecomment-660402821 Please check the error. It cannot be executed on GPU. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-tvm] MarisaKirisame commented on pull request #6080: [Relay][Dyn] Add dynamic reshape grad
MarisaKirisame commented on pull request #6080: URL: https://github.com/apache/incubator-tvm/pull/6080#issuecomment-660367022 Thanks @mbrookhart for the work! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-tvm] MarisaKirisame merged pull request #6080: [Relay][Dyn] Add dynamic reshape grad
MarisaKirisame merged pull request #6080: URL: https://github.com/apache/incubator-tvm/pull/6080 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[incubator-tvm] branch master updated (ccacb1e -> 3150db7)
This is an automated email from the ASF dual-hosted git repository. marisa pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/incubator-tvm.git. from ccacb1e Fixed point multiplication improvements for AArch64 (#5980) add 3150db7 [Relay][Dyn] Add dynamic reshape grad (#6080) No new revisions were added by this update. Summary of changes: python/tvm/relay/op/_tensor_grad.py | 12 python/tvm/relay/testing/__init__.py | 36 +--- tests/python/relay/dyn/test_dynamic_op_level3.py | 9 +- 3 files changed, 52 insertions(+), 5 deletions(-)
[GitHub] [incubator-tvm-vta] tmoreau89 commented on pull request #9: [Hardware][OpenCL] Intelfocl support
tmoreau89 commented on pull request #9: URL: https://github.com/apache/incubator-tvm-vta/pull/9#issuecomment-660364642 Thanks for the changes. Please apply the 0.0.2 and rename the vta target to something more specific, e.g. "arria10". Also there are some CI errors related to linting that could be addressed. Thanks! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-tvm] icemelon9 opened a new pull request #6089: [Relay][VM] Add ReshapeTensor instruction in the VM to replace the reshape op
icemelon9 opened a new pull request #6089: URL: https://github.com/apache/incubator-tvm/pull/6089 @jroesch @zhiics @mbrookhart Could you help review? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-tvm-vta] tmoreau89 commented on a change in pull request #9: [Hardware][OpenCL] Intelfocl support
tmoreau89 commented on a change in pull request #9:
URL: https://github.com/apache/incubator-tvm-vta/pull/9#discussion_r456699804
##
File path: config/intelfocl_sample.json
##
@@ -0,0 +1,13 @@
+{
+ "TARGET" : "intelfocl",
+ "HW_VER" : "0.0.1",
Review comment:
Please apply the 0.0.2 bump thanks
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [incubator-tvm-vta] tmoreau89 commented on a change in pull request #9: [Hardware][OpenCL] Intelfocl support
tmoreau89 commented on a change in pull request #9:
URL: https://github.com/apache/incubator-tvm-vta/pull/9#discussion_r456699307
##
File path: config/intelfocl_sample.json
##
@@ -0,0 +1,13 @@
+{
+ "TARGET" : "intelfocl",
Review comment:
SO I believe that the solution was to go with "arria10" in this context?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [incubator-tvm] junrushao1994 commented on a change in pull request #6077: [Ansor][AutoTVM v2.0] Phase 1: Add RPC Runner
junrushao1994 commented on a change in pull request #6077:
URL: https://github.com/apache/incubator-tvm/pull/6077#discussion_r456695376
##
File path: tests/python/unittest/test_auto_scheduler_measure.py
##
@@ -67,6 +67,26 @@ def test_measure_local_builder_runner():
assert mress[0].error_no == 0
+def test_measure_local_builder_rpc_runner():
+dag, s0 = get_tiled_matmul()
+
+if not tvm.runtime.enabled("llvm"):
+return
Review comment:
let's move these two lines to the beginning of this function
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [incubator-tvm] tmoreau89 commented on pull request #5842: [VTA][OpenCL] Cloud FPGA support
tmoreau89 commented on pull request #5842: URL: https://github.com/apache/incubator-tvm/pull/5842#issuecomment-660356504 I'm thinking that the `static auto-tune using profiling results` can also be separated out in a standalone PR This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-tvm] tmoreau89 commented on pull request #5842: [VTA][OpenCL] Cloud FPGA support
tmoreau89 commented on pull request #5842: URL: https://github.com/apache/incubator-tvm/pull/5842#issuecomment-660355629 @zhanghaohit I took a second look at the changes in this PR, and I believe it would be best to break the PR down into separate, smaller PRs: (1) modifications to VTA for OpenCL support (runtime, operator support etc.) (2) quantization support for conv2d_transpose which is device agnostic (3) modifications to device_annotation.cc (4) once (3) is merged and approved, updating GraphPack pass to use device annotation Let me know what you think. This will make the PRs easier to review and merge. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-tvm] tmoreau89 commented on a change in pull request #5842: [VTA][OpenCL] Cloud FPGA support
tmoreau89 commented on a change in pull request #5842: URL: https://github.com/apache/incubator-tvm/pull/5842#discussion_r456673553 ## File path: python/tvm/relay/op/strategy/generic.py ## @@ -323,8 +331,10 @@ def compute_conv2d_transpose(attrs, inputs, out_dtype): out = topi_compute( inputs[0], inputs[1], strides, padding, out_dtype) output_padding = get_const_tuple(attrs.output_padding) -out = topi.nn.pad(out, [0, 0, 0, 0], - [0, 0, output_padding[0], output_padding[1]]) +if output_padding[0] != 0 or output_padding[1] != 0: Review comment: Which operators was this breaking on previously? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-tvm] tmoreau89 commented on a change in pull request #5842: [VTA][OpenCL] Cloud FPGA support
tmoreau89 commented on a change in pull request #5842:
URL: https://github.com/apache/incubator-tvm/pull/5842#discussion_r456673094
##
File path: python/tvm/autotvm/measure/measure_methods.py
##
@@ -186,6 +189,16 @@ def __init__(self,
timeout=10, n_parallel=None,
number=4, repeat=3, min_repeat_ms=0, cooldown_interval=0.1,
check_correctness=False):
+static_tune = os.getenv("TVM_STATIC_TUNE_EXPERIMENTAL")
Review comment:
Can you elaborate on what TVM_STATIC_TUNE_EXPERIMENTAL is being used
for? I'm a little weary of inserting additional execution modes in AutoTVM
dictated by environmental variables.
(1) we either add additional flags in AutoTVM config
(2) we clean this experimental mode if it's not really necessary for the
OpenCL variant of VTA
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [incubator-tvm] tmoreau89 commented on a change in pull request #5842: [VTA][OpenCL] Cloud FPGA support
tmoreau89 commented on a change in pull request #5842:
URL: https://github.com/apache/incubator-tvm/pull/5842#discussion_r456671336
##
File path: vta/python/vta/transform.py
##
@@ -381,9 +381,10 @@ def _fold_buffer_dim(buf, scope, elem_block):
def _get_2d_pattern(buf, elem_width, elem_bytes, dtype, scope, allow_fold):
elem_block = elem_bytes * 8 // elem_width
-if buf.dtype != dtype:
-raise RuntimeError("Expect buffer type to be %s instead of %s" %
- (dtype, buf.dtype))
+# remove the checking as we have load_int8 insn
Review comment:
Please remove those lines
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [incubator-tvm] tmoreau89 commented on a change in pull request #5842: [VTA][OpenCL] Cloud FPGA support
tmoreau89 commented on a change in pull request #5842: URL: https://github.com/apache/incubator-tvm/pull/5842#discussion_r456670755 ## File path: vta/python/vta/environment.py ## @@ -241,7 +243,7 @@ def target_host(self): return "llvm -target=armv7-none-linux-gnueabihf" if self.TARGET == "ultra96": return "llvm -target=aarch64-linux-gnu" -if self.TARGET in ["sim", "tsim"]: +if self.TARGET in ["sim", "tsim", "intelfocl"]: Review comment: Could you follow up and rename to say, "arria10"? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-tvm] jroesch commented on pull request #5892: Add TVM application extension with WASM runtime
jroesch commented on pull request #5892: URL: https://github.com/apache/incubator-tvm/pull/5892#issuecomment-660317393 I have an open PR https://github.com/apache/incubator-tvm/pull/6011 that is trying to move all Rust code to the updated bindings it might make sense to land this against the new bindings, what do you think? the changes should be very minimal. I'm hoping to land that branch today or Monday. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-tvm] anijain2305 commented on pull request #6039: MXNet pre-quantized BERT
anijain2305 commented on pull request #6039: URL: https://github.com/apache/incubator-tvm/pull/6039#issuecomment-660275132 @icemelon9 Can you please manage this PR? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-tvm] csullivan commented on a change in pull request #6069: [TIR][BugFix] Avoid simplifying substituted tir.Any expressions for layout transformations
csullivan commented on a change in pull request #6069:
URL: https://github.com/apache/incubator-tvm/pull/6069#discussion_r456595165
##
File path: src/tir/ir/data_layout.cc
##
@@ -323,7 +323,12 @@ inline Array TransformShape(const
Array& src_shape,
if (symbolic_var_set.count(i)) {
result.push_back(tir::Any());
} else {
-result.push_back(ana.Simplify(tir::Substitute(rule, bind_map)));
+auto sub = tir::Substitute(rule, bind_map);
+if (sub.as()) {
+ result.push_back(tir::Any());
+} else {
+ result.push_back(ana.Simplify(sub));
+}
Review comment:
Good point, let me get my head around the semantics and first order
simplifications.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [incubator-tvm] merrymercy commented on a change in pull request #6073: [Ansor][AutoTVM v2.0] Phase 1: Add annotation/compute_at/compute_root/compute_inline steps
merrymercy commented on a change in pull request #6073:
URL: https://github.com/apache/incubator-tvm/pull/6073#discussion_r456549272
##
File path: src/auto_scheduler/compute_dag.cc
##
@@ -276,10 +276,18 @@ std::pair>
ComputeDAG::ApplySteps(
// return value, so the ApplyToSchedule is not able to be merged to single
interface
if (auto ps = step.as()) {
ps->ApplyToSchedule(stages, stage_to_axes);
+} else if (auto ps = step.as()) {
Review comment:
Why do you use a different order from the order in our internal repo?
I propose we can use one of the following two orders.
1. Alphabetical order
2. Logical order from easy to hard.
- primitives working on one stage: reorder, split, fuse, annotation
- primitives working on multiple stages: compute_at, compute_root,
compute_inline
- primitive adding new stages: cache_read, cache_write, ...
And we should keep the same order in all places.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [incubator-tvm] anijain2305 commented on pull request #5980: Fixed point multiplication improvements for AArch64
anijain2305 commented on pull request #5980: URL: https://github.com/apache/incubator-tvm/pull/5980#issuecomment-660197332 Thanks @giuseros @tmoreau89 @kparzysz-quic This is merged! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[incubator-tvm] branch master updated (3c12a5e -> ccacb1e)
This is an automated email from the ASF dual-hosted git repository. anijain2305 pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/incubator-tvm.git. from 3c12a5e [Test] Add missing test for fast erf (#6058) add ccacb1e Fixed point multiplication improvements for AArch64 (#5980) No new revisions were added by this update. Summary of changes: include/tvm/relay/attrs/transform.h | 13 include/tvm/tir/builtin.h | 8 + include/tvm/tir/op.h | 21 + python/tvm/relay/op/_tensor.py| 8 + python/tvm/relay/op/tensor.py | 21 + python/tvm/tir/__init__.py| 1 + python/tvm/tir/op.py | 28 + src/relay/op/tensor/unary.cc | 23 ++ src/relay/qnn/op/requantize.cc| 12 ++- src/relay/qnn/util.cc | 43 ++--- src/relay/qnn/util.h | 32 +++ src/relay/quantize/realize.cc | 20 ++-- src/relay/transforms/pattern_util.h | 8 + src/target/intrin_rule.cc | 46 +++ src/tir/op/builtin.cc | 5 +++ src/tir/op/op.cc | 6 src/tir/transforms/lower_intrin.cc| 13 ++-- tests/python/relay/test_op_level3.py | 17 ++ topi/python/topi/arm_cpu/conv2d_gemm.py | 13 +--- topi/python/topi/arm_cpu/conv2d_int8.py | 14 - topi/python/topi/arm_cpu/injective.py | 3 +- topi/python/topi/arm_cpu/tensor_intrin.py | 52 +++ topi/python/topi/math.py | 27 23 files changed, 382 insertions(+), 52 deletions(-)
[GitHub] [incubator-tvm] icemelon9 merged pull request #6058: [Test] Add missing test for fast erf
icemelon9 merged pull request #6058: URL: https://github.com/apache/incubator-tvm/pull/6058 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-tvm] tmoreau89 commented on pull request #5842: [VTA][OpenCL] Cloud FPGA support
tmoreau89 commented on pull request #5842: URL: https://github.com/apache/incubator-tvm/pull/5842#issuecomment-660185671 @zhanghaohit thanks for the follow up. I recommend to echo @liangfu to rebase so the PR can be in a mergeable state. In addition, if the target is a custom FPGA board, I think we can leave it as a generic target for now. Having it ported to AWS F1 should be a great addition. I understand having setup instructions at this time would not make the most sense. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-tvm] tmoreau89 commented on pull request #5980: Fixed point multiplication improvements for AArch64
tmoreau89 commented on pull request #5980: URL: https://github.com/apache/incubator-tvm/pull/5980#issuecomment-660183790 @anijain2305 please let us know if your requested changes have been addressed! thanks! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-tvm] merrymercy edited a comment on pull request #6040: [TOPI] Improve schedule for injective on x86
merrymercy edited a comment on pull request #6040: URL: https://github.com/apache/incubator-tvm/pull/6040#issuecomment-660172683 Currently, both c++ and python version work well. This PR is ready to be merged from my side. cc @icemelon9 I have no idea how to solve the mess of duplicated python and c++ topi functions. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-tvm] merrymercy commented on pull request #6040: [TOPI] Improve schedule for injective on x86
merrymercy commented on pull request #6040: URL: https://github.com/apache/incubator-tvm/pull/6040#issuecomment-660172510 Currently, both c++ and python versions work well. This PR is ready to be merged from my side. I have no idea how to solve the mess of duplicated python and c++ topi functions. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-tvm] merrymercy commented on pull request #6040: [TOPI] Improve schedule for injective on x86
merrymercy commented on pull request #6040: URL: https://github.com/apache/incubator-tvm/pull/6040#issuecomment-660172683 Currently, both c++ and python versions work well. This PR is ready to be merged from my side. cc @icemelon9 I have no idea how to solve the mess of duplicated python and c++ topi functions. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-tvm] merrymercy removed a comment on pull request #6040: [TOPI] Improve schedule for injective on x86
merrymercy removed a comment on pull request #6040: URL: https://github.com/apache/incubator-tvm/pull/6040#issuecomment-660172510 Currently, both c++ and python versions work well. This PR is ready to be merged from my side. I have no idea how to solve the mess of duplicated python and c++ topi functions. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-tvm] tqchen commented on pull request #6040: [TOPI] Improve schedule for injective on x86
tqchen commented on pull request #6040: URL: https://github.com/apache/incubator-tvm/pull/6040#issuecomment-660150798 If c++ version is good we can call into the c++ version from the pyone This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-tvm] tqchen commented on pull request #6083: [WIP] Fix LocalBuilder on macOS with python 3.8.
tqchen commented on pull request #6083: URL: https://github.com/apache/incubator-tvm/pull/6083#issuecomment-660150229 Thanks @tkonolige ! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-tvm] tqchen merged pull request #6083: [WIP] Fix LocalBuilder on macOS with python 3.8.
tqchen merged pull request #6083: URL: https://github.com/apache/incubator-tvm/pull/6083 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[incubator-tvm] branch master updated (3ddce13 -> 378a85a)
This is an automated email from the ASF dual-hosted git repository. tqchen pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/incubator-tvm.git. from 3ddce13 [Fix] Add missing expr visitor for any (#6082) add 378a85a Fix LocalBuilder on macos with python 3.8. (#6083) No new revisions were added by this update. Summary of changes: python/tvm/autotvm/measure/measure_methods.py | 24 +--- 1 file changed, 13 insertions(+), 11 deletions(-)
[GitHub] [incubator-tvm] tqchen commented on a change in pull request #6069: [TIR][BugFix] Avoid simplifying substituted tir.Any expressions for layout transformations
tqchen commented on a change in pull request #6069:
URL: https://github.com/apache/incubator-tvm/pull/6069#discussion_r456491809
##
File path: src/tir/ir/data_layout.cc
##
@@ -323,7 +323,12 @@ inline Array TransformShape(const
Array& src_shape,
if (symbolic_var_set.count(i)) {
result.push_back(tir::Any());
} else {
-result.push_back(ana.Simplify(tir::Substitute(rule, bind_map)));
+auto sub = tir::Substitute(rule, bind_map);
+if (sub.as()) {
+ result.push_back(tir::Any());
+} else {
+ result.push_back(ana.Simplify(sub));
+}
Review comment:
Perhaps in this case we should go and fix Simplify for Any in the arith
module and also add a testcase there
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[incubator-tvm] branch master updated (6e1b09e -> 3ddce13)
This is an automated email from the ASF dual-hosted git repository. tqchen pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/incubator-tvm.git. from 6e1b09e [TOPI] Fix the filter width parameter in depthwise_conv2d (#6081) add 3ddce13 [Fix] Add missing expr visitor for any (#6082) No new revisions were added by this update. Summary of changes: include/tvm/tir/expr_functor.h | 4 src/tir/ir/expr_functor.cc | 4 2 files changed, 8 insertions(+)
[GitHub] [incubator-tvm] tqchen merged pull request #6082: [Fix] Add missing expr visitor for any
tqchen merged pull request #6082: URL: https://github.com/apache/incubator-tvm/pull/6082 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-tvm] tqchen commented on pull request #6082: [Fix] Add missing expr visitor for any
tqchen commented on pull request #6082: URL: https://github.com/apache/incubator-tvm/pull/6082#issuecomment-660148843 Thanks @icemelon9 @junrushao1994 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-tvm] cam145 opened a new issue #6088: tflite frontend reshape operator AssertionError assert op.BuiltinOptionsType() == BuiltinOptions.ReshapeOptions
cam145 opened a new issue #6088: URL: https://github.com/apache/incubator-tvm/issues/6088 Thanks for participating in the TVM community! We use https://discuss.tvm.ai for any general usage questions and discussions. The issue tracker is used for actionable items such as feature proposals discussion, roadmaps, and bug tracking. You are always welcomed to post on the forum first :) Issues that are inactive for a period of time may get closed. We adopt this policy so that we won't lose track of actionable issues that may fall at the bottom of the pile. Feel free to reopen a new one if you feel there is an additional problem that needs attention when an old one gets closed. For bug reports, to help the developer act on the issues, please include a description of your environment, preferably a minimum script to reproduce the problem. Based on the latest master branch, I used the following script https://www.dropbox.com/s/eooupdzsnxm5i8u/test_tflite_frontend.py?dl=0 and model file: https://www.dropbox.com/s/qxiyfaiaefv57m8/simple_model_test_reshape.tflite?dl=0 Then got the following information: assert op.BuiltinOptionsType() == BuiltinOptions.ReshapeOptions AssertionError == Print out get the following information: op.BuiltinOptionsType() = 0 BuiltinOptions.ReshapeOptions = 17 It seems that the two 'reshape' operators are incompatible or something else. Thanks For feature proposals, list clear, small actionable items so we can track the progress of the change. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-tvm] siju-samuel commented on pull request #6023: [CI][Caffe Frontend] add caffe environment
siju-samuel commented on pull request #6023: URL: https://github.com/apache/incubator-tvm/pull/6023#issuecomment-660085200 > 1. We add caffe frontend based on SSD distribution, then this ci pr is good enough to test; > 2. We add caffe frontend based on our maintaining tvm caffe, then I will support the layers of BVLC distribution firstly. I vote for second option. BVLC caffe models will have more users compared to forked versions. Another concern is lot of dependancies are getting added for running the ci with custom distribution, i think bvlc have support for precompiled caffe installation. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-tvm] giuseros commented on pull request #5980: Fixed point multiplication improvements for AArch64
giuseros commented on pull request #5980: URL: https://github.com/apache/incubator-tvm/pull/5980#issuecomment-660067570 Hi @anijain2305 , @tmoreau89 , @kparzysz-quic I addressed the name change, reformatted the help text in `math.py` and addressed an integration change in the arm intrinsic (`q.val` -> `q.value`). Thank you all for your comments and your time! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-tvm] giuseros commented on a change in pull request #5980: Fixed point multiplication improvements for AArch64
giuseros commented on a change in pull request #5980: URL: https://github.com/apache/incubator-tvm/pull/5980#discussion_r456394839 ## File path: topi/python/topi/math.py ## @@ -612,6 +612,31 @@ def _compute(*indices): return tvm.te.max(tvm.te.min(value, const_max), const_min) return te.compute(x.shape, _compute) +@tvm.te.tag_scope(tag=tag.ELEMWISE) +def fixed_point_multiply(x, multiplier, shift): +""" + +Parameters +-- +x : tvm.te.Tensor or Expr + Input argument. Review comment: I reformatted it to follow the same style of other help texts. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-tvm] HakanKumdakci opened a new issue #6087: error building docker image
HakanKumdakci opened a new issue #6087: URL: https://github.com/apache/incubator-tvm/issues/6087 I used this commmand in new cloned repository of incubator-tvm. `docker build -t tvm.demo_android -f docker/Dockerfile.demo_android ./docker` And got this error  I couln't fix it. Its seems like sdkmanager can't find repo for extras;android;gapid;1 in repositories.cfg. So it can't download it and can't find it in device. I downloaded gapid and installed it. But it still doesn't work. Waiting for help.. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-tvm] FrozenGene edited a comment on pull request #5913: [ndarray][autotvm] support ndarray.non_empty
FrozenGene edited a comment on pull request #5913:
URL: https://github.com/apache/incubator-tvm/pull/5913#issuecomment-660027353
> see comments, we don't need to expose the(non_empty) function as a
primitive function, given that it is not a numpy function, and is only used in
AutoTVM, instead we can achieve the same goal by
>
> ```python
> x = nd.empty(..)
> random_fill = get_packed_func("contrib.random.random_fill")
> random_fill(x)
> ```
>
> Notably, the above solution is better because:
>
> * Minimum interface exposure(non autotvm devs don't need to be aware of
the change)
> * Random initialization is performed on the device, the current impl
actually will results in random number generated on the host then transfering
to the device.
@tqchen I restart this work. This way we could make the random
initialization on the device (i.e. `AllocaWorkSpace` use correct device api).
But as far as I know we still can not avoid generating random numbers on host
if we don't call specific apis like `cuRAND` (if we are for NV GPU) . That is
to say, we still have to generate random numbers and copy to device. However,
we should accomplish the goal we could generate random numbers on the remote
cpu (if we are in rpc mode) and copy the data from remote cpu (like arm) to
remote GPU (like mali) directly, not the path `x86 cpu -> arm cpu -> mali gpu`.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [incubator-tvm] FrozenGene commented on pull request #5913: [ndarray][autotvm] support ndarray.non_empty
FrozenGene commented on pull request #5913:
URL: https://github.com/apache/incubator-tvm/pull/5913#issuecomment-660027353
> see comments, we don't need to expose the(non_empty) function as a
primitive function, given that it is not a numpy function, and is only used in
AutoTVM, instead we can achieve the same goal by
>
> ```python
> x = nd.empty(..)
> random_fill = get_packed_func("contrib.random.random_fill")
> random_fill(x)
> ```
>
> Notably, the above solution is better because:
>
> * Minimum interface exposure(non autotvm devs don't need to be aware of
the change)
> * Random initialization is performed on the device, the current impl
actually will results in random number generated on the host then transfering
to the device.
@tqchen I restart to this work. This way we could make the random
initialization on the device (i.e. `AllocaWorkSpace` use correct device api).
But as far as I know we still can not avoid generating random numbers on host
if we don't call specific apis like `cuRAND` (if we are for NV GPU) . That is
to say, we still have to generate random numbers and copy to device. However,
we should accomplish the goal we could generate random numbers on the remote
cpu (if we are in rpc mode) and copy the data from remote cpu (like arm) to
remote GPU (like mali) directly, not the path `x86 cpu -> arm cpu -> mali gpu`.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [incubator-tvm] lhutton1 commented on a change in pull request #5915: [BYOC][Contrib] Arm Compute Library integration
lhutton1 commented on a change in pull request #5915:
URL: https://github.com/apache/incubator-tvm/pull/5915#discussion_r456353670
##
File path: src/relay/backend/contrib/arm_compute_lib/codegen.cc
##
@@ -0,0 +1,186 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file src/relay/backend/contrib/arm_compute_lib/codegen_acl.cc
+ * \brief Implementation of the Relay -> ACL JSON serializer.
+ */
+#include
+#include
+#include
+
+#include "../../utils.h"
+#include "codegen_acl.h"
+
+namespace tvm {
+namespace relay {
+namespace contrib {
+namespace arm_compute_lib {
+
+using JSONGraphNode = tvm::runtime::json::JSONGraphNode;
+using JSONGraphNodeEntry = tvm::runtime::json::JSONGraphNodeEntry;
+
+std::vector ACLJSONSerializer::VisitExpr_(const CallNode*
cn) {
+ Expr expr = GetRef(cn);
+ std::string name;
+ std::shared_ptr json_node;
+
+ if (cn->op.as()) {
+json_node = CreateOpJSONNode(cn);
+ } else if (const auto* fn = cn->op.as()) {
+auto comp = fn->GetAttr(attr::kComposite);
+CHECK(comp.defined()) << "Arm Compute Library JSON runtime only supports
composite functions.";
Review comment:
Currently functions are not supported by JSON codegen, although we can
always change this functionality in the ACL codegen. I think it's ok for now to
support only composite functions but we could add this in the future.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [incubator-tvm] manupa-arm commented on a change in pull request #5915: [BYOC][Contrib] Arm Compute Library integration
manupa-arm commented on a change in pull request #5915:
URL: https://github.com/apache/incubator-tvm/pull/5915#discussion_r456314032
##
File path: src/relay/backend/contrib/arm_compute_lib/codegen.cc
##
@@ -0,0 +1,186 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file src/relay/backend/contrib/arm_compute_lib/codegen_acl.cc
+ * \brief Implementation of the Relay -> ACL JSON serializer.
+ */
+#include
+#include
+#include
+
+#include "../../utils.h"
+#include "codegen_acl.h"
+
+namespace tvm {
+namespace relay {
+namespace contrib {
+namespace arm_compute_lib {
+
+using JSONGraphNode = tvm::runtime::json::JSONGraphNode;
+using JSONGraphNodeEntry = tvm::runtime::json::JSONGraphNodeEntry;
+
+std::vector ACLJSONSerializer::VisitExpr_(const CallNode*
cn) {
+ Expr expr = GetRef(cn);
+ std::string name;
+ std::shared_ptr json_node;
+
+ if (cn->op.as()) {
+json_node = CreateOpJSONNode(cn);
+ } else if (const auto* fn = cn->op.as()) {
+auto comp = fn->GetAttr(attr::kComposite);
+CHECK(comp.defined()) << "Arm Compute Library JSON runtime only supports
composite functions.";
Review comment:
Minor comment : Is there a downside going into nested function
hierarchies that are not composite functions. What if just use
JSONSerializer::VisiExpr_(cn) for every other case if its not a composite
function?
##
File path: python/tvm/relay/op/contrib/arm_compute_lib.py
##
@@ -0,0 +1,119 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+# pylint: disable=invalid-name, unused-argument
+"""ACL library supported operators."""
+import tvm
+from tvm.relay import transform
+from tvm.relay.build_module import bind_params_by_name
+
+from ...dataflow_pattern import wildcard, is_op, is_constant
+from .register import register_pattern_table
+
+
+def is_arm_compute_runtime_enabled():
+"""Check if the ACL graph runtime is present.
+
+Returns
+---
+ret: bool
+True if present, False if not.
+"""
+return tvm.get_global_func("relay.op.is_arm_compute_runtime_enabled", True)
+
+
+def partition_for_arm_compute_lib(mod, params=None):
+"""Partition the graph greedily offloading supported
+operators to Arm Compute Library.
+
+Parameters
+--
+mod : Module
+The module to run passes on.
+params : Optional[Dict[str, NDArray]]
+Constant input parameters.
+
+Returns
+---
+ret : annotated and partitioned module.
+"""
+if params:
+mod['main'] = bind_params_by_name(mod['main'], params)
+
+seq =
tvm.transform.Sequential([transform.MergeComposite(arm_compute_lib_pattern_table()),
+
transform.AnnotateTarget('arm_compute_lib'),
+transform.PartitionGraph()])
+
+return seq(mod)
+
+
+@register_pattern_table("arm_compute_lib")
+def arm_compute_lib_pattern_table():
+"""Get the ACL pattern table."""
+
+def conv_pattern():
+"""Create a convolution pattern.
+
+Returns
+---
+pattern : dataflow_pattern.AltPattern
+Denotes the convolution pattern.
+"""
+pattern = is_op('nn.pad')(wildcard()) | wildcard()
+pattern = is_op('nn.conv2d')(pattern, is_constant())
+pattern = pattern.optional(lambda x: is_op('nn.bias_add')(x,
is_constant()))
+pattern = pattern.optional(is_op('nn.relu'))
Review comment:
[GitHub] [incubator-tvm] jcf94 commented on a change in pull request #6073: [Ansor][AutoTVM v2.0] Phase 1: Add annotation/compute_at/compute_root/compute_inline steps
jcf94 commented on a change in pull request #6073:
URL: https://github.com/apache/incubator-tvm/pull/6073#discussion_r456321361
##
File path: src/auto_scheduler/utils.h
##
@@ -89,6 +89,15 @@ inline int GetIndex(const Array& array, const T&
to_locate) {
return -1;
}
+/*! \brief Delete the item in a std::vector. */
+template
+inline void DeleteItem(std::vector* array, const T& to_delete) {
Review comment:
Updated.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [incubator-tvm] jcf94 commented on a change in pull request #6073: [Ansor][AutoTVM v2.0] Phase 1: Add annotation/compute_at/compute_root/compute_inline steps
jcf94 commented on a change in pull request #6073: URL: https://github.com/apache/incubator-tvm/pull/6073#discussion_r456321060 ## File path: tests/python/unittest/test_auto_scheduler_loop_state.py ## @@ -61,5 +61,79 @@ def test_split_fuse_reorder(): assert s1[C].iters[4].range.extent == 8 assert s1[C].iters[5].range.extent == 2 +s1.parallel(C, j1) +s1.unroll(C, j2) +s1.vectorize(C, j3) +s1.bind(C, i1, "blockIdx.x") +s1.bind(C, i2, "vthread") +s1.bind(C, i3, "threadIdx.y") + + +def test_compute_at_root_inline(): +dag = auto_scheduler.ComputeDAG(conv2d_nchw_bn_relu(1, 224, 224, 3, 64, 7, 2, 3)) +s0 = dag.get_init_state() + +# data, padding, kernel = 0, 1, 2 +conv = s0.stage_ops[3] +# bias = 4 +bias_add = s0.stage_ops[5] +# bn_scale = 6 +bn_mul = s0.stage_ops[7] +# bn_offset = 8 +bn_add = s0.stage_ops[9] +relu = s0.stage_ops[10] + +s0.compute_inline(bn_add) +s0.compute_inline(bn_mul) +s0.compute_inline(bias_add) +s0.compute_at(conv, relu, s0[relu].iters[2]) +print(s0) +assert str(s0) == \ +"Placeholder: Data, Kernel, Bias, Bn_scale, Bn_offset\n" + \ +"for i1 (0,3)\n" + \ +" for i2 (0,230)\n" + \ +"for i3 (0,230)\n" + \ +" pad_temp = ...\n" + \ +"for i1 (0,64)\n" + \ +" for i2 (0,112)\n" + \ +"for nn (None)\n" + \ +" for ff (None)\n" + \ +"for yy (None)\n" + \ +" for xx (None)\n" + \ +"for rc (None)\n" + \ +" for ry (None)\n" + \ +"for rx (None)\n" + \ +" compute = ...\n" + \ +"for i3 (0,112)\n" + \ +" compute = ...\n" Review comment: Updated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-tvm] jcf94 commented on a change in pull request #6073: [Ansor][AutoTVM v2.0] Phase 1: Add annotation/compute_at/compute_root/compute_inline steps
jcf94 commented on a change in pull request #6073: URL: https://github.com/apache/incubator-tvm/pull/6073#discussion_r456320683 ## File path: tests/python/unittest/test_auto_scheduler_loop_state.py ## @@ -61,5 +61,79 @@ def test_split_fuse_reorder(): assert s1[C].iters[4].range.extent == 8 assert s1[C].iters[5].range.extent == 2 +s1.parallel(C, j1) +s1.unroll(C, j2) +s1.vectorize(C, j3) +s1.bind(C, i1, "blockIdx.x") +s1.bind(C, i2, "vthread") +s1.bind(C, i3, "threadIdx.y") Review comment: Check added, and this really helped me to find a hidden bug. :) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-tvm] jcf94 commented on a change in pull request #6073: [Ansor][AutoTVM v2.0] Phase 1: Add annotation/compute_at/compute_root/compute_inline steps
jcf94 commented on a change in pull request #6073:
URL: https://github.com/apache/incubator-tvm/pull/6073#discussion_r456319743
##
File path: src/auto_scheduler/loop_state.cc
##
@@ -143,12 +265,67 @@ void State::DoReorderStep(const ReorderStep& step) {
Stage(stage->op, stage->op_type, iters,
stage->compute_at, stage->attrs));
}
+void State::DoComputeAtStep(const ComputeAtStep& step) {
+ const Stage& stage = operator->()->stages[step->stage_id];
+
+ // After compute_at, we don't know the accurate length information any more
+ // If we do want to know the accurate lengths, we can call
ComputeDAG::InferBound
Review comment:
Doc updated.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [incubator-tvm] jcf94 commented on a change in pull request #6073: [Ansor][AutoTVM v2.0] Phase 1: Add annotation/compute_at/compute_root/compute_inline steps
jcf94 commented on a change in pull request #6073:
URL: https://github.com/apache/incubator-tvm/pull/6073#discussion_r456319452
##
File path: src/auto_scheduler/loop_state.cc
##
@@ -90,12 +90,69 @@ Stage::Stage(te::Operation op, StageKind op_type, const
Array& iters,
data_ = std::move(node);
}
+/** AttachMap **/
+void AttachMap::SetComputeAtIter(int stage_id, int target_stage_id, int
target_iter_id) {
+ AttachMapNode* pnode = CopyOnWrite();
+
+ // Delete the current entry of this stage
+ DeleteStageEntry(pnode, stage_id);
+
+ // Store the new relations to map
+ IterKey iter_key(target_stage_id, target_iter_id);
+ pnode->stage_to_attach_iter[stage_id] = iter_key;
+ pnode->iter_to_attached_stages[iter_key].push_back(stage_id);
+}
+
+void AttachMap::DeleteStage(int stage_id) {
+ AttachMapNode* pnode = CopyOnWrite();
+ // Delete the original stage entry
+ DeleteStageEntry(pnode, stage_id);
+}
+
+void AttachMap::UpdateIters(const std::vector& old_iters,
+const std::vector& new_iters) {
Review comment:
Moved this check to front.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [incubator-tvm] jcf94 commented on a change in pull request #6073: [Ansor][AutoTVM v2.0] Phase 1: Add annotation/compute_at/compute_root/compute_inline steps
jcf94 commented on a change in pull request #6073:
URL: https://github.com/apache/incubator-tvm/pull/6073#discussion_r456319021
##
File path: src/auto_scheduler/loop_state.h
##
@@ -217,6 +196,68 @@ class Stage : public ObjectRef {
TVM_DEFINE_OBJECT_REF_COW_METHOD(StageNode);
};
+/*! \brief Use stage_id to represent a stage. */
+using StageKey = int;
+/*! \brief Use stage_id and iter_id to represent a iterator. */
+using IterKey = std::pair;
+
+/*!
+ * \brief stores the compute_at relation between stages
+ * This stores a bi-directional mapping from stages and iter:
+ * 1. Stage to its attached iterator
+ * 2. Iterator to the stage attached to it
+ * You can use AttachMapNode::stage_to_attach_iter and
AttachMapNode::iter_to_attached_stages
+ * to query the relations
+ */
+class AttachMapNode : public Object {
+ public:
+ /*! \brief A Map to store the mapping of stage to its attached iterator. */
+ std::unordered_map stage_to_attach_iter;
+ /*! \brief A Map to store the mapping of iterator to the stage attached to
it. */
+ std::unordered_map> iter_to_attached_stages;
+
+ static constexpr const char* _type_key = "auto_scheduler.AttachMap";
+ TVM_DECLARE_FINAL_OBJECT_INFO(AttachMapNode, Object);
+};
+
+/*!
+ * \brief Managed reference to AttachMapNode.
+ * \sa AttachMapNode
+ */
+class AttachMap : public ObjectRef {
+ public:
+ /*!
+ * \brief Process the stage/iterator mapping after compute at.
+ * \param stage_id The index of the stage to be compute at.
+ * \param target_stage_id The index of stage that this step will compute at
to.
+ * \param target_iter_id The index of iterator in target stage that this
step will compute at to.
+ */
+ void SetComputeAtIter(int stage_id, int target_stage_id, int target_iter_id);
+ /*!
+ * \brief This is a public wrapper of `DeleteStageEntry`. To delete the
entry of a specific stage.
+ * \param stage_id The index of the stage to be compute at.
+ */
+ void DeleteStage(int stage_id);
+ /*!
+ * \brief Update the iterator relations in AttachMap.
Review comment:
Doc updated.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [incubator-tvm] jcf94 commented on a change in pull request #6073: [Ansor][AutoTVM v2.0] Phase 1: Add annotation/compute_at/compute_root/compute_inline steps
jcf94 commented on a change in pull request #6073: URL: https://github.com/apache/incubator-tvm/pull/6073#discussion_r456318718 ## File path: python/tvm/auto_scheduler/loop_state.py ## @@ -161,16 +202,116 @@ def fuse(self, stage, iters): The Stage to be fused, can be a Stage order index, Stage operation or stage output tensor. iters : List[Iterator] -The iterators to be fused +The iterators to be fused. + +Returns +--- +res_it : Iterator +The fused Iterator. +""" +self.state_object, res = _ffi_api.StateFuse(self.state_object, + self._resolve_stage_id(stage), iters) +return res + +def vectorize(self, stage, iterator): +""" Schedule primitive corresponds to te.vectorize. + +Parameters +-- +stage : Union[int, Operation, Tensor] +The Stage to be vectorized, can be a Stage order index, Stage operation or stage +output tensor. +iterator : Iterator +The iterator to be vectorized. Returns --- res_it : Iterator -The fused Iterator +The vectorized Iterator. """ -stage_id = self._resolve_stage_id(stage) +self.state_object, res = _ffi_api.StateVectorize(self.state_object, + self._resolve_stage_id(stage), iterator) +return res + +def parallel(self, stage, iterator): +""" Schedule primitive corresponds to te.parallel. -self.state_object, res = _ffi_api.StateFuse(self.state_object, stage_id, iters) +Parameters +-- +stage : Union[int, Operation, Tensor] +The Stage to be paralleled, can be a Stage order index, Stage operation or stage +output tensor. +iterator : Iterator +The iterator to be paralleled. + +Returns +--- +res_it : Iterator +The paralleled Iterator. +""" +self.state_object, res = _ffi_api.StateParallel(self.state_object, + self._resolve_stage_id(stage), iterator) +return res + +def unroll(self, stage, iterator, max_unroll=None): +""" Schedule primitive corresponds to te.unroll. + +Parameters +-- +stage : Union[int, Operation, Tensor] +The Stage to be unrolled, can be a Stage order index, Stage operation or stage +output tensor. +iterator : Iterator +The iterator to be unrolled. +max_unroll : Optional[int] +The max unroll limit. Iterator with extent larger than this limit will be skipped. + +Returns +--- +res_it : Iterator +The unrolled Iterator. +""" +self.state_object, res = _ffi_api.StateUnroll(self.state_object, + self._resolve_stage_id(stage), iterator, + max_unroll if max_unroll else -1) +return res + +def bind(self, stage, iterator, thread_name): +""" Schedule primitive corresponds to te.bind. + +Parameters +-- +stage : Union[int, Operation, Tensor] +The Stage to be binded, can be a Stage order index, Stage operation or stage +output tensor. +iterator : Iterator +The iterator to be binded. +thread_name : str +The thread type to be binded. Currently support: +- vthread +- blockIdx.x +- threadIdx.x +- blockIdx.y +- threadIdx.y Review comment: blockIdx.z & threadIdx.z Added. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-tvm] jcf94 commented on a change in pull request #6073: [Ansor][AutoTVM v2.0] Phase 1: Add annotation/compute_at/compute_root/compute_inline steps
jcf94 commented on a change in pull request #6073:
URL: https://github.com/apache/incubator-tvm/pull/6073#discussion_r456318375
##
File path: python/tvm/auto_scheduler/loop_state.py
##
@@ -161,16 +202,116 @@ def fuse(self, stage, iters):
The Stage to be fused, can be a Stage order index, Stage operation
or stage
output tensor.
iters : List[Iterator]
-The iterators to be fused
+The iterators to be fused.
+
+Returns
+---
+res_it : Iterator
+The fused Iterator.
+"""
+self.state_object, res = _ffi_api.StateFuse(self.state_object,
+
self._resolve_stage_id(stage), iters)
+return res
+
+def vectorize(self, stage, iterator):
+""" Schedule primitive corresponds to te.vectorize.
+
+Parameters
+--
+stage : Union[int, Operation, Tensor]
+The Stage to be vectorized, can be a Stage order index, Stage
operation or stage
+output tensor.
+iterator : Iterator
+The iterator to be vectorized.
Returns
---
res_it : Iterator
-The fused Iterator
+The vectorized Iterator.
"""
-stage_id = self._resolve_stage_id(stage)
+self.state_object, res = _ffi_api.StateVectorize(self.state_object,
+
self._resolve_stage_id(stage), iterator)
+return res
+
+def parallel(self, stage, iterator):
+""" Schedule primitive corresponds to te.parallel.
-self.state_object, res = _ffi_api.StateFuse(self.state_object,
stage_id, iters)
+Parameters
+--
+stage : Union[int, Operation, Tensor]
+The Stage to be paralleled, can be a Stage order index, Stage
operation or stage
+output tensor.
+iterator : Iterator
+The iterator to be paralleled.
+
+Returns
+---
+res_it : Iterator
+The paralleled Iterator.
+"""
+self.state_object, res = _ffi_api.StateParallel(self.state_object,
+
self._resolve_stage_id(stage), iterator)
+return res
+
+def unroll(self, stage, iterator, max_unroll=None):
+""" Schedule primitive corresponds to te.unroll.
+
+Parameters
+--
+stage : Union[int, Operation, Tensor]
+The Stage to be unrolled, can be a Stage order index, Stage
operation or stage
+output tensor.
+iterator : Iterator
+The iterator to be unrolled.
+max_unroll : Optional[int]
+The max unroll limit. Iterator with extent larger than this limit
will be skipped.
+
+Returns
+---
+res_it : Iterator
+The unrolled Iterator.
+"""
+self.state_object, res = _ffi_api.StateUnroll(self.state_object,
+
self._resolve_stage_id(stage), iterator,
+ max_unroll if max_unroll
else -1)
+return res
+
+def bind(self, stage, iterator, thread_name):
+""" Schedule primitive corresponds to te.bind.
+
+Parameters
+--
+stage : Union[int, Operation, Tensor]
+The Stage to be binded, can be a Stage order index, Stage
operation or stage
+output tensor.
+iterator : Iterator
+The iterator to be binded.
+thread_name : str
+The thread type to be binded. Currently support:
+- vthread
+- blockIdx.x
+- threadIdx.x
+- blockIdx.y
+- threadIdx.y
+
+Returns
+---
+res_it : Iterator
+The binded Iterator.
+"""
+trans_table = {
+"vthread": 4,
+"blockIdx.x": 5,
+"threadIdx.x": 6,
+"blockIdx.y": 7,
+"threadIdx.y": 8,
+}
Review comment:
Moved this to class static member.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [incubator-tvm] lhutton1 commented on a change in pull request #5915: [BYOC][Contrib] Arm Compute Library integration
lhutton1 commented on a change in pull request #5915:
URL: https://github.com/apache/incubator-tvm/pull/5915#discussion_r456313568
##
File path: src/runtime/contrib/arm_compute_lib/acl_runtime.cc
##
@@ -0,0 +1,310 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file src/runtime/contrib/arm_compute_lib/acl_runtime.cc
+ * \brief A simple JSON runtime for Arm Compute Library.
+ */
+
+#include
+#include
+
+#include "../../file_util.h"
+#include "../json/json_node.h"
+#include "../json/json_runtime.h"
+
+#ifdef TVM_GRAPH_RUNTIME_ARM_COMPUTE_LIB
+#include
+#include
+#include
+#include
+
+#include "acl_allocator.h"
+#include "acl_utils.h"
+#endif
+
+namespace tvm {
+namespace runtime {
+namespace contrib {
+
+using namespace tvm::runtime::json;
+
+#ifdef TVM_GRAPH_RUNTIME_ARM_COMPUTE_LIB
+using namespace arm_compute_lib;
+#endif
+
+class ACLRuntime : public JSONRuntimeBase {
+ public:
+ /*!
+ * \brief The ACL runtime module. Deserialize the provided functions
+ * on creation and store in the layer cache.
+ *
+ * \param symbol_name The name of the function.
+ * \param graph_json serialized JSON representation of a sub-graph.
+ * \param const_names The names of each constant in the sub-graph.
+ */
+ explicit ACLRuntime(const std::string& symbol_name, const std::string&
graph_json,
+ const Array& const_names)
+ : JSONRuntimeBase(symbol_name, graph_json, const_names) {}
+
+ /*!
+ * \brief The type key of the module.
+ *
+ * \return module type key.
+ */
+ const char* type_key() const override { return "arm_compute_lib"; }
+
+ /*!
+ * \brief Initialize runtime. Create ACL layer from JSON
+ * representation.
+ *
+ * \param consts The constant params from compiled model.
+ */
+ void Init(const Array& consts) override {
+CHECK_EQ(consts.size(), const_idx_.size())
+<< "The number of input constants must match the number of required.";
+SetupConstants(consts);
+BuildEngine();
+ }
+
+ /*!
+ * \brief Get the JSON generated by codegen.
+ *
+ * \param format the format to return (only JSON for the time being)
+ * \return A string of JSON.
+ */
+ std::string GetSource(const std::string& format) override {
+if (format == "json") {
+ return graph_json_;
+}
+LOG(FATAL) << "Format not supported by Arm Compute Library runtime.";
+return "";
+ }
+
+#ifdef TVM_GRAPH_RUNTIME_ARM_COMPUTE_LIB
+ /*!
+ * \brief Unpack inputs and outputs and run inference on a given layer.
+ *
+ * \param args Access inputs and outputs.
+ * \param function The layer to execute inference on.
+ * \return Status of inference.
+ */
+ void Run() override {
+for (size_t i = 0; i < input_nodes_.size(); ++i) {
+ auto nid = input_nodes_[i];
+ uint32_t eid = EntryID(nid, 0);
+ if (nodes_[nid].GetOpType() == "input") {
+void* data = data_entry_[eid]->data;
+CheckACLError(layer_.inputs[i].allocator()->import_memory(data));
+ }
+}
+
+for (size_t i = 0; i < outputs_.size(); ++i) {
+ uint32_t eid = EntryID(outputs_[i]);
+ void* data = data_entry_[eid]->data;
+ CheckACLError(layer_.outputs[i].allocator()->import_memory(data));
+}
+
+this->layer_.function->run();
+ }
+
+ private:
+ /*!
+ * \brief Build ACL layer from JSON representation and cache.
+ *
+ * \note For the time being only one layer or operator is supported
+ * per engine.
+ */
+ void BuildEngine() {
+std::shared_ptr mm =
MakeMemoryManager();
+int num_pools = 0;
+
+for (size_t i = 0; i < input_nodes_.size(); ++i) {
+ uint32_t nid = input_nodes_[i];
+ const auto& node = nodes_[nid];
+ if (node.GetOpType() == "input") {
+layer_.inputs.push_back(MakeTensor(node));
+ } else if (node.GetOpType() == "const") {
+uint32_t eid = EntryID(nid, 0);
+void* data = data_entry_[eid]->data;
+layer_.const_inputs.push_back(MakeTensor(node, data));
+ }
+}
+
+bool found_kernel_node = false;
+for (size_t nid = 0; nid < nodes_.size(); ++nid) {
+ const auto& node = nodes_[nid];
+ if (found_kernel_node) {
+LOG(FATAL)
+
[GitHub] [incubator-tvm] lhutton1 commented on a change in pull request #5915: [BYOC][Contrib] Arm Compute Library integration
lhutton1 commented on a change in pull request #5915:
URL: https://github.com/apache/incubator-tvm/pull/5915#discussion_r456312016
##
File path: src/runtime/contrib/arm_compute_lib/acl_runtime.cc
##
@@ -0,0 +1,310 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file src/runtime/contrib/arm_compute_lib/acl_runtime.cc
+ * \brief A simple JSON runtime for Arm Compute Library.
+ */
+
+#include
+#include
+
+#include "../../file_util.h"
+#include "../json/json_node.h"
+#include "../json/json_runtime.h"
+
+#ifdef TVM_GRAPH_RUNTIME_ARM_COMPUTE_LIB
+#include
+#include
+#include
+#include
+
+#include "acl_allocator.h"
+#include "acl_utils.h"
+#endif
+
+namespace tvm {
+namespace runtime {
+namespace contrib {
+
+using namespace tvm::runtime::json;
+
+#ifdef TVM_GRAPH_RUNTIME_ARM_COMPUTE_LIB
+using namespace arm_compute_lib;
+#endif
Review comment:
The utils functions and allocator are wrapped in another namespace
`arm_compute_lib`, using this namespace here avoids writing `arm_compute_lib`
when we want to use these functions. Probably best to remove the namespace
altogether.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [incubator-tvm] lhutton1 commented on a change in pull request #5915: [BYOC][Contrib] Arm Compute Library integration
lhutton1 commented on a change in pull request #5915:
URL: https://github.com/apache/incubator-tvm/pull/5915#discussion_r456310867
##
File path: src/relay/backend/contrib/arm_compute_lib/codegen.cc
##
@@ -0,0 +1,186 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file src/relay/backend/contrib/arm_compute_lib/codegen_acl.cc
+ * \brief Implementation of the Relay -> ACL JSON serializer.
+ */
+#include
+#include
+#include
+
+#include "../../utils.h"
+#include "codegen_acl.h"
+
+namespace tvm {
+namespace relay {
+namespace contrib {
+namespace arm_compute_lib {
+
+using JSONGraphNode = tvm::runtime::json::JSONGraphNode;
+using JSONGraphNodeEntry = tvm::runtime::json::JSONGraphNodeEntry;
+
+std::vector ACLJSONSerializer::VisitExpr_(const CallNode*
cn) {
+ Expr expr = GetRef(cn);
+ std::string name;
+ std::shared_ptr json_node;
+
+ if (cn->op.as()) {
+json_node = CreateOpJSONNode(cn);
+ } else if (const auto* fn = cn->op.as()) {
+auto comp = fn->GetAttr(attr::kComposite);
+CHECK(comp.defined()) << "Arm Compute Library JSON runtime only supports
composite functions.";
+name = comp.value();
+if (name == "arm_compute_lib.conv2d") {
+ json_node = CreateCompositeConvJSONNode(cn);
+} else {
+ LOG(FATAL) << "Unrecognized Arm Compute Library pattern: " << name;
+}
+ } else {
+LOG(FATAL) << "Arm Compute Library JSON runtime does not support calls to "
+ << cn->op->GetTypeKey();
+ }
+
+ return AddNode(json_node, GetRef(cn));
+}
+
+std::shared_ptr ACLJSONSerializer::CreateOpJSONNode(const
CallNode* cn) {
+ const auto* op = cn->op.as();
+ CHECK(op);
+ const std::string name = op->name;
+ // Collect inputs
+ std::vector inputs;
+ for (const auto& arg : cn->args) {
+auto res = VisitExpr(arg);
+inputs.insert(inputs.end(), res.begin(), res.end());
+ }
+ // Create JSON op
+ auto json_node = std::make_shared(name, "kernel", inputs, 1);
+ SetCallNodeAttribute(json_node, cn);
+ return json_node;
+}
+
+std::shared_ptr
ACLJSONSerializer::CreateCompositeConvJSONNode(const CallNode* cn) {
+ const std::string name = "arm_compute_lib.conv2d";
+ const CallNode* pad = nullptr;
+ const CallNode* conv;
+ const CallNode* bias = nullptr;
+ bool has_activation = false;
+
+ // Unpack composite function
+ const auto* fn = cn->op.as();
+ CHECK(fn);
+ const auto* current_call = fn->body.as();
+ if (backend::IsOp(current_call, "nn.relu")) {
+has_activation = true;
+current_call = current_call->args[0].as();
+ }
+ if (backend::IsOp(current_call, "nn.bias_add")) {
+bias = current_call;
+current_call = current_call->args[0].as();
+ }
+ CHECK(backend::IsOp(current_call, "nn.conv2d"));
+ conv = current_call;
+ if (!current_call->args.empty() &&
current_call->args[0]->IsInstance()) {
+current_call = current_call->args[0].as();
+if (backend::IsOp(current_call, "nn.pad")) {
+ pad = current_call;
+}
+ }
+
+ const auto* conv_attr = conv->attrs.as();
+ CHECK(conv_attr);
+ CHECK(conv_attr->kernel_layout == "OHWI")
+ << "Kernel layout must be OHWI, has the module been pre-processed
correctly?";
+
+ std::vector inputs;
+ inputs.push_back(VisitExpr(cn->args[0])[0]);
+ inputs.push_back(VisitExpr(conv->args[1])[0]);
+ if (bias) {
+inputs.push_back(VisitExpr(bias->args[1])[0]);
+ }
+
+ auto json_node = std::make_shared(name, "kernel", inputs, 1);
+ SetCallNodeAttribute(json_node, conv);
+
+ // Override attributes
+ if (pad) {
+const auto* pad_attr = pad->attrs.as();
+CHECK(pad_attr);
+auto p = pad_attr->pad_width;
+// Convert to TVM layout for now, conversion to ACL layout takes place in
runtime.
+// Standard convolution pad layout for TVM: top, left, bottom, right.
+std::vector padding =
{std::to_string(p[1][0].as()->value),
+
std::to_string(p[2][0].as()->value),
+
std::to_string(p[1][1].as()->value),
+
std::to_string(p[2][1].as()->value)};
+std::vector padding_attr;
+padding_attr.emplace_back(padding);
+json_node->SetAttr("padding", padding_attr);
+
[GitHub] [incubator-tvm] mbaret commented on a change in pull request #5915: [BYOC][Contrib] Arm Compute Library integration
mbaret commented on a change in pull request #5915:
URL: https://github.com/apache/incubator-tvm/pull/5915#discussion_r456115777
##
File path: python/tvm/relay/op/contrib/arm_compute_lib.py
##
@@ -0,0 +1,122 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+# pylint: disable=invalid-name, unused-argument
+"""Arm Compute Library supported operators."""
+import tvm
+from tvm.relay import transform
+from tvm.relay.build_module import bind_params_by_name
+
+from ...dataflow_pattern import wildcard, is_op, is_constant
+from .register import register_pattern_table
+
+
+def is_arm_compute_runtime_enabled():
+"""Check if the ACL graph runtime is present.
+
+Returns
+---
+ret: bool
+True if present, False if not.
+"""
+check_enabled =
tvm.get_global_func("relay.op.is_arm_compute_runtime_enabled", True)
+if check_enabled:
+return check_enabled()
+return False
+
+
+def partition_for_arm_compute_lib(mod, params=None):
+"""Partition the graph greedily offloading supported
+operators to Arm Compute Library.
+
+Parameters
+--
+mod : Module
+The module to run passes on.
+params : Optional[Dict[str, NDArray]]
+Constant input parameters.
+
+Returns
+---
+ret : annotated and partitioned module.
+"""
+if params:
+mod['main'] = bind_params_by_name(mod['main'], params)
+
+seq =
tvm.transform.Sequential([transform.MergeComposite(arm_compute_lib_pattern_table()),
+
transform.AnnotateTarget('arm_compute_lib'),
+transform.PartitionGraph()])
+
+return seq(mod)
+
+
+@register_pattern_table("arm_compute_lib")
+def arm_compute_lib_pattern_table():
+"""Get the ACL pattern table."""
+
+def conv_pattern():
+"""Create a convolution pattern.
+
+Returns
+---
+pattern : dataflow_pattern.AltPattern
+Denotes the convolution pattern.
+"""
+pattern = is_op('nn.pad')(wildcard()) | wildcard()
+pattern = is_op('nn.conv2d')(pattern, is_constant())
+pattern = pattern.optional(lambda x: is_op('nn.bias_add')(x,
is_constant()))
+pattern = pattern.optional(is_op('nn.relu'))
+return pattern
+
+def check_conv(extract):
+"""Check conv pattern is supported by ACL."""
+call = extract
+while call.op.name != "nn.conv2d":
+call = call.args[0]
+return conv2d(call.attrs, call.args)
+
+return [('arm_compute_lib.conv2d', conv_pattern(), check_conv)]
+
+
+def _register_external_op_helper(op_name, supported=True):
+@tvm.ir.register_op_attr(op_name, "target.arm_compute_lib")
+def _func_wrapper(attrs, args):
+return supported
+
+return _func_wrapper
+
+
+_register_external_op_helper("reshape")
+
+
+@tvm.ir.register_op_attr("nn.conv2d", "target.arm_compute_lib")
+def conv2d(attrs, args):
+"""Check if the external ACL codegen for conv2d should be used."""
+if attrs.groups != 1:
+return False
+if attrs.data_layout != "NHWC":
+return False
Review comment:
Pedantically you may want to check for data type here as well
(presumably = float32).
##
File path: src/relay/backend/contrib/arm_compute_lib/codegen.cc
##
@@ -0,0 +1,186 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file
[GitHub] [incubator-tvm-vta] remotego commented on a change in pull request #9: [Hardware][OpenCL] Intelfocl support
remotego commented on a change in pull request #9:
URL: https://github.com/apache/incubator-tvm-vta/pull/9#discussion_r456267226
##
File path: src/intelfocl/intelfocl_device.cc
##
@@ -0,0 +1,185 @@
+#include
+#include
+#include
+#include "intelfocl_device.h"
+#include "aoclutils/aocl_utils.h"
+
+#define CL_STATUS_SUCCESS(x) ((x) == CL_SUCCESS)
+
+void cleanup() {}
+
+int IntelFOCLDevice::init(size_t mem_size, std::string aocx_file)
+{
+cl_int status;
Review comment:
Sure. Thanks. We will update that.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [incubator-tvm-vta] remotego commented on a change in pull request #9: [Hardware][OpenCL] Intelfocl support
remotego commented on a change in pull request #9:
URL: https://github.com/apache/incubator-tvm-vta/pull/9#discussion_r456264532
##
File path: config/intelfocl_sample.json
##
@@ -0,0 +1,13 @@
+{
+ "TARGET" : "intelfocl",
+ "HW_VER" : "0.0.1",
Review comment:
Agreed. Let bump it up to 0.0.2
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [incubator-tvm-vta] remotego commented on a change in pull request #9: [Hardware][OpenCL] Intelfocl support
remotego commented on a change in pull request #9:
URL: https://github.com/apache/incubator-tvm-vta/pull/9#discussion_r456264177
##
File path: src/intelfocl/intelfocl_device.cc
##
@@ -0,0 +1,185 @@
+#include
+#include
+#include
+#include "intelfocl_device.h"
+#include "aoclutils/aocl_utils.h"
+
+#define CL_STATUS_SUCCESS(x) ((x) == CL_SUCCESS)
+
+void cleanup() {}
Review comment:
This cleanup() function is a callback function from 3rdparty code
aoclutils, it gives the user a chance to perform cleanups before calling
exit(), if required.
Here our code does not require any special cleanup before exit(), thus the
cleanup() was left blank by intention.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [incubator-tvm] hzfan commented on a change in pull request #5235: [RELAY][Fix] i64 indices
hzfan commented on a change in pull request #5235:
URL: https://github.com/apache/incubator-tvm/pull/5235#discussion_r456261216
##
File path: tests/python/relay/test_pass_fuse_ops.py
##
@@ -621,6 +621,81 @@ def expected():
after = run_opt_pass(expected(), transform.InferType())
assert tvm.ir.structural_equal(zz, after)
+
+def test_fuse_take():
+"""Test fusion case involving concat and take"""
+
+def before():
+shape = (tvm.tir.const(10, "int64"),
+ tvm.tir.const(1, "int64"))
+x = relay.var("x", shape=shape)
+concat = relay.concatenate([x,x], axis=-1)
+out = relay.op.take(concat, indices=relay.const([0], dtype="int64"))
+return relay.Function(relay.analysis.free_vars(out), out)
+
+def expected():
+shape1 = (tvm.tir.const(10, "int64"),
+ tvm.tir.const(1, "int64"))
+shape2 = (tvm.tir.const(1, "int64"),)
+x = relay.var("x", shape=shape1)
+p0 = relay.var("p0", shape=shape1)
+p1 = relay.var("p1", shape=shape2,
+ dtype="int64")
+c = relay.const([0], dtype="int64")
+concat = relay.concatenate([p0,p0], axis=-1)
+out = relay.op.take(concat, indices=p1)
+
+f0 = relay.Function([p0, p1], out)
+f0 = f0.with_attr("Primitive", tvm.tir.IntImm("int32", 1))
+
+y = relay.Call(f0, [x, c])
+return relay.Function([x], y)
+
+orig = before()
+fuse0(tvm.IRModule.from_expr(orig))
+m = fuse2(tvm.IRModule.from_expr(orig))
+relay.build(m, 'llvm')
+after = run_opt_pass(expected(), transform.InferType())
+assert tvm.ir.structural_equal(m["main"], after)
+
+
+def test_fuse_gather_nd():
+"""Test fusion case involving concat and gather_nd"""
+
+def before():
+shape = (tvm.tir.const(10, "int64"),
+ tvm.tir.const(1, "int64"))
+x = relay.var("x", shape=shape)
+concat = relay.concatenate([x,x], axis=-1)
+out = relay.gather_nd(concat, indices=relay.expr.const([[0,1],[1,0]],
dtype="int64"))
+return relay.Function(relay.analysis.free_vars(out), out)
+
+def expected():
+shape1 = (tvm.tir.const(10, "int64"),
+ tvm.tir.const(1, "int64"))
+shape2 = (tvm.tir.const(2, "int64"),
+ tvm.tir.const(2, "int64"))
+x = relay.var("x", shape=shape1)
+p0 = relay.var("p0", shape=shape1)
+p1 = relay.var("p1", shape=shape2, dtype="int64")
+c = relay.const([[0,1],[1,0]], dtype="int64")
+concat = relay.concatenate([p0,p0], axis=-1)
+out = relay.gather_nd(concat, indices=p1)
+
+f0 = relay.Function([p0, p1], out)
+f0 = f0.with_attr("Primitive", tvm.tir.IntImm("int32", 1))
+
+y = relay.Call(f0, [x, c])
+return relay.Function([x], y)
+
+orig = before()
+fuse0(tvm.IRModule.from_expr(orig))
Review comment:
Removed.
##
File path: tests/python/relay/test_pass_fuse_ops.py
##
@@ -621,6 +621,81 @@ def expected():
after = run_opt_pass(expected(), transform.InferType())
assert tvm.ir.structural_equal(zz, after)
+
+def test_fuse_take():
+"""Test fusion case involving concat and take"""
+
+def before():
+shape = (tvm.tir.const(10, "int64"),
+ tvm.tir.const(1, "int64"))
+x = relay.var("x", shape=shape)
+concat = relay.concatenate([x,x], axis=-1)
+out = relay.op.take(concat, indices=relay.const([0], dtype="int64"))
+return relay.Function(relay.analysis.free_vars(out), out)
+
+def expected():
+shape1 = (tvm.tir.const(10, "int64"),
+ tvm.tir.const(1, "int64"))
+shape2 = (tvm.tir.const(1, "int64"),)
+x = relay.var("x", shape=shape1)
+p0 = relay.var("p0", shape=shape1)
+p1 = relay.var("p1", shape=shape2,
+ dtype="int64")
+c = relay.const([0], dtype="int64")
+concat = relay.concatenate([p0,p0], axis=-1)
+out = relay.op.take(concat, indices=p1)
+
+f0 = relay.Function([p0, p1], out)
+f0 = f0.with_attr("Primitive", tvm.tir.IntImm("int32", 1))
+
+y = relay.Call(f0, [x, c])
+return relay.Function([x], y)
+
+orig = before()
+fuse0(tvm.IRModule.from_expr(orig))
Review comment:
Removed. Thanks @kazum .
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [incubator-tvm-vta] remotego commented on a change in pull request #9: [Hardware][OpenCL] Intelfocl support
remotego commented on a change in pull request #9:
URL: https://github.com/apache/incubator-tvm-vta/pull/9#discussion_r456259647
##
File path: config/intelfocl_sample.json
##
@@ -0,0 +1,13 @@
+{
+ "TARGET" : "intelfocl",
+ "HW_VER" : "0.0.1",
+ "LOG_INP_WIDTH" : 3,
Review comment:
Yes. LOG_INP_WIDTH and LOG_WGT_WIDTH are fixed at 3 in the current
version.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [incubator-tvm-vta] remotego commented on a change in pull request #9: [Hardware][OpenCL] Intelfocl support
remotego commented on a change in pull request #9: URL: https://github.com/apache/incubator-tvm-vta/pull/9#discussion_r456257771 ## File path: hardware/intelfocl/src/vta.cl ## @@ -0,0 +1,341 @@ +#pragma OPENCL EXTENSION cl_intel_channels: enable Review comment: The code that work for ultra96 is still inside hardware/xilinx This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-tvm] junrushao1994 commented on a change in pull request #6073: [Ansor][AutoTVM v2.0] Phase 1: Add annotation/compute_at/compute_root/compute_inline steps
junrushao1994 commented on a change in pull request #6073:
URL: https://github.com/apache/incubator-tvm/pull/6073#discussion_r456257744
##
File path: src/auto_scheduler/loop_state.h
##
@@ -217,6 +196,68 @@ class Stage : public ObjectRef {
TVM_DEFINE_OBJECT_REF_COW_METHOD(StageNode);
};
+/*! \brief Use stage_id to represent a stage. */
+using StageKey = int;
+/*! \brief Use stage_id and iter_id to represent a iterator. */
+using IterKey = std::pair;
+
+/*!
+ * \brief stores the compute_at relation between stages
+ * This stores a bi-directional mapping from stages and iter:
+ * 1. Stage to its attached iterator
+ * 2. Iterator to the stage attached to it
+ * You can use AttachMapNode::stage_to_attach_iter and
AttachMapNode::iter_to_attached_stages
+ * to query the relations
+ */
+class AttachMapNode : public Object {
+ public:
+ /*! \brief A Map to store the mapping of stage to its attached iterator. */
+ std::unordered_map stage_to_attach_iter;
+ /*! \brief A Map to store the mapping of iterator to the stage attached to
it. */
+ std::unordered_map> iter_to_attached_stages;
+
+ static constexpr const char* _type_key = "auto_scheduler.AttachMap";
+ TVM_DECLARE_FINAL_OBJECT_INFO(AttachMapNode, Object);
+};
+
+/*!
+ * \brief Managed reference to AttachMapNode.
+ * \sa AttachMapNode
+ */
+class AttachMap : public ObjectRef {
+ public:
+ /*!
+ * \brief Process the stage/iterator mapping after compute at.
+ * \param stage_id The index of the stage to be compute at.
+ * \param target_stage_id The index of stage that this step will compute at
to.
+ * \param target_iter_id The index of iterator in target stage that this
step will compute at to.
+ */
+ void SetComputeAtIter(int stage_id, int target_stage_id, int target_iter_id);
+ /*!
+ * \brief This is a public wrapper of `DeleteStageEntry`. To delete the
entry of a specific stage.
+ * \param stage_id The index of the stage to be compute at.
+ */
+ void DeleteStage(int stage_id);
+ /*!
+ * \brief Update the iterator relations in AttachMap.
Review comment:
I agree with Cody...Tried to understand the doc for a while but didn't
succeed without looking at the source code. That will be great if we have
clearer doc and method name :-)
##
File path: src/auto_scheduler/utils.h
##
@@ -89,6 +89,15 @@ inline int GetIndex(const Array& array, const T&
to_locate) {
return -1;
}
+/*! \brief Delete the item in a std::vector. */
+template
+inline void DeleteItem(std::vector* array, const T& to_delete) {
Review comment:
Maybe `FindAndDeleteItem` sounds more informative?
##
File path: src/auto_scheduler/loop_state.cc
##
@@ -90,12 +90,69 @@ Stage::Stage(te::Operation op, StageKind op_type, const
Array& iters,
data_ = std::move(node);
}
+/** AttachMap **/
+void AttachMap::SetComputeAtIter(int stage_id, int target_stage_id, int
target_iter_id) {
+ AttachMapNode* pnode = CopyOnWrite();
+
+ // Delete the current entry of this stage
+ DeleteStageEntry(pnode, stage_id);
+
+ // Store the new relations to map
+ IterKey iter_key(target_stage_id, target_iter_id);
+ pnode->stage_to_attach_iter[stage_id] = iter_key;
+ pnode->iter_to_attached_stages[iter_key].push_back(stage_id);
+}
+
+void AttachMap::DeleteStage(int stage_id) {
+ AttachMapNode* pnode = CopyOnWrite();
+ // Delete the original stage entry
+ DeleteStageEntry(pnode, stage_id);
+}
+
+void AttachMap::UpdateIters(const std::vector& old_iters,
+const std::vector& new_iters) {
Review comment:
Shall we move the check to the beginning of this function before CoW?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [incubator-tvm] hzfan commented on a change in pull request #5235: [RELAY][Fix] i64 indices
hzfan commented on a change in pull request #5235:
URL: https://github.com/apache/incubator-tvm/pull/5235#discussion_r456257305
##
File path: tests/python/relay/test_pass_fuse_ops.py
##
@@ -621,6 +621,81 @@ def expected():
after = run_opt_pass(expected(), transform.InferType())
assert tvm.ir.structural_equal(zz, after)
+
+def test_fuse_take():
+"""Test fusion case involving concat and take"""
+
+def before():
+shape = (tvm.tir.const(10, "int64"),
+ tvm.tir.const(1, "int64"))
+x = relay.var("x", shape=shape)
+concat = relay.concatenate([x,x], axis=-1)
+out = relay.op.take(concat, indices=relay.const([0], dtype="int64"))
+return relay.Function(relay.analysis.free_vars(out), out)
+
+def expected():
+shape1 = (tvm.tir.const(10, "int64"),
+ tvm.tir.const(1, "int64"))
+shape2 = (tvm.tir.const(1, "int64"),)
+x = relay.var("x", shape=shape1)
+p0 = relay.var("p0", shape=shape1)
+p1 = relay.var("p1", shape=shape2,
+ dtype="int64")
+c = relay.const([0], dtype="int64")
+concat = relay.concatenate([p0,p0], axis=-1)
+out = relay.op.take(concat, indices=p1)
+
+f0 = relay.Function([p0, p1], out)
+f0 = f0.with_attr("Primitive", tvm.tir.IntImm("int32", 1))
+
+y = relay.Call(f0, [x, c])
+return relay.Function([x], y)
+
+orig = before()
+fuse0(tvm.IRModule.from_expr(orig))
+m = fuse2(tvm.IRModule.from_expr(orig))
+relay.build(m, 'llvm')
+after = run_opt_pass(expected(), transform.InferType())
+assert tvm.ir.structural_equal(m["main"], after)
+
+
+def test_fuse_gather_nd():
+"""Test fusion case involving concat and gather_nd"""
+
+def before():
+shape = (tvm.tir.const(10, "int64"),
+ tvm.tir.const(1, "int64"))
+x = relay.var("x", shape=shape)
+concat = relay.concatenate([x,x], axis=-1)
+out = relay.gather_nd(concat, indices=relay.expr.const([[0,1],[1,0]],
dtype="int64"))
+return relay.Function(relay.analysis.free_vars(out), out)
+
+def expected():
+shape1 = (tvm.tir.const(10, "int64"),
+ tvm.tir.const(1, "int64"))
+shape2 = (tvm.tir.const(2, "int64"),
+ tvm.tir.const(2, "int64"))
+x = relay.var("x", shape=shape1)
+p0 = relay.var("p0", shape=shape1)
+p1 = relay.var("p1", shape=shape2, dtype="int64")
+c = relay.const([[0,1],[1,0]], dtype="int64")
+concat = relay.concatenate([p0,p0], axis=-1)
+out = relay.gather_nd(concat, indices=p1)
+
+f0 = relay.Function([p0, p1], out)
+f0 = f0.with_attr("Primitive", tvm.tir.IntImm("int32", 1))
Review comment:
I explore a bit, and find that the data type matters here.
Here `f0` is a manually fused function. It is compared with the fusion pass
to verify its result. Because the result generated by fusion pass uses `int32`
for `Primitive`, using `int64` or `bool` here results in an assertion failure,
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [incubator-tvm] comaniac commented on a change in pull request #6073: [Ansor][AutoTVM v2.0] Phase 1: Add annotation/compute_at/compute_root/compute_inline steps
comaniac commented on a change in pull request #6073:
URL: https://github.com/apache/incubator-tvm/pull/6073#discussion_r456255907
##
File path: src/auto_scheduler/loop_state.cc
##
@@ -90,12 +90,69 @@ Stage::Stage(te::Operation op, StageKind op_type, const
Array& iters,
data_ = std::move(node);
}
+/** AttachMap **/
+void AttachMap::SetComputeAtIter(int stage_id, int target_stage_id, int
target_iter_id) {
+ AttachMapNode* pnode = CopyOnWrite();
+
+ // Delete the current entry of this stage
+ DeleteStageEntry(pnode, stage_id);
+
+ // Store the new relations to map
+ IterKey iter_key(target_stage_id, target_iter_id);
+ pnode->stage_to_attach_iter[stage_id] = iter_key;
+ pnode->iter_to_attached_stages[iter_key].push_back(stage_id);
+}
+
+void AttachMap::DeleteStage(int stage_id) {
+ AttachMapNode* pnode = CopyOnWrite();
+ // Delete the original stage entry
+ DeleteStageEntry(pnode, stage_id);
+}
+
+void AttachMap::UpdateIters(const std::vector& old_iters,
+const std::vector& new_iters) {
Review comment:
Ah I found it. Sorry for missing that line.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [incubator-tvm] hzfan commented on a change in pull request #5235: [RELAY][Fix] i64 indices
hzfan commented on a change in pull request #5235:
URL: https://github.com/apache/incubator-tvm/pull/5235#discussion_r456252011
##
File path: src/te/schedule/operation_inline.cc
##
@@ -63,7 +63,9 @@ class OperationInliner final : public StmtExprMutator {
} else {
Map vmap;
for (size_t i = 0; i < args_.size(); ++i) {
- vmap.Set(args_[i], op->indices[i]);
+ // indices into `operation_` must be in the range of its output
shape,
+ // so we can safely cast the indices without worrying about overflow
Review comment:
You are right. The comment is a bit misleading. I think I can change it
to 'cast indices to the type of the original indexing variable'
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [incubator-tvm] jcf94 commented on a change in pull request #6073: [Ansor][AutoTVM v2.0] Phase 1: Add annotation/compute_at/compute_root/compute_inline steps
jcf94 commented on a change in pull request #6073:
URL: https://github.com/apache/incubator-tvm/pull/6073#discussion_r456249617
##
File path: src/auto_scheduler/loop_state.cc
##
@@ -90,12 +90,69 @@ Stage::Stage(te::Operation op, StageKind op_type, const
Array& iters,
data_ = std::move(node);
}
+/** AttachMap **/
+void AttachMap::SetComputeAtIter(int stage_id, int target_stage_id, int
target_iter_id) {
+ AttachMapNode* pnode = CopyOnWrite();
+
+ // Delete the current entry of this stage
+ DeleteStageEntry(pnode, stage_id);
+
+ // Store the new relations to map
+ IterKey iter_key(target_stage_id, target_iter_id);
+ pnode->stage_to_attach_iter[stage_id] = iter_key;
+ pnode->iter_to_attached_stages[iter_key].push_back(stage_id);
+}
+
+void AttachMap::DeleteStage(int stage_id) {
+ AttachMapNode* pnode = CopyOnWrite();
+ // Delete the original stage entry
+ DeleteStageEntry(pnode, stage_id);
+}
+
+void AttachMap::UpdateIters(const std::vector& old_iters,
+const std::vector& new_iters) {
Review comment:
We already have a vector size check in this function. :)
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [incubator-tvm] lixiaoquan opened a new pull request #6086: [Relay] Fix interpreter for dyanmic shape input of ndarray_size
lixiaoquan opened a new pull request #6086: URL: https://github.com/apache/incubator-tvm/pull/6086 @icemelon9 Could you please take a look? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
