[GitHub] [incubator-tvm] liangfu commented on a change in pull request #4703: [VTA] Support network which have no unique operator as start/stop name for graph pack.

[GitBox]

liangfu commented on a change in pull request #4703: [VTA] Support network 
which have no unique operator as start/stop name for graph pack.
URL: https://github.com/apache/incubator-tvm/pull/4703#discussion_r366724818
 
 

 ##
 File path: vta/python/vta/top/graphpack.py
 ##
 @@ -246,32 +246,42 @@ def visit_call(self, call):
 
 class BT(Exception):
 pass
-def get_subgraph(expr, start_name, stop_name):
+def get_subgraph(expr, start_name, stop_name, start_name_indx=0, 
stop_name_indx=0):
 """ We assume stop_name only appears once for simplicity.
 This constraint will be lifted in the future.
 bitpack_start and bitpack_end are both inclusive.
 """
 bitpack_start = op.op.get('annotation.bitpack_start')
 bitpack_end = op.op.get('annotation.bitpack_end')
 anf = run_opt_pass(expr, transform.ToANormalForm())
-def _recursion(anf, start_found, stop_found):
+start_name_num = 0
+stop_name_num = 0
+def _recursion(anf, start_found, stop_found, start_name_num, 
stop_name_num):
 
 Review comment:
   CHECK: when start_name==stopname, start index should be smaller than stop 
index.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-tvm] masahi commented on issue #4497: [WIP] [Relay] Add a PyTorch to Relay Parser

[GitBox]

masahi commented on issue #4497: [WIP] [Relay] Add a PyTorch to Relay Parser
URL: https://github.com/apache/incubator-tvm/pull/4497#issuecomment-574517425
 
 
   ok I found a dirty hack to remove prim::CallMethod. Add
   ```torch._C._jit_pass_inline(trace.graph)```
   after tracing. This will move operators hidden inside each prim::CallMethod 
into top level.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[incubator-tvm] branch master updated (81e03ee -> 4eecd2a)

[tqchen]

This is an automated email from the ASF dual-hosted git repository.

tqchen pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-tvm.git.


from 81e03ee  Revert "[Relay][TOPI]Fix meaning of conv2d_transpose 
output_padding parameter (#4318)" (#4708)
 add 4eecd2a  link the math library by default (#4713)

No new revisions were added by this update.

Summary of changes:
 python/tvm/contrib/ndk.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

[GitHub] [incubator-tvm] tqchen merged pull request #4713: link the math library by default

[GitBox]

tqchen merged pull request #4713: link the math library by default
URL: https://github.com/apache/incubator-tvm/pull/4713
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-tvm] FrozenGene commented on issue #4713: link the math library by default

[GitBox]

FrozenGene commented on issue #4713: link the math library by default
URL: https://github.com/apache/incubator-tvm/pull/4713#issuecomment-574487666
 
 
   @tqchen could you help to merge it next? Thanks.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-tvm] FrozenGene commented on issue #4713: link the math library by default

[GitBox]

FrozenGene commented on issue #4713: link the math library by default
URL: https://github.com/apache/incubator-tvm/pull/4713#issuecomment-574487438
 
 
   Thanks @zhigaowu !


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-tvm] zhigaowu opened a new pull request #4713: link the math library by default

[GitBox]

zhigaowu opened a new pull request #4713: link the math library by default
URL: https://github.com/apache/incubator-tvm/pull/4713
 
 
   
Https://discuss.tvm.ai/t/cannot-locate-symbol-expf-error-on-android-host-system-ubuntu/5387
 
   @FrozenGene link the math library by default


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-tvm] tqchen merged pull request #4708: Revert "[Relay][TOPI]Fix meaning of conv2d_transpose output_padding parameter"

[GitBox]

tqchen merged pull request #4708: Revert "[Relay][TOPI]Fix meaning of 
conv2d_transpose output_padding parameter"
URL: https://github.com/apache/incubator-tvm/pull/4708
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[incubator-tvm] branch master updated (7f7dc07 -> 81e03ee)

[tqchen]

This is an automated email from the ASF dual-hosted git repository.

tqchen pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-tvm.git.


from 7f7dc07  use packed func macro for external codegen (#4710)
 add 81e03ee  Revert "[Relay][TOPI]Fix meaning of conv2d_transpose 
output_padding parameter (#4318)" (#4708)

No new revisions were added by this update.

Summary of changes:
 python/tvm/autotvm/tophub.py   |  6 +++---
 python/tvm/relay/op/nn/_nn.py  | 12 +++
 tests/python/relay/test_op_level2.py   | 25 +++---
 topi/python/topi/arm_cpu/conv2d_transpose.py   | 21 ++
 topi/python/topi/cuda/conv1d_transpose_ncw.py  |  7 +++---
 topi/python/topi/cuda/conv2d_transpose_nchw.py | 12 +--
 topi/python/topi/nn/conv1d_transpose.py|  6 ++
 topi/python/topi/nn/conv2d_transpose.py| 24 +
 .../topi/testing/conv1d_transpose_ncw_python.py|  7 +++---
 .../python/topi/testing/conv2d_transpose_python.py | 17 ++-
 topi/python/topi/x86/conv2d_transpose.py   |  4 ++--
 .../tests/python/test_topi_conv1d_transpose_ncw.py |  2 +-
 .../python/test_topi_conv2d_transpose_nchw.py  | 18 +++-
 vta/python/vta/top/vta_conv2d_transpose.py | 16 ++
 vta/scripts/tune_conv2d_transpose.py   | 19 ++--
 .../test_benchmark_topi_conv2d_transpose.py| 16 ++
 16 files changed, 92 insertions(+), 120 deletions(-)

[GitHub] [incubator-tvm] zhigaowu closed pull request #4712: V0.6

[GitBox]

zhigaowu closed pull request #4712: V0.6
URL: https://github.com/apache/incubator-tvm/pull/4712
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-tvm] zhigaowu opened a new pull request #4712: V0.6

[GitBox]

zhigaowu opened a new pull request #4712: V0.6
URL: https://github.com/apache/incubator-tvm/pull/4712
 
 
   
Https://discuss.tvm.ai/t/cannot-locate-symbol-expf-error-on-android-host-system-ubuntu/5387
 
   @FrozenGene link the math library by default


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[incubator-tvm] branch master updated (ce807fe -> 7f7dc07)

[liuyizhi]

This is an automated email from the ASF dual-hosted git repository.

liuyizhi pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-tvm.git.


from ce807fe  [REFACTOR][IR] Unify IntImm and UIntImm (#4706)
 add 7f7dc07  use packed func macro for external codegen (#4710)

No new revisions were added by this update.

Summary of changes:
 python/tvm/_ffi/libinfo.py  |  11 ++-
 src/relay/backend/contrib/codegen_c/codegen.cc  |   5 +-
 src/relay/backend/contrib/codegen_c/codegen_c.h |  57 +---
 src/relay/backend/contrib/dnnl/codegen.cc   |   1 +
 tests/python/relay/test_external_runtime.py | 111 ++--
 5 files changed, 81 insertions(+), 104 deletions(-)

[GitHub] [incubator-tvm] yzhliu merged pull request #4710: [relay][external_codgen] use packed func macro for external codegen

[GitBox]

yzhliu merged pull request #4710: [relay][external_codgen] use packed func 
macro for external codegen
URL: https://github.com/apache/incubator-tvm/pull/4710
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-tvm] liaha opened a new pull request #4711: [Relay][Frontend][TF] fix _parse_param bug

[GitBox]

liaha opened a new pull request #4711: [Relay][Frontend][TF] fix _parse_param 
bug
URL: https://github.com/apache/incubator-tvm/pull/4711
 
 
   * fix parsing bug that DT_STRING Const name might not be defined in shape 
dictionary specified in from_tensorflow parameter
   * update test case


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[incubator-tvm] branch master updated (f4c4fde -> ce807fe)

[tqchen]

This is an automated email from the ASF dual-hosted git repository.

tqchen pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-tvm.git.


from f4c4fde  [REFACTOR][IR] Polish ir/type (#4705)
 add ce807fe  [REFACTOR][IR] Unify IntImm and UIntImm (#4706)

No new revisions were added by this update.

Summary of changes:
 include/tvm/attrs.h|  4 --
 include/tvm/expr.h | 48 ++--
 include/tvm/expr_operator.h| 53 +++---
 include/tvm/ir.h   | 27 +---
 include/tvm/ir/expr.h  | 50 +
 include/tvm/ir_functor_ext.h   |  4 --
 python/tvm/api.py  |  3 ++
 python/tvm/autotvm/task/task.py|  4 +-
 python/tvm/autotvm/util.py |  8 ++--
 python/tvm/expr.py | 17 ---
 python/tvm/hybrid/calls.py |  2 +-
 python/tvm/hybrid/parser.py|  4 +-
 python/tvm/hybrid/util.py  |  2 +-
 python/tvm/relay/frontend/tensorflow.py|  2 +-
 src/api/api_ir.cc  |  2 -
 src/api/api_lang.cc|  3 ++
 src/arithmetic/analyzer.cc |  6 +--
 src/arithmetic/canonical_simplify.cc   |  8 ++--
 src/arithmetic/const_fold.h| 61 --
 src/arithmetic/const_int_bound.cc  | 10 +
 src/arithmetic/int_set.cc  |  6 +--
 src/arithmetic/modular_set.cc  | 10 +
 src/arithmetic/pattern_match.h |  6 +--
 src/arithmetic/rewrite_simplify.cc | 26 +--
 src/autotvm/touch_extractor.cc |  2 +-
 src/codegen/codegen_c.cc   | 21 +
 src/codegen/codegen_c.h|  1 -
 src/codegen/codegen_opengl.cc  |  5 ---
 src/codegen/codegen_opengl.h   |  1 -
 src/codegen/llvm/codegen_arm.cc| 22 +-
 src/codegen/llvm/codegen_llvm.cc   | 18 
 src/codegen/llvm/codegen_llvm.h|  1 -
 src/codegen/llvm/codegen_x86_64.cc |  4 +-
 src/codegen/llvm/intrin_rule_llvm.h|  8 ++--
 src/codegen/spirv/codegen_spirv.cc | 13 +++---
 src/codegen/spirv/codegen_spirv.h  |  1 -
 src/codegen/spirv/intrin_rule_spirv.cc |  2 +-
 src/codegen/spirv/ir_builder.cc|  4 +-
 src/codegen/spirv/ir_builder.h |  4 +-
 src/codegen/stackvm/codegen_stackvm.cc |  6 ---
 src/codegen/stackvm/codegen_stackvm.h  |  1 -
 src/contrib/hybrid/codegen_hybrid.cc   |  5 +--
 src/contrib/hybrid/codegen_hybrid.h|  1 -
 src/ir/expr.cc | 19 
 src/lang/attr_functor.h|  4 --
 src/lang/attrs.cc  | 11 -
 src/lang/expr.cc   | 11 +
 src/lang/expr_operator.cc  | 55 ++-
 src/lang/ir.cc | 16 +--
 src/pass/arg_binder.cc | 16 +++
 src/pass/ir_deep_compare.cc|  4 --
 src/pass/ir_functor.cc |  2 -
 src/pass/lift_attr_scope.cc|  3 --
 src/pass/lower_intrin.cc   |  2 +-
 src/pass/lower_thread_allreduce.cc |  2 +-
 src/pass/lower_tvm_builtin.cc  |  4 +-
 src/pass/make_api.cc   |  6 +--
 src/pass/rewrite_unsafe_select.cc  |  1 -
 src/pass/tensor_core.cc| 14 +++---
 src/pass/unroll_loop.cc|  4 --
 src/relay/backend/compile_engine.cc|  8 ++--
 src/relay/ir/expr.cc   |  2 +-
 src/relay/ir/pretty_printer.cc |  4 --
 src/relay/op/tensor/transform.cc   |  2 +-
 src/relay/pass/type_solver.cc  |  2 +-
 src/relay/qnn/util.h   | 12 +
 tests/cpp/pattern_match_test.cc|  4 +-
 tests/python/unittest/test_codegen_device.py   | 27 
 tests/python/unittest/test_codegen_llvm.py | 20 +
 tests/python/unittest/test_hybrid_script.py|  2 +-
 tests/python/unittest/test_lang_constructor.py |  7 +--
 tests/python/unittest/test_lang_operator.py|  2 +-
 topi/include/topi/detail/constant_utils.h  | 10 ++---
 topi/python/topi/util.py   | 12 ++---
 74 files changed, 361 insertions(+), 413 deletions(-)

[GitHub] [incubator-tvm] tqchen merged pull request #4706: [REFACTOR][IR] Unify IntImm and UIntImm

[GitBox]

tqchen merged pull request #4706: [REFACTOR][IR] Unify IntImm and UIntImm
URL: https://github.com/apache/incubator-tvm/pull/4706
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-tvm] zhiics commented on a change in pull request #4602: [Docs] Bring Your Own Codegen Guide -- Part 1

[GitBox]

zhiics commented on a change in pull request #4602: [Docs] Bring Your Own 
Codegen Guide -- Part 1
URL: https://github.com/apache/incubator-tvm/pull/4602#discussion_r35208
 
 

 ##
 File path: docs/dev/relay_bring_your_own_codegen.rst
 ##
 @@ -0,0 +1,529 @@
+..  Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+..http://www.apache.org/licenses/LICENSE-2.0
+
+..  Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied.  See the License for the
+specific language governing permissions and limitations
+under the License.
+
+=
+Bring Your Own Codegen To TVM
+=
+**Author**: `Zhi Chen `_, `Cody Hao Yu 
`_
+
+As the number of hardware devices targeted by deep learning workloads keeps 
increasing, the required knowledge for users to achieve high performance on 
various devices keeps increasing as well. To free data scientists from worrying 
about the performance when developing a new model, hardware vendors either 
provide libraries such as MKLDNN or cuDNN with many commonly used deep learning 
operators, or provide frameworks such as TensorRT to let users describe their 
models in a certain way to achieve high performance. However, users have to 
learn a new programming interface when they attempt to work on a new library or 
device. As a result, the demand for a unified programming interface becomes 
more and more important to 1) let all users and hardware vendors stand on the 
same page, and 2) provide a feasible solution to allow specialized hardware or 
library to only support widely used operators with extremely high performance, 
but fallback unsupported operators to general devices like CPU/GPU.
+
+In this developer guide, we demonstrate how you, as a hardware vendor, can 
easily implement your own codegen and register it as a Relay backend compiler 
to support your hardware device/library. This guide covers two types of codegen 
based on different graph representations you need:
+
+**1. You want to generate C code.**
+
+If your hardware already has a well-optimized C/C++ library, such as Intel 
CBLAS/MKL to CPU and NVIDIA CUBLAS to GPU, then this is what you are looking 
for. Fortunately, C source code module is fully compatible with TVM runtime 
module, which means the generated code could be compiled by any C/C++ compiler 
with proper compilation flags, so the only task you have is to implement a 
codegen that generates C code for subgraphs and a C source module to integrate 
into TVM runtime module. We will demonstrate how to implement a C code 
generator for your hardware in the following section.
 
 Review comment:
   see #4710 


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-tvm] zhiics commented on issue #4710: [relay][external_codgen] use packed func macro for external codegen

[GitBox]

zhiics commented on issue #4710: [relay][external_codgen] use packed func macro 
for external codegen
URL: https://github.com/apache/incubator-tvm/pull/4710#issuecomment-574461129
 
 
   cc @tqchen @comaniac 


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-tvm] zhiics opened a new pull request #4710: [relay][external_codgen] use packed func macro for external codegen

[GitBox]

zhiics opened a new pull request #4710: [relay][external_codgen] use packed 
func macro for external codegen
URL: https://github.com/apache/incubator-tvm/pull/4710
 
 
   #4602 Simplify the wrapper using `TVM_DLL_EXPORT_TYPED_FUNC` macro.
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-tvm] comaniac commented on a change in pull request #4602: [Docs] Bring Your Own Codegen Guide -- Part 1

[GitBox]

comaniac commented on a change in pull request #4602: [Docs] Bring Your Own 
Codegen Guide -- Part 1
URL: https://github.com/apache/incubator-tvm/pull/4602#discussion_r366648208
 
 

 ##
 File path: docs/dev/relay_bring_your_own_codegen.rst
 ##
 @@ -0,0 +1,529 @@
+..  Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+..http://www.apache.org/licenses/LICENSE-2.0
+
+..  Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied.  See the License for the
+specific language governing permissions and limitations
+under the License.
+
+=
+Bring Your Own Codegen To TVM
+=
+**Author**: `Zhi Chen `_, `Cody Hao Yu 
`_
+
+As the number of hardware devices targeted by deep learning workloads keeps 
increasing, the required knowledge for users to achieve high performance on 
various devices keeps increasing as well. To free data scientists from worrying 
about the performance when developing a new model, hardware vendors either 
provide libraries such as MKLDNN or cuDNN with many commonly used deep learning 
operators, or provide frameworks such as TensorRT to let users describe their 
models in a certain way to achieve high performance. However, users have to 
learn a new programming interface when they attempt to work on a new library or 
device. As a result, the demand for a unified programming interface becomes 
more and more important to 1) let all users and hardware vendors stand on the 
same page, and 2) provide a feasible solution to allow specialized hardware or 
library to only support widely used operators with extremely high performance, 
but fallback unsupported operators to general devices like CPU/GPU.
+
+In this developer guide, we demonstrate how you, as a hardware vendor, can 
easily implement your own codegen and register it as a Relay backend compiler 
to support your hardware device/library. This guide covers two types of codegen 
based on different graph representations you need:
+
+**1. You want to generate C code.**
+
+If your hardware already has a well-optimized C/C++ library, such as Intel 
CBLAS/MKL to CPU and NVIDIA CUBLAS to GPU, then this is what you are looking 
for. Fortunately, C source code module is fully compatible with TVM runtime 
module, which means the generated code could be compiled by any C/C++ compiler 
with proper compilation flags, so the only task you have is to implement a 
codegen that generates C code for subgraphs and a C source module to integrate 
into TVM runtime module. We will demonstrate how to implement a C code 
generator for your hardware in the following section.
 
 Review comment:
   OK we will first file a PR to include dmlc path and then come back to modify 
the codegen accordingly.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-tvm] tqchen opened a new pull request #4709: [REFACTOR][IR] attrs.h -> ir

[GitBox]

tqchen opened a new pull request #4709: [REFACTOR][IR] attrs.h -> ir
URL: https://github.com/apache/incubator-tvm/pull/4709
 
 
   This PR moves attrs.h into the ir folder as it
   can serve as a common infra for building ir dats structures.
   
   We also moved common container(FloatImm) into ir/expr.h
   
   depends on #4706 


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-tvm] tqchen commented on issue #4709: [REFACTOR][IR] attrs.h -> ir

[GitBox]

tqchen commented on issue #4709: [REFACTOR][IR] attrs.h -> ir
URL: https://github.com/apache/incubator-tvm/pull/4709#issuecomment-574436966
 
 
   cc @yzhliu @zhiics @ZihengJiang 


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-tvm] tqchen commented on a change in pull request #4602: [Docs] Bring Your Own Codegen Guide -- Part 1

[GitBox]

tqchen commented on a change in pull request #4602: [Docs] Bring Your Own 
Codegen Guide -- Part 1
URL: https://github.com/apache/incubator-tvm/pull/4602#discussion_r366640373
 
 

 ##
 File path: docs/dev/relay_bring_your_own_codegen.rst
 ##
 @@ -0,0 +1,529 @@
+..  Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+..http://www.apache.org/licenses/LICENSE-2.0
+
+..  Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied.  See the License for the
+specific language governing permissions and limitations
+under the License.
+
+=
+Bring Your Own Codegen To TVM
+=
+**Author**: `Zhi Chen `_, `Cody Hao Yu 
`_
+
+As the number of hardware devices targeted by deep learning workloads keeps 
increasing, the required knowledge for users to achieve high performance on 
various devices keeps increasing as well. To free data scientists from worrying 
about the performance when developing a new model, hardware vendors either 
provide libraries such as MKLDNN or cuDNN with many commonly used deep learning 
operators, or provide frameworks such as TensorRT to let users describe their 
models in a certain way to achieve high performance. However, users have to 
learn a new programming interface when they attempt to work on a new library or 
device. As a result, the demand for a unified programming interface becomes 
more and more important to 1) let all users and hardware vendors stand on the 
same page, and 2) provide a feasible solution to allow specialized hardware or 
library to only support widely used operators with extremely high performance, 
but fallback unsupported operators to general devices like CPU/GPU.
+
+In this developer guide, we demonstrate how you, as a hardware vendor, can 
easily implement your own codegen and register it as a Relay backend compiler 
to support your hardware device/library. This guide covers two types of codegen 
based on different graph representations you need:
+
+**1. You want to generate C code.**
+
+If your hardware already has a well-optimized C/C++ library, such as Intel 
CBLAS/MKL to CPU and NVIDIA CUBLAS to GPU, then this is what you are looking 
for. Fortunately, C source code module is fully compatible with TVM runtime 
module, which means the generated code could be compiled by any C/C++ compiler 
with proper compilation flags, so the only task you have is to implement a 
codegen that generates C code for subgraphs and a C source module to integrate 
into TVM runtime module. We will demonstrate how to implement a C code 
generator for your hardware in the following section.
 
 Review comment:
   The specific macro handles quite a few cases that might be useful in c 
codegen packages (e.g. handle Object) 
   
   Given that this is part of the TVM runtime, i feel it is OK to introduce the 
additional deps as long as they are deps of the tvm runtime


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-tvm] tqchen commented on a change in pull request #4602: [Docs] Bring Your Own Codegen Guide -- Part 1

[GitBox]

tqchen commented on a change in pull request #4602: [Docs] Bring Your Own 
Codegen Guide -- Part 1
URL: https://github.com/apache/incubator-tvm/pull/4602#discussion_r366639821
 
 

 ##
 File path: docs/dev/relay_bring_your_own_codegen.rst
 ##
 @@ -0,0 +1,529 @@
+..  Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+..http://www.apache.org/licenses/LICENSE-2.0
+
+..  Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied.  See the License for the
+specific language governing permissions and limitations
+under the License.
+
+=
+Bring Your Own Codegen To TVM
+=
+**Author**: `Zhi Chen `_, `Cody Hao Yu 
`_
+
+As the number of hardware devices targeted by deep learning workloads keeps 
increasing, the required knowledge for users to achieve high performance on 
various devices keeps increasing as well. To free data scientists from worrying 
about the performance when developing a new model, hardware vendors either 
provide libraries such as MKLDNN or cuDNN with many commonly used deep learning 
operators, or provide frameworks such as TensorRT to let users describe their 
models in a certain way to achieve high performance. However, users have to 
learn a new programming interface when they attempt to work on a new library or 
device. As a result, the demand for a unified programming interface becomes 
more and more important to 1) let all users and hardware vendors stand on the 
same page, and 2) provide a feasible solution to allow specialized hardware or 
library to only support widely used operators with extremely high performance, 
but fallback unsupported operators to general devices like CPU/GPU.
+
+In this developer guide, we demonstrate how you, as a hardware vendor, can 
easily implement your own codegen and register it as a Relay backend compiler 
to support your hardware device/library. This guide covers two types of codegen 
based on different graph representations you need:
+
+**1. You want to generate C code.**
+
+If your hardware already has a well-optimized C/C++ library, such as Intel 
CBLAS/MKL to CPU and NVIDIA CUBLAS to GPU, then this is what you are looking 
for. Fortunately, C source code module is fully compatible with TVM runtime 
module, which means the generated code could be compiled by any C/C++ compiler 
with proper compilation flags, so the only task you have is to implement a 
codegen that generates C code for subgraphs and a C source module to integrate 
into TVM runtime module. We will demonstrate how to implement a C code 
generator for your hardware in the following section.
 
 Review comment:
   I do think in this case perhaps it is better to introduce dep to dmlc-core 
and DLPack


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-tvm] tqchen commented on a change in pull request #4602: [Docs] Bring Your Own Codegen Guide -- Part 1

[GitBox]

tqchen commented on a change in pull request #4602: [Docs] Bring Your Own 
Codegen Guide -- Part 1
URL: https://github.com/apache/incubator-tvm/pull/4602#discussion_r366639821
 
 

 ##
 File path: docs/dev/relay_bring_your_own_codegen.rst
 ##
 @@ -0,0 +1,529 @@
+..  Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+..http://www.apache.org/licenses/LICENSE-2.0
+
+..  Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied.  See the License for the
+specific language governing permissions and limitations
+under the License.
+
+=
+Bring Your Own Codegen To TVM
+=
+**Author**: `Zhi Chen `_, `Cody Hao Yu 
`_
+
+As the number of hardware devices targeted by deep learning workloads keeps 
increasing, the required knowledge for users to achieve high performance on 
various devices keeps increasing as well. To free data scientists from worrying 
about the performance when developing a new model, hardware vendors either 
provide libraries such as MKLDNN or cuDNN with many commonly used deep learning 
operators, or provide frameworks such as TensorRT to let users describe their 
models in a certain way to achieve high performance. However, users have to 
learn a new programming interface when they attempt to work on a new library or 
device. As a result, the demand for a unified programming interface becomes 
more and more important to 1) let all users and hardware vendors stand on the 
same page, and 2) provide a feasible solution to allow specialized hardware or 
library to only support widely used operators with extremely high performance, 
but fallback unsupported operators to general devices like CPU/GPU.
+
+In this developer guide, we demonstrate how you, as a hardware vendor, can 
easily implement your own codegen and register it as a Relay backend compiler 
to support your hardware device/library. This guide covers two types of codegen 
based on different graph representations you need:
+
+**1. You want to generate C code.**
+
+If your hardware already has a well-optimized C/C++ library, such as Intel 
CBLAS/MKL to CPU and NVIDIA CUBLAS to GPU, then this is what you are looking 
for. Fortunately, C source code module is fully compatible with TVM runtime 
module, which means the generated code could be compiled by any C/C++ compiler 
with proper compilation flags, so the only task you have is to implement a 
codegen that generates C code for subgraphs and a C source module to integrate 
into TVM runtime module. We will demonstrate how to implement a C code 
generator for your hardware in the following section.
 
 Review comment:
   I do think in this case perhaps it is better to introduce dep to dmlc-core 
and DLPack so simplify the code gen part


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-tvm] icemelon9 edited a comment on issue #4318: [Relay][TOPI]Fix meaning of conv2d_transpose output_padding parameter

[GitBox]

icemelon9 edited a comment on issue #4318: [Relay][TOPI]Fix meaning of 
conv2d_transpose output_padding parameter
URL: https://github.com/apache/incubator-tvm/pull/4318#issuecomment-574434976
 
 
   @abergeron @vinx13 @tmoreau89 
   I found two problems in this PR. 
   1. In this 
[line](https://github.com/apache/incubator-tvm/pull/4318/files#diff-8be7003a84f663f3f6b0dbb3bf1f5ba6R105),
 `h+dh` can be potentially out of boundary. `max(h) = out_h - 1 = in_h - 
filter_h + output_padding[0]` and `max(dh) = filter_h - 1`, therefore, 
`max(h+dh) = in_h + output_padding[0] - 1`. When `output_padding[0] >= 1`, 
`max(h+dh) >= in_h`, which is out of `data_pad` height boundary. Similar to 
`w+dw`.
   2. The x86 conv2d_transpose implementation is different from generic 
conv2d_tranpose. In x86, after you call `conv2d_transpose_nchw_preprocess`, you 
directly call the normal conv2d without using `output_padding`.
   
   I'll revert this PR for now. Could you fix these two bugs and double check 
whether cuda and arm_cpu implementation are correct? Further, could you 
investigate why CI doesn't catch these bugs?
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-tvm] icemelon9 edited a comment on issue #4318: [Relay][TOPI]Fix meaning of conv2d_transpose output_padding parameter

[GitBox]

icemelon9 edited a comment on issue #4318: [Relay][TOPI]Fix meaning of 
conv2d_transpose output_padding parameter
URL: https://github.com/apache/incubator-tvm/pull/4318#issuecomment-574434976
 
 
   @abergeron @vinx13 @tmoreau89 
   I found out two problems in this PR. 
   1. In this 
[line](https://github.com/apache/incubator-tvm/pull/4318/files#diff-8be7003a84f663f3f6b0dbb3bf1f5ba6R105),
 `h+dh` can be potentially out of boundary. `max(h) = out_h - 1 = in_h - 
filter_h + output_padding[0]` and `max(dh) = filter_h - 1`, therefore, 
`max(h+dh) = in_h + output_padding[0] - 1`. When `output_padding[0] >= 1`, 
`max(h+dh) >= in_h`, which is out of `data_pad` height boundary. Similar to 
`w+dw`.
   2. The x86 conv2d_transpose implementation is different from generic 
conv2d_tranpose. In x86, after you call `conv2d_transpose_nchw_preprocess`, you 
directly call the normal conv2d without using `output_padding`.
   
   I'll revert this PR for now. Could you fix these two bugs and double check 
whether cuda and arm_cpu implementation are correct? Further, could you 
investigate why CI doesn't catch these bugs?
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-tvm] icemelon9 edited a comment on issue #4318: [Relay][TOPI]Fix meaning of conv2d_transpose output_padding parameter

[GitBox]

icemelon9 edited a comment on issue #4318: [Relay][TOPI]Fix meaning of 
conv2d_transpose output_padding parameter
URL: https://github.com/apache/incubator-tvm/pull/4318#issuecomment-574434976
 
 
   @abergeron @vinx13 @tmoreau89 
   I found out two problems in this PR. 
   1. In this 
[line](https://github.com/apache/incubator-tvm/pull/4318/files#diff-8be7003a84f663f3f6b0dbb3bf1f5ba6R105),
 `h+dh` can be potentially out of boundary. `max(h) = out_h - 1 = in_h - 
filter_h + output_padding[0]` and `max(dh) = filter_h - 1`, therefore, 
`max(h+dh) = in_h + output_padding[0] - 1`. When `output_padding[0] >= 1`, 
`max(h+dh) >= in_h`, which is out of `data_pad` height boundary. Similar to 
`w+dw`.
   2. The x86 conv2d_transpose implementation is different from generic 
conv2d_tranpose. In x86, after you call `conv2d_transpose_nchw_preprocess`, you 
directly call the normal conv2d without using `output_padding`.
   
   I'll revert this PR for now. Could you fix these two bugs and double check 
whether cuda and arm_cpu implementation are correct? Further, could you 
investigate why CI doesn't catch these errors?
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[incubator-tvm] 01/01: Revert "[Relay][TOPI]Fix meaning of conv2d_transpose output_padding parameter (#4318)"

[haichen]

This is an automated email from the ASF dual-hosted git repository.

haichen pushed a commit to branch revert-4318-conv2d_transpose_fix
in repository https://gitbox.apache.org/repos/asf/incubator-tvm.git

commit ebfff4419fc714d173a1e365739badee1d79adfa
Author: Haichen Shen 
AuthorDate: Tue Jan 14 16:13:22 2020 -0800

Revert "[Relay][TOPI]Fix meaning of conv2d_transpose output_padding 
parameter (#4318)"

This reverts commit dcf7fbf1f962569e78c624755b2d612fffa81ada.
---
 python/tvm/autotvm/tophub.py   |  6 +++---
 python/tvm/relay/op/nn/_nn.py  | 12 +++
 tests/python/relay/test_op_level2.py   | 25 +++---
 topi/python/topi/arm_cpu/conv2d_transpose.py   | 21 ++
 topi/python/topi/cuda/conv1d_transpose_ncw.py  |  7 +++---
 topi/python/topi/cuda/conv2d_transpose_nchw.py | 12 +--
 topi/python/topi/nn/conv1d_transpose.py|  6 ++
 topi/python/topi/nn/conv2d_transpose.py| 24 +
 .../topi/testing/conv1d_transpose_ncw_python.py|  7 +++---
 .../python/topi/testing/conv2d_transpose_python.py | 17 ++-
 topi/python/topi/x86/conv2d_transpose.py   |  4 ++--
 .../tests/python/test_topi_conv1d_transpose_ncw.py |  2 +-
 .../python/test_topi_conv2d_transpose_nchw.py  | 18 +++-
 vta/python/vta/top/vta_conv2d_transpose.py | 16 ++
 vta/scripts/tune_conv2d_transpose.py   | 19 ++--
 .../test_benchmark_topi_conv2d_transpose.py| 16 ++
 16 files changed, 92 insertions(+), 120 deletions(-)

diff --git a/python/tvm/autotvm/tophub.py b/python/tvm/autotvm/tophub.py
index 2f07f72..d953eaa 100644
--- a/python/tvm/autotvm/tophub.py
+++ b/python/tvm/autotvm/tophub.py
@@ -46,16 +46,16 @@ AUTOTVM_TOPHUB_ROOT_PATH = 
os.path.join(os.path.expanduser('~'), ".tvm", "tophub
 
 # the version of each package
 PACKAGE_VERSION = {
-'arm_cpu':  "v0.05",
+'arm_cpu':  "v0.04",
 'llvm': "v0.03",
 
-'cuda': "v0.07",
+'cuda': "v0.06",
 'rocm': "v0.03",
 'opencl':   "v0.03",
 'mali': "v0.05",
 'intel_graphics':   "v0.01",
 
-'vta':  "v0.07",
+'vta':  "v0.06",
 }
 
 logger = logging.getLogger('autotvm')
diff --git a/python/tvm/relay/op/nn/_nn.py b/python/tvm/relay/op/nn/_nn.py
index e405fee..275e09c 100644
--- a/python/tvm/relay/op/nn/_nn.py
+++ b/python/tvm/relay/op/nn/_nn.py
@@ -339,7 +339,6 @@ def compute_conv2d_transpose(attrs, inputs, out_dtype, 
target):
 padding = get_const_tuple(attrs.padding)
 strides = get_const_tuple(attrs.strides)
 dilation = get_const_tuple(attrs.dilation)
-output_padding = get_const_tuple(attrs.output_padding)
 groups = attrs.groups
 layout = attrs.data_layout
 out_dtype = attrs.out_dtype
@@ -349,7 +348,10 @@ def compute_conv2d_transpose(attrs, inputs, out_dtype, 
target):
 assert dilation == (1, 1), "not support dilate now"
 assert groups == 1, "only support groups == 1 for now"
 out = topi.nn.conv2d_transpose_nchw(
-inputs[0], inputs[1], strides, padding, out_dtype, output_padding)
+inputs[0], inputs[1], strides, padding, out_dtype)
+output_padding = get_const_tuple(attrs.output_padding)
+out = topi.nn.pad(out,
+  [0, 0, 0, 0], [0, 0, output_padding[0], 
output_padding[1]])
 return [out]
 
 
@@ -442,8 +444,10 @@ def compute_conv1d_transpose(attrs, inputs, out_dtype, 
target):
 assert dilation == (1,), "conv1d_transpose dilation is not supported"
 assert groups == 1, "conv1d_transpose groups == 1 only supported"
 out = topi.nn.conv1d_transpose_ncw(
-inputs[0], inputs[1], strides, padding, out_dtype,
-get_const_tuple(attrs.output_padding))
+inputs[0], inputs[1], strides, padding, out_dtype)
+output_padding = get_const_tuple(attrs.output_padding)
+out = topi.nn.pad(out,
+  [0, 0, 0], [0, 0, output_padding[0]])
 return [out]
 
 
diff --git a/tests/python/relay/test_op_level2.py 
b/tests/python/relay/test_op_level2.py
index 68f3983..4b914ee 100644
--- a/tests/python/relay/test_op_level2.py
+++ b/tests/python/relay/test_op_level2.py
@@ -570,8 +570,11 @@ def test_conv2d_transpose_nchw_run():
 dtype = "float32"
 data = np.random.uniform(size=dshape).astype(dtype)
 kernel = np.random.uniform(size=kshape).astype(dtype)
-ref_res = topi.testing.conv2d_transpose_nchw_python(
-data, kernel, 2, 1, (2, 2))
+c_np = topi.testing.conv2d_transpose_nchw_python(
+data, kernel, 2, 1)
+d_np = np.zeros(shape=oshape)
+d_np[:,:,0:c_np.shape[2],0:c_np.shape[3]] = c_np
+ref_res = d_np
 
 for target, ctx in ctx_list():
 intrp1 = relay.create_executor("graph", ctx=ctx, target=target)
@@ -596,14 +599,9 @@ def 

[GitHub] [incubator-tvm] icemelon9 opened a new pull request #4708: Revert "[Relay][TOPI]Fix meaning of conv2d_transpose output_padding parameter"

[GitBox]

icemelon9 opened a new pull request #4708: Revert "[Relay][TOPI]Fix meaning of 
conv2d_transpose output_padding parameter"
URL: https://github.com/apache/incubator-tvm/pull/4708
 
 
   Reverts apache/incubator-tvm#4318
   
   Bugs in the PR.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[incubator-tvm] branch revert-4318-conv2d_transpose_fix created (now ebfff44)

[haichen]

This is an automated email from the ASF dual-hosted git repository.

haichen pushed a change to branch revert-4318-conv2d_transpose_fix
in repository https://gitbox.apache.org/repos/asf/incubator-tvm.git.


  at ebfff44  Revert "[Relay][TOPI]Fix meaning of conv2d_transpose 
output_padding parameter (#4318)"

This branch includes the following new commits:

 new ebfff44  Revert "[Relay][TOPI]Fix meaning of conv2d_transpose 
output_padding parameter (#4318)"

The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails.  The revisions
listed as "add" were already present in the repository and have only
been added to this reference.

[GitHub] [incubator-tvm] icemelon9 commented on issue #4318: [Relay][TOPI]Fix meaning of conv2d_transpose output_padding parameter

[GitBox]

icemelon9 commented on issue #4318: [Relay][TOPI]Fix meaning of 
conv2d_transpose output_padding parameter
URL: https://github.com/apache/incubator-tvm/pull/4318#issuecomment-574434976
 
 
   @abergeron @vinx13 @tmoreau89 
   I found out two problems in this PR. 
   1. In this 
[line](https://github.com/apache/incubator-tvm/pull/4318/files#diff-8be7003a84f663f3f6b0dbb3bf1f5ba6R105),
 `h+dh` can be potentially out of boundary. `max(h) = out_h - 1 = in_h - 
filter_h + output_padding[0]` and `max(dh) = filter_h - 1`, therefore, 
`max(h+dh) = in_h + output_padding[0] - 1`. When `output_padding[0] >= 1`, 
`max(h+dh) >= in_h`, which is out of `data_pad` height boundary. Similar to 
`w+dw`.
   2. The x86 conv2d_transpose implementation is different from generic 
conv2d_tranpose. In x86, after you call `conv2d_transpose_nchw_preprocess`, you 
directly call the normal conv2d without using `output_padding`.
   
   I'll revert this PR for now. Could you fix these two bugs and double check 
whether cuda and arm_cpu are correct? Further, could you investigate why CI 
doesn't catch these errors?
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-tvm] hcho3 opened a new pull request #4707: [Relay][Frontend][TF] Fix handling of 0D scalar Constant

[GitBox]

hcho3 opened a new pull request #4707: [Relay][Frontend][TF] Fix handling of 0D 
scalar Constant
URL: https://github.com/apache/incubator-tvm/pull/4707
 
 
   Consider the following snippet of a TensorFlow model:
   ```
   node {
 name: "flatten_1/stack/0"
 op: "Const"
 attr {
   key: "dtype"
   value {
 type: DT_INT32
   }
 }
 attr {
   key: "value"
   value {
 tensor {
   dtype: DT_INT32
   tensor_shape {
   }
   int_val: -1
 }
   }
 }
   }
   node {
 name: "flatten_1/stack"
 op: "Pack"
 input: "flatten_1/stack/0"
 input: "flatten_1/Prod"
 attr {
   key: "N"
   value {
 i: 2
   }
 }
 attr {
   key: "T"
   value {
 type: DT_INT32
   }
 }
 attr {
   key: "axis"
   value {
 i: 0
   }
 }
   }
   ```
   
   The Pack operator accepts two inputs: the 0D constant `flatten_1/stack/0` 
and a 0D tensor  `flatten_1/Prod`. Currently, the TF frontend cannot handle 
this model because it turns the 0D constant `flatten_1/stack/0` into a 1D 
constant of dimension `(1,)`:
   
https://github.com/apache/incubator-tvm/blob/f4c4fde4a07d2346c159f1d5c6ccd00fb8fb7c7d/python/tvm/relay/frontend/tensorflow.py#L2402-L2405
   
   This PR modifies the frontend to handle 0D constants properly.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-tvm] comaniac commented on a change in pull request #4602: [Docs] Bring Your Own Codegen Guide -- Part 1

[GitBox]

comaniac commented on a change in pull request #4602: [Docs] Bring Your Own 
Codegen Guide -- Part 1
URL: https://github.com/apache/incubator-tvm/pull/4602#discussion_r366634417
 
 

 ##
 File path: docs/dev/relay_bring_your_own_codegen.rst
 ##
 @@ -0,0 +1,529 @@
+..  Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+..http://www.apache.org/licenses/LICENSE-2.0
+
+..  Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied.  See the License for the
+specific language governing permissions and limitations
+under the License.
+
+=
+Bring Your Own Codegen To TVM
+=
+**Author**: `Zhi Chen `_, `Cody Hao Yu 
`_
+
+As the number of hardware devices targeted by deep learning workloads keeps 
increasing, the required knowledge for users to achieve high performance on 
various devices keeps increasing as well. To free data scientists from worrying 
about the performance when developing a new model, hardware vendors either 
provide libraries such as MKLDNN or cuDNN with many commonly used deep learning 
operators, or provide frameworks such as TensorRT to let users describe their 
models in a certain way to achieve high performance. However, users have to 
learn a new programming interface when they attempt to work on a new library or 
device. As a result, the demand for a unified programming interface becomes 
more and more important to 1) let all users and hardware vendors stand on the 
same page, and 2) provide a feasible solution to allow specialized hardware or 
library to only support widely used operators with extremely high performance, 
but fallback unsupported operators to general devices like CPU/GPU.
+
+In this developer guide, we demonstrate how you, as a hardware vendor, can 
easily implement your own codegen and register it as a Relay backend compiler 
to support your hardware device/library. This guide covers two types of codegen 
based on different graph representations you need:
+
+**1. You want to generate C code.**
+
+If your hardware already has a well-optimized C/C++ library, such as Intel 
CBLAS/MKL to CPU and NVIDIA CUBLAS to GPU, then this is what you are looking 
for. Fortunately, C source code module is fully compatible with TVM runtime 
module, which means the generated code could be compiled by any C/C++ compiler 
with proper compilation flags, so the only task you have is to implement a 
codegen that generates C code for subgraphs and a C source module to integrate 
into TVM runtime module. We will demonstrate how to implement a C code 
generator for your hardware in the following section.
 
 Review comment:
   Thanks for the advice. We can modify the C codegen to generate the following 
working code:
   
   ```c++
   extern "C" int json_rt_1_(DLTensor* arg0, DLTensor* arg1, DLTensor* 
arg2, DLTensor* arg3,
 DLTensor* out) {
   // User provided function.
   gcc_1_(static_cast(arg0->data), 
static_cast(arg1->data),
   static_cast(arg2->data), 
static_cast(arg3->data),
   static_cast(out->data));
   return 0;
   }
   
   TVM_DLL_EXPORT_TYPED_FUNC(json_rt_1, json_rt_1_);
   ```
   
   On the other hand, here is still a concern of including this marco. Since 
the macro is defined in `packed_func.h`, we need to include this file in the 
generated C code. However, `packed_func.h` includes `dmlc/logging.h`, which is 
not one of the default 3rd party include paths. As a result, users have to pass 
an additional argument for compilation:
   
   ```python
   options=[ "-O2", "-std=c++11", "-I",  "./3rdparty/dmlc-core/include/"]
   ```
   
   This seems not intuitive to users as they never use dmlc directly.
   
   The solutions are: 1) Keep the current version. Since this wrapper is 
generated by ours, users will not see those complicated wrappers. 2) Put dmlc 
include path to libinfo, but it will increase the code size for all compiled 
modules. 3) Modify `packed_func.h` to make it dmlc indpendent. Please let us 
know which solution you prefer.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:

[GitHub] [incubator-tvm] tqchen commented on a change in pull request #4602: [Docs] Bring Your Own Codegen Guide -- Part 1

[GitBox]

tqchen commented on a change in pull request #4602: [Docs] Bring Your Own 
Codegen Guide -- Part 1
URL: https://github.com/apache/incubator-tvm/pull/4602#discussion_r366616774
 
 

 ##
 File path: docs/dev/relay_bring_your_own_codegen.rst
 ##
 @@ -0,0 +1,529 @@
+..  Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+..http://www.apache.org/licenses/LICENSE-2.0
+
+..  Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied.  See the License for the
+specific language governing permissions and limitations
+under the License.
+
+=
+Bring Your Own Codegen To TVM
+=
+**Author**: `Zhi Chen `_, `Cody Hao Yu 
`_
+
+As the number of hardware devices targeted by deep learning workloads keeps 
increasing, the required knowledge for users to achieve high performance on 
various devices keeps increasing as well. To free data scientists from worrying 
about the performance when developing a new model, hardware vendors either 
provide libraries such as MKLDNN or cuDNN with many commonly used deep learning 
operators, or provide frameworks such as TensorRT to let users describe their 
models in a certain way to achieve high performance. However, users have to 
learn a new programming interface when they attempt to work on a new library or 
device. As a result, the demand for a unified programming interface becomes 
more and more important to 1) let all users and hardware vendors stand on the 
same page, and 2) provide a feasible solution to allow specialized hardware or 
library to only support widely used operators with extremely high performance, 
but fallback unsupported operators to general devices like CPU/GPU.
+
+In this developer guide, we demonstrate how you, as a hardware vendor, can 
easily implement your own codegen and register it as a Relay backend compiler 
to support your hardware device/library. This guide covers two types of codegen 
based on different graph representations you need:
+
+**1. You want to generate C code.**
+
+If your hardware already has a well-optimized C/C++ library, such as Intel 
CBLAS/MKL to CPU and NVIDIA CUBLAS to GPU, then this is what you are looking 
for. Fortunately, C source code module is fully compatible with TVM runtime 
module, which means the generated code could be compiled by any C/C++ compiler 
with proper compilation flags, so the only task you have is to implement a 
codegen that generates C code for subgraphs and a C source module to integrate 
into TVM runtime module. We will demonstrate how to implement a C code 
generator for your hardware in the following section.
 
 Review comment:
   The API will looks like:
   ```c++
   void json_rt_(DLTensor* x, DLTensor *y, DLTensor* out) {
  // followup code
   } 
   
   TVM_DLL_EXPORT_TYPED(json_rt, json_rt_); 
   ```


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-tvm] tqchen commented on a change in pull request #4602: [Docs] Bring Your Own Codegen Guide -- Part 1

[GitBox]

tqchen commented on a change in pull request #4602: [Docs] Bring Your Own 
Codegen Guide -- Part 1
URL: https://github.com/apache/incubator-tvm/pull/4602#discussion_r366616774
 
 

 ##
 File path: docs/dev/relay_bring_your_own_codegen.rst
 ##
 @@ -0,0 +1,529 @@
+..  Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+..http://www.apache.org/licenses/LICENSE-2.0
+
+..  Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied.  See the License for the
+specific language governing permissions and limitations
+under the License.
+
+=
+Bring Your Own Codegen To TVM
+=
+**Author**: `Zhi Chen `_, `Cody Hao Yu 
`_
+
+As the number of hardware devices targeted by deep learning workloads keeps 
increasing, the required knowledge for users to achieve high performance on 
various devices keeps increasing as well. To free data scientists from worrying 
about the performance when developing a new model, hardware vendors either 
provide libraries such as MKLDNN or cuDNN with many commonly used deep learning 
operators, or provide frameworks such as TensorRT to let users describe their 
models in a certain way to achieve high performance. However, users have to 
learn a new programming interface when they attempt to work on a new library or 
device. As a result, the demand for a unified programming interface becomes 
more and more important to 1) let all users and hardware vendors stand on the 
same page, and 2) provide a feasible solution to allow specialized hardware or 
library to only support widely used operators with extremely high performance, 
but fallback unsupported operators to general devices like CPU/GPU.
+
+In this developer guide, we demonstrate how you, as a hardware vendor, can 
easily implement your own codegen and register it as a Relay backend compiler 
to support your hardware device/library. This guide covers two types of codegen 
based on different graph representations you need:
+
+**1. You want to generate C code.**
+
+If your hardware already has a well-optimized C/C++ library, such as Intel 
CBLAS/MKL to CPU and NVIDIA CUBLAS to GPU, then this is what you are looking 
for. Fortunately, C source code module is fully compatible with TVM runtime 
module, which means the generated code could be compiled by any C/C++ compiler 
with proper compilation flags, so the only task you have is to implement a 
codegen that generates C code for subgraphs and a C source module to integrate 
into TVM runtime module. We will demonstrate how to implement a C code 
generator for your hardware in the following section.
 
 Review comment:
   The API will looks like:
   ```c++
   void json_rt_(DLTensor* x, DLTensor *y, DLTensor* out) {
  // followup code
   } 
   
   TVM_DLL_EXPORT_TYPED_FUNC(json_rt, json_rt_); 
   ```


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-tvm] tqchen commented on a change in pull request #4602: [Docs] Bring Your Own Codegen Guide -- Part 1

[GitBox]

tqchen commented on a change in pull request #4602: [Docs] Bring Your Own 
Codegen Guide -- Part 1
URL: https://github.com/apache/incubator-tvm/pull/4602#discussion_r366615883
 
 

 ##
 File path: docs/dev/relay_bring_your_own_codegen.rst
 ##
 @@ -0,0 +1,529 @@
+..  Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+..http://www.apache.org/licenses/LICENSE-2.0
+
+..  Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied.  See the License for the
+specific language governing permissions and limitations
+under the License.
+
+=
+Bring Your Own Codegen To TVM
+=
+**Author**: `Zhi Chen `_, `Cody Hao Yu 
`_
+
+As the number of hardware devices targeted by deep learning workloads keeps 
increasing, the required knowledge for users to achieve high performance on 
various devices keeps increasing as well. To free data scientists from worrying 
about the performance when developing a new model, hardware vendors either 
provide libraries such as MKLDNN or cuDNN with many commonly used deep learning 
operators, or provide frameworks such as TensorRT to let users describe their 
models in a certain way to achieve high performance. However, users have to 
learn a new programming interface when they attempt to work on a new library or 
device. As a result, the demand for a unified programming interface becomes 
more and more important to 1) let all users and hardware vendors stand on the 
same page, and 2) provide a feasible solution to allow specialized hardware or 
library to only support widely used operators with extremely high performance, 
but fallback unsupported operators to general devices like CPU/GPU.
+
+In this developer guide, we demonstrate how you, as a hardware vendor, can 
easily implement your own codegen and register it as a Relay backend compiler 
to support your hardware device/library. This guide covers two types of codegen 
based on different graph representations you need:
+
+**1. You want to generate C code.**
+
+If your hardware already has a well-optimized C/C++ library, such as Intel 
CBLAS/MKL to CPU and NVIDIA CUBLAS to GPU, then this is what you are looking 
for. Fortunately, C source code module is fully compatible with TVM runtime 
module, which means the generated code could be compiled by any C/C++ compiler 
with proper compilation flags, so the only task you have is to implement a 
codegen that generates C code for subgraphs and a C source module to integrate 
into TVM runtime module. We will demonstrate how to implement a C code 
generator for your hardware in the following section.
+
+**2. You want to generate any other graph representations.**
+
+Your hardware may require other forms of graph representation, such as JSON. 
In this case, you need to implement not only a codegen but also a customized 
TVM runtime module to let TVM runtime know how this graph representation should 
be executed. If you already have a complete graph execution engine for your 
hardware, such as TensorRT for GPU, then this is a solution you can consider.
+
+After you finish the codegen and runtime, you can then let your customers 
annotate their models with your customized tag to make use of them. The 
tutorial for end-users to annotate and launch a specific codegen is **here 
(TBA)**.
+
+*
+Implement a C Codegen
+*
+
+In this part, we demonstrate how to implement a codegen that generates C code 
with pre-implemented operator functions. To simplify, our example codegen does 
not depend on third-party libraries. Instead, we manually implement two macros 
in C:
+
+.. code-block:: c++
+
+#define CSOURCE_BINARY_OP_1D(p_ID_, p_OP_, p_DIM1_) \
 
 Review comment:
   It is possible now to put NDArray as a return value. Although in most cases, 
we want to expose libraries that modifies NDArray inplace (e.g. pass return 
value through argument and ask the lib to modify inplace)


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-tvm] zhiics commented on a change in pull request #4706: [REFACTOR][IR] Unify IntImm and UIntImm

[GitBox]

zhiics commented on a change in pull request #4706: [REFACTOR][IR] Unify IntImm 
and UIntImm
URL: https://github.com/apache/incubator-tvm/pull/4706#discussion_r366610155
 
 

 ##
 File path: include/tvm/expr_operator.h
 ##
 @@ -597,6 +583,15 @@ TVM_DLL PrimExpr nearbyint(PrimExpr x);
  */
 TVM_DLL PrimExpr trunc(PrimExpr x);
 
+/*!
+ * \brief Construct a big uint constant by its low 32 bits and high 32bits.
+ * \param dtype The final data type.
+ * \param low The lower 32 bits.
+ * \param high The higher 32 bits.
+ * \return The constructed expression.
+ */
+TVM_DLL PrimExpr BigUIntImm(DataType dtype, int64_t low, int64_t high);
 
 Review comment:
   BigIntImm -> LargeUIntImm ?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[incubator-tvm] branch master updated (3f2abfb -> f4c4fde)

[tqchen]

This is an automated email from the ASF dual-hosted git repository.

tqchen pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-tvm.git.


from 3f2abfb  [relay] Relay annotation and partitioning for external 
compilers (#4570)
 add f4c4fde  [REFACTOR][IR] Polish ir/type (#4705)

No new revisions were added by this update.

Summary of changes:
 include/tvm/ir/adt.h|   2 +-
 include/tvm/{node => ir}/env_func.h |  24 ++---
 include/tvm/ir/expr.h   |   2 +-
 include/tvm/ir/module.h |  22 ++--
 include/tvm/ir/op.h |  10 +-
 include/tvm/ir/transform.h  |  26 ++---
 include/tvm/ir/type.h   | 185 +---
 include/tvm/ir/type_relation.h  |  84 +++
 include/tvm/relay/type.h|  69 ++--
 src/api/api_test.cc |   2 +-
 src/{node => ir}/env_func.cc|   2 +-
 src/ir/type.cc  |  73 ++---
 src/ir/type_relation.cc |  39 +--
 src/relay/backend/compile_engine.cc |   2 +-
 src/relay/ir/expr.cc|   2 +-
 src/relay/ir/type.cc|  36 ---
 src/relay/ir/type_functor.cc|   8 +-
 src/relay/op/algorithm/topk.cc  |   2 +-
 src/relay/op/memory/memory.cc   |  20 ++--
 src/relay/op/nn/nn.cc   |   4 +-
 src/relay/op/nn/sparse.cc   |   2 +-
 src/relay/op/tensor/transform.cc|   4 +-
 src/relay/op/vision/multibox_op.cc  |   2 +-
 src/relay/op/vision/nms.cc  |   2 +-
 src/relay/pass/de_duplicate.cc  |   2 +-
 src/relay/pass/eta_expand.cc|   4 +-
 src/relay/pass/gradient.cc  |  18 ++--
 src/relay/pass/to_cps.cc|   8 +-
 src/relay/pass/type_infer.cc|  22 ++--
 src/relay/pass/type_solver.cc   |   8 +-
 tests/cpp/relay_pass_type_infer_test.cc |   2 +-
 31 files changed, 415 insertions(+), 273 deletions(-)
 rename include/tvm/{node => ir}/env_func.h (94%)
 rename src/{node => ir}/env_func.cc (98%)

[GitHub] [incubator-tvm] tqchen merged pull request #4705: [REFACTOR][IR] Polish ir/type

[GitBox]

tqchen merged pull request #4705: [REFACTOR][IR] Polish ir/type
URL: https://github.com/apache/incubator-tvm/pull/4705
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-tvm] masahi commented on issue #4497: [WIP] [Relay] Add a PyTorch to Relay Parser

[GitBox]

masahi commented on issue #4497: [WIP] [Relay] Add a PyTorch to Relay Parser
URL: https://github.com/apache/incubator-tvm/pull/4497#issuecomment-574403934
 
 
   I'm happy to merge this even if we can only support v1.3 models for now. I 
want to send other op converters I need.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-tvm] yzhliu commented on issue #4684: [Arith] add ShapeVar representing non-neg valued variable in a tensor shape

[GitBox]

yzhliu commented on issue #4684: [Arith] add ShapeVar representing non-neg 
valued variable in a tensor shape
URL: https://github.com/apache/incubator-tvm/pull/4684#issuecomment-574397147
 
 
   https://discuss.tvm.ai/t/rfc-name-for-non-negative-variable


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-tvm] comaniac commented on a change in pull request #4602: [Docs] Bring Your Own Codegen Guide -- Part 1

[GitBox]

comaniac commented on a change in pull request #4602: [Docs] Bring Your Own 
Codegen Guide -- Part 1
URL: https://github.com/apache/incubator-tvm/pull/4602#discussion_r366589820
 
 

 ##
 File path: docs/dev/relay_bring_your_own_codegen.rst
 ##
 @@ -0,0 +1,529 @@
+..  Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+..http://www.apache.org/licenses/LICENSE-2.0
+
+..  Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied.  See the License for the
+specific language governing permissions and limitations
+under the License.
+
+=
+Bring Your Own Codegen To TVM
+=
+**Author**: `Zhi Chen `_, `Cody Hao Yu 
`_
+
+As the number of hardware devices targeted by deep learning workloads keeps 
increasing, the required knowledge for users to achieve high performance on 
various devices keeps increasing as well. To free data scientists from worrying 
about the performance when developing a new model, hardware vendors either 
provide libraries such as MKLDNN or cuDNN with many commonly used deep learning 
operators, or provide frameworks such as TensorRT to let users describe their 
models in a certain way to achieve high performance. However, users have to 
learn a new programming interface when they attempt to work on a new library or 
device. As a result, the demand for a unified programming interface becomes 
more and more important to 1) let all users and hardware vendors stand on the 
same page, and 2) provide a feasible solution to allow specialized hardware or 
library to only support widely used operators with extremely high performance, 
but fallback unsupported operators to general devices like CPU/GPU.
+
+In this developer guide, we demonstrate how you, as a hardware vendor, can 
easily implement your own codegen and register it as a Relay backend compiler 
to support your hardware device/library. This guide covers two types of codegen 
based on different graph representations you need:
+
+**1. You want to generate C code.**
+
+If your hardware already has a well-optimized C/C++ library, such as Intel 
CBLAS/MKL to CPU and NVIDIA CUBLAS to GPU, then this is what you are looking 
for. Fortunately, C source code module is fully compatible with TVM runtime 
module, which means the generated code could be compiled by any C/C++ compiler 
with proper compilation flags, so the only task you have is to implement a 
codegen that generates C code for subgraphs and a C source module to integrate 
into TVM runtime module. We will demonstrate how to implement a C code 
generator for your hardware in the following section.
 
 Review comment:
   I've tried but it seems cannot reduce the code size much. Since 
TVM_DLL_EXPORT_PACKED_FUNC packs args, type_code, num_args to TVMArgs, we still 
have to have another wrapper to unpack them back to TVMValue. As a result, 
[this 
wrapper](https://github.com/apache/incubator-tvm/blob/master/tests/python/relay/test_external_runtime.py#L147)
 is still required.
   
   On the other hand, TVM_DLL_EXPORT_TYPED_FUNC does unpack TVMArgs according 
to native function signiture, but it seems can only return scalar values. 
Please correct me if I misused this marco.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-tvm] comaniac commented on a change in pull request #4602: [Docs] Bring Your Own Codegen Guide -- Part 1

[GitBox]

comaniac commented on a change in pull request #4602: [Docs] Bring Your Own 
Codegen Guide -- Part 1
URL: https://github.com/apache/incubator-tvm/pull/4602#discussion_r366584308
 
 

 ##
 File path: docs/dev/relay_bring_your_own_codegen.rst
 ##
 @@ -0,0 +1,529 @@
+..  Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+..http://www.apache.org/licenses/LICENSE-2.0
+
+..  Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied.  See the License for the
+specific language governing permissions and limitations
+under the License.
+
+=
+Bring Your Own Codegen To TVM
+=
+**Author**: `Zhi Chen `_, `Cody Hao Yu 
`_
+
+As the number of hardware devices targeted by deep learning workloads keeps 
increasing, the required knowledge for users to achieve high performance on 
various devices keeps increasing as well. To free data scientists from worrying 
about the performance when developing a new model, hardware vendors either 
provide libraries such as MKLDNN or cuDNN with many commonly used deep learning 
operators, or provide frameworks such as TensorRT to let users describe their 
models in a certain way to achieve high performance. However, users have to 
learn a new programming interface when they attempt to work on a new library or 
device. As a result, the demand for a unified programming interface becomes 
more and more important to 1) let all users and hardware vendors stand on the 
same page, and 2) provide a feasible solution to allow specialized hardware or 
library to only support widely used operators with extremely high performance, 
but fallback unsupported operators to general devices like CPU/GPU.
+
+In this developer guide, we demonstrate how you, as a hardware vendor, can 
easily implement your own codegen and register it as a Relay backend compiler 
to support your hardware device/library. This guide covers two types of codegen 
based on different graph representations you need:
+
+**1. You want to generate C code.**
+
+If your hardware already has a well-optimized C/C++ library, such as Intel 
CBLAS/MKL to CPU and NVIDIA CUBLAS to GPU, then this is what you are looking 
for. Fortunately, C source code module is fully compatible with TVM runtime 
module, which means the generated code could be compiled by any C/C++ compiler 
with proper compilation flags, so the only task you have is to implement a 
codegen that generates C code for subgraphs and a C source module to integrate 
into TVM runtime module. We will demonstrate how to implement a C code 
generator for your hardware in the following section.
+
+**2. You want to generate any other graph representations.**
+
+Your hardware may require other forms of graph representation, such as JSON. 
In this case, you need to implement not only a codegen but also a customized 
TVM runtime module to let TVM runtime know how this graph representation should 
be executed. If you already have a complete graph execution engine for your 
hardware, such as TensorRT for GPU, then this is a solution you can consider.
+
+After you finish the codegen and runtime, you can then let your customers 
annotate their models with your customized tag to make use of them. The 
tutorial for end-users to annotate and launch a specific codegen is **here 
(TBA)**.
+
+*
+Implement a C Codegen
+*
+
+In this part, we demonstrate how to implement a codegen that generates C code 
with pre-implemented operator functions. To simplify, our example codegen does 
not depend on third-party libraries. Instead, we manually implement two macros 
in C:
+
+.. code-block:: c++
+
+#define CSOURCE_BINARY_OP_1D(p_ID_, p_OP_, p_DIM1_) \
 
 Review comment:
   From my understanding this macro cannot be replaced with 
TVM_DLL_EXPORT_TYPED because it returns a tensor (float *).


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-tvm] alexwong commented on issue #4497: [WIP] [Relay] Add a PyTorch to Relay Parser

[GitBox]

alexwong commented on issue #4497: [WIP] [Relay] Add a PyTorch to Relay Parser
URL: https://github.com/apache/incubator-tvm/pull/4497#issuecomment-574383474
 
 
   > @alexwong I tried your PR locally. With pytorch v1.3 it works, but they 
introduced big change in 
[#28408](https://github.com/pytorch/pytorch/pull/28408) and 
[#28409](https://github.com/pytorch/pytorch/pull/28409), and it broke your PR 
(I may be wrong about which PR broke it). Below is how their IR looks like for 
resnet18 now. My pytorch version is '1.5.0a0+0dbd5c0' (the output of 
torch.**version**).
   > 
   > ```
   > graph(%self.1 : 
__torch__.torch.nn.modules.module.___torch_mangle_66.Module,
   >   %input.1 : Float(1, 3, 224, 224)):
   >   %1452 : __torch__.torch.nn.modules.module.___torch_mangle_65.Module = 
prim::GetAttr[name="fc"](%self.1)
   >   %1449 : __torch__.torch.nn.modules.module.___torch_mangle_64.Module = 
prim::GetAttr[name="avgpool"](%self.1)
   >   %1448 : __torch__.torch.nn.modules.module.___torch_mangle_63.Module = 
prim::GetAttr[name="layer4"](%self.1)
   >   %1402 : __torch__.torch.nn.modules.module.___torch_mangle_47.Module = 
prim::GetAttr[name="layer3"](%self.1)
   >   %1356 : __torch__.torch.nn.modules.module.___torch_mangle_31.Module = 
prim::GetAttr[name="layer2"](%self.1)
   >   %1310 : __torch__.torch.nn.modules.module.___torch_mangle_15.Module = 
prim::GetAttr[name="layer1"](%self.1)
   >   %1273 : __torch__.torch.nn.modules.module.___torch_mangle_2.Module = 
prim::GetAttr[name="maxpool"](%self.1)
   >   %1272 : __torch__.torch.nn.modules.module.___torch_mangle_1.Module = 
prim::GetAttr[name="relu"](%self.1)
   >   %1271 : __torch__.torch.nn.modules.module.___torch_mangle_0.Module = 
prim::GetAttr[name="bn1"](%self.1)
   >   %1265 : __torch__.torch.nn.modules.module.Module = 
prim::GetAttr[name="conv1"](%self.1)
   >   %1528 : Tensor = prim::CallMethod[name="forward"](%1265, %input.1)
   >   %1529 : Tensor = prim::CallMethod[name="forward"](%1271, %1528)
   >   %1530 : Tensor = prim::CallMethod[name="forward"](%1272, %1529)
   >   %1531 : Tensor = prim::CallMethod[name="forward"](%1273, %1530)
   >   %1532 : Tensor = prim::CallMethod[name="forward"](%1310, %1531)
   >   %1533 : Tensor = prim::CallMethod[name="forward"](%1356, %1532)
   >   %1534 : Tensor = prim::CallMethod[name="forward"](%1402, %1533)
   >   %1535 : Tensor = prim::CallMethod[name="forward"](%1448, %1534)
   >   %1536 : Tensor = prim::CallMethod[name="forward"](%1449, %1535)
   >   %1182 : int = prim::Constant[value=1]() # 
/home/masa/anaconda3/lib/python3.7/site-packages/torchvision-0.5.0a0+07cbb46-py3.7-linux-x86_64.egg/torchvision/models/resnet.py:210:0
   >   %1183 : int = prim::Constant[value=-1]() # 
/home/masa/anaconda3/lib/python3.7/site-packages/torchvision-0.5.0a0+07cbb46-py3.7-linux-x86_64.egg/torchvision/models/resnet.py:210:0
   >   %input : Float(1, 512) = aten::flatten(%1536, %1182, %1183) # 
/home/masa/anaconda3/lib/python3.7/site-packages/torchvision-0.5.0a0+07cbb46-py3.7-linux-x86_64.egg/torchvision/models/resnet.py:210:0
   >   %1537 : Tensor = prim::CallMethod[name="forward"](%1452, %input)
   >   return (%1537)
   > ```
   
   I can take a look again in the next few days. Will probably move to at least 
support PT 1.4 which was just released (or sometime in the next few days) and 
may have those IR changes as well. Some to-do's remaining are making sure 
different types are working, cleaning up the tests based off @jwfromm comments, 
and updating the parser to work for 1.4>. 


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-tvm] tqchen commented on a change in pull request #4602: [Docs] Bring Your Own Codegen Guide -- Part 1

[GitBox]

tqchen commented on a change in pull request #4602: [Docs] Bring Your Own 
Codegen Guide -- Part 1
URL: https://github.com/apache/incubator-tvm/pull/4602#discussion_r366582394
 
 

 ##
 File path: docs/dev/relay_bring_your_own_codegen.rst
 ##
 @@ -0,0 +1,529 @@
+..  Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+..http://www.apache.org/licenses/LICENSE-2.0
+
+..  Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied.  See the License for the
+specific language governing permissions and limitations
+under the License.
+
+=
+Bring Your Own Codegen To TVM
+=
+**Author**: `Zhi Chen `_, `Cody Hao Yu 
`_
+
+As the number of hardware devices targeted by deep learning workloads keeps 
increasing, the required knowledge for users to achieve high performance on 
various devices keeps increasing as well. To free data scientists from worrying 
about the performance when developing a new model, hardware vendors either 
provide libraries such as MKLDNN or cuDNN with many commonly used deep learning 
operators, or provide frameworks such as TensorRT to let users describe their 
models in a certain way to achieve high performance. However, users have to 
learn a new programming interface when they attempt to work on a new library or 
device. As a result, the demand for a unified programming interface becomes 
more and more important to 1) let all users and hardware vendors stand on the 
same page, and 2) provide a feasible solution to allow specialized hardware or 
library to only support widely used operators with extremely high performance, 
but fallback unsupported operators to general devices like CPU/GPU.
+
+In this developer guide, we demonstrate how you, as a hardware vendor, can 
easily implement your own codegen and register it as a Relay backend compiler 
to support your hardware device/library. This guide covers two types of codegen 
based on different graph representations you need:
 
 Review comment:
   While hardware vendor could be potential audience of this post. I feel it is 
a bit too narrow to only talk about "hardware vendor". Rename "hardware vendor" 
to hardware backend provider.
   
   Note that the later name would include academias who wrote their own hw stack


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-tvm] tqchen commented on a change in pull request #4602: [Docs] Bring Your Own Codegen Guide -- Part 1

[GitBox]

tqchen commented on a change in pull request #4602: [Docs] Bring Your Own 
Codegen Guide -- Part 1
URL: https://github.com/apache/incubator-tvm/pull/4602#discussion_r366581709
 
 

 ##
 File path: docs/dev/relay_bring_your_own_codegen.rst
 ##
 @@ -0,0 +1,529 @@
+..  Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+..http://www.apache.org/licenses/LICENSE-2.0
+
+..  Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied.  See the License for the
+specific language governing permissions and limitations
+under the License.
+
+=
+Bring Your Own Codegen To TVM
+=
+**Author**: `Zhi Chen `_, `Cody Hao Yu 
`_
+
+As the number of hardware devices targeted by deep learning workloads keeps 
increasing, the required knowledge for users to achieve high performance on 
various devices keeps increasing as well. To free data scientists from worrying 
about the performance when developing a new model, hardware vendors either 
provide libraries such as MKLDNN or cuDNN with many commonly used deep learning 
operators, or provide frameworks such as TensorRT to let users describe their 
models in a certain way to achieve high performance. However, users have to 
learn a new programming interface when they attempt to work on a new library or 
device. As a result, the demand for a unified programming interface becomes 
more and more important to 1) let all users and hardware vendors stand on the 
same page, and 2) provide a feasible solution to allow specialized hardware or 
library to only support widely used operators with extremely high performance, 
but fallback unsupported operators to general devices like CPU/GPU.
+
+In this developer guide, we demonstrate how you, as a hardware vendor, can 
easily implement your own codegen and register it as a Relay backend compiler 
to support your hardware device/library. This guide covers two types of codegen 
based on different graph representations you need:
+
+**1. You want to generate C code.**
+
+If your hardware already has a well-optimized C/C++ library, such as Intel 
CBLAS/MKL to CPU and NVIDIA CUBLAS to GPU, then this is what you are looking 
for. Fortunately, C source code module is fully compatible with TVM runtime 
module, which means the generated code could be compiled by any C/C++ compiler 
with proper compilation flags, so the only task you have is to implement a 
codegen that generates C code for subgraphs and a C source module to integrate 
into TVM runtime module. We will demonstrate how to implement a C code 
generator for your hardware in the following section.
 
 Review comment:
   consider update to talk about TVM_DLL_EXPORT_PACKED and TVM_DLL_EXPORT_TYPED


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-tvm] tqchen commented on a change in pull request #4602: [Docs] Bring Your Own Codegen Guide -- Part 1

[GitBox]

tqchen commented on a change in pull request #4602: [Docs] Bring Your Own 
Codegen Guide -- Part 1
URL: https://github.com/apache/incubator-tvm/pull/4602#discussion_r366580897
 
 

 ##
 File path: docs/dev/relay_bring_your_own_codegen.rst
 ##
 @@ -0,0 +1,529 @@
+..  Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+..http://www.apache.org/licenses/LICENSE-2.0
+
+..  Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied.  See the License for the
+specific language governing permissions and limitations
+under the License.
+
+=
+Bring Your Own Codegen To TVM
+=
+**Author**: `Zhi Chen `_, `Cody Hao Yu 
`_
+
+As the number of hardware devices targeted by deep learning workloads keeps 
increasing, the required knowledge for users to achieve high performance on 
various devices keeps increasing as well. To free data scientists from worrying 
about the performance when developing a new model, hardware vendors either 
provide libraries such as MKLDNN or cuDNN with many commonly used deep learning 
operators, or provide frameworks such as TensorRT to let users describe their 
models in a certain way to achieve high performance. However, users have to 
learn a new programming interface when they attempt to work on a new library or 
device. As a result, the demand for a unified programming interface becomes 
more and more important to 1) let all users and hardware vendors stand on the 
same page, and 2) provide a feasible solution to allow specialized hardware or 
library to only support widely used operators with extremely high performance, 
but fallback unsupported operators to general devices like CPU/GPU.
+
+In this developer guide, we demonstrate how you, as a hardware vendor, can 
easily implement your own codegen and register it as a Relay backend compiler 
to support your hardware device/library. This guide covers two types of codegen 
based on different graph representations you need:
+
+**1. You want to generate C code.**
+
+If your hardware already has a well-optimized C/C++ library, such as Intel 
CBLAS/MKL to CPU and NVIDIA CUBLAS to GPU, then this is what you are looking 
for. Fortunately, C source code module is fully compatible with TVM runtime 
module, which means the generated code could be compiled by any C/C++ compiler 
with proper compilation flags, so the only task you have is to implement a 
codegen that generates C code for subgraphs and a C source module to integrate 
into TVM runtime module. We will demonstrate how to implement a C code 
generator for your hardware in the following section.
+
+**2. You want to generate any other graph representations.**
+
+Your hardware may require other forms of graph representation, such as JSON. 
In this case, you need to implement not only a codegen but also a customized 
TVM runtime module to let TVM runtime know how this graph representation should 
be executed. If you already have a complete graph execution engine for your 
hardware, such as TensorRT for GPU, then this is a solution you can consider.
+
+After you finish the codegen and runtime, you can then let your customers 
annotate their models with your customized tag to make use of them. The 
tutorial for end-users to annotate and launch a specific codegen is **here 
(TBA)**.
+
+*
+Implement a C Codegen
+*
+
+In this part, we demonstrate how to implement a codegen that generates C code 
with pre-implemented operator functions. To simplify, our example codegen does 
not depend on third-party libraries. Instead, we manually implement two macros 
in C:
+
+.. code-block:: c++
+
+#define CSOURCE_BINARY_OP_1D(p_ID_, p_OP_, p_DIM1_) \
+extern "C" void p_ID_(float* a, float* b, float* out) { \
+for (int64_t i = 0; i < p_DIM1_; ++i) { \
+out[i] = a[i] p_OP_ b[i];   \
+}   \
+}
+
+#define CSOURCE_BINARY_OP_2D(p_ID_, p_OP_, p_DIM1_, p_DIM2_)  \
+extern "C" void p_ID_(float* a, float* b, float* out) {   \
+for (int64_t i = 0; i < p_DIM1_; ++i) {   \
+for (int64_t j = 0; j < p_DIM2_; ++j) {   \
+int64_t k = i * p_DIM2_ + j;  \
+out[k] = a[k] p_OP_ b[k]; 

[GitHub] [incubator-tvm] tqchen commented on a change in pull request #4602: [Docs] Bring Your Own Codegen Guide -- Part 1

[GitBox]

tqchen commented on a change in pull request #4602: [Docs] Bring Your Own 
Codegen Guide -- Part 1
URL: https://github.com/apache/incubator-tvm/pull/4602#discussion_r366580758
 
 

 ##
 File path: docs/dev/relay_bring_your_own_codegen.rst
 ##
 @@ -0,0 +1,529 @@
+..  Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+..http://www.apache.org/licenses/LICENSE-2.0
+
+..  Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied.  See the License for the
+specific language governing permissions and limitations
+under the License.
+
+=
+Bring Your Own Codegen To TVM
+=
+**Author**: `Zhi Chen `_, `Cody Hao Yu 
`_
+
+As the number of hardware devices targeted by deep learning workloads keeps 
increasing, the required knowledge for users to achieve high performance on 
various devices keeps increasing as well. To free data scientists from worrying 
about the performance when developing a new model, hardware vendors either 
provide libraries such as MKLDNN or cuDNN with many commonly used deep learning 
operators, or provide frameworks such as TensorRT to let users describe their 
models in a certain way to achieve high performance. However, users have to 
learn a new programming interface when they attempt to work on a new library or 
device. As a result, the demand for a unified programming interface becomes 
more and more important to 1) let all users and hardware vendors stand on the 
same page, and 2) provide a feasible solution to allow specialized hardware or 
library to only support widely used operators with extremely high performance, 
but fallback unsupported operators to general devices like CPU/GPU.
+
+In this developer guide, we demonstrate how you, as a hardware vendor, can 
easily implement your own codegen and register it as a Relay backend compiler 
to support your hardware device/library. This guide covers two types of codegen 
based on different graph representations you need:
+
+**1. You want to generate C code.**
+
+If your hardware already has a well-optimized C/C++ library, such as Intel 
CBLAS/MKL to CPU and NVIDIA CUBLAS to GPU, then this is what you are looking 
for. Fortunately, C source code module is fully compatible with TVM runtime 
module, which means the generated code could be compiled by any C/C++ compiler 
with proper compilation flags, so the only task you have is to implement a 
codegen that generates C code for subgraphs and a C source module to integrate 
into TVM runtime module. We will demonstrate how to implement a C code 
generator for your hardware in the following section.
+
+**2. You want to generate any other graph representations.**
+
+Your hardware may require other forms of graph representation, such as JSON. 
In this case, you need to implement not only a codegen but also a customized 
TVM runtime module to let TVM runtime know how this graph representation should 
be executed. If you already have a complete graph execution engine for your 
hardware, such as TensorRT for GPU, then this is a solution you can consider.
+
+After you finish the codegen and runtime, you can then let your customers 
annotate their models with your customized tag to make use of them. The 
tutorial for end-users to annotate and launch a specific codegen is **here 
(TBA)**.
+
+*
+Implement a C Codegen
+*
+
+In this part, we demonstrate how to implement a codegen that generates C code 
with pre-implemented operator functions. To simplify, our example codegen does 
not depend on third-party libraries. Instead, we manually implement two macros 
in C:
+
+.. code-block:: c++
+
+#define CSOURCE_BINARY_OP_1D(p_ID_, p_OP_, p_DIM1_) \
 
 Review comment:
   Is this macro necessary? or can we simplify using TVM_DLL_EXPORT_TYPED?
   
   Just to think about reduce the number of concepts


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-tvm] tqchen commented on issue #4684: [Arith] add ShapeVar representing non-neg valued variable in a tensor shape

[GitBox]

tqchen commented on issue #4684: [Arith] add ShapeVar representing non-neg 
valued variable in a tensor shape
URL: https://github.com/apache/incubator-tvm/pull/4684#issuecomment-574380538
 
 
   We can ask other's thoughts about it(perhaps list a few candidates and send 
an rfc?). TIndex seems to be a reasonable name, although it is a bit ambiguous 
because people need to guess what does T mean). 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-tvm] masahi edited a comment on issue #4497: [WIP] [Relay] Add a PyTorch to Relay Parser

[GitBox]

masahi edited a comment on issue #4497: [WIP] [Relay] Add a PyTorch to Relay 
Parser
URL: https://github.com/apache/incubator-tvm/pull/4497#issuecomment-574076780
 
 
   @alexwong I tried your PR locally. With pytorch v1.3 it works, but they 
introduced big change in 
[#28408](https://github.com/pytorch/pytorch/pull/28408) and 
[#28409](https://github.com/pytorch/pytorch/pull/28409), and it broke your PR 
(I may be wrong about which PR broke it). Below is how their IR looks like for 
resnet18 now. My pytorch version is '1.5.0a0+0dbd5c0' (the output of 
torch.__version__). 
   
   UPDATE: The breaking change might have come from 
[#25089](https://github.com/pytorch/pytorch/pull/25089). Not sure if this 
commit was part of v1.3 release.
   ```
   graph(%self.1 : __torch__.torch.nn.modules.module.___torch_mangle_66.Module,
 %input.1 : Float(1, 3, 224, 224)):
 %1452 : __torch__.torch.nn.modules.module.___torch_mangle_65.Module = 
prim::GetAttr[name="fc"](%self.1)
 %1449 : __torch__.torch.nn.modules.module.___torch_mangle_64.Module = 
prim::GetAttr[name="avgpool"](%self.1)
 %1448 : __torch__.torch.nn.modules.module.___torch_mangle_63.Module = 
prim::GetAttr[name="layer4"](%self.1)
 %1402 : __torch__.torch.nn.modules.module.___torch_mangle_47.Module = 
prim::GetAttr[name="layer3"](%self.1)
 %1356 : __torch__.torch.nn.modules.module.___torch_mangle_31.Module = 
prim::GetAttr[name="layer2"](%self.1)
 %1310 : __torch__.torch.nn.modules.module.___torch_mangle_15.Module = 
prim::GetAttr[name="layer1"](%self.1)
 %1273 : __torch__.torch.nn.modules.module.___torch_mangle_2.Module = 
prim::GetAttr[name="maxpool"](%self.1)
 %1272 : __torch__.torch.nn.modules.module.___torch_mangle_1.Module = 
prim::GetAttr[name="relu"](%self.1)
 %1271 : __torch__.torch.nn.modules.module.___torch_mangle_0.Module = 
prim::GetAttr[name="bn1"](%self.1)
 %1265 : __torch__.torch.nn.modules.module.Module = 
prim::GetAttr[name="conv1"](%self.1)
 %1528 : Tensor = prim::CallMethod[name="forward"](%1265, %input.1)
 %1529 : Tensor = prim::CallMethod[name="forward"](%1271, %1528)
 %1530 : Tensor = prim::CallMethod[name="forward"](%1272, %1529)
 %1531 : Tensor = prim::CallMethod[name="forward"](%1273, %1530)
 %1532 : Tensor = prim::CallMethod[name="forward"](%1310, %1531)
 %1533 : Tensor = prim::CallMethod[name="forward"](%1356, %1532)
 %1534 : Tensor = prim::CallMethod[name="forward"](%1402, %1533)
 %1535 : Tensor = prim::CallMethod[name="forward"](%1448, %1534)
 %1536 : Tensor = prim::CallMethod[name="forward"](%1449, %1535)
 %1182 : int = prim::Constant[value=1]() # 
/home/masa/anaconda3/lib/python3.7/site-packages/torchvision-0.5.0a0+07cbb46-py3.7-linux-x86_64.egg/torchvision/models/resnet.py:210:0
 %1183 : int = prim::Constant[value=-1]() # 
/home/masa/anaconda3/lib/python3.7/site-packages/torchvision-0.5.0a0+07cbb46-py3.7-linux-x86_64.egg/torchvision/models/resnet.py:210:0
 %input : Float(1, 512) = aten::flatten(%1536, %1182, %1183) # 
/home/masa/anaconda3/lib/python3.7/site-packages/torchvision-0.5.0a0+07cbb46-py3.7-linux-x86_64.egg/torchvision/models/resnet.py:210:0
 %1537 : Tensor = prim::CallMethod[name="forward"](%1452, %input)
 return (%1537)
   ```
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[incubator-tvm] branch master updated (d7d2a9b -> 3f2abfb)

[zhic]

This is an automated email from the ASF dual-hosted git repository.

zhic pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-tvm.git.


from d7d2a9b  [REFACTOR][IR] Initialize Unified IR Pass Infra. (#4702)
 add 3f2abfb  [relay] Relay annotation and partitioning for external 
compilers (#4570)

No new revisions were added by this update.

Summary of changes:
 include/tvm/relay/attrs/annotation.h|  13 +
 include/tvm/relay/op_attr_types.h   |   6 +-
 include/tvm/relay/transform.h   |   8 +
 python/tvm/relay/op/annotation/annotation.py|  41 +++
 python/tvm/relay/transform.py   |  12 +
 src/relay/backend/contrib/dnnl/codegen.cc   |   4 +-
 src/relay/op/annotation/annotation.cc   |  50 +++
 src/relay/pass/fuse_ops.cc  |   3 +-
 src/relay/pass/partition_graph.cc   | 386 +
 tests/python/relay/test_pass_partition_graph.py | 434 
 10 files changed, 950 insertions(+), 7 deletions(-)
 create mode 100644 src/relay/pass/partition_graph.cc
 create mode 100644 tests/python/relay/test_pass_partition_graph.py

[GitHub] [incubator-tvm] zhiics commented on issue #4570: [relay] Relay annotation and partitioning for external compilers

[GitBox]

zhiics commented on issue #4570: [relay] Relay annotation and partitioning for 
external compilers
URL: https://github.com/apache/incubator-tvm/pull/4570#issuecomment-574338792
 
 
   Thanks @tqchen @masahi @comaniac 


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-tvm] zhiics merged pull request #4570: [relay] Relay annotation and partitioning for external compilers

[GitBox]

zhiics merged pull request #4570: [relay] Relay annotation and partitioning for 
external compilers
URL: https://github.com/apache/incubator-tvm/pull/4570
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-tvm] comaniac commented on issue #4602: [Docs] Bring Your Own Codegen Guide -- Part 1

[GitBox]

comaniac commented on issue #4602: [Docs] Bring Your Own Codegen Guide -- Part 1
URL: https://github.com/apache/incubator-tvm/pull/4602#issuecomment-574337997
 
 
   Should we merge this part first if no one wants to further review it for now?
   cc @masahi @tqchen 


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-tvm] huajsj edited a comment on issue #4703: [VTA] Support network which have no unique operator as start/stop name for graph pack.

[GitBox]

huajsj edited a comment on issue #4703: [VTA] Support network which have no 
unique operator as start/stop name for graph pack.
URL: https://github.com/apache/incubator-tvm/pull/4703#issuecomment-574325187
 
 
   Hi Liangfu, Thanks for the comments, the reason here to use operator name 
and indices is to first compatible with existing implementation like resnet18 
example, secondly try to make the logic to
   be more readable for that from my point operator name is more friendly  than 
indices only. please kindly let me know how you think.
   
   Regards
   Hua  
   
   > This is a favorable feature, since there could be more than one operator 
with the given name.
   > 
   > I wonder whether we can bring the unique name of the layer, instead of the 
name of the operator, to define the subgraph. I think using indices of the 
named operators are less intuitive, comparing to unique names of the layers.
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-tvm] zhiics commented on issue #4693: [runtime][refactor] Unify vm and interpreter objects

[GitBox]

zhiics commented on issue #4693: [runtime][refactor] Unify vm and interpreter 
objects
URL: https://github.com/apache/incubator-tvm/pull/4693#issuecomment-574328741
 
 
   ping reviewers.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-tvm] huajsj commented on issue #4703: [VTA] Support network which have no unique operator as start/stop name for graph pack.

[GitBox]

huajsj commented on issue #4703: [VTA] Support network which have no unique 
operator as start/stop name for graph pack.
URL: https://github.com/apache/incubator-tvm/pull/4703#issuecomment-574325187
 
 
   Hi Liangfu, Thanks for the comments, the reason here to use operator name 
and indices is to first compatible with existing implementation like resnet18 
example, secondly try to make the logic to
   be more readable for that operator name is more friendly  than indices only.
   
   Regards
   Hua  
   
   > This is a favorable feature, since there could be more than one operator 
with the given name.
   > 
   > I wonder whether we can bring the unique name of the layer, instead of the 
name of the operator, to define the subgraph. I think using indices of the 
named operators are less intuitive, comparing to unique names of the layers.
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-tvm] tqchen opened a new pull request #4705: [REFACTOR][IR] Polish ir/type

[GitBox]

tqchen opened a new pull request #4705: [REFACTOR][IR] Polish ir/type
URL: https://github.com/apache/incubator-tvm/pull/4705
 
 
   - Use consistent constructor style to construct objects.
   - Move env_func to ir as it is mainly used to construct IRs.
   - Make docs consistent.
   cc @zhiics @wweic @jroesch 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-tvm] tqchen commented on a change in pull request #4684: [Arith] add ShapeVar representing non-neg valued variable in a tensor shape

[GitBox]

tqchen commented on a change in pull request #4684: [Arith] add ShapeVar 
representing non-neg valued variable in a tensor shape
URL: https://github.com/apache/incubator-tvm/pull/4684#discussion_r366473075
 
 

 ##
 File path: include/tvm/expr.h
 ##
 @@ -115,6 +117,45 @@ class Var : public PrimExpr {
   using ContainerType = VarNode;
 };
 
+class ShapeVar;
+/*!
+ * \brief A variable node represent a tensor shape size,
+ * whose value must be non-negative.
+ */
+class ShapeVarNode : public VarNode {
+ public:
+  /*! \brief constructor */
+  ShapeVarNode() {}
+  ShapeVarNode(DataType dtype, std::string name_hint);
+
+  static constexpr const char* _type_key = "ShapeVar";
+  TVM_DECLARE_FINAL_OBJECT_INFO(ShapeVarNode, VarNode);
+};
+
+/*! \brief a named variable represents a tensor shape size */
+class ShapeVar : public Var {
+ public:
+  explicit ShapeVar(ObjectPtr n) : Var(n) {}
+  TVM_DLL explicit ShapeVar(std::string name_hint = "s",
 
 Review comment:
   Document the constructor


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-tvm] tqchen commented on issue #4684: [Arith] add ShapeVar representing non-neg valued variable in a tensor shape

[GitBox]

tqchen commented on issue #4684: [Arith] add ShapeVar representing non-neg 
valued variable in a tensor shape
URL: https://github.com/apache/incubator-tvm/pull/4684#issuecomment-574285396
 
 
   I don't have a better idea for name. Indeed shape could indicate a tuple 
rather than an integer. 
   
   We could potentially rename the relay's shape template variable for clarity 
as it is really type var. But we could think about a better name.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-tvm] tqchen merged pull request #4702: [REFACTOR][IR] Initialize Unified IR Pass Infra

[GitBox]

tqchen merged pull request #4702: [REFACTOR][IR] Initialize Unified IR Pass 
Infra
URL: https://github.com/apache/incubator-tvm/pull/4702
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[incubator-tvm] branch master updated (edc3674 -> d7d2a9b)

[tqchen]

This is an automated email from the ASF dual-hosted git repository.

tqchen pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-tvm.git.


from edc3674  [REFACTOR][IR] Move error.h into ir (#4701)
 add d7d2a9b  [REFACTOR][IR] Initialize Unified IR Pass Infra. (#4702)

No new revisions were added by this update.

Summary of changes:
 include/tvm/ir/transform.h | 330 +
 include/tvm/relay/transform.h  | 311 +--
 .../pass/pass_manager.cc => ir/transform.cc}   | 202 +++--
 src/relay/ir/transform.cc  | 170 +++
 4 files changed, 549 insertions(+), 464 deletions(-)
 create mode 100644 include/tvm/ir/transform.h
 rename src/{relay/pass/pass_manager.cc => ir/transform.cc} (69%)
 create mode 100644 src/relay/ir/transform.cc

[GitHub] [incubator-tvm] FrozenGene commented on a change in pull request #4543: [FRONTEND][TFLITE] Add support for TFLite_Detection_PostProcess

[GitBox]

FrozenGene commented on a change in pull request #4543: [FRONTEND][TFLITE] Add 
support for TFLite_Detection_PostProcess
URL: https://github.com/apache/incubator-tvm/pull/4543#discussion_r366438454
 
 

 ##
 File path: tests/python/frontend/tflite/test_forward.py
 ##
 @@ -1113,6 +1113,49 @@ def test_forward_fully_connected():
 _test_fully_connected([5, 1, 1, 150], [150, 100], [100])
 
 
+###
+# Custom Operators
+# ---
+
+def test_detection_postprocess():
+tf_model_file = tf_testing.get_workload_official(
+"http://download.tensorflow.org/models/object_detection/;
+"ssd_mobilenet_v2_quantized_300x300_coco_2019_01_03.tar.gz",
 
 Review comment:
   do you mean the issue of quantized rounding here? 
https://github.com/apache/incubator-tvm/pull/3900#discussion_r334334418


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-tvm] mbarrett97 commented on a change in pull request #4543: [FRONTEND][TFLITE] Add support for TFLite_Detection_PostProcess

[GitBox]

mbarrett97 commented on a change in pull request #4543: [FRONTEND][TFLITE] Add 
support for TFLite_Detection_PostProcess
URL: https://github.com/apache/incubator-tvm/pull/4543#discussion_r366410943
 
 

 ##
 File path: tests/python/frontend/tflite/test_forward.py
 ##
 @@ -1113,6 +1113,49 @@ def test_forward_fully_connected():
 _test_fully_connected([5, 1, 1, 150], [150, 100], [100])
 
 
+###
+# Custom Operators
+# ---
+
+def test_detection_postprocess():
+tf_model_file = tf_testing.get_workload_official(
+"http://download.tensorflow.org/models/object_detection/;
+"ssd_mobilenet_v2_quantized_300x300_coco_2019_01_03.tar.gz",
 
 Review comment:
   The test looks non-trivial to add because quite a small difference in the 
convolutional part of the network can result in significant changes to the 
ordering of the output tensor (eg. we might see at different detection at the 
cut off threshold). I'm not sure what the best way is to proceed, do you have 
any thoughts?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-tvm] inadob commented on issue #4704: [Relay][Frontend][TFLite] Add parser support for arg_min_max

[GitBox]

inadob commented on issue #4704: [Relay][Frontend][TFLite] Add parser support 
for arg_min_max
URL: https://github.com/apache/incubator-tvm/pull/4704#issuecomment-574223969
 
 
   can you please review this PR @anijain2305 @FrozenGene


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-tvm] inadob opened a new pull request #4704: [Relay][Frontend][TFLite] Add parser support for arg_min_max

[GitBox]

inadob opened a new pull request #4704: [Relay][Frontend][TFLite] Add parser 
support for arg_min_max
URL: https://github.com/apache/incubator-tvm/pull/4704
 
 
   * this implementation supports only the case when the axis is a scalar
   * tflite 1.13 removes all dims of size 1, Relay doesn't do this
   * WARNING: every newer version of tflite > 1.13 needs keepdims=TRUE
 in the corresponding Relay operation. If there is a way to check the 
tflite version that was used to 
 convert the model passed to the parser, I will be happy to implement that.
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-tvm] masahi commented on issue #4497: [WIP] [Relay] Add a PyTorch to Relay Parser

[GitBox]

masahi commented on issue #4497: [WIP] [Relay] Add a PyTorch to Relay Parser
URL: https://github.com/apache/incubator-tvm/pull/4497#issuecomment-574076780
 
 
   @alexwong I tried your PR locally. With pytorch v1.3 it works, but they 
introduced big change in 
[#28408](https://github.com/pytorch/pytorch/pull/28408) and 
[#28409](https://github.com/pytorch/pytorch/pull/28409), and it broke your PR 
(I may be wrong about which PR broke it). Below is how their IR looks like for 
resnet18 now. My pytorch version is '1.5.0a0+0dbd5c0' (the output of 
torch.__version__). 
   ```
   graph(%self.1 : __torch__.torch.nn.modules.module.___torch_mangle_66.Module,
 %input.1 : Float(1, 3, 224, 224)):
 %1452 : __torch__.torch.nn.modules.module.___torch_mangle_65.Module = 
prim::GetAttr[name="fc"](%self.1)
 %1449 : __torch__.torch.nn.modules.module.___torch_mangle_64.Module = 
prim::GetAttr[name="avgpool"](%self.1)
 %1448 : __torch__.torch.nn.modules.module.___torch_mangle_63.Module = 
prim::GetAttr[name="layer4"](%self.1)
 %1402 : __torch__.torch.nn.modules.module.___torch_mangle_47.Module = 
prim::GetAttr[name="layer3"](%self.1)
 %1356 : __torch__.torch.nn.modules.module.___torch_mangle_31.Module = 
prim::GetAttr[name="layer2"](%self.1)
 %1310 : __torch__.torch.nn.modules.module.___torch_mangle_15.Module = 
prim::GetAttr[name="layer1"](%self.1)
 %1273 : __torch__.torch.nn.modules.module.___torch_mangle_2.Module = 
prim::GetAttr[name="maxpool"](%self.1)
 %1272 : __torch__.torch.nn.modules.module.___torch_mangle_1.Module = 
prim::GetAttr[name="relu"](%self.1)
 %1271 : __torch__.torch.nn.modules.module.___torch_mangle_0.Module = 
prim::GetAttr[name="bn1"](%self.1)
 %1265 : __torch__.torch.nn.modules.module.Module = 
prim::GetAttr[name="conv1"](%self.1)
 %1528 : Tensor = prim::CallMethod[name="forward"](%1265, %input.1)
 %1529 : Tensor = prim::CallMethod[name="forward"](%1271, %1528)
 %1530 : Tensor = prim::CallMethod[name="forward"](%1272, %1529)
 %1531 : Tensor = prim::CallMethod[name="forward"](%1273, %1530)
 %1532 : Tensor = prim::CallMethod[name="forward"](%1310, %1531)
 %1533 : Tensor = prim::CallMethod[name="forward"](%1356, %1532)
 %1534 : Tensor = prim::CallMethod[name="forward"](%1402, %1533)
 %1535 : Tensor = prim::CallMethod[name="forward"](%1448, %1534)
 %1536 : Tensor = prim::CallMethod[name="forward"](%1449, %1535)
 %1182 : int = prim::Constant[value=1]() # 
/home/masa/anaconda3/lib/python3.7/site-packages/torchvision-0.5.0a0+07cbb46-py3.7-linux-x86_64.egg/torchvision/models/resnet.py:210:0
 %1183 : int = prim::Constant[value=-1]() # 
/home/masa/anaconda3/lib/python3.7/site-packages/torchvision-0.5.0a0+07cbb46-py3.7-linux-x86_64.egg/torchvision/models/resnet.py:210:0
 %input : Float(1, 512) = aten::flatten(%1536, %1182, %1183) # 
/home/masa/anaconda3/lib/python3.7/site-packages/torchvision-0.5.0a0+07cbb46-py3.7-linux-x86_64.egg/torchvision/models/resnet.py:210:0
 %1537 : Tensor = prim::CallMethod[name="forward"](%1452, %input)
 return (%1537)
   ```
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services