[GitHub] [incubator-tvm] CoinCheung commented on issue #4361: I think the dependency of topi is broken

[GitBox]

CoinCheung commented on issue #4361: I think the dependency of topi is broken
URL: https://github.com/apache/incubator-tvm/issues/4361#issuecomment-555379475
 
 
   Thanks, the method of exporting `PYTHONPATH` is really good. I am closing 
this.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-tvm] CoinCheung closed issue #4361: I think the dependency of topi is broken

[GitBox]

CoinCheung closed issue #4361: I think the dependency of topi is broken
URL: https://github.com/apache/incubator-tvm/issues/4361
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-tvm] jackwish commented on a change in pull request #4351: [QNN] Lowering for Depthwise Convolution.

[GitBox]

jackwish commented on a change in pull request #4351: [QNN] Lowering for 
Depthwise Convolution.
URL: https://github.com/apache/incubator-tvm/pull/4351#discussion_r347752773
 
 

 ##
 File path: src/relay/qnn/op/convolution.cc
 ##
 @@ -391,7 +526,20 @@ Expr Conv2DCombineTerms(const Expr& term1, const Expr& 
term2, const Expr& term3,
  * gives an opportunity to reuse alter_op_layout infrastructure.
  * 3) For dilated conv, in current lowering, we need dilated pool. So 
as
  * a workaround, we fall back to simpler lowering using int32 conv if
- * the conv is dilated. We fallback also in case of depthwise conv.
+ * the conv is dilated. We fallback also in case of grouped conv.
+ *
+ *   For depthwise, we can similarly unroll the computation. The intial 
compute is as follows
+ *   wehere cm = channel_multiplier
+ *
+ *   Qc(n, oc, oh, ow) = Sigma(r, s) (Qw(oc/m, oc%/m, r, s) - zp_w)
 
 Review comment:
   Suggesting to rewrite related formulas with tex: [block 
example](https://stackoverflow.com/questions/10081633/using-displaymath-directives-in-docstrings-formulas),
 or [inline 
style](https://github.com/apache/incubator-tvm/blob/master/topi/python/topi/nn/dense.py#L66).


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-tvm] jackwish commented on a change in pull request #4351: [QNN] Lowering for Depthwise Convolution.

[GitBox]

jackwish commented on a change in pull request #4351: [QNN] Lowering for 
Depthwise Convolution.
URL: https://github.com/apache/incubator-tvm/pull/4351#discussion_r347756284
 
 

 ##
 File path: src/relay/qnn/op/convolution.cc
 ##
 @@ -417,23 +565,33 @@ Expr QnnConv2DCanonicalize(const Attrs& attrs, const 
Array& new_args,
 param->kernel_layout == "HWOI")
   << "qnn.conv2d supports only OIHW/HWIO/HWOI kernel data layout.";
 
-  int batch_size, in_channels, out_channels, kernel_h, kernel_w;
-  std::tie(batch_size, in_channels, out_channels, kernel_h, kernel_w) =
+  int batch_size, in_channels, out_channels, kernel_h, kernel_w, 
channel_multiplier;
+  std::tie(batch_size, in_channels, out_channels, kernel_h, kernel_w, 
channel_multiplier) =
   GetWorkload(arg_types, param);
 
-  // Fallback to int32 conv if there is dilation or depthwise conv2d
+  // Fallback to int32 conv if there is dilation or grouped conv2d
+
   CHECK_EQ(param->dilation.size(), 2) << "qnn.conv2d only supports 2D 
dilation";
   auto dilation_h = get_const_int(param->dilation[0]);
   auto dilation_w = get_const_int(param->dilation[1]);
-  if (dilation_h != 1 || dilation_w != 1 || param->groups != 1) {
+  if (dilation_h != 1 || dilation_w != 1 || (param->groups != 1 && 
!is_depthwise(param))) {
 return Conv2DFallBack(data, weight, param);
+  } else if (is_depthwise(param)) {
+CHECK_NE(channel_multiplier, -1);
+auto padded_data = Conv2DPadInput(data, param);
+auto term1 = Conv2DFirstTerm(padded_data, weight, param);
+auto term2 =
+DepthwiseConv2DSecondTerm(padded_data, param, kernel_h, kernel_w, 
channel_multiplier);
+auto term3 = DepthwiseConv2DThirdTerm(weight, param, out_channels, 
channel_multiplier);
+auto term4 = DepthwiseConv2DFourthTerm(param, kernel_h, kernel_w);
+return Conv2DCombineTerms(term1, term2, term3, term4, param);
   }
 
   auto padded_data = Conv2DPadInput(data, param);
   auto term1 = Conv2DFirstTerm(padded_data, weight, param);
   auto term2 = Conv2DSecondTerm(padded_data, param, kernel_h, kernel_w, 
out_channels);
-  auto term3 = Conv2DThirdTerm(weight, param, batch_size, out_channels);
-  auto term4 = Conv2DFourthTerm(param, batch_size, in_channels, kernel_h, 
kernel_w);
+  auto term3 = Conv2DThirdTerm(weight, param, out_channels);
+  auto term4 = Conv2DFourthTerm(param, in_channels, kernel_h, kernel_w);
 
 Review comment:
   That is interesting, why removing batch semantic?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-tvm] jackwish commented on a change in pull request #4351: [QNN] Lowering for Depthwise Convolution.

[GitBox]

jackwish commented on a change in pull request #4351: [QNN] Lowering for 
Depthwise Convolution.
URL: https://github.com/apache/incubator-tvm/pull/4351#discussion_r347745090
 
 

 ##
 File path: src/relay/qnn/op/convolution.cc
 ##
 @@ -84,26 +87,38 @@ WorkloadType GetWorkload(const Array& 
arg_types, const QnnConv
 
   const auto kernel_shape = get_shape(arg_types[1]);
   int out_channels, kernel_h, kernel_w;
+  int channel_multiplier = -1;
   if (param->kernel_layout == "OIHW") {
 out_channels = get_const_int(kernel_shape[0]);
 kernel_h = get_const_int(kernel_shape[2]);
 kernel_w = get_const_int(kernel_shape[3]);
+if (is_depthwise(param)) {
 
 Review comment:
   What about using a variable to hold this?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-tvm] apivovarov commented on a change in pull request #4370: [WIP] Relay visualization: exporter + visualizer

[GitBox]

apivovarov commented on a change in pull request #4370: [WIP] Relay 
visualization: exporter + visualizer
URL: https://github.com/apache/incubator-tvm/pull/4370#discussion_r347760145
 
 

 ##
 File path: python/tvm/relay/analysis.py
 ##
 @@ -22,11 +22,14 @@
 """
 from . import _analysis
 from . import _make
-from .expr import Expr
+from .expr import Expr, Function, Var, Call, TupleGetItem
 
 Review comment:
   Sort imports


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-tvm] apivovarov opened a new pull request #4373: Compare all outputs in TFLite test_forward_ssd_mobilenet_v1

[GitBox]

apivovarov opened a new pull request #4373: Compare all outputs in TFLite 
test_forward_ssd_mobilenet_v1
URL: https://github.com/apache/incubator-tvm/pull/4373
 
 
   `forward_ssd_mobilenet_v1` has two raw outputs. Currently we only test 
output[0].
   This PR makes the test to compare both outputs.
   
   I also needed to double Absolute tolerance `atol` to make output comparison 
`assert_allclose` to pass.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-tvm] lsy643 opened a new pull request #4372: [Relay][Frontend][TF] Fix slice when begin or size is not Const

[GitBox]

lsy643 opened a new pull request #4372: [Relay][Frontend][TF] Fix slice when 
begin or size is not Const
URL: https://github.com/apache/incubator-tvm/pull/4372
 
 
   For Relay Frontend of Tensorflow, def _slice can not handle begin or size is 
not Const. _infer_value_simulated is used to handle these conditions
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-tvm] tmoreau89 commented on a change in pull request #4369: Add cusparse for sparse dense

[GitBox]

tmoreau89 commented on a change in pull request #4369: Add cusparse for sparse 
dense
URL: https://github.com/apache/incubator-tvm/pull/4369#discussion_r347745819
 
 

 ##
 File path: python/tvm/contrib/cusparse.py
 ##
 @@ -0,0 +1,54 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""External function interface to cuBLAS libraries."""
+from __future__ import absolute_import as _abs
+
+from .. import api as _api
+from .. import intrin as _intrin
+
+def matmul(lhs, rhs_data, rhs_indices, rhs_indptr, transb=False):
+"""Create an extern op that compute matrix mult of 
+   lhs and rhs with cuSPARSE
+
+Parameters
+--
+lhs : Tensor
+The left matrix operand
+rhs is a sparse matrix in csr format
 
 Review comment:
   There is no `rhs` argument in the func above


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-tvm] tqchen commented on issue #4344: [ThreadPool] Solve thread transitions issue

[GitBox]

tqchen commented on issue #4344: [ThreadPool] Solve thread transitions issue
URL: https://github.com/apache/incubator-tvm/pull/4344#issuecomment-555332772
 
 
   I think there seems to be two concerns:
   - Whether we want to use cpu as worker0
   - Whether we want bind the master thread to CPU0
   
   The problematic case seems can be resolved by not binding the master thread, 
but still include as part of the worker thread.  By default, we bind the rest 
of the threads but not the master one.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-tvm] minminsun commented on a change in pull request #4353: [Perf] Enhance cudnn and cublas backend and enable TensorCore

[GitBox]

minminsun commented on a change in pull request #4353: [Perf] Enhance cudnn and 
cublas backend and enable TensorCore
URL: https://github.com/apache/incubator-tvm/pull/4353#discussion_r347728939
 
 

 ##
 File path: src/runtime/contrib/cublas/cublas.cc
 ##
 @@ -124,35 +169,203 @@ struct CublasDgemmBatchOp {
   }
 };
 
+// Check cublas supported mix-precision computation type and return computeType
+bool CheckMixPrecisionType(DLDataType in_dtype, DLDataType out_dtype, bool 
int_support = true) {
+  if (int_support && TypeMatch(out_dtype, kDLInt, 32)) {
+return TypeMatch(in_dtype, kDLInt, 8);
+  } else if (TypeMatch(out_dtype, kDLFloat, 32)) {
+return TypeMatch(in_dtype, kDLInt, 8) ||
+   TypeMatch(in_dtype, kDLFloat, 16);
+  } else {
+return false;
+  }
+}
+
+inline void CallGemmEx(TVMArgs args, TVMRetValue *ret, cublasHandle_t hdl) {
+  DLTensor *A = args[0];
+  DLTensor *B = args[1];
+  DLTensor *C = args[2];
+  bool transa = args[3];
+  bool transb = args[4];
+  CHECK_EQ(A->ndim, 2);
+  CHECK_EQ(B->ndim, 2);
+  CHECK_EQ(C->ndim, 2);
+
+  CHECK_EQ(ElementStride(A), 1);
+  CHECK_EQ(ElementStride(B), 1);
+  CHECK_EQ(ElementStride(C), 1);
+
+  CHECK(TypeEqual(A->dtype, B->dtype));
+
+  // C can never be transposed.
+  CHECK(!IsInPlaceTransposed(C));
+
+  // Reversed strides indicates an in-place transpose operation.
+  transa = IsInPlaceTransposed(A) ? !transa : transa;
+  transb = IsInPlaceTransposed(B) ? !transb : transb;
+
+  CHECK(CheckMixPrecisionType(A->dtype, C->dtype)) << "Unsupported data type";
+  CHECK(!TypeMatch(A->dtype, kDLInt, 8) ||
+  ColumnStride(A) % 4 == 0) << "leading dimension must divide 4 for int8 
gemm";
+  CHECK(!TypeMatch(B->dtype, kDLInt, 8) ||
+  ColumnStride(B) % 4 == 0) << "leading dimension must divide 4 for int8 
gemm";
+  double alpha = args.size() > 5 ? args[5] : 1.0;
+  double beta = args.size() > 6 ? args[6] : 0.0;
+
+  cudaDataType_t cuda_in_type = GetCudaDataType(A->dtype);
+  cudaDataType_t cuda_out_type = GetCudaDataType(C->dtype);
+  cublasGemmAlgo_t algo = CUBLAS_GEMM_DEFAULT;
+  void *alpha_ptr = nullptr, *beta_ptr = nullptr;
+  auto alpha_int = static_cast(alpha);
+  auto beta_int = static_cast(beta);
+  auto alpha_float = static_cast(alpha);
+  auto beta_float = static_cast(beta);
+  if (C->dtype.code == kDLInt) {
+alpha_ptr = _int;
+beta_ptr = _int;
+  } else if (C->dtype.code == kDLFloat) {
+alpha_ptr = _float;
+beta_ptr = _float;
+  }
+
+  auto A_data = reinterpret_cast(static_cast(A->data) + 
A->byte_offset);
+  auto B_data = reinterpret_cast(static_cast(B->data) + 
B->byte_offset);
+  auto C_data = reinterpret_cast(static_cast(C->data) + 
C->byte_offset);
+
+  CHECK_CUBLAS_ERROR(cublasGemmEx(hdl,
+ BooleanToTranspose(transb),
+ BooleanToTranspose(transa),
+ ColumnCount(B, transb),
+ RowCount(A, transa),
+ ColumnCount(A, transa),
+ alpha_ptr,
+ B_data, cuda_in_type, ColumnStride(B),
+ A_data, cuda_in_type, ColumnStride(A),
+ beta_ptr,
+ C_data, cuda_out_type, ColumnStride(C),
+ cuda_out_type, algo));
 
 Review comment:
   OK, no problem since you are aware of the precision difference. It was just 
a reminder of the risk of precision drop.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-tvm] FrozenGene commented on issue #4344: [ThreadPool] Solve thread transitions issue

[GitBox]

FrozenGene commented on issue #4344: [ThreadPool] Solve thread transitions issue
URL: https://github.com/apache/incubator-tvm/pull/4344#issuecomment-555329941
 
 
   Hmm...I think Auto Tuning and OpenCV + TVM is very common and many users 
will forget this things, so I prefer it be `false` by default. However, we 
could add comment on `exclude_worker_0` and suggest to be `true` when to use 
single thread. Also open to listen more people's comment. @vinx13 @tqchen guys 
could tag anyone interested.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[incubator-tvm] branch master updated (f1d6f33 -> 26eb405)

[tqchen]

This is an automated email from the ASF dual-hosted git repository.

tqchen pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-tvm.git.


from f1d6f33  add rule for clean (#4364)
 add 26eb405  [Relay tests] AlterOpLayout - Temporary attr update (#4357)

No new revisions were added by this update.

Summary of changes:
 include/tvm/relay/op.h  |6 +
 python/tvm/relay/op/op.py   |   10 +
 python/tvm/relay/testing/__init__.py|1 +
 python/tvm/relay/testing/temp_op_attr.py|   63 ++
 src/relay/ir/op.cc  |   28 +-
 tests/python/relay/test_ir_op.py|   47 ++
 tests/python/relay/test_op_qnn_conv2d.py| 1008 ---
 tests/python/relay/test_op_qnn_dense.py |   33 +-
 tests/python/relay/test_pass_alter_op_layout.py |  243 +++---
 tests/python/relay/test_pass_legalize.py|   40 +-
 tests/python/relay/test_pass_qnn_legalize.py|   21 +-
 11 files changed, 829 insertions(+), 671 deletions(-)
 create mode 100644 python/tvm/relay/testing/temp_op_attr.py

[GitHub] [incubator-tvm] tqchen merged pull request #4357: [Relay tests] Temporary Attr Update for Order-Independent Testing

[GitBox]

tqchen merged pull request #4357: [Relay tests] Temporary Attr Update for 
Order-Independent Testing
URL: https://github.com/apache/incubator-tvm/pull/4357
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-tvm] apivovarov commented on issue #4320: Fix TFLite RESHAPE assert

[GitBox]

apivovarov commented on issue #4320: Fix TFLite RESHAPE assert
URL: https://github.com/apache/incubator-tvm/pull/4320#issuecomment-555319420
 
 
   @FrozenGene All checks have passed - Can you approve?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-tvm] apivovarov opened a new issue #4371: ImportError: cannot import name 'bilinear_sample_nchw'

[GitBox]

apivovarov opened a new issue #4371: ImportError: cannot import name 
'bilinear_sample_nchw'
URL: https://github.com/apache/incubator-tvm/issues/4371
 
 
   Same issue, tvm, nnvm and topi were installed after the build
   ```
   ~/tvm/python$ sudo python3 setup.py install
   ~/tvm/nnvm/python$ sudo python3 setup.py install
   ~/tvm/topi/python$ sudo python3 setup.py install
   ```
   
   But model compilation fails
   ```
   
   Traceback (most recent call last):
   
 File "./compile.py", line 79, in 
   import tvm.relay.testing.tf as tf_testing
   
 File 
"/usr/local/lib/python3.6/dist-packages/tvm-0.6.dev0-py3.6-linux-x86_64.egg/tvm/relay/__init__.py",
 line 27, in 
   from . import expr_functor
   
 File 
"/usr/local/lib/python3.6/dist-packages/tvm-0.6.dev0-py3.6-linux-x86_64.egg/tvm/relay/expr_functor.py",
 line 24, in 
   from .op import Op
   
 File 
"/usr/local/lib/python3.6/dist-packages/tvm-0.6.dev0-py3.6-linux-x86_64.egg/tvm/relay/op/__init__.py",
 line 20, in 
   from .op import get, register, register_schedule, register_compute, 
register_gradient, \
   
 File 
"/usr/local/lib/python3.6/dist-packages/tvm-0.6.dev0-py3.6-linux-x86_64.egg/tvm/relay/op/op.py",
 line 19, in 
   import topi
   
 File 
"/usr/local/lib/python3.6/dist-packages/topi-0.6.dev0-py3.6.egg/topi/__init__.py",
 line 43, in 
   from . import nn
   
 File 
"/usr/local/lib/python3.6/dist-packages/topi-0.6.dev0-py3.6.egg/topi/nn/__init__.py",
 line 23, in 
   from .deformable_conv2d import *
   
 File 
"/usr/local/lib/python3.6/dist-packages/topi-0.6.dev0-py3.6.egg/topi/nn/deformable_conv2d.py",
 line 23, in 
   from ..cpp.image import bilinear_sample_nchw
   
   ImportError: cannot import name 'bilinear_sample_nchw'
   ```
   
   If tvm, nnvm and topi were installed using setup.py we do not need to set 
`LD_LIBRARY_PATH`
   
   Anyone looking at the issue?
   
   Discussion: 
https://discuss.tvm.ai/t/cannot-import-name-bilinear-sample-nchw/2510


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-tvm] yzhliu commented on issue #4368: [tutorial][benchmark] nnvm -> relay

[GitBox]

yzhliu commented on issue #4368: [tutorial][benchmark] nnvm -> relay
URL: https://github.com/apache/incubator-tvm/pull/4368#issuecomment-555317160
 
 
   sorry I need to verify rpc once I get a device tomorrow.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-tvm] anijain2305 commented on issue #4357: [Relay tests] Temporary Attr Update for Order-Independent Testing

[GitBox]

anijain2305 commented on issue #4357: [Relay tests] Temporary Attr Update for 
Order-Independent Testing
URL: https://github.com/apache/incubator-tvm/pull/4357#issuecomment-555316537
 
 
   @tqchen comments addressed


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-tvm] FrozenGene commented on issue #4344: [ThreadPool] Solve thread transitions issue

[GitBox]

FrozenGene commented on issue #4344: [ThreadPool] Solve thread transitions issue
URL: https://github.com/apache/incubator-tvm/pull/4344#issuecomment-555313133
 
 
   ping @vinx13 @yidawang @tqchen have any comments?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-tvm] comaniac edited a comment on issue #4280: [TVM][RUNTIME] A minimum example to generate external library wrappers for DSOModule

[GitBox]

comaniac edited a comment on issue #4280: [TVM][RUNTIME] A minimum example to 
generate external library wrappers for DSOModule
URL: https://github.com/apache/incubator-tvm/pull/4280#issuecomment-555303900
 
 
   @tqchen we have moved the example JSON runtime as well as the other two 
examplese to `apps/ext_runtime` as suggested. Please see the following code 
snippet (in `test_rt.py`:520-526) for the use case of the example JSON runtime:
   
   ```python
   ext_lib = ext_json_rt.create_json_rt(subgraph_json, "")
   lib.import_module(ext_lib)
   lib.export_library('external.so')
   
   # load module for execution.
   lib = tvm.module.load('external.so')
   mod = tvm.contrib.graph_runtime.create(graph_json, lib, tvm.cpu(0))
   ```
   
   Please note that we need to change [get function 
call](https://github.com/apache/incubator-tvm/pull/4280/files#diff-5243f3909dfa484d85d3d8471f259169R399)
 in the graph runtime in order to search the implementation in the imported 
module. Please also help check if it's ok to do so.
   
   Thanks.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-tvm] cylinbao opened a new pull request #4369: Add cusparse for sparse dense

[GitBox]

cylinbao opened a new pull request #4369: Add cusparse for sparse dense
URL: https://github.com/apache/incubator-tvm/pull/4369
 
 
   Add a simple gpu scheduling and cusparse support for topi.nn.sparse_dense() 
operator.
   
   Please review. Thanks!
   @ZihengJiang @tmoreau89 @yuluny2 


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-tvm] vinx13 commented on issue #4295: [Relay][Quantize] Integrate data-aware calibration into quantization

[GitBox]

vinx13 commented on issue #4295: [Relay][Quantize] Integrate data-aware 
calibration into quantization
URL: https://github.com/apache/incubator-tvm/pull/4295#issuecomment-555281603
 
 
   @yzhliu comments addressed


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-tvm] masahi commented on issue #4368: [tutorial][benchmark] nnvm -> relay

[GitBox]

masahi commented on issue #4368: [tutorial][benchmark] nnvm -> relay
URL: https://github.com/apache/incubator-tvm/pull/4368#issuecomment-555262197
 
 
   Shouldn't we update get_network() too?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-tvm] yzhliu opened a new pull request #4368: [tutorial][benchmark] nnvm -> relay

[GitBox]

yzhliu opened a new pull request #4368: [tutorial][benchmark] nnvm -> relay
URL: https://github.com/apache/incubator-tvm/pull/4368
 
 
   @tqchen @masahi 


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-tvm] yzhliu commented on issue #4365: reminding message for TVM_REGISTER_NODE_TYPE

[GitBox]

yzhliu commented on issue #4365: reminding message for TVM_REGISTER_NODE_TYPE
URL: https://github.com/apache/incubator-tvm/pull/4365#issuecomment-555253086
 
 
   Thanks guys @hcho3 @anijain2305 @tqchen @junrushao1994 


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[incubator-tvm] branch master updated (3914be5 -> 7efb72e)

[liuyizhi]

This is an automated email from the ASF dual-hosted git repository.

liuyizhi pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-tvm.git.


from 3914be5  fix Android and OpenCL docker install (#4363)
 add 7efb72e  reminding message for TVM_REGISTER_NODE_TYPE (#4365)

No new revisions were added by this update.

Summary of changes:
 src/runtime/object.cc | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

[GitHub] [incubator-tvm] yzhliu merged pull request #4365: reminding message for TVM_REGISTER_NODE_TYPE

[GitBox]

yzhliu merged pull request #4365: reminding message for TVM_REGISTER_NODE_TYPE
URL: https://github.com/apache/incubator-tvm/pull/4365
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-tvm] anijain2305 commented on a change in pull request #4357: [Relay tests] Temporary Attr Update for Order-Independent Testing

[GitBox]

anijain2305 commented on a change in pull request #4357: [Relay tests] 
Temporary Attr Update for Order-Independent Testing
URL: https://github.com/apache/incubator-tvm/pull/4357#discussion_r347653451
 
 

 ##
 File path: python/tvm/relay/testing/temp_op_attr.py
 ##
 @@ -0,0 +1,37 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+""" Defines a TempOpAttr class that allows temporarily changing an attr of the
+operator to allow unit testing. This is useful for AlterOpLayout and Legalize
+tests."""
+
+from tvm import relay
+
+class TempOpAttr(object):
+""" Temporarily changes the attr of an op. """
+def __init__(self, op_name, attr_key):
 
 Review comment:
   Good idea! Will update soon.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-tvm] tqchen commented on a change in pull request #4357: [Relay tests] Temporary Attr Update for Order-Independent Testing

[GitBox]

tqchen commented on a change in pull request #4357: [Relay tests] Temporary 
Attr Update for Order-Independent Testing
URL: https://github.com/apache/incubator-tvm/pull/4357#discussion_r347649506
 
 

 ##
 File path: python/tvm/relay/testing/temp_op_attr.py
 ##
 @@ -0,0 +1,37 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+""" Defines a TempOpAttr class that allows temporarily changing an attr of the
+operator to allow unit testing. This is useful for AlterOpLayout and Legalize
+tests."""
+
+from tvm import relay
+
+class TempOpAttr(object):
+""" Temporarily changes the attr of an op. """
+def __init__(self, op_name, attr_key):
 
 Review comment:
   document all the arguments 
https://docs.tvm.ai/contribute/document.html#document-python
   
   ```python
   """
   Parameters
   --
   
   Examples
   --
   """
   ```


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-tvm] tqchen commented on a change in pull request #4357: [Relay tests] Temporary Attr Update for Order-Independent Testing

[GitBox]

tqchen commented on a change in pull request #4357: [Relay tests] Temporary 
Attr Update for Order-Independent Testing
URL: https://github.com/apache/incubator-tvm/pull/4357#discussion_r347649668
 
 

 ##
 File path: src/relay/ir/op.cc
 ##
 @@ -152,6 +169,15 @@ TVM_REGISTER_API("relay.op._OpSetAttr")
 reg.set_attr(attr_name, value, plevel);
   });
 
+TVM_REGISTER_API("relay.op._OpResetAttr")
+.set_body([](TVMArgs args, TVMRetValue* rv) {
+Op op = args[0];
+std::string attr_name = args[1];
+auto& reg =
+OpRegistry::Registry()->__REGISTER_OR_GET__(op->name).set_name();
 
 Review comment:
   .set_name() is not necessary.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-tvm] tqchen commented on a change in pull request #4357: [Relay tests] Temporary Attr Update for Order-Independent Testing

[GitBox]

tqchen commented on a change in pull request #4357: [Relay tests] Temporary 
Attr Update for Order-Independent Testing
URL: https://github.com/apache/incubator-tvm/pull/4357#discussion_r347649770
 
 

 ##
 File path: tests/python/relay/test_ir_op.py
 ##
 @@ -27,6 +27,32 @@ def test(x):
 assert log_op.get_attr("ftest") is None
 assert relay.op.get("exp").get_attr("ftest")(1) == 2
 
+def test_op_reset_attr():
+""" Tests reset_attr functionality. """
+def add1(x):
+return x + 1
+
+def add2(x):
+return x + 2
+
+# Register fadd1 and fadd2 attributes.
+relay.op.register("exp", "fadd1", add1)
+relay.op.register("log", "fadd1", add1)
+relay.op.register("log", "fadd2", add2)
+
+# Reset log fadd1 attr.
+log_op = relay.op.get("log")
+log_op.reset_attr("fadd1")
 
 Review comment:
   Let us also test TempOpAttr


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-tvm] anijain2305 opened a new pull request #4367: [Tests] Dump duration of Pytests.

[GitBox]

anijain2305 opened a new pull request #4367: [Tests] Dump duration of Pytests.
URL: https://github.com/apache/incubator-tvm/pull/4367
 
 
   Dumping test durations can help identify any misbehaving tests.
   
   @tqchen @icemelon9 @wweic @broune 


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-tvm] tqchen commented on issue #4346: [Runtime] Make ADTObject POD container type

[GitBox]

tqchen commented on issue #4346: [Runtime] Make ADTObject POD container type
URL: https://github.com/apache/incubator-tvm/pull/4346#issuecomment-555223817
 
 
   Interesting, sorry I didn't realize that due to injection the child class 
was not complete. It is fine to turn into runtime constant then, but we still 
need the static assert. An alternative way to do that is to introduce an 
auxiliary traits class that defines these constants and derived types, and then 
make use of these constants inside the functions(which will then know the 
complete type)


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-tvm] anijain2305 commented on issue #4357: [Relay tests] Temporary Attr Update for Order-Independent Testing

[GitBox]

anijain2305 commented on issue #4357: [Relay tests] Temporary Attr Update for 
Order-Independent Testing
URL: https://github.com/apache/incubator-tvm/pull/4357#issuecomment-555221188
 
 
   @tqchen Can you take a look now? Thanks for your patience.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-tvm] tqchen commented on a change in pull request #4346: [Runtime] Make ADTObject POD container type

[GitBox]

tqchen commented on a change in pull request #4346: [Runtime] Make ADTObject 
POD container type
URL: https://github.com/apache/incubator-tvm/pull/4346#discussion_r347622247
 
 

 ##
 File path: include/tvm/runtime/container.h
 ##
 @@ -0,0 +1,220 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file tvm/runtime/container.h
+ * \brief Common POD(plain old data) container types.
+ */
+#ifndef TVM_RUNTIME_CONTAINER_H_
+#define TVM_RUNTIME_CONTAINER_H_
+#include 
+#include 
+#include 
+#include 
+
+namespace tvm {
+namespace runtime {
+
+/**
+ * @brief Base template for classes with array like memory layout.
+ *
+ *It provides general methods to access the memory. The memory
+ *layout is ArrayType + [ElemType]. The alignment of ArrayType
+ *and ElemType is handled by the memory allocator.
+ *
+ * @tparam ArrayType
+ * @tparam ElemType
+ */
+template 
+class InplaceArrayBase {
+ public:
+  /**
+   * @brief Initialize the elements in the array.
+   */
+  void Init() {
 
 Review comment:
   No we won't support dynamic growth of the object, but perhaps will need to 
support push_back when capacity() and size() are different. 
   
   Ideally, InplaceArrayBase will be a base class for all containers(Array, ADT 
String) :)


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-tvm] anwang2009 opened a new pull request #4366: add benchmark log format doc

[GitBox]

anwang2009 opened a new pull request #4366: add benchmark log format doc
URL: https://github.com/apache/incubator-tvm/pull/4366
 
 
   Adds benchmark log format as discussed in 
   
   https://discuss.tvm.ai/t/rfc-benchmark-performance-log-format/4610
   
   https://github.com/apache/incubator-tvm/issues/4304
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-tvm] wweic commented on issue #4346: [Runtime] Make ADTObject POD container type

[GitBox]

wweic commented on issue #4346: [Runtime] Make ADTObject POD container type
URL: https://github.com/apache/incubator-tvm/pull/4346#issuecomment-555198920
 
 
   @tqchen The c++ compiler in CI does not support defining static constexpr 
from sizeof of ADTObj, since its definition is not complete. I saw the error 
last time so I changed it to runtime constant. Do you want to upgrade the 
compiler version or I'll change it back to runtime constant?
   
   
   /workspace/include/tvm/runtime/container.h:138:40: 
   error: invalid application of 'sizeof' to an incomplete type 
'tvm::runtime::ADTObj'
   
 static constexpr size_t kDataStart = sizeof(ArrayType);
   
   /workspace/include/tvm/runtime/container.h:163:7: note: 
   definition of 'tvm::runtime::ADTObj' is not complete until the closing '}'
   
   class ADTObj : public Object, public InplaceArrayBase {
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-tvm] wweic commented on a change in pull request #4346: [Runtime] Make ADTObject POD container type

[GitBox]

wweic commented on a change in pull request #4346: [Runtime] Make ADTObject POD 
container type
URL: https://github.com/apache/incubator-tvm/pull/4346#discussion_r347595585
 
 

 ##
 File path: include/tvm/runtime/container.h
 ##
 @@ -0,0 +1,220 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file tvm/runtime/container.h
+ * \brief Common POD(plain old data) container types.
+ */
+#ifndef TVM_RUNTIME_CONTAINER_H_
+#define TVM_RUNTIME_CONTAINER_H_
+#include 
+#include 
+#include 
+#include 
+
+namespace tvm {
+namespace runtime {
+
+/**
+ * @brief Base template for classes with array like memory layout.
+ *
+ *It provides general methods to access the memory. The memory
+ *layout is ArrayType + [ElemType]. The alignment of ArrayType
+ *and ElemType is handled by the memory allocator.
+ *
+ * @tparam ArrayType
+ * @tparam ElemType
+ */
+template 
+class InplaceArrayBase {
+ public:
+  /**
+   * @brief Initialize the elements in the array.
+   */
+  void Init() {
 
 Review comment:
   Oh, I was not aware of this use case. I think it would be great if we can 
just provide `std::vector` interface(`operator[]`, `push_back`, `size`) in 
`InplaceArrayBase`. 
   
   To support dynamic insertion, it can query child class with `size()` and 
`capacity()` in `push_back` implementation. And I won't support dynamically 
grow the underlying storage since the actual memory management is done by child 
class, it's also difficult to grow this memory. 


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-tvm] yzhliu opened a new pull request #4365: reminding message for TVM_REGISTER_NODE_TYPE

[GitBox]

yzhliu opened a new pull request #4365: reminding message for 
TVM_REGISTER_NODE_TYPE
URL: https://github.com/apache/incubator-tvm/pull/4365
 
 
   per https://discuss.tvm.ai/t/i-spent-5hr-today-add-a-new-node/4754/9
   @tqchen @hcho3 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-tvm] vmiheer commented on issue #4364: add rule for clean

[GitBox]

vmiheer commented on issue #4364: add rule for clean
URL: https://github.com/apache/incubator-tvm/pull/4364#issuecomment-555179355
 
 
   cc: @tqchen please review.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-tvm] yzhliu merged pull request #4363: [Docker] Fix TVM folder name for installing on Android and OpenCL

[GitBox]

yzhliu merged pull request #4363: [Docker] Fix TVM folder name for installing 
on Android and OpenCL
URL: https://github.com/apache/incubator-tvm/pull/4363
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[incubator-tvm] branch master updated (00521fa -> 3914be5)

[liuyizhi]

This is an automated email from the ASF dual-hosted git repository.

liuyizhi pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-tvm.git.


from 00521fa  [SOURCE] Add ASF header to __init__.py files (#4359)
 add 3914be5  fix Android and OpenCL docker install (#4363)

No new revisions were added by this update.

Summary of changes:
 docker/Dockerfile.demo_android | 2 +-
 docker/Dockerfile.demo_opencl  | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

[GitHub] [incubator-tvm] yzhliu commented on issue #4363: [Docker] Fix TVM folder name for installing on Android and OpenCL

[GitBox]

yzhliu commented on issue #4363: [Docker] Fix TVM folder name for installing on 
Android and OpenCL
URL: https://github.com/apache/incubator-tvm/pull/4363#issuecomment-555169679
 
 
   Thanks @comaniac @tqchen 


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-tvm] tqchen commented on issue #4362: About model conversion to TVM

[GitBox]

tqchen commented on issue #4362: About model conversion to TVM 
URL: https://github.com/apache/incubator-tvm/issues/4362#issuecomment-555154249
 
 
   Thanks for bringing this up. Please open threads on https://discuss.tvm.ai/ 
:)


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-tvm] tqchen closed issue #4362: About model conversion to TVM

[GitBox]

tqchen closed issue #4362: About model conversion to TVM 
URL: https://github.com/apache/incubator-tvm/issues/4362
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-tvm] Hzfengsy commented on a change in pull request #4353: [Perf] Enhance cudnn and cublas backend and enable TensorCore

[GitBox]

Hzfengsy commented on a change in pull request #4353: [Perf] Enhance cudnn and 
cublas backend and enable TensorCore
URL: https://github.com/apache/incubator-tvm/pull/4353#discussion_r347546810
 
 

 ##
 File path: src/runtime/contrib/cublas/cublas.cc
 ##
 @@ -124,35 +169,203 @@ struct CublasDgemmBatchOp {
   }
 };
 
+// Check cublas supported mix-precision computation type and return computeType
+bool CheckMixPrecisionType(DLDataType in_dtype, DLDataType out_dtype, bool 
int_support = true) {
+  if (int_support && TypeMatch(out_dtype, kDLInt, 32)) {
+return TypeMatch(in_dtype, kDLInt, 8);
+  } else if (TypeMatch(out_dtype, kDLFloat, 32)) {
+return TypeMatch(in_dtype, kDLInt, 8) ||
+   TypeMatch(in_dtype, kDLFloat, 16);
+  } else {
+return false;
+  }
+}
+
+inline void CallGemmEx(TVMArgs args, TVMRetValue *ret, cublasHandle_t hdl) {
+  DLTensor *A = args[0];
+  DLTensor *B = args[1];
+  DLTensor *C = args[2];
+  bool transa = args[3];
+  bool transb = args[4];
+  CHECK_EQ(A->ndim, 2);
+  CHECK_EQ(B->ndim, 2);
+  CHECK_EQ(C->ndim, 2);
+
+  CHECK_EQ(ElementStride(A), 1);
+  CHECK_EQ(ElementStride(B), 1);
+  CHECK_EQ(ElementStride(C), 1);
+
+  CHECK(TypeEqual(A->dtype, B->dtype));
+
+  // C can never be transposed.
+  CHECK(!IsInPlaceTransposed(C));
+
+  // Reversed strides indicates an in-place transpose operation.
+  transa = IsInPlaceTransposed(A) ? !transa : transa;
+  transb = IsInPlaceTransposed(B) ? !transb : transb;
+
+  CHECK(CheckMixPrecisionType(A->dtype, C->dtype)) << "Unsupported data type";
+  CHECK(!TypeMatch(A->dtype, kDLInt, 8) ||
+  ColumnStride(A) % 4 == 0) << "leading dimension must divide 4 for int8 
gemm";
+  CHECK(!TypeMatch(B->dtype, kDLInt, 8) ||
+  ColumnStride(B) % 4 == 0) << "leading dimension must divide 4 for int8 
gemm";
+  double alpha = args.size() > 5 ? args[5] : 1.0;
+  double beta = args.size() > 6 ? args[6] : 0.0;
+
+  cudaDataType_t cuda_in_type = GetCudaDataType(A->dtype);
+  cudaDataType_t cuda_out_type = GetCudaDataType(C->dtype);
+  cublasGemmAlgo_t algo = CUBLAS_GEMM_DEFAULT;
+  void *alpha_ptr = nullptr, *beta_ptr = nullptr;
+  auto alpha_int = static_cast(alpha);
+  auto beta_int = static_cast(beta);
+  auto alpha_float = static_cast(alpha);
+  auto beta_float = static_cast(beta);
+  if (C->dtype.code == kDLInt) {
+alpha_ptr = _int;
+beta_ptr = _int;
+  } else if (C->dtype.code == kDLFloat) {
+alpha_ptr = _float;
+beta_ptr = _float;
+  }
+
+  auto A_data = reinterpret_cast(static_cast(A->data) + 
A->byte_offset);
+  auto B_data = reinterpret_cast(static_cast(B->data) + 
B->byte_offset);
+  auto C_data = reinterpret_cast(static_cast(C->data) + 
C->byte_offset);
+
+  CHECK_CUBLAS_ERROR(cublasGemmEx(hdl,
+ BooleanToTranspose(transb),
+ BooleanToTranspose(transa),
+ ColumnCount(B, transb),
+ RowCount(A, transa),
+ ColumnCount(A, transa),
+ alpha_ptr,
+ B_data, cuda_in_type, ColumnStride(B),
+ A_data, cuda_in_type, ColumnStride(A),
+ beta_ptr,
+ C_data, cuda_out_type, ColumnStride(C),
+ cuda_out_type, algo));
 
 Review comment:
   Thank you! Here are two things worth noting:
   - If we have fp16 input and output, we will use `CublasHgemm` rather than 
`cublasGemmEx`.
   - Currently, I can not any int8 output support in `cublasGemmEx`. Please see 
details at https://docs.nvidia.com/cuda/cublas/index.html#cublas-GemmEx
   
   Hence, I don't think there is a precision problem for now. Please figure out 
if I have missed anything.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-tvm] tqchen commented on issue #4304: [RFC] Benchmark Performance Log Format WIP

[GitBox]

tqchen commented on issue #4304: [RFC] Benchmark Performance Log Format WIP
URL: https://github.com/apache/incubator-tvm/issues/4304#issuecomment-555150731
 
 
   Awesome, let us bring it to the official document by sending a pr to 
docs/dev/benchmark.rst


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-tvm] tqchen commented on a change in pull request #4346: [Runtime] Make ADTObject POD container type

[GitBox]

tqchen commented on a change in pull request #4346: [Runtime] Make ADTObject 
POD container type
URL: https://github.com/apache/incubator-tvm/pull/4346#discussion_r347543087
 
 

 ##
 File path: include/tvm/runtime/memory.h
 ##
 @@ -73,6 +74,26 @@ class ObjAllocatorBase {
 ptr->deleter_ = Handler::Deleter();
 return ObjectPtr(ptr);
   }
+
+  /*!
+   * \tparam T The type to be allocated.
+   * \tparam ElemType The type to array element.
+   * \tparam Args The constructor signature.
+   * \param num_elems The number of array elements.
+   * \param args The arguments.
+   */
+  template
+  inline ObjectPtr make_array(size_t num_elems, Args&&... args) {
 
 Review comment:
   make_inplace_array_object


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-tvm] tqchen commented on a change in pull request #4346: [Runtime] Make ADTObject POD container type

[GitBox]

tqchen commented on a change in pull request #4346: [Runtime] Make ADTObject 
POD container type
URL: https://github.com/apache/incubator-tvm/pull/4346#discussion_r347542358
 
 

 ##
 File path: include/tvm/runtime/container.h
 ##
 @@ -0,0 +1,220 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file tvm/runtime/container.h
+ * \brief Common POD(plain old data) container types.
+ */
+#ifndef TVM_RUNTIME_CONTAINER_H_
+#define TVM_RUNTIME_CONTAINER_H_
+#include 
+#include 
+#include 
+#include 
+
+namespace tvm {
+namespace runtime {
+
+/**
+ * @brief Base template for classes with array like memory layout.
+ *
+ *It provides general methods to access the memory. The memory
+ *layout is ArrayType + [ElemType]. The alignment of ArrayType
+ *and ElemType is handled by the memory allocator.
+ *
+ * @tparam ArrayType
+ * @tparam ElemType
+ */
+template 
+class InplaceArrayBase {
+ public:
+  /**
+   * @brief Initialize the elements in the array.
+   */
+  void Init() {
+CHECK_EQ(sizeof(ArrayType) % alignof(ElemType), 0);
+for (size_t i = 0; i < Self()->size(); ++i) {
+  void* field_ptr = AddressOf(i);
+  new (field_ptr) ElemType();
+}
+  }
+
+  /**
+   * @brief Initialize the elements in the array.
+   *
+   * @tparam Iterator Iterator type of the array.
+   * @param begin The begin iterator.
+   * @param end The end iterator.
+   */
+  template 
+  void Init(Iterator begin, Iterator end) {
+CHECK_EQ(sizeof(ArrayType) % alignof(ElemType), 0);
+ArrayType* self = Self();
+size_t num_elems = std::distance(begin, end);
+if (num_elems != self->size()) {
+  LOG(FATAL)
+  << "Number of initializer values does not match number of 
elements\n";
+}
+auto it = begin;
+for (size_t i = 0; i < num_elems; ++i) {
+  void* field_ptr = AddressOf(i);
+  new (field_ptr) ElemType(*it);
+  ++it;
+}
+  }
+
+  /**
+   * @brief Initialize the elements in the array.
+   *
+   * @param init The initializer list of elements.
+   */
+  void Init(std::initializer_list init) {
+CHECK_EQ(sizeof(ArrayType) % alignof(ElemType), 0);
+Init(init.begin(), init.end());
+  }
+
+  /*!
+   * \brief Access element at index
+   * \param idx The index of the element.
+   * \return Reference to ElemType at the index.
+   */
+  ElemType& operator[](size_t idx) const {
+size_t size = Self()->size();
+if (idx > size) {
+  LOG(FATAL) << "Index " << idx << " out of bounds " << size << "\n";
+}
+return *(reinterpret_cast(AddressOf(idx)));
+  }
+
+  /**
+   * @brief Destroy the Inplace Array Base object
+   */
+  virtual ~InplaceArrayBase() {
+if (!IsPOD()) {
+  size_t size = Self()->size();
+  for (size_t i = 0; i < size; ++i) {
+ElemType* fp = reinterpret_cast(AddressOf(i));
+fp->ElemType::~ElemType();
+  }
+}
+  }
+
+ private:
+  /**
+   * @brief Check if the ElemType is Plain Old Data.
+   *
+   * @return If ElemType is POD.
+   */
+  inline bool IsPOD() const {
+return std::is_standard_layout::value &&
+   std::is_trivial::value;
+  }
+
+  /**
+   * @brief Return the self object for the array.
+   *
+   * @return Pointer to ArrayType.
+   */
+  inline ArrayType* Self() const {
+return static_cast(const_cast(this));
+  }
+
+  /**
+   * @brief Return the raw pointer to the element at idx.
+   *
+   * @param idx The index of the element.
+   * @return Raw pointer to the element.
+   */
+  void* AddressOf(size_t idx) const {
+const size_t kDataStart = sizeof(ArrayType);
+ArrayType* self = Self();
 
 Review comment:
   You need to run the alignment static assertions somewhere to make sure the 
kDataStart gives you the right alignment


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-tvm] tqchen commented on a change in pull request #4346: [Runtime] Make ADTObject POD container type

[GitBox]

tqchen commented on a change in pull request #4346: [Runtime] Make ADTObject 
POD container type
URL: https://github.com/apache/incubator-tvm/pull/4346#discussion_r347541893
 
 

 ##
 File path: include/tvm/runtime/container.h
 ##
 @@ -0,0 +1,220 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file tvm/runtime/container.h
+ * \brief Common POD(plain old data) container types.
+ */
+#ifndef TVM_RUNTIME_CONTAINER_H_
+#define TVM_RUNTIME_CONTAINER_H_
+#include 
+#include 
+#include 
+#include 
+
+namespace tvm {
+namespace runtime {
+
+/**
+ * @brief Base template for classes with array like memory layout.
+ *
+ *It provides general methods to access the memory. The memory
+ *layout is ArrayType + [ElemType]. The alignment of ArrayType
+ *and ElemType is handled by the memory allocator.
+ *
+ * @tparam ArrayType
+ * @tparam ElemType
+ */
+template 
+class InplaceArrayBase {
+ public:
+  /**
+   * @brief Initialize the elements in the array.
+   */
+  void Init() {
 
 Review comment:
   Think a bit more, what functions do you need to support a dynamic shaped 
array? where you might need to insert elements  :) ?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-tvm] tqchen commented on a change in pull request #4346: [Runtime] Make ADTObject POD container type

[GitBox]

tqchen commented on a change in pull request #4346: [Runtime] Make ADTObject 
POD container type
URL: https://github.com/apache/incubator-tvm/pull/4346#discussion_r347540754
 
 

 ##
 File path: include/tvm/runtime/container.h
 ##
 @@ -0,0 +1,220 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file tvm/runtime/container.h
+ * \brief Common POD(plain old data) container types.
+ */
+#ifndef TVM_RUNTIME_CONTAINER_H_
+#define TVM_RUNTIME_CONTAINER_H_
+#include 
+#include 
+#include 
+#include 
+
+namespace tvm {
+namespace runtime {
+
+/**
+ * @brief Base template for classes with array like memory layout.
+ *
+ *It provides general methods to access the memory. The memory
+ *layout is ArrayType + [ElemType]. The alignment of ArrayType
+ *and ElemType is handled by the memory allocator.
+ *
+ * @tparam ArrayType
+ * @tparam ElemType
+ */
+template 
+class InplaceArrayBase {
+ public:
+  /**
+   * @brief Initialize the elements in the array.
+   */
+  void Init() {
+CHECK_EQ(sizeof(ArrayType) % alignof(ElemType), 0);
+for (size_t i = 0; i < Self()->size(); ++i) {
+  void* field_ptr = AddressOf(i);
+  new (field_ptr) ElemType();
+}
+  }
+
+  /**
+   * @brief Initialize the elements in the array.
+   *
+   * @tparam Iterator Iterator type of the array.
+   * @param begin The begin iterator.
+   * @param end The end iterator.
+   */
+  template 
+  void Init(Iterator begin, Iterator end) {
+CHECK_EQ(sizeof(ArrayType) % alignof(ElemType), 0);
+ArrayType* self = Self();
+size_t num_elems = std::distance(begin, end);
+if (num_elems != self->size()) {
+  LOG(FATAL)
+  << "Number of initializer values does not match number of 
elements\n";
+}
+auto it = begin;
+for (size_t i = 0; i < num_elems; ++i) {
+  void* field_ptr = AddressOf(i);
+  new (field_ptr) ElemType(*it);
+  ++it;
+}
+  }
+
+  /**
+   * @brief Initialize the elements in the array.
+   *
+   * @param init The initializer list of elements.
+   */
+  void Init(std::initializer_list init) {
+CHECK_EQ(sizeof(ArrayType) % alignof(ElemType), 0);
+Init(init.begin(), init.end());
+  }
+
+  /*!
+   * \brief Access element at index
+   * \param idx The index of the element.
+   * \return Reference to ElemType at the index.
+   */
+  ElemType& operator[](size_t idx) const {
+size_t size = Self()->size();
+if (idx > size) {
+  LOG(FATAL) << "Index " << idx << " out of bounds " << size << "\n";
+}
+return *(reinterpret_cast(AddressOf(idx)));
+  }
+
+  /**
+   * @brief Destroy the Inplace Array Base object
+   */
+  virtual ~InplaceArrayBase() {
+if (!IsPOD()) {
+  size_t size = Self()->size();
+  for (size_t i = 0; i < size; ++i) {
+ElemType* fp = reinterpret_cast(AddressOf(i));
+fp->ElemType::~ElemType();
+  }
+}
+  }
+
+ private:
+  /**
+   * @brief Check if the ElemType is Plain Old Data.
+   *
+   * @return If ElemType is POD.
+   */
+  inline bool IsPOD() const {
+return std::is_standard_layout::value &&
+   std::is_trivial::value;
+  }
+
+  /**
+   * @brief Return the self object for the array.
+   *
+   * @return Pointer to ArrayType.
+   */
+  inline ArrayType* Self() const {
+return static_cast(const_cast(this));
+  }
+
+  /**
+   * @brief Return the raw pointer to the element at idx.
 
 Review comment:
   \brief(just to be consistent with rest of the codebase).


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-tvm] tqchen commented on a change in pull request #4346: [Runtime] Make ADTObject POD container type

[GitBox]

tqchen commented on a change in pull request #4346: [Runtime] Make ADTObject 
POD container type
URL: https://github.com/apache/incubator-tvm/pull/4346#discussion_r347541168
 
 

 ##
 File path: include/tvm/runtime/container.h
 ##
 @@ -0,0 +1,220 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file tvm/runtime/container.h
+ * \brief Common POD(plain old data) container types.
+ */
+#ifndef TVM_RUNTIME_CONTAINER_H_
+#define TVM_RUNTIME_CONTAINER_H_
+#include 
+#include 
+#include 
+#include 
+
+namespace tvm {
+namespace runtime {
+
+/**
+ * @brief Base template for classes with array like memory layout.
+ *
+ *It provides general methods to access the memory. The memory
+ *layout is ArrayType + [ElemType]. The alignment of ArrayType
+ *and ElemType is handled by the memory allocator.
+ *
+ * @tparam ArrayType
+ * @tparam ElemType
+ */
+template 
+class InplaceArrayBase {
+ public:
+  /**
+   * @brief Initialize the elements in the array.
+   */
+  void Init() {
+CHECK_EQ(sizeof(ArrayType) % alignof(ElemType), 0);
+for (size_t i = 0; i < Self()->size(); ++i) {
+  void* field_ptr = AddressOf(i);
+  new (field_ptr) ElemType();
+}
+  }
+
+  /**
+   * @brief Initialize the elements in the array.
+   *
+   * @tparam Iterator Iterator type of the array.
+   * @param begin The begin iterator.
+   * @param end The end iterator.
+   */
+  template 
+  void Init(Iterator begin, Iterator end) {
+CHECK_EQ(sizeof(ArrayType) % alignof(ElemType), 0);
+ArrayType* self = Self();
+size_t num_elems = std::distance(begin, end);
+if (num_elems != self->size()) {
+  LOG(FATAL)
+  << "Number of initializer values does not match number of 
elements\n";
+}
+auto it = begin;
+for (size_t i = 0; i < num_elems; ++i) {
+  void* field_ptr = AddressOf(i);
+  new (field_ptr) ElemType(*it);
+  ++it;
+}
+  }
+
+  /**
+   * @brief Initialize the elements in the array.
+   *
+   * @param init The initializer list of elements.
+   */
+  void Init(std::initializer_list init) {
+CHECK_EQ(sizeof(ArrayType) % alignof(ElemType), 0);
+Init(init.begin(), init.end());
+  }
+
+  /*!
+   * \brief Access element at index
+   * \param idx The index of the element.
+   * \return Reference to ElemType at the index.
+   */
+  ElemType& operator[](size_t idx) const {
 
 Review comment:
   if is const reference, shall we return non-constant elem?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-tvm] tqchen edited a comment on issue #4280: [TVM][RUNTIME] A minimum example to generate external library wrappers for DSOModule

[GitBox]

tqchen edited a comment on issue #4280: [TVM][RUNTIME] A minimum example to 
generate external library wrappers for DSOModule
URL: https://github.com/apache/incubator-tvm/pull/4280#issuecomment-555146621
 
 
   Comments about json runtime. I would consider move the json runtime to 
   ```apps/extern_runtime``` to allow us to write examples even in python.  It 
would be great to see examples like:
   
   ```python
   mod = DSOModule()
   mod.import_module(JSONExampleModule())
   
   mod.export_library("xyz.so")
   
   mod = tvm.module.load("xyz.so")
   # run that he code that can uses functions in the JSONExampleModule
   ```
   
   The main goal is to provide readers an easy reference point about how to add 
their own runtimes.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-tvm] tqchen commented on a change in pull request #4280: [TVM][RUNTIME] A minimum example to generate external library wrappers for DSOModule

[GitBox]

tqchen commented on a change in pull request #4280: [TVM][RUNTIME] A minimum 
example to generate external library wrappers for DSOModule
URL: https://github.com/apache/incubator-tvm/pull/4280#discussion_r347537299
 
 

 ##
 File path: tests/cpp/external_runtime_test.cc
 ##
 @@ -0,0 +1,245 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file external_runtime_test.cc
+ * \brief Test an example runtime module to interpreting a json string.
+ */
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+
+#include 
+#include 
+#include 
+#include 
+#include 
+
+using tvm::runtime::Module;
+using tvm::runtime::ModuleNode;
+using tvm::runtime::NDArray;
+using tvm::runtime::Object;
+using tvm::runtime::ObjectPtr;
+using tvm::runtime::PackedFunc;
+using tvm::runtime::TVMArgs;
+using tvm::runtime::TVMArgsSetter;
+using tvm::runtime::TVMRetValue;
+
+void Add_(float* a, int len_a, float* b, int len_b, float* c) {
+  for (int i = 0; i < len_a * len_b; i++) {
+c[i] = a[i] + b[i];
+  }
+}
+
+int Add(TVMValue* value, int* type_code, int nargs) {
+  CHECK_EQ(nargs, 3U) << "Expect 3 args, but get " << nargs << "\n";
+  DLTensor* arg0 = static_cast(value[0].v_handle);
+  DLTensor* arg1 = static_cast(value[1].v_handle);
+  DLTensor* out = static_cast(value[2].v_handle);
+  Add_(static_cast(arg0->data), arg0->shape[0],
+   static_cast(arg1->data), arg1->shape[0],
+   static_cast(out->data));
+  return 0;
+}
+
+void Sub_(float* a, int len_a, float* b, int len_b, float* c) {
+  for (int i = 0; i < len_a * len_b; i++) {
+c[i] = a[i] - b[i];
+  }
+}
+
+int Sub(TVMValue* value, int* type_code, int nargs) {
+  CHECK_EQ(nargs, 3U) << "Expect 3 args, but get " << nargs << "\n";
+  DLTensor* arg0 = static_cast(value[0].v_handle);
+  DLTensor* arg1 = static_cast(value[1].v_handle);
+  DLTensor* out = static_cast(value[2].v_handle);
+  Sub_(static_cast(arg0->data), arg0->shape[0],
+   static_cast(arg1->data), arg1->shape[0],
+   static_cast(out->data));
+  return 0;
+}
+
+class ExampleJSonModule : public ModuleNode {
+ public:
+  PackedFunc GetFunction(const std::string& name,
+ const ObjectPtr& sptr_to_self) final {
+if (this->graph_.find(name) != this->graph_.end()) {
+  this->curr_subgraph_ = name;
+  return PackedFunc([sptr_to_self, this](TVMArgs args, TVMRetValue* rv) {
+for (auto i = 0; i < args.size(); ++i) {
+  NDArray arg = args[i];
+  this->data_entry_[i].CopyFrom(arg);
+}
+for (const auto& it : this->graph_[this->curr_subgraph_]) {
+  this->run(it.first, it.second);
+}
+*rv = data_entry_.back();
+  });
+} else {
+  LOG(FATAL) << "Unkown runtime type: " << name << "\n";
+  return PackedFunc();
+}
+  }
+
+  void run(int id, const std::vector& inputs) {
 
 Review comment:
   Run, as per GoogleC style(camelcase)


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-tvm] tqchen commented on a change in pull request #4280: [TVM][RUNTIME] A minimum example to generate external library wrappers for DSOModule

[GitBox]

tqchen commented on a change in pull request #4280: [TVM][RUNTIME] A minimum 
example to generate external library wrappers for DSOModule
URL: https://github.com/apache/incubator-tvm/pull/4280#discussion_r347536983
 
 

 ##
 File path: include/tvm/runtime/module.h
 ##
 @@ -111,7 +111,7 @@ class Module : public ObjectRef {
  *
  * \endcode
  */
-class ModuleNode : public Object {
+class TVM_DLL ModuleNode : public Object {
 
 Review comment:
   sgtm


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[incubator-tvm] branch master updated (a226973 -> 00521fa)

[tqchen]

This is an automated email from the ASF dual-hosted git repository.

tqchen pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-tvm.git.


from a226973  [Frontend]Add TensorFlow FloorMod (#4308)
 add 00521fa  [SOURCE] Add ASF header to __init__.py files (#4359)

No new revisions were added by this update.

Summary of changes:
 apps/extension/python/tvm_ext/__init__.py  |  17 +
 cmake/config.cmake |  17 +
 .../java/org/apache/tvm/rpc/SocketChannel.java |  19 +
 nnvm/python/nnvm/__init__.py   |  17 +
 nnvm/python/nnvm/_ctypes/__init__.py   |  17 +
 nnvm/python/nnvm/_cy2/__init__.py  |  17 +
 nnvm/python/nnvm/_cy3/__init__.py  |  17 +
 nnvm/python/nnvm/compiler/__init__.py  |  17 +
 nnvm/python/nnvm/frontend/__init__.py  |  17 +
 nnvm/python/nnvm/testing/__init__.py   |  17 +
 nnvm/python/nnvm/top/__init__.py   |  17 +
 nnvm/tests/lint/pylintrc   | 406 -
 .../python/frontend/caffe2/model_zoo/__init__.py   |  17 +
 .../python/frontend/coreml/model_zoo/__init__.py   |  17 +
 .../python/frontend/mxnet/model_zoo/__init__.py|  17 +
 .../python/frontend/onnx/model_zoo/__init__.py |  17 +
 python/tvm/contrib/debugger/__init__.py|  16 +
 python/tvm/micro/__init__.py   |  17 +
 tests/lint/add_asf_header.py   |   1 +
 tests/lint/check_file_type.py  |   1 -
 tests/lint/pylintrc|  17 +
 tests/lint/rat-excludes|   7 +-
 tests/python/frontend/caffe2/model_zoo/__init__.py |  17 +
 tests/python/frontend/coreml/model_zoo/__init__.py |  17 +
 tests/python/frontend/mxnet/model_zoo/__init__.py  |  17 +
 topi/python/topi/__init__.py   |  17 +
 topi/python/topi/arm_cpu/__init__.py   |  17 +
 topi/python/topi/bifrost/__init__.py   |  17 +
 topi/python/topi/cpp/__init__.py   |  17 +
 topi/python/topi/cpp/vision/__init__.py|  17 +
 topi/python/topi/cuda/__init__.py  |  17 +
 topi/python/topi/cuda/rcnn/__init__.py |  17 +
 topi/python/topi/cuda/ssd/__init__.py  |  17 +
 topi/python/topi/generic/__init__.py   |  17 +
 topi/python/topi/hls/__init__.py   |  17 +
 topi/python/topi/image/__init__.py |  17 +
 topi/python/topi/intel_graphics/__init__.py|  17 +
 topi/python/topi/mali/__init__.py  |  17 +
 topi/python/topi/nn/__init__.py|  17 +
 topi/python/topi/opengl/__init__.py|  17 +
 topi/python/topi/rocm/__init__.py  |  17 +
 topi/python/topi/sparse/__init__.py|  17 +
 topi/python/topi/testing/__init__.py   |  17 +
 topi/python/topi/vision/__init__.py|  17 +
 topi/python/topi/vision/rcnn/__init__.py   |  17 +
 topi/python/topi/vision/ssd/__init__.py|  17 +
 topi/python/topi/x86/__init__.py   |  17 +
 vta/apps/gemm/python/__init__.py   |  17 +
 vta/apps/tsim_example/python/__init__.py   |  17 +
 vta/python/vta/__init__.py |  17 +
 vta/python/vta/exec/__init__.py|  17 +
 vta/python/vta/testing/__init__.py |  17 +
 vta/python/vta/top/__init__.py |  17 +
 53 files changed, 836 insertions(+), 413 deletions(-)
 delete mode 100644 nnvm/tests/lint/pylintrc

[GitHub] [incubator-tvm] tqchen merged pull request #4359: [SOURCE] Add ASF header to __init__.py files

[GitBox]

tqchen merged pull request #4359: [SOURCE] Add ASF header to __init__.py files
URL: https://github.com/apache/incubator-tvm/pull/4359
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-tvm] comaniac commented on issue #4360: Build docker container failed

[GitBox]

comaniac commented on issue #4360: Build docker container failed
URL: https://github.com/apache/incubator-tvm/issues/4360#issuecomment-555135914
 
 
   PR filed #4363 


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-tvm] comaniac opened a new pull request #4363: [Docker] Fix TVM folder name for installing on Android and OpenCL

[GitBox]

comaniac opened a new pull request #4363: [Docker] Fix TVM folder name for 
installing on Android and OpenCL
URL: https://github.com/apache/incubator-tvm/pull/4363
 
 
   After cloning `incubator-tvm`, we need to rename the folder back to `tvm` to 
make the script working.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-tvm] junrushao1994 commented on issue #4361: I think the dependency of topi is broken

[GitBox]

junrushao1994 commented on issue #4361: I think the dependency of topi is broken
URL: https://github.com/apache/incubator-tvm/issues/4361#issuecomment-555135543
 
 
   Hmmm I am not sure, but I think probably the best way to use TVM is just 
setup PYTHONPATH, but not to pip install it, as it is not formally a pip 
package yet.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-tvm] OriAlpha commented on issue #4361: I think the dependency of topi is broken

[GitBox]

OriAlpha commented on issue #4361: I think the dependency of topi is broken
URL: https://github.com/apache/incubator-tvm/issues/4361#issuecomment-555010443
 
 
   ya it's true, but there is a temporary fix for that  
   
   export 
LD_LIBRARY_PATH=/home/user/.local/lib/python3.6/site-packages/topi-0.6.dev0-py3.6.egg/topi
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-tvm] OriAlpha opened a new issue #4362: About model conversion to TVM

[GitBox]

OriAlpha opened a new issue #4362: About model conversion to TVM 
URL: https://github.com/apache/incubator-tvm/issues/4362
 
 
   Hallo, 
   
   I have tried to compile model to TVM, and the model is consists of a 
1SimpleRNN layer and dense layers. but when I try to compile it gives out me 
error. 
   
   AttributeError:  has no attribute 
type_annotation
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-tvm] CoinCheung opened a new issue #4361: I think the dependency of topi is break

[GitBox]

CoinCheung opened a new issue #4361: I think the dependency of topi is break
URL: https://github.com/apache/incubator-tvm/issues/4361
 
 
   After building following the 
(documentation)[https://docs.tvm.ai/install/from_source.html]. I tried to 
import topi, and I got the error: 
   
   ```python
   import topi
   ```
   
   Traceback (most recent call last):
   
 File "", line 1, in 
   
 File "/root/build/tvm/topi/python/topi/__init__.py", line 26, in 
   from . import nn
   
 File "/root/build/tvm/topi/python/topi/nn/__init__.py", line 6, in 
   from .deformable_conv2d import *
   
 File "/root/build/tvm/topi/python/topi/nn/deformable_conv2d.py", line 23, 
in 
   from ..cpp.image import bilinear_sample_nchw
   
   ImportError: cannot import name 'bilinear_sample_nchw'
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-tvm] cszhz opened a new issue #4360: Build docker container failed

[GitBox]

cszhz opened a new issue #4360: Build docker container failed
URL: https://github.com/apache/incubator-tvm/issues/4360
 
 
   Build docker container failed with the file "docker\Dockerfile.demo_android"
   
   please add below in line 60:
   mv incubator-tvm tvm && \
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-tvm] minminsun commented on a change in pull request #4353: [Perf] Enhance cudnn and cublas backend and enable TensorCore

[GitBox]

minminsun commented on a change in pull request #4353: [Perf] Enhance cudnn and 
cublas backend and enable TensorCore
URL: https://github.com/apache/incubator-tvm/pull/4353#discussion_r347291047
 
 

 ##
 File path: src/runtime/contrib/cublas/cublas.cc
 ##
 @@ -124,35 +169,203 @@ struct CublasDgemmBatchOp {
   }
 };
 
+// Check cublas supported mix-precision computation type and return computeType
+bool CheckMixPrecisionType(DLDataType in_dtype, DLDataType out_dtype, bool 
int_support = true) {
+  if (int_support && TypeMatch(out_dtype, kDLInt, 32)) {
+return TypeMatch(in_dtype, kDLInt, 8);
+  } else if (TypeMatch(out_dtype, kDLFloat, 32)) {
+return TypeMatch(in_dtype, kDLInt, 8) ||
+   TypeMatch(in_dtype, kDLFloat, 16);
+  } else {
+return false;
+  }
+}
+
+inline void CallGemmEx(TVMArgs args, TVMRetValue *ret, cublasHandle_t hdl) {
+  DLTensor *A = args[0];
+  DLTensor *B = args[1];
+  DLTensor *C = args[2];
+  bool transa = args[3];
+  bool transb = args[4];
+  CHECK_EQ(A->ndim, 2);
+  CHECK_EQ(B->ndim, 2);
+  CHECK_EQ(C->ndim, 2);
+
+  CHECK_EQ(ElementStride(A), 1);
+  CHECK_EQ(ElementStride(B), 1);
+  CHECK_EQ(ElementStride(C), 1);
+
+  CHECK(TypeEqual(A->dtype, B->dtype));
+
+  // C can never be transposed.
+  CHECK(!IsInPlaceTransposed(C));
+
+  // Reversed strides indicates an in-place transpose operation.
+  transa = IsInPlaceTransposed(A) ? !transa : transa;
+  transb = IsInPlaceTransposed(B) ? !transb : transb;
+
+  CHECK(CheckMixPrecisionType(A->dtype, C->dtype)) << "Unsupported data type";
+  CHECK(!TypeMatch(A->dtype, kDLInt, 8) ||
+  ColumnStride(A) % 4 == 0) << "leading dimension must divide 4 for int8 
gemm";
+  CHECK(!TypeMatch(B->dtype, kDLInt, 8) ||
+  ColumnStride(B) % 4 == 0) << "leading dimension must divide 4 for int8 
gemm";
+  double alpha = args.size() > 5 ? args[5] : 1.0;
+  double beta = args.size() > 6 ? args[6] : 0.0;
+
+  cudaDataType_t cuda_in_type = GetCudaDataType(A->dtype);
+  cudaDataType_t cuda_out_type = GetCudaDataType(C->dtype);
+  cublasGemmAlgo_t algo = CUBLAS_GEMM_DEFAULT;
+  void *alpha_ptr = nullptr, *beta_ptr = nullptr;
+  auto alpha_int = static_cast(alpha);
+  auto beta_int = static_cast(beta);
+  auto alpha_float = static_cast(alpha);
+  auto beta_float = static_cast(beta);
+  if (C->dtype.code == kDLInt) {
+alpha_ptr = _int;
+beta_ptr = _int;
+  } else if (C->dtype.code == kDLFloat) {
+alpha_ptr = _float;
+beta_ptr = _float;
+  }
+
+  auto A_data = reinterpret_cast(static_cast(A->data) + 
A->byte_offset);
+  auto B_data = reinterpret_cast(static_cast(B->data) + 
B->byte_offset);
+  auto C_data = reinterpret_cast(static_cast(C->data) + 
C->byte_offset);
+
+  CHECK_CUBLAS_ERROR(cublasGemmEx(hdl,
+ BooleanToTranspose(transb),
+ BooleanToTranspose(transa),
+ ColumnCount(B, transb),
+ RowCount(A, transa),
+ ColumnCount(A, transa),
+ alpha_ptr,
+ B_data, cuda_in_type, ColumnStride(B),
+ A_data, cuda_in_type, ColumnStride(A),
+ beta_ptr,
+ C_data, cuda_out_type, ColumnStride(C),
+ cuda_out_type, algo));
 
 Review comment:
   The second to last arg of cublasGemmEx is computation type. In the case 
where both input type and output type are fp16, we noticed that computation 
type fp16 results in much lower precision than computation type fp32. So 
setting computation type the same as output type here may lead to precision 
dropping for output type fp16/int8.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services