[GitHub] [incubator-tvm] jackwish commented on a change in pull request #4798: [QNN] Optimize lowering for requantize and FixedPointMultiply.
jackwish commented on a change in pull request #4798: [QNN] Optimize lowering
for requantize and FixedPointMultiply.
URL: https://github.com/apache/incubator-tvm/pull/4798#discussion_r376682469
##
File path: src/relay/qnn/op/requantize.cc
##
@@ -103,7 +103,11 @@ Expr RequantizeLower(const Expr& input_tensor, const
Expr& input_scale,
shifted_int64_t = Add(Cast(output_zero_point, hp_dtype), scaled_int64_t);
}
- // 4) Clip to the out_dtype min/max.
+ // 4) Clip to the out_dtype min/max. Skip clipping if out_dtype is Int32.
The fixed point
+ // multiplication keeps the value in int32 range.
+ if (out_dtype == DataType::Int(32)) {
+return Cast(shifted_int64_t, out_dtype);
+ }
Review comment:
I see, thank you for the detailed explain!
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] FrozenGene edited a comment on issue #4828: [QNN][TFLite] TFLite rounding mode support
FrozenGene edited a comment on issue #4828: [QNN][TFLite] TFLite rounding mode support URL: https://github.com/apache/incubator-tvm/pull/4828#issuecomment-583691425 > just modified add/mul/concat requantize rounding mode and tested, no luck. will change the default rounding behavior for a later test. > > update: I forced FixedPointMultiply(PerChannel) rounding mode to be TFLITE, but still unable to get bit-exact results. > > one more thing, setting tflite default rounding mode to TFLITE seems to break GPU test of mobilenet_v2, maybe you guys have any ideas/suggestions? > > @FrozenGene @anijain2305 Let us break the model into layer by layer and compare with tflite. I want to describe my development way, maybe it could help you. For example, we have mobilenetv2 quantized model, you could get the quantized tensorflow and tflite model. Then you could call `tflite_convert` (feed it quantized tensorflow model) and set the output layer (for example, just the first convolution layer), then you get one quantized tflite model only contain the first convolution layer of mobilenet v2. After you verify it correctly, you could go on until you finish the whole model e2e correctly. Command example: `tflite_convert --input_format=TENSORFLOW_GRAPHDEF --graph_def_file="xx.pb" --output_file=xx.tflite --output_format=TFLITE --input_arrays=input --input_shapes=1,224,224,3 --std_dev_values=127 --mean_values=127 --inference_type=QUANTIZED_UINT8 --inference_input_type=QUANTIZED_UINT8 --default_ranges_min=0 --default_ranges_max=6 --output_arrays=xx` I think when you verify, you could run on cpu firstly locally to find issue, then consider gpu ci issue. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-tvm] FrozenGene commented on issue #4828: [QNN][TFLite] TFLite rounding mode support
FrozenGene commented on issue #4828: [QNN][TFLite] TFLite rounding mode support URL: https://github.com/apache/incubator-tvm/pull/4828#issuecomment-583691425 > just modified add/mul/concat requantize rounding mode and tested, no luck. will change the default rounding behavior for a later test. > > update: I forced FixedPointMultiply(PerChannel) rounding mode to be TFLITE, but still unable to get bit-exact results. > > one more thing, setting tflite default rounding mode to TFLITE seems to break GPU test of mobilenet_v2, maybe you guys have any ideas/suggestions? > > @FrozenGene @anijain2305 Let us break the model into layer by layer and compare with tflite. I want to describe my development way, maybe it could help you. For example, we have mobilenetv2 quantized model, you could get the quantized tensorflow and tflite model. Then you could call `tflite_convert` (feed it quantized tensorflow model) and set the output layer (for example, just the first convolution layer), then you get one quantized tflite model only contain the first convolution layer of mobilenet v2. After you verify it correctly, you could go on until you finish the whole model e2e correctly. I think when you verify, you could run on cpu firstly locally to find issue, then consider gpu ci issue. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-tvm] u99127 commented on a change in pull request #4788: [FRONTEND][TFLITE]Gather, StridedSlice op support added
u99127 commented on a change in pull request #4788: [FRONTEND][TFLITE]Gather,
StridedSlice op support added
URL: https://github.com/apache/incubator-tvm/pull/4788#discussion_r376673796
##
File path: tests/python/frontend/tflite/test_forward.py
##
@@ -244,6 +244,74 @@ def test_forward_slice():
_test_slice(np.arange(8, dtype=np.int32).reshape((2, 4)), begin=[0,
1], size=[-1, -1])
_test_slice(np.arange(5, dtype=np.int32).reshape((5, )), begin=[4],
size=[-1])
+###
+# Gather
+# --
+
+def _test_gather(dshape, indices, axis, dtype):
+""" One iteration of Gather """
+data = np.random.uniform(1, 10, size=dshape).astype(dtype)
+indices = np.asarray(indices).astype('int32')
+
+with tf.Graph().as_default():
+in_data = array_ops.placeholder(shape=data.shape, dtype=data.dtype)
+out = array_ops.gather(in_data, indices, axis=axis)
+compare_tflite_with_tvm(data, 'Placeholder:0', [in_data], [out])
+
+#Test quantized input
+data = np.random.uniform(1, 10, size=dshape).astype(np.uint8)
+with tf.Graph().as_default():
+in_data = array_ops.placeholder(shape=data.shape, dtype=data.dtype,
name="in_data")
+out = array_ops.gather(in_data, indices, axis=axis)
+compare_tflite_with_tvm([data], ['in_data:0'], [in_data], [out],
quantized=True)
+
+def test_forward_gather():
+""" GATHER """
+_test_gather((4,), [1], 0, 'float32')
+_test_gather((1, 4), [0], 0, 'int32')
+_test_gather((4,), [[[1, 0], [0, 1]]], 0, 'float32')
+_test_gather((2, 2), [[[1, 0], [0, 1]]], 0, 'int32')
+_test_gather((2, 2), [[[1, 0], [0, 1]]], 1, 'int32')
+_test_gather((2, 2), [[[1, 0], [0, 1]]], 0, 'float32')
+_test_gather((3, 3, 3), [[[1, 0]]], 0, 'int32')
+_test_gather((3, 3, 3), [[[1, 0]]], 2, 'int32')
+_test_gather((4, 3, 5, 6), [[2, 1, 0, 0]], 0, 'float32')
+
+###
+# StridedSlice
+#
+
+def _test_stridedslice(ip_shape, begin, end, stride, dtype,
+ begin_mask=0, end_mask=0, new_axis_mask=0,
+ shrink_axis_mask=0, ellipsis_mask=0):
+""" One iteration of a Stridedslice """
+data = np.random.uniform(size=ip_shape).astype(dtype)
+with tf.Graph().as_default():
+in_data = tf.placeholder(dtype, ip_shape, name="in_data")
+out = array_ops.strided_slice(in_data, begin, end, stride,
+ begin_mask=begin_mask,
+ end_mask=end_mask,
new_axis_mask=new_axis_mask,
+ shrink_axis_mask=shrink_axis_mask,
+ ellipsis_mask=ellipsis_mask)
+compare_tflite_with_tvm(data, 'in_data:0', [in_data], [out])
+
+#Test with quantized inputs
+data = np.random.uniform(size=ip_shape).astype(np.uint8)
+with tf.Graph().as_default():
+in_data = tf.placeholder(dtype, ip_shape, name="in_data")
+out = array_ops.strided_slice(in_data, begin, end, stride,
+ begin_mask=begin_mask,
+ end_mask=end_mask,
new_axis_mask=new_axis_mask,
+ shrink_axis_mask=shrink_axis_mask,
+ ellipsis_mask=ellipsis_mask)
+compare_tflite_with_tvm([data], ['in_data:0'], [in_data], [out],
quantized=True)
+
+def test_forward_stridedslice():
+'''test StridedSlice'''
Review comment:
@wyc-ruiker - might help to specify what situations you see missing to help
with actionable feedback ?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] u99127 commented on a change in pull request #4828: [QNN][TFLite] TFLite rounding mode support
u99127 commented on a change in pull request #4828: [QNN][TFLite] TFLite
rounding mode support
URL: https://github.com/apache/incubator-tvm/pull/4828#discussion_r376672825
##
File path: src/relay/qnn/util.cc
##
@@ -100,35 +134,54 @@ Expr FixedPointMultiply(Expr tensor, double multiplier,
const Array&
// (from the right, rightmost bit is bit 0). The computation is performed in
// higher precision to avoid overflow in multiplying two int32 values.
Expr scalar = MakeConstantScalar(hp_dtype, fixed_point_multiplier);
- tensor = Multiply(tensor, scalar);
+ Expr scaled_tensor = Multiply(tensor, scalar);
// 4) Find the rounding scalar. This depends on where the final decimal
// point sits. As we will be right shifting the multiplied_t, we need to
// first calculate the total_right_shift.
int total_right_shift = right_shift + 31;
int64_t pos_rounding_value = (1ll << (total_right_shift - 1));
+ auto nearest_rounding_scalar =
+[&](const Expr& input_tensor, int right_shift) -> Expr {
+ int64_t pos_rounding_value = (1ll << (right_shift - 1));
+ auto pos_rounder = MakeConstantScalar(hp_dtype, pos_rounding_value);
+ auto neg_rounder = MakeConstantScalar(hp_dtype, pos_rounding_value - 1);
+ auto pos_rounder_t = Full(pos_rounder, input_shape, hp_dtype);
+ auto neg_rounder_t = Full(neg_rounder, input_shape, hp_dtype);
+
+ auto zero_t = Zeros(input_shape, hp_dtype);
+ return Where(
+GreaterEqual(input_tensor, zero_t), pos_rounder_t, neg_rounder_t);
Review comment:
I'm missing an update to the comment above.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] u99127 commented on a change in pull request #4828: [QNN][TFLite] TFLite rounding mode support
u99127 commented on a change in pull request #4828: [QNN][TFLite] TFLite
rounding mode support
URL: https://github.com/apache/incubator-tvm/pull/4828#discussion_r376671194
##
File path: include/tvm/relay/qnn/attrs.h
##
@@ -44,14 +44,15 @@ struct RequantizeAttrs : public
tvm::AttrsNode {
.set_default(-1);
TVM_ATTR_FIELD(rounding).set_default("UPWARD")
.describe("Defines the rounding direction when the value is midway
between"
- "two representable values. There are two supported modes -
UPWARD"
- "or TONEAREST. Both modes behave exactly same except at the"
+ "two representable values. There are two 3 modes - UPWARD,
TONEAREST"
+ "or TFLITE. UP/TONEAREST modes behave exactly same except at
the"
"midpoints between the two representable values. At the
midpoint,"
"UPWARD rounds towards positive infinity (for example -1.5
will be"
"rounded to -1). TONEAREST is the standard rounding where
the"
"value is rounded away from zero at midpoints (for example,
-1.5"
"rounds to -2). More context can be found at following gblic
manual"
-
"https://www.gnu.org/software/libc/manual/html_node/Rounding.html.;);
+
"https://www.gnu.org/software/libc/manual/html_node/Rounding.html.;
+ "TFLITE mode is more complicated, referring to tflite
implementation.");
Review comment:
tflite is a moving target, what version of tflite implementation are we
targeting here ? I would give an exact version and implementation here.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] alexwong commented on a change in pull request #4497: [Relay] Add a PyTorch to Relay Parser
alexwong commented on a change in pull request #4497: [Relay] Add a PyTorch to
Relay Parser
URL: https://github.com/apache/incubator-tvm/pull/4497#discussion_r376657697
##
File path: tests/python/frontend/pytorch/test_forward.py
##
@@ -0,0 +1,767 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+# pylint: disable=import-self, invalid-name, unused-argument
+"""Unit tests for various models and operators"""
+from time import time
+import os
+import sys
+from tempfile import TemporaryDirectory
+from scipy.stats import t as tdistr
+import numpy as np
+import torch
+from torch.nn import Module
+import tvm
+import torchvision
+
+from tvm import relay
+from tvm.contrib import graph_runtime
+from tvm.relay.testing.config import ctx_list
+
+sys.setrecursionlimit(1)
+
+def _vectorize(ten):
+return ten.reshape(-1)
+
+def atol(tru, est):
+def _atol_elt(tru, est):
+return abs(tru - est)
+tru = _vectorize(tru)
+est = _vectorize(est)
+return max([_atol_elt(x, y) for x, y in zip(tru, est)])
+
+def rtol(tru, est):
+def _rtol_elt(tru, est):
+return abs(tru - est) / min(abs(tru), abs(est))
+tru = _vectorize(tru)
+est = _vectorize(est)
+return max([_rtol_elt(x, y) for x, y in zip(tru, est)])
+
+def assert_shapes_match(tru, est):
+if tru.shape != est.shape:
+msg = "Output shapes {} and {} don't match"
+raise AssertionError(msg.format(tru.shape, est.shape))
+
+def load_torchvision(model_name):
+"""Given a model name, returns a Torchvision model in eval mode as well
+as an example input."""
+with torch.no_grad():
+if model_name.startswith("inception"):
+height = width = 299
+mean = [0.5, 0.5, 0.5]
+std = [0.5, 0.5, 0.5]
+else:
+height = width = 224
+mean = [0.485, 0.456, 0.406]
+std = [0.229, 0.224, 0.225]
+input_shape = [1, 3, height, width]
+input_data = torch.randn(input_shape).float()
+for channel in range(3):
+input_data[:, channel] -= mean[channel]
+input_data[:, channel] /= std[channel]
+model = getattr(torchvision.models, model_name)(pretrained=True)
+model = model.float().eval()
+return model, input_data
+
+def load_pretrainedmodels(model_name):
+"""Given a model name, returns a pretrainedmodels.pytorch model in eval
+mode as well as an example input."""
+import pretrainedmodels #
https://github.com/Cadene/pretrained-models.pytorch
+model = getattr(pretrainedmodels, model_name)().float().eval()
+input_shape = [1, *model.input_size]
+input_data = torch.rand(input_shape).float() * 256
+for channel in range(3):
+input_data[:, channel] -= model.mean[channel]
+input_data[:, channel] /= model.std[channel]
+return model, input_data
+
+def load_model(model_name):
+"""Given a model name, returns a model as well as an example input."""
+if hasattr(torchvision.models, model_name):
+return load_torchvision(model_name)
+try:
+if hasattr(pretrainedmodels, model_name):
+return load_pretrainedmodels(model_name)
+except ModuleNotFoundError:
+raise ModuleNotFoundError("Please install pretrainedmodels.pytorch")
+raise RuntimeError("Model not supported")
+
+
+def confidence_interval(mean, stdev, count, alpha=.01):
+"""Returns the lower and upper bounds of the confidence interval of a
random
+variable. Confidence is 1 - alpha (default confidence is 99%)."""
+stdval = tdistr.ppf(1 - alpha / 2, count - 1)
+lower, upper = mean + np.array([-1, 1]) * stdval * stdev / np.sqrt(count)
+return lower, upper
+
+def measure_latency(model, input_shapes, output_shapes, thresh, dryruns=40):
+"""Compute the latency of the given model"""
+latencies = []
+count = 0
+while True:
+if isinstance(model, torch.nn.Module):
+input_data = [torch.rand(shape).float() for shape in input_shapes]
+if torch.cuda.is_available():
+input_data = list(map(lambda x: x.cuda(), input_data))
+model = model.cuda()
+t_start = time()
+with
[GitHub] [incubator-tvm] tqchen commented on issue #4824: Tflite frontend needs to use zero point of input tensor while lowering qnn.conv2d for padding
tqchen commented on issue #4824: Tflite frontend needs to use zero point of input tensor while lowering qnn.conv2d for padding URL: https://github.com/apache/incubator-tvm/issues/4824#issuecomment-583661376 @u99127 perhaps we can open a discuss thread on the forum. I agree with most of your points :) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-tvm] u99127 commented on issue #4824: Tflite frontend needs to use zero point of input tensor while lowering qnn.conv2d for padding
u99127 commented on issue #4824: Tflite frontend needs to use zero point of input tensor while lowering qnn.conv2d for padding URL: https://github.com/apache/incubator-tvm/issues/4824#issuecomment-583660260 @tqchen , @FrozenGene [NEED-BACKPORT-0.6] ? since there's no way of reporting what issues affect what versions in github issues ? This is probably worth a discuss post, however I'll say my piece here in response to the comment about point releases. Releasing from a release branch is the next step in my opinion. The first set of steps towards this are to: 1. Be reasonably clear about the criteria for backports. There is not always a clear answer but correctness issues are likely to need backports but then you don't want a too risky backport so that we are not chasing bug tails by introducing a new bug to fix an old one. Thus there needs to be a risk vs reward evaluation by either a group of release managers or the reviewer community in an objective manner. For example this fix and the related fix to fixup the tests is appropriate, however large fixes that require refactoring changes will not be at which point you need a different fix. 2. Reviewers need to help that contributors improve descriptions in the pull request and ask the question which release branches a particular issue affects. Further if pull requests combine bug fixes with new features especially regression fixes, such pull requests need to be split up as our current policy is to squash commits and then it's not obvious what came from where if we want someone to do a bit of archeology. 3. Optionally add this to the template of the Pull Request reminding developers submitting bug fixes that they could help the project by considering whether a pull request requires a backport or not. 4. We need an update to the contributor's guide 5. How do we help the reporter report the version that the issue was reported in ? Currently it's free form text but if that were a drop down list that would be great and thus keeping that information there. 6. Finally decide how long we would keep a branch going - what's the branch management lifecycle ? My 2 cents. regards Ramana This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-tvm] icemelon9 commented on a change in pull request #4644: [WIP] Relay op strategy
icemelon9 commented on a change in pull request #4644: [WIP] Relay op strategy
URL: https://github.com/apache/incubator-tvm/pull/4644#discussion_r376652263
##
File path: python/tvm/autotvm/task/space.py
##
@@ -944,11 +939,10 @@ def from_json_dict(json_dict):
raise RuntimeError("Invalid config knob type: " + knob_type)
entity_map[str(key)] = entity
-return ConfigEntity(index, code_hash, template_key, entity_map,
constraints)
+return ConfigEntity(index, code_hash, entity_map, constraints)
def __repr__(self):
-return "%s,%s,%s,%d" % (str(self._entity_map)[12:-1],
self.template_key,
-self.code_hash, self.index)
+return "%s,%s,%d" % (str(self._entity_map)[12:-1], self.code_hash,
self.index)
Review comment:
Sure. Do you have any proposal for the change?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] ZihengJiang commented on a change in pull request #4644: [WIP] Relay op strategy
ZihengJiang commented on a change in pull request #4644: [WIP] Relay op strategy
URL: https://github.com/apache/incubator-tvm/pull/4644#discussion_r376651491
##
File path: python/tvm/autotvm/task/space.py
##
@@ -944,11 +939,10 @@ def from_json_dict(json_dict):
raise RuntimeError("Invalid config knob type: " + knob_type)
entity_map[str(key)] = entity
-return ConfigEntity(index, code_hash, template_key, entity_map,
constraints)
+return ConfigEntity(index, code_hash, entity_map, constraints)
def __repr__(self):
-return "%s,%s,%s,%d" % (str(self._entity_map)[12:-1],
self.template_key,
-self.code_hash, self.index)
+return "%s,%s,%d" % (str(self._entity_map)[12:-1], self.code_hash,
self.index)
Review comment:
Then we may need to bump the log format version, if we cannot ensure the
compatibility, we should raise error when read the old version. Also, we can
take this opportunity to fix our previous design, like changing the shortcut
`i`, `r`, `v` to the full word:
https://github.com/apache/incubator-tvm/blob/master/python/tvm/autotvm/record.py#L91
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] comaniac commented on issue #4187: [AutoTVM] Selective Tuning
comaniac commented on issue #4187: [AutoTVM] Selective Tuning URL: https://github.com/apache/incubator-tvm/pull/4187#issuecomment-583651669 Close for now as the feature is not in the development process anymore. Will integrate this feature with a more comprehensive AutoTVM improvements later. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-tvm] comaniac closed pull request #4187: [AutoTVM] Selective Tuning
comaniac closed pull request #4187: [AutoTVM] Selective Tuning URL: https://github.com/apache/incubator-tvm/pull/4187 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[incubator-tvm] branch master updated (0186ca4 -> e33de24)
This is an automated email from the ASF dual-hosted git repository. wuwei pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/incubator-tvm.git. from 0186ca4 [COMMUNITY] comaniac -> reviewer (#4841) add e33de24 [Doc][AutoTVM] Fix bugs that override n_trials (#4842) No new revisions were added by this update. Summary of changes: tutorials/autotvm/tune_relay_arm.py| 9 ++-- tutorials/autotvm/tune_relay_cuda.py | 9 ++-- tutorials/autotvm/tune_relay_mobile_gpu.py | 9 ++-- vta/tutorials/autotvm/tune_relay_vta.py| 80 -- 4 files changed, 59 insertions(+), 48 deletions(-)
[GitHub] [incubator-tvm] icemelon9 commented on a change in pull request #4644: [WIP] Relay op strategy
icemelon9 commented on a change in pull request #4644: [WIP] Relay op strategy
URL: https://github.com/apache/incubator-tvm/pull/4644#discussion_r376621668
##
File path: python/tvm/autotvm/task/space.py
##
@@ -944,11 +939,10 @@ def from_json_dict(json_dict):
raise RuntimeError("Invalid config knob type: " + knob_type)
entity_map[str(key)] = entity
-return ConfigEntity(index, code_hash, template_key, entity_map,
constraints)
+return ConfigEntity(index, code_hash, entity_map, constraints)
def __repr__(self):
-return "%s,%s,%s,%d" % (str(self._entity_map)[12:-1],
self.template_key,
-self.code_hash, self.index)
+return "%s,%s,%d" % (str(self._entity_map)[12:-1], self.code_hash,
self.index)
Review comment:
Yes. Anyway this PR changes the workload name. So old tuning logs can no
longer be used.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] ZihengJiang merged pull request #4841: [COMMUNITY] comaniac -> reviewer
ZihengJiang merged pull request #4841: [COMMUNITY] comaniac -> reviewer URL: https://github.com/apache/incubator-tvm/pull/4841 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[incubator-tvm] branch master updated (e0122c0 -> 0186ca4)
This is an automated email from the ASF dual-hosted git repository. ziheng pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/incubator-tvm.git. from e0122c0 [REFACTOR][PY][API-Change] Polish tvm.runtime, tvm.runtime.module API update (#4837) add 0186ca4 [COMMUNITY] comaniac -> reviewer (#4841) No new revisions were added by this update. Summary of changes: CONTRIBUTORS.md | 1 + 1 file changed, 1 insertion(+)
[GitHub] [incubator-tvm] videetparekh opened a new issue #4843: Emscripten Runtime Error while linking
videetparekh opened a new issue #4843: Emscripten Runtime Error while linking URL: https://github.com/apache/incubator-tvm/issues/4843 I'm trying to get the following tutorial (https://github.com/apache/incubator-tvm/tree/master/web) up and running within a Docker container. I'm using a pre-installed version of TVM, installing Emscripten(clang-tag-e1.38.30_32bit) and patching the two through TVM's config. I'm able to generate the `test_add_one.bc` file, but the linking fails at Runtime with the following error: ``` RuntimeError: Compilation error: Traceback (most recent call last): File "/usr/emsdk/upstream/emscripten/emcc.py", line 3841, in sys.exit(run(sys.argv)) File "/usr/emsdk/upstream/emscripten/emcc.py", line 2312, in run final = do_emscripten(final, shared.replace_or_append_suffix(target, '.mem')) File "/usr/emsdk/upstream/emscripten/emcc.py", line 475, in do_emscripten emscripten.run(infile, outfile, memfile) File "/usr/emsdk/upstream/emscripten/emscripten.py", line 2792, in run return temp_files.run_and_clean(lambda: emscripter( File "/usr/emsdk/upstream/emscripten/tools/tempfiles.py", line 105, in run_and_clean return func() File "/usr/emsdk/upstream/emscripten/emscripten.py", line 2793, in infile, outfile_obj, memfile, shared.NODE_JS, temp_files, shared.DEBUG) File "/usr/emsdk/upstream/emscripten/emscripten.py", line 95, in emscript_fastcomp glue, forwarded_data = compiler_glue(metadata, compiler_engine, temp_files, DEBUG) File "/usr/emsdk/upstream/emscripten/emscripten.py", line 242, in compiler_glue assert not (metadata['simd'] and shared.Settings.WASM), 'SIMD is used, but not supported in WASM mode yet' AssertionError: SIMD is used, but not supported in WASM mode yet ``` I believe Emscripten no longer supports fastcomp and so clang-tag-e1.38.30_32bit would be the version to use? Please let me know how I can resolve this? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-tvm] ZihengJiang commented on a change in pull request #4644: [WIP] Relay op strategy
ZihengJiang commented on a change in pull request #4644: [WIP] Relay op strategy
URL: https://github.com/apache/incubator-tvm/pull/4644#discussion_r376570164
##
File path: python/tvm/autotvm/task/space.py
##
@@ -944,11 +939,10 @@ def from_json_dict(json_dict):
raise RuntimeError("Invalid config knob type: " + knob_type)
entity_map[str(key)] = entity
-return ConfigEntity(index, code_hash, template_key, entity_map,
constraints)
+return ConfigEntity(index, code_hash, entity_map, constraints)
def __repr__(self):
-return "%s,%s,%s,%d" % (str(self._entity_map)[12:-1],
self.template_key,
-self.code_hash, self.index)
+return "%s,%s,%d" % (str(self._entity_map)[12:-1], self.code_hash,
self.index)
Review comment:
Will this change our autotvm log format?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] tqchen closed issue #4839: Nvcc fatal : Value ‘sm_75’ is not defined for option ‘gpu-architecture’
tqchen closed issue #4839: Nvcc fatal : Value ‘sm_75’ is not defined for option ‘gpu-architecture’ URL: https://github.com/apache/incubator-tvm/issues/4839 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-tvm] comaniac opened a new pull request #4842: [Doc][AutoTVM] Fix bugs that overrides n_trials
comaniac opened a new pull request #4842: [Doc][AutoTVM] Fix bugs that overrides n_trials URL: https://github.com/apache/incubator-tvm/pull/4842 In the loop of tuning a list of tasks, `n_trials` will be overwritten and result in incorrect maximum trial number for rest tasks. cc @dati91 @merrymercy @eqy This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-tvm] tqchen commented on issue #4839: Nvcc fatal : Value ‘sm_75’ is not defined for option ‘gpu-architecture’
tqchen commented on issue #4839: Nvcc fatal : Value ‘sm_75’ is not defined for option ‘gpu-architecture’ URL: https://github.com/apache/incubator-tvm/issues/4839#issuecomment-583567644 Please open a new thread on https://discuss.tvm.ai/ for related questions. In your case, it is due to too low version of your cuda This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-tvm] ZihengJiang opened a new pull request #4841: [COMMUNITY] comaniac -> reviewer
ZihengJiang opened a new pull request #4841: [COMMUNITY] comaniac -> reviewer URL: https://github.com/apache/incubator-tvm/pull/4841 Please join me to welcome @comaniac as a new reviewer. He has been working on improving operators in topi and quite active in code review for the community. - [Commits History](https://github.com/dmlc/tvm/commits?author=comaniac) - [Code Review](https://github.com/dmlc/tvm/pulls?utf8=%E2%9C%93=reviewed-by%3Acomaniac) - [Community Forum Summary](https://discuss.tvm.ai/u/comaniac/summary) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-tvm] anijain2305 commented on a change in pull request #4816: [TFLite] Using real image for QNN testing.
anijain2305 commented on a change in pull request #4816: [TFLite] Using real image for QNN testing. URL: https://github.com/apache/incubator-tvm/pull/4816#discussion_r376528698 ## File path: tests/python/frontend/tflite/test_forward.py ## @@ -1100,16 +1117,26 @@ def test_forward_squeeze(): # Pad # --- -def _test_pad(data): +def _test_pad(data, quantized=False): """ One iteration of PAD """ assert len(data) == 2 # Test with tensor and constant with tf.Graph().as_default(): -in_data = [array_ops.placeholder(shape=data[0].shape, dtype=data[0].dtype, name='in')] -out = array_ops.pad(in_data[0], ops.convert_to_tensor(data[1], dtype=data[1].dtype)) -compare_tflite_with_tvm([data[0]], ['in:0'], in_data, [out]) +in_data = [array_ops.placeholder(shape=data[0].shape, dtype='float32', name='in')] + +if quantized: +# fake_quant will keep the tensors in float32 until the conversion in the session +inq_data = [tf.quantization.fake_quant_with_min_max_args(in_data[0], + min=-100, + max=100, + name="inq_0")] Review comment: Thanks @inadob for the review. Will incorporate your comments in few hours. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-tvm] anijain2305 commented on issue #4828: [QNN][TFLite] TFLite rounding mode support
anijain2305 commented on issue #4828: [QNN][TFLite] TFLite rounding mode support URL: https://github.com/apache/incubator-tvm/pull/4828#issuecomment-583531400 > update: I forced FixedPointMultiply(PerChannel) rounding mode to be TFLITE, but still unable to get bit-exact results. :( This will be a challenging task. I think, instead of trying to fix the full e2e test, we should now focus on unit tests. I can help with improving unit test coverage. > one more thing, setting tflite default rounding mode to TFLITE seems to break GPU test of mobilenet_v2, maybe you guys have any ideas/suggestions? Do we run TFLIte tests on GPU? Little confused. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-tvm] icemelon9 merged pull request #4837: [REFACTOR][PY][API-Change] Polish tvm.runtime, tvm.runtime.module API update
icemelon9 merged pull request #4837: [REFACTOR][PY][API-Change] Polish tvm.runtime, tvm.runtime.module API update URL: https://github.com/apache/incubator-tvm/pull/4837 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[incubator-tvm] branch master updated (30b7d83 -> e0122c0)
This is an automated email from the ASF dual-hosted git repository.
haichen pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-tvm.git.
from 30b7d83 [Frontend][TFLite] Add MIRROR_PAD operator (#4822)
add e0122c0 [REFACTOR][PY][API-Change] Polish tvm.runtime,
tvm.runtime.module API update (#4837)
No new revisions were added by this update.
Summary of changes:
apps/bundle_deploy/bundle.cc | 2 +-
apps/dso_plugin_module/README.md | 2 +-
apps/dso_plugin_module/test_plugin_module.py | 2 +-
apps/extension/tests/test_ext.py | 4 +-
apps/howto_deploy/cpp_deploy.cc| 6 +-
apps/howto_deploy/python_deploy.py | 14 +-
apps/sgx/run_model.py | 2 +-
docs/api/python/bridge.rst | 24 ---
docs/api/python/contrib.rst| 15 +-
docs/api/python/dev.rst| 7 -
docs/api/python/index.rst | 4 +-
docs/api/python/module.rst | 21 ---
docs/api/python/ndarray.rst| 20 +--
docs/api/python/{function.rst => runtime.rst} | 27 +++-
docs/deploy/aocl_fpga.md | 4 +-
docs/deploy/aws_fpga.md| 6 +-
docs/dev/introduction_to_module_serialization.rst | 4 +-
docs/dev/relay_bring_your_own_codegen.rst | 16 +-
include/tvm/runtime/module.h | 7 +
jvm/core/src/test/scripts/test_add_gpu.py | 2 +-
python/tvm/__init__.py | 4 -
python/tvm/_ffi/_ctypes/object.py | 19 +++
python/tvm/_ffi/_cython/object.pxi | 17 ++
python/tvm/api.py | 7 +-
python/tvm/build_module.py | 4 +-
python/tvm/container.py| 15 +-
python/tvm/contrib/debugger/debug_result.py| 2 +-
python/tvm/contrib/dlpack.py | 2 +-
python/tvm/contrib/mxnet.py| 12 +-
python/tvm/contrib/sparse.py | 5 +-
python/tvm/expr.py | 4 +-
python/tvm/micro/base.py | 8 +-
python/tvm/relay/backend/compile_engine.py | 4 +-
python/tvm/relay/backend/interpreter.py| 4 +-
python/tvm/relay/backend/vm.py | 14 +-
python/tvm/relay/frontend/caffe2.py| 4 +-
python/tvm/relay/frontend/coreml.py| 2 +-
python/tvm/relay/frontend/darknet.py | 2 +-
python/tvm/relay/frontend/keras.py | 2 +-
python/tvm/relay/frontend/mxnet.py | 2 +-
python/tvm/relay/frontend/onnx.py | 2 +-
python/tvm/relay/frontend/tensorflow.py| 4 +-
python/tvm/relay/frontend/tflite.py| 2 +-
python/tvm/rpc/server.py | 2 +-
python/tvm/runtime/__init__.py | 6 +-
.../{relay/_analysis.py => runtime/_ffi_api.py}| 6 +-
python/tvm/runtime/_ffi_node_api.py| 50 ++
python/tvm/runtime/module.py | 47 +++---
python/tvm/runtime/ndarray.py | 6 +-
python/tvm/runtime/object.py | 22 +--
python/tvm/runtime/object_generic.py | 12 +-
python/tvm/runtime/packed_func.py | 2 +-
python/tvm/target.py | 2 +-
rust/frontend/README.md| 2 +-
rust/frontend/examples/resnet/src/build_resnet.py | 4 +-
src/api/api_base.cc| 87 --
src/api/api_codegen.cc | 48 --
src/api/api_lang.cc| 169 +---
src/node/container.cc | 177 +
src/node/reflection.cc | 5 +-
src/node/repr_printer.cc | 8 +
src/node/serialization.cc | 8 +-
src/relay/backend/contrib/codegen_c/codegen.cc | 2 +-
src/relay/backend/contrib/dnnl/codegen.cc | 2 +-
src/runtime/c_runtime_api.cc | 4 +
.../example_ext_runtime/example_ext_runtime.cc | 4 +-
src/runtime/cuda/cuda_module.cc| 6 +-
src/runtime/dso_library.cc | 2 +-
src/runtime/library_module.cc | 2 +-
src/runtime/metal/metal_module.mm | 4 +-
src/runtime/micro/micro_module.cc | 2 +-
src/runtime/module.cc | 69
src/runtime/object.cc | 6 +
src/runtime/opencl/aocl/aocl_module.cc | 6 +-
[GitHub] [incubator-tvm] jikechao removed a comment on issue #4764: [CI] ci-gpu update blockers
jikechao removed a comment on issue #4764: [CI] ci-gpu update blockers URL: https://github.com/apache/incubator-tvm/issues/4764#issuecomment-583411064 got it. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-tvm] jikechao commented on issue #4764: [CI] ci-gpu update blockers
jikechao commented on issue #4764: [CI] ci-gpu update blockers URL: https://github.com/apache/incubator-tvm/issues/4764#issuecomment-583411064 got it. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-tvm] uday60 commented on issue #4830: libnnvm_compiler.so file missing | Commands inside
uday60 commented on issue #4830: libnnvm_compiler.so file missing | Commands
inside
URL: https://github.com/apache/incubator-tvm/issues/4830#issuecomment-583401837
@tqchen
I have been using an earlier version of TVM. Now things have changed the
following script is giving me a segmentation fault. I have already raised the
issue here but no replies.
https://discuss.tvm.ai/t/conversion-script-terminate-called-after-throwing-an-instance-of-std-bad-alloc/5601
```
import onnx
import numpy as np
import tvm
import tvm.relay as relay
import nnvm.compiler
import nnvm.testing
onnx_model = onnx.load('./centerface.onnx')
opt_level = 3
target = tvm.target.create("llvm -mcpu=haswell")
image_size = (224, 224)
shape_dict = {'data': (1, 3, *image_size)}
print(shape_dict)
nnvm_sym, nnvm_params = relay.frontend.from_onnx(onnx_model, shape_dict)
with nnvm.compiler.build_config(opt_level=opt_level):
graph, lib, params = nnvm.compiler.build(nnvm_sym, target, shape_dict,
params=nnvm_params)
lib.export_library("./centerface_deploy_lib.so")
print('lib export succeefully')
with open("./centerface_deploy_graph.json", "w") as fo:
fo.write(graph.json())
with open("./centerface_deploy_param.params", "wb") as fo:
fo.write(nnvm.compiler.save_param_dict(params))
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] wyc-ruiker opened a new pull request #4840: [Frontend][TFlite] use qnn helper function in softmax
wyc-ruiker opened a new pull request #4840: [Frontend][TFlite] use qnn helper function in softmax URL: https://github.com/apache/incubator-tvm/pull/4840 Thanks for contributing to TVM! Please refer to guideline https://docs.tvm.ai/contribute/ for useful information and tips. After the pull request is submitted, please request code reviews from [Reviewers](https://github.com/apache/incubator-tvm/blob/master/CONTRIBUTORS.md#reviewers) by @ them in the pull request thread. Use qnn helper function in softmax to make calculation logic more centralized. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-tvm] inadob commented on a change in pull request #4816: [TFLite] Using real image for QNN testing.
inadob commented on a change in pull request #4816: [TFLite] Using real image
for QNN testing.
URL: https://github.com/apache/incubator-tvm/pull/4816#discussion_r376337649
##
File path: tests/python/frontend/tflite/test_forward.py
##
@@ -1100,16 +1117,26 @@ def test_forward_squeeze():
# Pad
# ---
-def _test_pad(data):
+def _test_pad(data, quantized=False):
""" One iteration of PAD """
assert len(data) == 2
# Test with tensor and constant
with tf.Graph().as_default():
-in_data = [array_ops.placeholder(shape=data[0].shape,
dtype=data[0].dtype, name='in')]
-out = array_ops.pad(in_data[0], ops.convert_to_tensor(data[1],
dtype=data[1].dtype))
-compare_tflite_with_tvm([data[0]], ['in:0'], in_data, [out])
+in_data = [array_ops.placeholder(shape=data[0].shape, dtype='float32',
name='in')]
+
+if quantized:
+# fake_quant will keep the tensors in float32 until the conversion
in the session
+inq_data =
[tf.quantization.fake_quant_with_min_max_args(in_data[0],
+ min=-100,
+ max=100,
+
name="inq_0")]
Review comment:
ping @anijain2305
add input_range = {'inq_0': (-100, 100)} and pass that to
compare_tflite_with_tvm()
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] inadob commented on a change in pull request #4816: [TFLite] Using real image for QNN testing.
inadob commented on a change in pull request #4816: [TFLite] Using real image for QNN testing. URL: https://github.com/apache/incubator-tvm/pull/4816#discussion_r375547063 ## File path: tests/python/frontend/tflite/test_forward.py ## @@ -1100,16 +1117,26 @@ def test_forward_squeeze(): # Pad # --- -def _test_pad(data): +def _test_pad(data, quantized=False): """ One iteration of PAD """ assert len(data) == 2 # Test with tensor and constant with tf.Graph().as_default(): -in_data = [array_ops.placeholder(shape=data[0].shape, dtype=data[0].dtype, name='in')] -out = array_ops.pad(in_data[0], ops.convert_to_tensor(data[1], dtype=data[1].dtype)) -compare_tflite_with_tvm([data[0]], ['in:0'], in_data, [out]) +in_data = [array_ops.placeholder(shape=data[0].shape, dtype='float32', name='in')] + +if quantized: +# fake_quant will keep the tensors in float32 until the conversion in the session +inq_data = [tf.quantization.fake_quant_with_min_max_args(in_data[0], + min=-100, + max=100, + name="inq_0")] Review comment: Can you please rebase and pass the 'input_range' to the convertor as in here #4789 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-tvm] LiangHao151941 edited a comment on issue #4828: [QNN][TFLite] TFLite rounding mode support
LiangHao151941 edited a comment on issue #4828: [QNN][TFLite] TFLite rounding mode support URL: https://github.com/apache/incubator-tvm/pull/4828#issuecomment-583328143 just modified add/mul/concat requantize rounding mode and tested, no luck. will change the default rounding behavior for a later test. update: I forced FixedPointMultiply(PerChannel) rounding mode to be TFLITE, but still unable to get bit-exact results. one more thing, setting tflite default rounding mode to TFLITE seems to break GPU test of mobilenet_v2, maybe you guys have any ideas/suggestions? @FrozenGene @anijain2305 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-tvm] jikechao opened a new issue #4839: Nvcc fatal : Value ‘sm_75’ is not defined for option ‘gpu-architecture’
jikechao opened a new issue #4839: Nvcc fatal : Value ‘sm_75’ is not defined for option ‘gpu-architecture’ URL: https://github.com/apache/incubator-tvm/issues/4839 Thanks for participating in the TVM community! We use https://discuss.tvm.ai for any general usage questions and discussions. The issue tracker is used for actionable items such as feature proposals discussion, roadmaps, and bug tracking. You are always welcomed to post on the forum first :) Issues that are inactive for a period of time may get closed. We adopt this policy so that we won't lose track of actionable issues that may fall at the bottom of the pile. Feel free to reopen a new one if you feel there is an additional problem that needs attention when an old one gets closed. For bug reports, to help the developer act on the issues, please include a description of your environment, preferably a minimum script to reproduce the problem. For feature proposals, list clear, small actionable items so we can track the progress of the change. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-tvm] masahi opened a new pull request #4838: [Frontend, ONNX] Add Resize op converter
masahi opened a new pull request #4838: [Frontend, ONNX] Add Resize op converter URL: https://github.com/apache/incubator-tvm/pull/4838 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-tvm] FrozenGene commented on issue #4822: [Frontend][TFLite] Add MIRROR_PAD operator
FrozenGene commented on issue #4822: [Frontend][TFLite] Add MIRROR_PAD operator URL: https://github.com/apache/incubator-tvm/pull/4822#issuecomment-583360475 Thanks @wyc-ruiker @inadob It is merged now. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-tvm] FrozenGene merged pull request #4822: [Frontend][TFLite] Add MIRROR_PAD operator
FrozenGene merged pull request #4822: [Frontend][TFLite] Add MIRROR_PAD operator URL: https://github.com/apache/incubator-tvm/pull/4822 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[incubator-tvm] branch master updated (5a13575 -> 30b7d83)
This is an automated email from the ASF dual-hosted git repository. zhaowu pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/incubator-tvm.git. from 5a13575 [Relay][Frontend][TFlite] Add support for quantized LOGISTIC (#4696) add 30b7d83 [Frontend][TFLite] Add MIRROR_PAD operator (#4822) No new revisions were added by this update. Summary of changes: python/tvm/relay/frontend/tflite.py | 43 ++-- tests/python/frontend/tflite/test_forward.py | 8 -- 2 files changed, 47 insertions(+), 4 deletions(-)
[GitHub] [incubator-tvm] FrozenGene commented on issue #4822: [Frontend][TFLite] Add MIRROR_PAD operator
FrozenGene commented on issue #4822: [Frontend][TFLite] Add MIRROR_PAD operator URL: https://github.com/apache/incubator-tvm/pull/4822#issuecomment-583360008 > LGTM @inadob Could you do the approve operation as this guide? https://docs.tvm.ai/contribute/code_review.html#approve-and-request-changes-explicitly Thanks! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-tvm] inadob commented on issue #4822: [Frontend][TFLite] Add MIRROR_PAD operator
inadob commented on issue #4822: [Frontend][TFLite] Add MIRROR_PAD operator URL: https://github.com/apache/incubator-tvm/pull/4822#issuecomment-583359091 LGTM This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-tvm] LiangHao151941 commented on issue #4828: [QNN][TFLite] TFLite rounding mode support
LiangHao151941 commented on issue #4828: [QNN][TFLite] TFLite rounding mode support URL: https://github.com/apache/incubator-tvm/pull/4828#issuecomment-583328143 > cc @LiangHao151941 Could you try this way to do bit-extract testing? For conveniently, you could simply change the C++ default rounding value to "TFLITE" to test. just modified add/mul/concat requantize rounding mode and tested, no luck. will change the default rounding behavior for a later test. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[incubator-tvm] branch master updated (4c961dd -> 5a13575)
This is an automated email from the ASF dual-hosted git repository. zhaowu pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/incubator-tvm.git. from 4c961dd [Doc] ConvertLayout - Call RemoveUnunsedFunctions. add 5a13575 [Relay][Frontend][TFlite] Add support for quantized LOGISTIC (#4696) No new revisions were added by this update. Summary of changes: python/tvm/relay/frontend/tflite.py | 26 ++ tests/python/frontend/tflite/test_forward.py | 18 +- 2 files changed, 39 insertions(+), 5 deletions(-)
[GitHub] [incubator-tvm] FrozenGene merged pull request #4696: [Relay][Frontend][TFlite] Add support for quantized LOGISTIC
FrozenGene merged pull request #4696: [Relay][Frontend][TFlite] Add support for quantized LOGISTIC URL: https://github.com/apache/incubator-tvm/pull/4696 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-tvm] FrozenGene commented on issue #4696: [Relay][Frontend][TFlite] Add support for quantized LOGISTIC
FrozenGene commented on issue #4696: [Relay][Frontend][TFlite] Add support for quantized LOGISTIC URL: https://github.com/apache/incubator-tvm/pull/4696#issuecomment-583327668 Thanks @inadob @anijain2305 @u99127 This is merged. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-tvm] FrozenGene commented on issue #4828: [QNN][TFLite] TFLite rounding mode support
FrozenGene commented on issue #4828: [QNN][TFLite] TFLite rounding mode support URL: https://github.com/apache/incubator-tvm/pull/4828#issuecomment-583326430 > I like this way > > > Another way is we provide rounding args for qnn.add / qnn.mul / qnn.conconcate, because they use requantize in fact too, so they need rounding. cc @LiangHao151941 Could you try this way to do bit-extract testing? For conveniently, you could simply change the C++ default rounding value to "TFLITE" to test. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-tvm] LiangHao151941 commented on a change in pull request #4828: [QNN][TFLite] TFLite rounding mode support
LiangHao151941 commented on a change in pull request #4828: [QNN][TFLite]
TFLite rounding mode support
URL: https://github.com/apache/incubator-tvm/pull/4828#discussion_r376313583
##
File path: include/tvm/relay/qnn/attrs.h
##
@@ -44,14 +44,16 @@ struct RequantizeAttrs : public
tvm::AttrsNode {
.set_default(-1);
TVM_ATTR_FIELD(rounding).set_default("UPWARD")
.describe("Defines the rounding direction when the value is midway
between"
- "two representable values. There are two supported modes -
UPWARD"
- "or TONEAREST. Both modes behave exactly same except at the"
+ "two representable values. There are two 3 modes - UPWARD,
TONEAREST"
+ "or TFLITE. Both modes behave exactly same except at the"
Review comment:
updated
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] LiangHao151941 commented on a change in pull request #4828: [QNN][TFLite] TFLite rounding mode support
LiangHao151941 commented on a change in pull request #4828: [QNN][TFLite]
TFLite rounding mode support
URL: https://github.com/apache/incubator-tvm/pull/4828#discussion_r376311994
##
File path: src/relay/qnn/util.cc
##
@@ -22,13 +22,49 @@
* \brief Utility functions for QNN.
*/
+#include
+
#include "util.h"
#include "../pass/pattern_util.h"
namespace tvm {
namespace relay {
namespace qnn {
+/* \brief This function implements the rounding part of ARMv7 NEON VQRDMULH
+ * instruction. For code reuse, the multiplied tensor is directly passed in
+ * as parameter. Reference implementation:
+ *
https://github.com/google/gemmlowp/blob/2483d846ad865dd4190fe4a1a1ba2d9cfcea78e1/fixedpoint/fixedpoint.h#L337
Review comment:
Sure, I'll respect that
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] FrozenGene commented on a change in pull request #4828: [QNN][TFLite] TFLite rounding mode support
FrozenGene commented on a change in pull request #4828: [QNN][TFLite] TFLite
rounding mode support
URL: https://github.com/apache/incubator-tvm/pull/4828#discussion_r376286808
##
File path: include/tvm/relay/qnn/attrs.h
##
@@ -44,14 +44,16 @@ struct RequantizeAttrs : public
tvm::AttrsNode {
.set_default(-1);
TVM_ATTR_FIELD(rounding).set_default("UPWARD")
.describe("Defines the rounding direction when the value is midway
between"
- "two representable values. There are two supported modes -
UPWARD"
- "or TONEAREST. Both modes behave exactly same except at the"
+ "two representable values. There are two 3 modes - UPWARD,
TONEAREST"
+ "or TFLITE. Both modes behave exactly same except at the"
Review comment:
Both modes -> UPWARD / TONEAREST
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] anijain2305 commented on issue #4828: [QNN][TFLite] TFLite rounding mode support
anijain2305 commented on issue #4828: [QNN][TFLite] TFLite rounding mode support URL: https://github.com/apache/incubator-tvm/pull/4828#issuecomment-583292788 I like this way > Another way is we provide rounding args for qnn.add / qnn.mul / qnn.conconcate, because they use requantize in fact too, so they need rounding. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-tvm] wweic merged pull request #4834: [Doc] ConvertLayout - Call RemoveUnunsedFunctions.
wweic merged pull request #4834: [Doc] ConvertLayout - Call RemoveUnunsedFunctions. URL: https://github.com/apache/incubator-tvm/pull/4834 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[incubator-tvm] branch master updated (e578777 -> 4c961dd)
This is an automated email from the ASF dual-hosted git repository. wweic pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/incubator-tvm.git. from e578777 Improve tol to resolve flaky case (#4836) add 4c961dd [Doc] ConvertLayout - Call RemoveUnunsedFunctions. No new revisions were added by this update. Summary of changes: docs/dev/convert_layout.rst | 6 +- 1 file changed, 5 insertions(+), 1 deletion(-)
[GitHub] [incubator-tvm] FrozenGene commented on issue #4828: [QNN][TFLite] TFLite rounding mode support
FrozenGene commented on issue #4828: [QNN][TFLite] TFLite rounding mode support URL: https://github.com/apache/incubator-tvm/pull/4828#issuecomment-583285949 > I thought little more about the bit exact problem. One source of discrepancy for certain is QNN add, and QNN concatenate ops. These call Requantize internally, and they will have default rounding in C++ (UPWARD). For testing, @LiangHao151941 , you can set the C++ default to TFLIte to see if it helps. Meanwhile, we can also think how to can make that C++ rounding visible at python level. Maybe we could use `@register_func` to register one function named as `@register_func(qnn.requantize.rounding)` in TFLite parser, then we could get the value in `RequantizeQnnCanonicalize` from C++. However, this will have one problem that if users specify the rounding value, it won't work, because we don't know whether the rounding value is setting by default or by user explicitly. So, if we support it this, we will constraint we will only one rounding setting by TFLite parser. Another way is we provide `rounding` args for `qnn.add / qnn.mul / qnn.conconcate`, because they use `requantize` in fact too, so they need `rounding`. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-tvm] kazum commented on a change in pull request #4497: [Relay] Add a PyTorch to Relay Parser
kazum commented on a change in pull request #4497: [Relay] Add a PyTorch to
Relay Parser
URL: https://github.com/apache/incubator-tvm/pull/4497#discussion_r376260979
##
File path: python/tvm/relay/frontend/pytorch.py
##
@@ -0,0 +1,1036 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+# pylint: disable=import-self, too-many-lines, len-as-condition,
no-else-return, unused-variable, too-many-nested-blocks
+# pylint: disable=consider-iterating-dictionary, invalid-name,
unused-argument, unused-variable, broad-except
+"""PT: PyTorch frontend."""
+import numpy as np
+
+import tvm
+
+from .. import analysis as _analysis
+from .. import expr as _expr
+from .. import module as _module
+from .. import op as _op
+from .common import get_relay_op
+from .common import infer_shape as _infer_shape
+
+__all__ = ["from_pytorch"]
+
+# operator implementation
+def _elemwise(name):
+def _impl(inputs, input_types):
+# TODO: Figure out a better way to get typing to work for tensor +
scalar
+type0 = input_types[0]
+if isinstance(inputs[1], _expr.Expr):
+type0 = input_types[1]
+
+type1 = input_types[1]
+if isinstance(inputs[0], _expr.Expr):
+type1 = input_types[0]
+
+data0 = _convert_elemwise_input(inputs[0], type0)
+data1 = _convert_elemwise_input(inputs[1], type1)
+
+return get_relay_op(name)(data0, data1)
+return _impl
+
+def _unsqueeze():
+def _impl(inputs, input_types):
+data = inputs[0]
+axis = inputs[1]
+
+return _op.transform.expand_dims(data, int(axis), 1)
+return _impl
+
+def _concatenate():
+def _impl(inputs, input_types):
+data = inputs[0]
+axis = inputs[1]
+
+if isinstance(data, _expr.Expr):
+data = [data]
+
+return _op.tensor.concatenate(data, int(axis))
+return _impl
+
+def _slice():
+def _impl(inputs, input_types):
+data = inputs[0]
+strides = []
+
+if isinstance(data, _expr.Expr):
+inferred_shape = _infer_shape(data)
+end = []
+for infer in inferred_shape:
+end.append(int(infer))
+if isinstance(data, _expr.Var):
+end = inferred_shape
+end = list(end)
+else:
+end = data.shape
+
+begin = [0]*len(end)
+dim = int(inputs[1])
+begin[dim] = int(inputs[2])
+
+if isinstance(inputs[3], str) and inputs[3].isdigit():
+end[dim] = min(end[dim], int(inputs[3]))
+else:
+end[dim] = inputs[3]
+
+strides.append(int(inputs[4]))
+return _op.transform.strided_slice(data, begin, end, strides)
+return _impl
+
+def _select():
+def _impl(inputs, input_types):
+data = inputs[0]
+dim = int(inputs[1])
+index = int(inputs[2])
+
+return _op.transform.take(data, _expr.const(index, dtype="int32"),
axis=dim)
+return _impl
+
+def _ones():
+def _impl(inputs, input_types):
+data = inputs[0]
+
+import torch
+if isinstance(data, _expr.Expr):
+shape = _infer_shape(data)
+elif isinstance(data, list):
+shape = data
+elif isinstance(data, (torch.Tensor, np.ndarray)):
+shape = data.shape
+else:
+assert "data type {} could not be parsed in ones op" % (type(data))
+
+return _op.full(_expr.const(1), shape,
dtype=_convert_data_type(input_types[0]))
+return _impl
+
+def _zeros():
+def _impl(inputs, input_types):
+data = inputs[0]
+
+import torch
+if isinstance(data, _expr.Expr):
+shape = _infer_shape(data)
+elif isinstance(data, list):
+shape = data
+elif isinstance(data, (torch.Tensor, np.ndarray)):
+shape = data.shape
+else:
+assert "data type {} could not be parsed in zeros op" %
(type(data))
+
+return _op.full(_expr.const(0), shape,
dtype=_convert_data_type(input_types[0]))
+return _impl
+
+def _relu():
+def _impl(inputs, input_types):
+data = inputs[0]
+return _op.nn.relu(data)
+return _impl
+
+def
[GitHub] [incubator-tvm] FrozenGene commented on a change in pull request #4828: [QNN][TFLite] TFLite rounding mode support
FrozenGene commented on a change in pull request #4828: [QNN][TFLite] TFLite
rounding mode support
URL: https://github.com/apache/incubator-tvm/pull/4828#discussion_r376258392
##
File path: python/tvm/relay/frontend/tflite.py
##
@@ -1812,7 +1816,7 @@ def get_tensor_name(subgraph, tensor_idx):
return subgraph.Tensors(tensor_idx).Name().decode("utf-8")
-def from_tflite(model, shape_dict, dtype_dict):
+def from_tflite(model, shape_dict, dtype_dict, rounding="TFLITE"):
Review comment:
Please update the comment doc of `from_tflite`.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] FrozenGene commented on a change in pull request #4828: [QNN][TFLite] TFLite rounding mode support
FrozenGene commented on a change in pull request #4828: [QNN][TFLite] TFLite
rounding mode support
URL: https://github.com/apache/incubator-tvm/pull/4828#discussion_r376257846
##
File path: src/relay/qnn/util.cc
##
@@ -22,13 +22,49 @@
* \brief Utility functions for QNN.
*/
+#include
+
#include "util.h"
#include "../pass/pattern_util.h"
namespace tvm {
namespace relay {
namespace qnn {
+/* \brief This function implements the rounding part of ARMv7 NEON VQRDMULH
+ * instruction. For code reuse, the multiplied tensor is directly passed in
+ * as parameter. Reference implementation:
+ *
https://github.com/google/gemmlowp/blob/2483d846ad865dd4190fe4a1a1ba2d9cfcea78e1/fixedpoint/fixedpoint.h#L337
Review comment:
We would be better remove this so that we won't worry about it. The same
thing is happened in other PRs too. We all suggest authors removing these
links. Thanks.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
