FrozenGene commented on a change in pull request #4277: [ARM][Topi] Improving
Int8 Perf in Spatial Conv2D schedule.
URL: https://github.com/apache/incubator-tvm/pull/4277#discussion_r344568267
##
File path: topi/python/topi/arm_cpu/conv2d_spatial_pack.py
##
@@ -93,24 +93,35 @@ def conv2d_spatial_pack_nchw(cfg, data, kernel, strides,
padding, dilation,
ovshape = (N, CO // VC, OH // VH, OW // VW, VH, VW, VC)
oshape = (N, CO, OH, OW)
+# For Integer convs, upcasting to int16 leads to faster implementation
+# because LLVM is able to better interleave vmlal.s16 and vldr
instructions,
+# leading to higher CPU utilization.
+adjusted_dtype = data.dtype
+if 'int8' in data.dtype and 'int8' in kernel.dtype and out_dtype ==
'int32':
+adjusted_dtype = 'int16'
+
if dilation_h != 1 or dilation_w != 1:
# undilate input data
dvshape = (N, OH // VH, OW // VW, CI, KH, KW, VH, VW)
data_vec = tvm.compute(dvshape, lambda n, h, w, ci, kh, kw, vh, vw:
data_pad[n][ci][(h*VH+vh)*HSTR+kh*dilation_h]
- [(w*VW+vw)*WSTR+kw*dilation_w],
+
[(w*VW+vw)*WSTR+kw*dilation_w].astype(adjusted_dtype),
name='data_vec_undilated')
else:
dvshape = (N, OH // VH, OW // VW, CI, VH*HSTR + KH-1, VW*WSTR + KW-1)
data_vec = tvm.compute(dvshape, lambda n, h, w, ci, vh, vw:
- data_pad[n][ci][h*VH*HSTR+vh][w*VW*WSTR+vw],
+
data_pad[n][ci][h*VH*HSTR+vh][w*VW*WSTR+vw].astype(adjusted_dtype),
name='data_vec')
if pre_packed:
kernel_vec = kernel
+if adjusted_dtype != kernel.dtype:
+kernel_vec = tvm.compute(kvshape, lambda co, ci, kh, kw, vc:
Review comment:
we only do parallel for kernel_vec before, however, we introduce one compute
now, better way is compute_inline. Could you try this schedule:
```
s[kernel_vec].unroll(kh)
s[kernel_vec].unroll(kw)
s[kernel_vec].vectorize(vc)
s[kernel_vec].parallel(co)
s[kernel_vec].compute_inline()
```
Which is used in our schedule internally and could produce SMLAL instruction
when to cast into int16. However, I can not make sure whether to work here,
because our computation and schedule is not the same.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
