anijain2305 commented on a change in pull request #4277: [ARM][Topi] Improving
Int8 Perf in Spatial Conv2D schedule.
URL: https://github.com/apache/incubator-tvm/pull/4277#discussion_r343954134
##########
File path: topi/python/topi/arm_cpu/conv2d_spatial_pack.py
##########
@@ -93,24 +93,35 @@ def conv2d_spatial_pack_nchw(cfg, data, kernel, strides,
padding, dilation,
ovshape = (N, CO // VC, OH // VH, OW // VW, VH, VW, VC)
oshape = (N, CO, OH, OW)
+ # For Integer convs, upcasting to int16 leads to faster implementation
+ # because LLVM is able to better interleave vmlal.s16 and vldr
instructions,
+ # leading to higher CPU utilization.
+ adjusted_dtype = data.dtype
+ if 'int8' in data.dtype and 'int8' in kernel.dtype and out_dtype ==
'int32':
+ adjusted_dtype = 'int16'
+
if dilation_h != 1 or dilation_w != 1:
# undilate input data
dvshape = (N, OH // VH, OW // VW, CI, KH, KW, VH, VW)
data_vec = tvm.compute(dvshape, lambda n, h, w, ci, kh, kw, vh, vw:
data_pad[n][ci][(h*VH+vh)*HSTR+kh*dilation_h]
- [(w*VW+vw)*WSTR+kw*dilation_w],
+
[(w*VW+vw)*WSTR+kw*dilation_w].astype(adjusted_dtype),
name='data_vec_undilated')
else:
dvshape = (N, OH // VH, OW // VW, CI, VH*HSTR + KH-1, VW*WSTR + KW-1)
data_vec = tvm.compute(dvshape, lambda n, h, w, ci, vh, vw:
- data_pad[n][ci][h*VH*HSTR+vh][w*VW*WSTR+vw],
+
data_pad[n][ci][h*VH*HSTR+vh][w*VW*WSTR+vw].astype(adjusted_dtype),
name='data_vec')
if pre_packed:
kernel_vec = kernel
+ if adjusted_dtype != kernel.dtype:
+ kernel_vec = tvm.compute(kvshape, lambda co, ci, kh, kw, vc:
Review comment:
An alternative to this PR can be a legalize pass - which when sees a conv2d
with int8 inputs, inserts an upcast to int16. In this case, the upcast will
show up at Relay level, and the topi will remain unchanged.
In both the cases, the memory footprint takes a hit as we upcast weight to
int16. For the alternate implementation, they show up in artifacts (or disk).
For current PR, the allocation happens internally in the conv operator.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
