Qianshui-Jiang commented on code in PR #13642:
URL: https://github.com/apache/tvm/pull/13642#discussion_r1058782058
##########
python/tvm/topi/x86/tensor_intrin.py:
##########
@@ -348,3 +348,227 @@ def _instr(index):
binds={data: a_buffer, kernel: b_buffer},
default_buffer_params=buffer_params,
)
+
+
+def dot_32x128x32_u8s8s32_sapphirerapids(LDA):
+ """
+ Int8 dot product by every 16x64 elements using AMX-TMUL Sapphire Rapids
instructions.
+ The tdpxxd instruction takes two tile of uint8 and int8 datatype --
data[16][64] and
+ kernel[1][16][16][4] -- and computes a dot product of data[16][16] in
int32 datatype.
+
+ (Physically, to efficiently leveraging the tile register, we constructing
a 2x2 tiles
+ matmul which performs 32x128x32 in total)
+
+ The pseudo code is as follows:
+ for(k=0; k<2; k++){
+ for(n=0; n<2; n++){
+ tileload64(tmm_b, B)
+ for(m=0; m<2; m++){
+ if(n==0)
+ tileload64(tmm_a, A)
Review Comment:
> Have you considered tensorizing at finer granularity? Meaning, instead of
tensoring the whole load + compute as one tensor intrin, tensorize each load
and compute separately. That could make the hard coded outer-loop factors (2 in
this intrin) tunable.
@masahi Yes we considered to make it works like tensor core which mentioned
here: https://github.com/apache/tvm/issues/4052, but the problem is AMX
intrinsics in LLVM need user to assigned register(tmm) number by hand craft. If
we seperate the load and compute, we have to leave the register assginment to
schedule method, that would bring much redundancy work. So finnally we decide
to follow BRGEMM and leave the whole smallest comute unit for tensorizing.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
