[tvm] branch unity updated: [Unity][DLight] Update GEMV Rule for Mali GPUs (#15531)

[tqchen]

This is an automated email from the ASF dual-hosted git repository.

tqchen pushed a commit to branch unity
in repository https://gitbox.apache.org/repos/asf/tvm.git


The following commit(s) were added to refs/heads/unity by this push:
     new a012fe0df4 [Unity][DLight] Update GEMV Rule for Mali GPUs (#15531)
a012fe0df4 is described below

commit a012fe0df48b75297fe7862bfbb9238128b4df6d
Author: Siyuan Feng <hzfen...@sjtu.edu.cn>
AuthorDate: Fri Aug 11 20:57:29 2023 +0800

    [Unity][DLight] Update GEMV Rule for Mali GPUs (#15531)
    
    this PR updates the GEMV rule for Mali GPUs, improving the performance:
    Llama-7B: 2.2 toks/s -> 2.8 toks/s
    RedPajama-3B: 4.5 toks/s -> 5.0 toks/s
---
 python/tvm/dlight/gpu/gemv.py | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/python/tvm/dlight/gpu/gemv.py b/python/tvm/dlight/gpu/gemv.py
index b063883800..c65e6b4f6b 100644
--- a/python/tvm/dlight/gpu/gemv.py
+++ b/python/tvm/dlight/gpu/gemv.py
@@ -447,6 +447,12 @@ class GEMV(ScheduleRule):
                     TS, TR = 4, 32
                 else:
                     TS, TR = 16, 32
+        elif target.kind.name == "opencl" and "mali" in str(target.attrs):
+            VEC_C = 8
+            LOAD_V_SHARED = False
+            LOAD_V_VEC = -1
+            UNROLL = 64
+            TS, TR = 1, 64
         else:
             VEC_C = 1
             LOAD_V_SHARED = False