yux created FLINK-37741:
---------------------------

             Summary: Severe performance regression of transform operator node
                 Key: FLINK-37741
                 URL: https://issues.apache.org/jira/browse/FLINK-37741
             Project: Flink
          Issue Type: Bug
          Components: Flink CDC
            Reporter: yux
         Attachments: image-2025-04-28-15-31-18-308.png

FLINK-37104 refactored transform module with various code cleanup and logic 
simplification. However, a List<DataType> object is repeatedly created from 
pre/post transformed schemas, which significantly affects the RPS throughput.

!image-2025-04-28-15-31-18-308.png!

Based on my local testing, even a simple transform rule would cause a ~40% 
performance regression.
|{color:#000000}*Version*{color}|*{color:#000000}Transform 
Rule{color}*|*{color:#000000}Time Elapsed{color}*|{color:#172b4d}*RPS*{color}|
|{color:#000000}{*}3.4-SNAPSHOT{*}{*}{*}{color}|{color:#000000}Complex{color}|{color:#000000}4m
 21s 86ms{color}|{color:#a30003}{*}38,314.18{*}{*}{*}{color}|
|{color:#000000}{*}3.4-SNAPSHOT{*}{*}{*}{color}|{color:#000000}Simple{color}|{color:#000000}2m
 45s 488ms{color}|{color:#a30003}{*}60,606.06{*}{*}{*}{color}|
|{color:#000000}{*}3.4-SNAPSHOT{*}{*}{*}{color}|{color:#000000}Empty{color}|{color:#000000}1m
 31s 869ms{color}|{color:#000000}109,890.11{color}|
|{color:#000000}{*}3.4-SNAPSHOT{*}{*}{*}{color}|{color:#000000}Off{color}|{color:#000000}1m
 45s 587ms{color}|{color:#000000}95,238.10{color}|
|{color:#000000}{*}3.3{*}{*}{*}{color}|{color:#000000}Complex{color}|{color:#000000}1m
 46s 54ms{color}|{color:#000000}94,339.62{color}|
|{color:#000000}{*}3.3{*}{*}{*}{color}|{color:#000000}Simple{color}|{color:#000000}1m
 34s 971ms{color}|{color:#000000}106,382.98{color}|
|{color:#000000}{*}3.3{*}{*}{*}{color}|{color:#000000}Empty{color}|{color:#000000}1m
 21s 546ms{color}|{color:#000000}123,456.79{color}|
|{color:#000000}{*}3.3{*}{*}{*}{color}|{color:#000000}Off{color}|{color:#000000}1m
 34s 5ms{color}|{color:#000000}106,382.98{color}|



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Reply via email to