yux created FLINK-37741: --------------------------- Summary: Severe performance regression of transform operator node Key: FLINK-37741 URL: https://issues.apache.org/jira/browse/FLINK-37741 Project: Flink Issue Type: Bug Components: Flink CDC Reporter: yux Attachments: image-2025-04-28-15-31-18-308.png
FLINK-37104 refactored transform module with various code cleanup and logic simplification. However, a List<DataType> object is repeatedly created from pre/post transformed schemas, which significantly affects the RPS throughput. !image-2025-04-28-15-31-18-308.png! Based on my local testing, even a simple transform rule would cause a ~40% performance regression. |{color:#000000}*Version*{color}|*{color:#000000}Transform Rule{color}*|*{color:#000000}Time Elapsed{color}*|{color:#172b4d}*RPS*{color}| |{color:#000000}{*}3.4-SNAPSHOT{*}{*}{*}{color}|{color:#000000}Complex{color}|{color:#000000}4m 21s 86ms{color}|{color:#a30003}{*}38,314.18{*}{*}{*}{color}| |{color:#000000}{*}3.4-SNAPSHOT{*}{*}{*}{color}|{color:#000000}Simple{color}|{color:#000000}2m 45s 488ms{color}|{color:#a30003}{*}60,606.06{*}{*}{*}{color}| |{color:#000000}{*}3.4-SNAPSHOT{*}{*}{*}{color}|{color:#000000}Empty{color}|{color:#000000}1m 31s 869ms{color}|{color:#000000}109,890.11{color}| |{color:#000000}{*}3.4-SNAPSHOT{*}{*}{*}{color}|{color:#000000}Off{color}|{color:#000000}1m 45s 587ms{color}|{color:#000000}95,238.10{color}| |{color:#000000}{*}3.3{*}{*}{*}{color}|{color:#000000}Complex{color}|{color:#000000}1m 46s 54ms{color}|{color:#000000}94,339.62{color}| |{color:#000000}{*}3.3{*}{*}{*}{color}|{color:#000000}Simple{color}|{color:#000000}1m 34s 971ms{color}|{color:#000000}106,382.98{color}| |{color:#000000}{*}3.3{*}{*}{*}{color}|{color:#000000}Empty{color}|{color:#000000}1m 21s 546ms{color}|{color:#000000}123,456.79{color}| |{color:#000000}{*}3.3{*}{*}{*}{color}|{color:#000000}Off{color}|{color:#000000}1m 34s 5ms{color}|{color:#000000}106,382.98{color}| -- This message was sent by Atlassian Jira (v8.20.10#820010)