rocketguan opened a new issue, #6983: URL: https://github.com/apache/seatunnel/issues/6983
### Search before asking - [X] I had searched in the [issues](https://github.com/apache/seatunnel/issues?q=is%3Aissue+label%3A%22bug%22) and found no similar issues. ### What happened 我在本地使用seatunnel engine和datax进行性能比对验证，我配置了一个mysql到mysql数据采集任务，配置信息如下 env { execution.parallelism = 5 job.mode = "BATCH" } source{ Jdbc { parallelism=5 partition_num="20" url = "jdbc:mysql://127.0.0.1:3306/mysql_src" driver = "com.mysql.cj.jdbc.Driver" connection_check_timeout_sec = 100 user = "root" password = "111111" table_path="mysql_src.product_catalog" split.size = 10000 } } transform { } sink { jdbc { url = "jdbc:mysql://127.0.0.1:3306/mysql_dest" driver = "com.mysql.cj.jdbc.Driver" connection_check_timeout_sec = 100 user = "root" password = "*****" enable_upsert = false query = "insert into product_catalog(product_id,catalog_name) values(?,?)" } } 运行后发现速度很慢，每秒处理速度只有几百条，具体统计信息如下 *********************************************** Job Progress Information *********************************************** Job Id : 853820592841818113 Read Count So Far : 25365 Write Count So Far : 15120 Average Read Count : 422/s Average Write Count : 252/s Last Statistic Time : 2024-06-14 10:18:13 Current Statistic Time : 2024-06-14 10:19:13 *********************************************** 我同时尝试用datax采集同样的源表并插入到相同的目标表 { "content":[ { "reader":{ "name":"mysqlreader", "parameter":{ "column":[ "product_id", "catalog_name" ], "connection":[ { "jdbcUrl":[ "jdbc:mysql://localhost:3306/mysql_src" ], "table":[ "product_catalog" ] } ], "password":"******", "splitPk":"product_id", "username":"root" } }, "writer":{ "name":"mysqlwriter", "parameter":{ "batchSize":1024, "column":[ "product_id", "catalog_name" ], "connection":[ { "jdbcUrl":"jdbc:mysql://localhost:3306/mysql_dest", "table":[ "product_catalog" ] } ], "password":"******", "username":"root", "writeMode":"insert" } } } ], "setting":{ "speed":{ "channel":"1" } } } 任务启动时刻 : 2024-06-14 09:02:49 任务结束时刻 : 2024-06-14 09:04:00 任务总计耗时 : 70s 任务平均流量 : 572.77KB/s 记录写入速度 : 37449rec/s 读出记录总数 : 2621440 读写失败总数 : 0 每秒处理速度能达到3W+ seatunnel engine运行速度严重低于预期，想请教下我应该如何调整配置，提升运行速度呢？ ### SeaTunnel Version 2.3.5 ### SeaTunnel Config ```conf env { execution.parallelism = 5 job.mode = "BATCH" } source{ Jdbc { parallelism=5 partition_num="20" url = "jdbc:mysql://127.0.0.1:3306/mysql_src" driver = "com.mysql.cj.jdbc.Driver" connection_check_timeout_sec = 100 user = "root" password = "111111" table_path="mysql_src.product_catalog" split.size = 10000 } } transform { } sink { jdbc { url = "jdbc:mysql://127.0.0.1:3306/mysql_dest" driver = "com.mysql.cj.jdbc.Driver" connection_check_timeout_sec = 100 user = "root" password = "*****" enable_upsert = false query = "insert into product_catalog(product_id,catalog_name) values(?,?)" } } ``` ### Running Command ```shell bin/seatunnel.sh --config jobs/mysql2mysql.conf ``` ### Error Exception ```log 运行速度慢 ``` ### Zeta or Flink or Spark Version _No response_ ### Java or Scala Version _No response_ ### Screenshots _No response_ ### Are you willing to submit PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
