15810785091 opened a new issue, #9723: URL: https://github.com/apache/seatunnel/issues/9723
### Search before asking - [x] I had searched in the [issues](https://github.com/apache/seatunnel/issues?q=is%3Aissue+label%3A%22bug%22) and found no similar issues. ### What happened For the same configuration file, there is no response to data synchronization after transform is added, and synchronization can be completed in 14 seconds after transfrom is removed. The data set size is 250W,transform only the timestamp of several fields is simply processed and converted into standard time format. I have encountered the same situation when dealing with json before. 之前我解析json的时候以为是处理逻辑太过复杂导致没有响应，然后就放弃了使用seatunnel处理json数据，而专用了别的方法，我发现在数据集比较大的时候transform如果使用就会造成这个现象，或者字段过多？具体可能得测试一下。 ### SeaTunnel Version 2.3.11 ### SeaTunnel Config ```conf env { parallelism = 10 job.mode = "BATCH" job.name = "weiq_cpt_task" } source { jdbc { url = "jdbc:mysql://172.16.111.71:3306/weiq?useSSL=false&useCursorFetch=true&useServerPrepStmts=true&fetchSize=2000&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false" driver = "com.mysql.cj.jdbc.Driver" user = "hadoop_sel" password = "p#sdord8" table_path = "weiq.weiq_cpt_task" partition_column = "id" split.size = 50000 fetch_size = 2000 plugin_output = "mysql_table" } } transform { Sql{ plugin_input = "mysql_table" plugin_output = "sql_table" query = """ SELECT id , taskname , wxtitle , wx_author , paytype , type , plattype , category , FROM_UNIXTIME(createtime, 'yyyy-MM-dd HH:mm:ss') createtime , FROM_UNIXTIME(audittime, 'yyyy-MM-dd HH:mm:ss') audittime , FROM_UNIXTIME(canceltime, 'yyyy-MM-dd HH:mm:ss') canceltime , FROM_UNIXTIME(deltime, 'yyyy-MM-dd HH:mm:ss') deltime , FROM_UNIXTIME(starttime, 'yyyy-MM-dd HH:mm:ss') starttime , FROM_UNIXTIME(stoptime, 'yyyy-MM-dd HH:mm:ss') stoptime , is_fee , sort , FROM_UNIXTIME(endtime, 'yyyy-MM-dd HH:mm:ss') endtime , FROM_UNIXTIME(realusetime, 'yyyy-MM-dd HH:mm:ss') realusetime , status , payerid , agentid , saleid , managerid , reason , resource_status , feestatus , FROM_UNIXTIME(feetime, 'yyyy-MM-dd HH:mm:ss') feetime , operatorid , mediamanagerid , identifier , planid , affiliatedid , idstr , source , customerid , customer_remarksid , pid_complex , discount_rate , discount_code , coupon_id , isonline , isdnr , remarks , process_type , bd_brand_id , weiq_brand_id , kf_brand_id , purchase_name , '${date}' as dt from mysql_table """ } } sink { Console{ plugin_input = "sql_table" } } env { parallelism = 10 job.mode = "BATCH" job.name = "weiq_cpt_task" } source { jdbc { url = "jdbc:mysql://xxx:3306/weiq?useSSL=false&useCursorFetch=true&useServerPrepStmts=true&fetchSize=2000&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false" driver = "com.mysql.cj.jdbc.Driver" user = "xxxx" password = "xxxx" table_path = "weiq.weiq_cpt_task" split.size = 50000 fetch_size = 2000 plugin_output = "mysql_table" } } transform { Sql{ plugin_input = "mysql_table" plugin_output = "sql_table" query = """ SELECT id , taskname , wxtitle , wx_author , paytype , type , plattype , category , FROM_UNIXTIME(createtime, 'yyyy-MM-dd HH:mm:ss') createtime , FROM_UNIXTIME(audittime, 'yyyy-MM-dd HH:mm:ss') audittime , FROM_UNIXTIME(canceltime, 'yyyy-MM-dd HH:mm:ss') canceltime , FROM_UNIXTIME(deltime, 'yyyy-MM-dd HH:mm:ss') deltime , FROM_UNIXTIME(starttime, 'yyyy-MM-dd HH:mm:ss') starttime , FROM_UNIXTIME(stoptime, 'yyyy-MM-dd HH:mm:ss') stoptime , is_fee , sort , FROM_UNIXTIME(endtime, 'yyyy-MM-dd HH:mm:ss') endtime , FROM_UNIXTIME(realusetime, 'yyyy-MM-dd HH:mm:ss') realusetime , status , payerid , agentid , saleid , managerid , reason , resource_status , feestatus , FROM_UNIXTIME(feetime, 'yyyy-MM-dd HH:mm:ss') feetime , operatorid , mediamanagerid , identifier , planid , affiliatedid , idstr , source , customerid , customer_remarksid , pid_complex , discount_rate , discount_code , coupon_id , isonline , isdnr , remarks , process_type , bd_brand_id , weiq_brand_id , kf_brand_id , purchase_name from mysql_table """ } } sink { Hive { plugin_input = "sql_table" table_name = "temp.org_task_cpt_weiq" metastore_uri = "thrift://xxxx:9083" hdfs_site_path = "/etc/hadoop/conf/hdfs-site.xml" overwrite = "true" } } ``` ### Running Command ```shell /bin/seatunnel.sh --config /tmp/dolphinscheduler/exec/process/root/8400880170624/15089838574336_106/135251/491568/seatunnel_135251_491568.conf --deploy-mode cluster ``` ### Error Exception ```log No errors reported.Only a long wait！ ``` ### Zeta or Flink or Spark Version Zeta ### Java or Scala Version _No response_ ### Screenshots <img width="1595" height="919" alt="Image" src="https://github.com/user-attachments/assets/32a9f712-5479-49e3-839b-ebc065d3059f"; /> No response, finally killed ### Are you willing to submit PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
