ponyliuh opened a new issue, #5416: URL: https://github.com/apache/seatunnel/issues/5416
### Search before asking - [X] I had searched in the [issues](https://github.com/apache/seatunnel/issues?q=is%3Aissue+label%3A%22bug%22) and found no similar issues. ### What happened Hello when i use seatunnel2.3.3 Export data from hive to MySQL，got a error。 but when i use 2.3.2，it does not happen ### SeaTunnel Version 2.3.3 ### SeaTunnel Config ```conf env { parallelism = 12 job.mode = "STREAMING" job.name = "SeaTunnel-dm_eagle.eagle_migration_rate_month-hiveToMysql" hive.execution.engine=spark spark.yarn.queue=root.daily spark.executor.memory=4g spark.num-executors=4 spark.executor.cores=2 spark.driver.memory=4g spark.executor.instances=4 } source { Hive { table_name = "dm_eagle.eagle_migration_rate_month" metastore_uri = "thrift://xxx" } } transform { } sink { jdbc { url = "jdbc:mysql://xxxx/dm_eagle?rewriteBatchedStatements=true&autoReconnect=true&useCompression=true&timeout=3000&idle-timeout=3000" driver = "com.mysql.cj.jdbc.Driver" user = "xx" password = "xxxxxx" query = "insert into eagle_migration_rate_month (capital_id,channel_id,product_id,loan_terms,overdue_stage,mob,loan_month,loan_amount,remain_principal,biz_month,project_id) values(?,?,?,?,?,?,?,?,?,?,?)" } } ``` ### Running Command ```shell Execute SeaTunnel Spark Job: ${SPARK_HOME}/bin/spark-submit --class "org.apache.seatunnel.core.starter.spark.SeaTunnelSpark" --name "SeaTunnel" --master "yarn" --deploy-mode "cluster" --jars "/data/share/apache-seatunnel-2.3.3/lib/mysql-connector-java-8.0.28.jar,/data/share/apache-seatunnel-2.3.3/lib/seatunnel-hadoop3-3.1.4-uber-2.3.3-optional.jar,/data/share/apache-seatunnel-2.3.3/lib/seatunnel-transforms-v2.jar,/data/share/apache-seatunnel-2.3.3/connectors/seatunnel/connector-jdbc-2.3.3.jar,/data/share/apache-seatunnel-2.3.3/connectors/seatunnel/connector-hive-2.3.3.jar" --files "/data/share/apache-seatunnel-2.3.3/plugins.tar.gz,/data/share/apache-seatunnel-2.3.3/config/spark.batch.dm_eagle.eagle_migration_rate_month.hive_to_mysql.template.mY4Q6C" --conf "spark.executor.memory=4g" --conf "spark.driver.memory=4g" --conf "job.mode=STREAMING" --conf "spark.num-executors=4" --conf "parallelism=12" --conf "spark.executor.cores=2" --conf "job.name=SeaTunnel-dm_eagle.eagle_migration_ra te_month-hiveToMysql" --conf "spark.executor.instances=4" --conf "spark.yarn.queue=root.daily" --conf "hive.execution.engine=spark" /data/share/apache-seatunnel-2.3.3/starter/seatunnel-spark-2-starter.jar --config "/data/share/apache-seatunnel-2.3.3/config/spark.batch.dm_eagle.eagle_migration_rate_month.hive_to_mysql.template.mY4Q6C" --master "yarn" --deploy-mode "cluster" --name "SeaTunnel" ``` ### Error Exception ```log Warning: Ignoring non-spark config property: hive.execution.engine=spark Warning: Ignoring non-spark config property: job.mode=STREAMING Warning: Ignoring non-spark config property: job.name=SeaTunnel-dm_eagle.eagle_migration_rate_month-hiveToMysql Warning: Ignoring non-spark config property: parallelism=12 23/09/01 14:40:32 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 23/09/01 14:40:32 WARN DomainSocketFactory: The short-circuit local reads feature cannot be used because libhadoop cannot be loaded. 23/09/01 14:40:33 WARN Client: Neither spark.yarn.jars nor spark.yarn.archive is set, falling back to uploading libraries under SPARK_HOME. 23/09/01 14:41:28 ERROR Client: Application diagnostics message: User class threw exception: org.apache.seatunnel.core.starter.exception.CommandExecuteException: ErrorCode:[FILE-04], ErrorDescription:[File list is empty] - The target file list is empty,SeaTunnel will not be able to sync empty table, please check the configuration parameters such as: [file_filter_pattern] at org.apache.seatunnel.core.starter.spark.command.SparkTaskExecuteCommand.execute(SparkTaskExecuteCommand.java:63) at org.apache.seatunnel.core.starter.SeaTunnel.run(SeaTunnel.java:40) at org.apache.seatunnel.core.starter.spark.SeaTunnelSpark.main(SeaTunnelSpark.java:35) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.spark.deploy.yarn.ApplicationMaster$$anon$2.run(ApplicationMaster.scala:684) Exception in thread "main" org.apache.spark.SparkException: Application application_1638495856834_4700196 finished with failed status at org.apache.spark.deploy.yarn.Client.run(Client.scala:1149) at org.apache.spark.deploy.yarn.YarnClusterApplication.start(Client.scala:1526) at org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:849) at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:167) at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:195) at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:86) at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:924) at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:933) at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala) ``` ### Zeta or Flink or Spark Version spark2.4.3 ### Java or Scala Version jdk1.8 ### Screenshots Execute SeaTunnel Spark Job: ${SPARK_HOME}/bin/spark-submit --class "org.apache.seatunnel.core.starter.spark.SeaTunnelSpark" --name "SeaTunnel" --master "yarn" --deploy-mode "cluster" --jars "/data/share/apache-seatunnel-2.3.3/lib/mysql-connector-java-8.0.28.jar,/data/share/apache-seatunnel-2.3.3/lib/seatunnel-hadoop3-3.1.4-uber-2.3.3-optional.jar,/data/share/apache-seatunnel-2.3.3/lib/seatunnel-transforms-v2.jar,/data/share/apache-seatunnel-2.3.3/connectors/seatunnel/connector-jdbc-2.3.3.jar,/data/share/apache-seatunnel-2.3.3/connectors/seatunnel/connector-hive-2.3.3.jar" --files "/data/share/apache-seatunnel-2.3.3/plugins.tar.gz,/data/share/apache-seatunnel-2.3.3/config/spark.batch.dm_eagle.eagle_migration_rate_month.hive_to_mysql.template.mY4Q6C" --conf "spark.executor.memory=4g" --conf "spark.driver.memory=4g" --conf "job.mode=STREAMING" --conf "spark.num-executors=4" --conf "parallelism=12" --conf "spark.executor.cores=2" --conf "job.name=SeaTunnel-dm_eagle.eagle_migration_ra te_month-hiveToMysql" --conf "spark.executor.instances=4" --conf "spark.yarn.queue=root.daily" --conf "hive.execution.engine=spark" /data/share/apache-seatunnel-2.3.3/starter/seatunnel-spark-2-starter.jar --config "/data/share/apache-seatunnel-2.3.3/config/spark.batch.dm_eagle.eagle_migration_rate_month.hive_to_mysql.template.mY4Q6C" --master "yarn" --deploy-mode "cluster" --name "SeaTunnel" Warning: Ignoring non-spark config property: hive.execution.engine=spark Warning: Ignoring non-spark config property: job.mode=STREAMING Warning: Ignoring non-spark config property: job.name=SeaTunnel-dm_eagle.eagle_migration_rate_month-hiveToMysql Warning: Ignoring non-spark config property: parallelism=12 23/09/01 14:40:32 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 23/09/01 14:40:32 WARN DomainSocketFactory: The short-circuit local reads feature cannot be used because libhadoop cannot be loaded. 23/09/01 14:40:33 WARN Client: Neither spark.yarn.jars nor spark.yarn.archive is set, falling back to uploading libraries under SPARK_HOME. 23/09/01 14:41:28 ERROR Client: Application diagnostics message: User class threw exception: org.apache.seatunnel.core.starter.exception.CommandExecuteException: ErrorCode:[FILE-04], ErrorDescription:[File list is empty] - The target file list is empty,SeaTunnel will not be able to sync empty table, please check the configuration parameters such as: [file_filter_pattern] at org.apache.seatunnel.core.starter.spark.command.SparkTaskExecuteCommand.execute(SparkTaskExecuteCommand.java:63) at org.apache.seatunnel.core.starter.SeaTunnel.run(SeaTunnel.java:40) at org.apache.seatunnel.core.starter.spark.SeaTunnelSpark.main(SeaTunnelSpark.java:35) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.spark.deploy.yarn.ApplicationMaster$$anon$2.run(ApplicationMaster.scala:684) Exception in thread "main" org.apache.spark.SparkException: Application application_1638495856834_4700196 finished with failed status at org.apache.spark.deploy.yarn.Client.run(Client.scala:1149) at org.apache.spark.deploy.yarn.YarnClusterApplication.start(Client.scala:1526) at org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:849) at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:167) at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:195) at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:86) at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:924) at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:933) at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala) ### Are you willing to submit PR? - [X] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
