LiJie20190102 opened a new issue, #10032: URL: https://github.com/apache/seatunnel/issues/10032
### Search before asking - [x] I had searched in the [issues](https://github.com/apache/seatunnel/issues?q=is%3Aissue+label%3A%22bug%22) and found no similar issues. ### What happened When Spark submits tasks in the form of '--master yarn --deploy-mode cluster', the transform plugin cannot be discovered,and errors will occur. The log shows that 'seatunnel-transforms-v2-2.3.11. jar' was not uploaded： <img width="2185" height="622" alt="Image" src="https://github.com/user-attachments/assets/09ce4521-115d-4b81-9fb8-9076ca183d06"; /> An error has occurred: <img width="1600" height="1057" alt="Image" src="https://github.com/user-attachments/assets/f6f6a4d6-e2dd-49fd-9bac-e2f200ac968f"; /> ### SeaTunnel Version 2.3.11 ### SeaTunnel Config ```conf transform { Sql { query="SELECT _0 AS _0, _1 AS _1, _2 AS _2 FROM source_table" "source_table_name"="source_table" "result_table_name"="sink_table" } } sink { HdfsFile { "fs.defaultFS"="hdfs://cmss" "file_name_expression"="test_zip" "path "="/tmp/st/sink/hdfs/1014" "file_format_type"=text "source_table_name"="sink_table" "is_enable_transaction"=false "custom_filename"=true "data_save_mode"="ERROR_WHEN_DATA_EXISTS" } } source { HdfsFile { "fs.defaultFS"="hdfs://cmss" schema { fields { "_0"=string "_1"=int "_2"=int } } path="/tmp/st/source/hdfs3" "file_format_type"=text "archive_compress_codec"=ZIP "result_table_name"="source_table" encoding=UTF-8 "field_delimiter"="\\001" } } env { "job.mode"=BATCH "execution.parallelism"="2" } ``` ### Running Command ```shell bin/start-seatunnel-spark-3-connector-v2.sh \ --master yarn \ --deploy-mode cluster \ --config ./config/hdfs2hdfs ``` ### Error Exception ```log 25/11/05 20:02:44 ERROR ApplicationMaster: User class threw exception: org.apache.seatunnel.core.starter.exception.CommandExecuteException: Run SeaTunnel on spark failed org.apache.seatunnel.core.starter.exception.CommandExecuteException: Run SeaTunnel on spark failed at org.apache.seatunnel.core.starter.spark.command.SparkTaskExecuteCommand.execute(SparkTaskExecuteCommand.java:62) at org.apache.seatunnel.core.starter.SeaTunnel.run(SeaTunnel.java:40) at org.apache.seatunnel.core.starter.spark.SeaTunnelSpark.main(SeaTunnelSpark.java:35) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.spark.deploy.yarn.ApplicationMaster$$anon$2.run(ApplicationMaster.scala:739) Caused by: java.lang.RuntimeException: Plugin PluginIdentifier{engineType='seatunnel', pluginType='transform', pluginName='Sql'} not found. at org.apache.seatunnel.plugin.discovery.AbstractPluginDiscovery.createPluginInstance(AbstractPluginDiscovery.java:242) at org.apache.seatunnel.plugin.discovery.AbstractPluginDiscovery.createPluginInstance(AbstractPluginDiscovery.java:182) at org.apache.seatunnel.core.starter.spark.execution.TransformExecuteProcessor.lambda$initializePlugins$0(TransformExecuteProcessor.java:101) at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193) at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1374) at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481) at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471) at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708) at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499) at org.apache.seatunnel.core.starter.spark.execution.TransformExecuteProcessor.initializePlugins(TransformExecuteProcessor.java:111) at org.apache.seatunnel.core.starter.spark.execution.SparkAbstractPluginExecuteProcessor.<init>(SparkAbstractPluginExecuteProcessor.java:60) at org.apache.seatunnel.core.starter.spark.execution.TransformExecuteProcessor.<init>(TransformExecuteProcessor.java:73) at org.apache.seatunnel.core.starter.spark.execution.SparkExecution.<init>(SparkExecution.java:62) at org.apache.seatunnel.core.starter.spark.command.SparkTaskExecuteCommand.execute(SparkTaskExecuteCommand.java:59) ... 7 more 25/11/05 20:02:44 INFO ApplicationMaster: Final app status: FAILED, exitCode: 15, (reason: User class threw exception: org.apache.seatunnel.core.starter.exception.CommandExecuteException: Run SeaTunnel on spark failed at org.apache.seatunnel.core.starter.spark.command.SparkTaskExecuteCommand.execute(SparkTaskExecuteCommand.java:62) at org.apache.seatunnel.core.starter.SeaTunnel.run(SeaTunnel.java:40) at org.apache.seatunnel.core.starter.spark.SeaTunnelSpark.main(SeaTunnelSpark.java:35) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.spark.deploy.yarn.ApplicationMaster$$anon$2.run(ApplicationMaster.scala:739) Caused by: java.lang.RuntimeException: Plugin PluginIdentifier{engineType='seatunnel', pluginType='transform', pluginName='Sql'} not found. at org.apache.seatunnel.plugin.discovery.AbstractPluginDiscovery.createPluginInstance(AbstractPluginDiscovery.java:242) at org.apache.seatunnel.plugin.discovery.AbstractPluginDiscovery.createPluginInstance(AbstractPluginDiscovery.java:182) at org.apache.seatunnel.core.starter.spark.execution.TransformExecuteProcessor.lambda$initializePlugins$0(TransformExecuteProcessor.java:101) at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193) at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1374) at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481) at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471) at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708) at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499) at org.apache.seatunnel.core.starter.spark.execution.TransformExecuteProcessor.initializePlugins(TransformExecuteProcessor.java:111) at org.apache.seatunnel.core.starter.spark.execution.SparkAbstractPluginExecuteProcessor.<init>(SparkAbstractPluginExecuteProcessor.java:60) at org.apache.seatunnel.core.starter.spark.execution.TransformExecuteProcessor.<init>(TransformExecuteProcessor.java:73) at org.apache.seatunnel.core.starter.spark.execution.SparkExecution.<init>(SparkExecution.java:62) at org.apache.seatunnel.core.starter.spark.command.SparkTaskExecuteCommand.execute(SparkTaskExecuteCommand.java:59) ... 7 more ``` ### Zeta or Flink or Spark Version spark 3.x ### Java or Scala Version java 1.8.0_202 ### Screenshots _No response_ ### Are you willing to submit PR? - [x] Yes I am willing to submit a PR! ### Code of Conduct - [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
