CommandCV opened a new issue, #7864: URL: https://github.com/apache/seatunnel/issues/7864
### Search before asking - [X] I had searched in the [issues](https://github.com/apache/seatunnel/issues?q=is%3Aissue+label%3A%22bug%22) and found no similar issues. ### What happened I submitted a OssFile sink job to seatunnel cluster, then seatunnel cluster throw a `ClassNotFoundException`. ### SeaTunnel Version 2.3.8 ### SeaTunnel Config ```conf { "env": { "job.name": "oss-test", "job.mode": "batch", "parallelism": 1 }, "source": [ { "plugin_name": "OSSFile", "bucket": "oss://test-dev", "path": "/teacher.csv", "access_key": "************************", "access_secret": "******************************", "endpoint": "oss-cn-hongkong.aliyuncs.com", "file_format_type": "csv", "schema": { "fields": { "name": "string" } }, "read_columns": [ "name" ], "parse_partition_from_path": true, "date_format": "yyyy-MM-dd", "datetime_format": "yyyy-MM-dd HH:mm:ss", "time_format": "HH:mm:ss", "compress_codec": "none", "encoding": "UTF-8", "delimiter": ",", "skip_header_row_number": 0, "xml_use_attr_format": true } ], "transform": [ { "plugin_name": "Copy", "fields": { "copy_name": "name" } } ], "sink": [ { "plugin_name": "OSSFile", "bucket": "oss://test-dev", "path": "/test", "tmp_path": "/tmp", "access_key": "************************", "access_secret": "******************************", "endpoint": "oss-cn-hongkong.aliyuncs.com", "file_format_type": "csv", "custom_filename": true, "file_name_expression": "teacher_copy_${transactionId}.csv", "filename_time_format": "yyyy.MM.dd", "have_partition": false, "partition_dir_expression": "${k0}=${v0}/${k1}=${v1}/.../${kn}=${vn}/", "is_partition_field_write_in_file": false, "sink_columns": [ "name", "copy_name" ], "is_enable_transaction": true, "batch_size": 100000, "compress_codec": "none", "encoding": "UTF-8", "max_rows_in_memory": 10000, "xml_use_attr_format": true, "parquet_avro_write_timestamp_as_int96": false } ] } ``` ### Running Command ```shell 1. Install plugins and dependencies. cd ${SEATUNNEL_HOME} && sh bin/install-plugin.sh \ && wget -P ${SEATUNNEL_HOME}/lib https://repo1.maven.org/maven2/com/mysql/mysql-connector-j/8.0.33/mysql-connector-j-8.0.33.jar \ && wget -P ${SEATUNNEL_HOME}/lib https://repo1.maven.org/maven2/org/apache/hive/hive-jdbc/3.1.3/hive-jdbc-3.1.3-standalone.jar \ && wget -P ${SEATUNNEL_HOME}/lib https://repo1.maven.org/maven2/org/apache/hadoop/hadoop-aws/3.1.4/hadoop-aws-3.1.4.jar \ && wget -P ${SEATUNNEL_HOME}/lib https://repo1.maven.org/maven2/com/amazonaws/aws-java-sdk-bundle/1.12.692/aws-java-sdk-bundle-1.12.692.jar \ && wget -P ${SEATUNNEL_HOME}/lib https://repo1.maven.org/maven2/com/aliyun/oss/aliyun-sdk-oss/3.4.1/aliyun-sdk-oss-3.4.1.jar \ && wget -P ${SEATUNNEL_HOME}/lib https://repo1.maven.org/maven2/org/apache/hadoop/hadoop-aliyun/3.1.4/hadoop-aliyun-3.1.4.jar \ && wget -P ${SEATUNNEL_HOME}/lib https://repo1.maven.org/maven2/org/jdom/jdom/1.1/jdom-1.1.jar ``` 2.Start seatunnel cluster `seatunnel-cluster.sh --cluster seatunnel -d` ``` ### Error Exception ```log 2024-10-16 20:29:54,845 ERROR [o.a.s.c.s.SeaTunnel ] [main] - =============================================================================== Exception in thread "main" org.apache.seatunnel.core.starter.exception.CommandExecuteException: SeaTunnel job executed failed at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:213) at org.apache.seatunnel.core.starter.SeaTunnel.run(SeaTunnel.java:40) at org.apache.seatunnel.core.starter.seatunnel.SeaTunnelClient.main(SeaTunnelClient.java:34) Caused by: java.util.concurrent.CompletionException: org.apache.seatunnel.engine.common.exception.JobException: java.lang.NoClassDefFoundError: org/apache/seatunnel/api/kerberos/KerberosConfig at java.base/java.lang.ClassLoader.defineClass1(Native Method) at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1027) at java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:150) at java.base/java.net.URLClassLoader.defineClass(URLClassLoader.java:524) at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:427) at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:421) at java.base/java.security.AccessController.doPrivileged(AccessController.java:714) at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:420) at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:593) at org.apache.seatunnel.engine.common.loader.SeaTunnelBaseClassLoader.loadClassWithoutExceptionHandling(SeaTunnelBaseClassLoader.java:56) at org.apache.seatunnel.engine.common.loader.SeaTunnelChildFirstClassLoader.loadClassWithoutExceptionHandling(SeaTunnelChildFirstClassLoader.java:88) at org.apache.seatunnel.engine.common.loader.SeaTunnelBaseClassLoader.loadClass(SeaTunnelBaseClassLoader.java:47) at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:526) at org.apache.seatunnel.connectors.seatunnel.file.sink.BaseMultipleTableFileSink.getSaveModeHandler(BaseMultipleTableFileSink.java:131) at org.apache.seatunnel.engine.server.master.JobMaster.handleSaveMode(JobMaster.java:372) at org.apache.seatunnel.engine.server.master.JobMaster.handleSaveMode(JobMaster.java:383) at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184) at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197) at java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:179) at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197) at java.base/java.util.Iterator.forEachRemaining(Iterator.java:133) at java.base/java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1939) at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509) at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499) at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151) at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174) at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:596) at org.apache.seatunnel.engine.server.master.JobMaster.init(JobMaster.java:242) at org.apache.seatunnel.engine.server.CoordinatorService.lambda$submitJob$4(CoordinatorService.java:499) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:572) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) at java.base/java.lang.Thread.run(Thread.java:1583) Caused by: java.lang.ClassNotFoundException: org.apache.seatunnel.api.kerberos.KerberosConfig at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:445) at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:593) at org.apache.seatunnel.engine.common.loader.SeaTunnelBaseClassLoader.loadClassWithoutExceptionHandling(SeaTunnelBaseClassLoader.java:56) at org.apache.seatunnel.engine.common.loader.SeaTunnelChildFirstClassLoader.loadClassWithoutExceptionHandling(SeaTunnelChildFirstClassLoader.java:88) at org.apache.seatunnel.engine.common.loader.SeaTunnelBaseClassLoader.loadClass(SeaTunnelBaseClassLoader.java:47) at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:526) ... 35 more at com.hazelcast.spi.impl.AbstractInvocationFuture.wrapInCompletionException(AbstractInvocationFuture.java:1347) at com.hazelcast.spi.impl.AbstractInvocationFuture.cascadeException(AbstractInvocationFuture.java:1340) at com.hazelcast.spi.impl.AbstractInvocationFuture.access$200(AbstractInvocationFuture.java:65) at com.hazelcast.spi.impl.AbstractInvocationFuture$ApplyNode.execute(AbstractInvocationFuture.java:1478) at com.hazelcast.spi.impl.AbstractInvocationFuture.unblockOtherNode(AbstractInvocationFuture.java:797) at com.hazelcast.spi.impl.AbstractInvocationFuture.unblockAll(AbstractInvocationFuture.java:759) at com.hazelcast.spi.impl.AbstractInvocationFuture.complete0(AbstractInvocationFuture.java:1235) at com.hazelcast.spi.impl.AbstractInvocationFuture.completeExceptionallyInternal(AbstractInvocationFuture.java:1223) at com.hazelcast.spi.impl.AbstractInvocationFuture.completeExceptionally(AbstractInvocationFuture.java:709) at com.hazelcast.client.impl.spi.impl.ClientInvocation.completeExceptionally(ClientInvocation.java:294) at com.hazelcast.client.impl.spi.impl.ClientInvocation.notifyExceptionWithOwnedPermission(ClientInvocation.java:321) at com.hazelcast.client.impl.spi.impl.ClientInvocation.notifyException(ClientInvocation.java:304) at com.hazelcast.client.impl.spi.impl.ClientResponseHandlerSupplier.handleResponse(ClientResponseHandlerSupplier.java:164) at com.hazelcast.client.impl.spi.impl.ClientResponseHandlerSupplier.process(ClientResponseHandlerSupplier.java:141) at com.hazelcast.client.impl.spi.impl.ClientResponseHandlerSupplier.access$300(ClientResponseHandlerSupplier.java:60) at com.hazelcast.client.impl.spi.impl.ClientResponseHandlerSupplier$DynamicResponseHandler.accept(ClientResponseHandlerSupplier.java:251) at com.hazelcast.client.impl.spi.impl.ClientResponseHandlerSupplier$DynamicResponseHandler.accept(ClientResponseHandlerSupplier.java:243) at com.hazelcast.client.impl.connection.tcp.TcpClientConnection.handleClientMessage(TcpClientConnection.java:245) at com.hazelcast.client.impl.protocol.util.ClientMessageDecoder.handleMessage(ClientMessageDecoder.java:135) at com.hazelcast.client.impl.protocol.util.ClientMessageDecoder.onRead(ClientMessageDecoder.java:89) at com.hazelcast.internal.networking.nio.NioInboundPipeline.process(NioInboundPipeline.java:136) at com.hazelcast.internal.networking.nio.NioThread.processSelectionKey(NioThread.java:383) at com.hazelcast.internal.networking.nio.NioThread.processSelectionKeys(NioThread.java:368) at com.hazelcast.internal.networking.nio.NioThread.selectLoop(NioThread.java:294) at com.hazelcast.internal.networking.nio.NioThread.executeRun(NioThread.java:249) at com.hazelcast.internal.util.executor.HazelcastManagedThread.run(HazelcastManagedThread.java:102) Caused by: org.apache.seatunnel.engine.common.exception.JobException: java.lang.NoClassDefFoundError: org/apache/seatunnel/api/kerberos/KerberosConfig at java.base/java.lang.ClassLoader.defineClass1(Native Method) at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1027) at java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:150) at java.base/java.net.URLClassLoader.defineClass(URLClassLoader.java:524) at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:427) at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:421) at java.base/java.security.AccessController.doPrivileged(AccessController.java:714) at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:420) at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:593) at org.apache.seatunnel.engine.common.loader.SeaTunnelBaseClassLoader.loadClassWithoutExceptionHandling(SeaTunnelBaseClassLoader.java:56) at org.apache.seatunnel.engine.common.loader.SeaTunnelChildFirstClassLoader.loadClassWithoutExceptionHandling(SeaTunnelChildFirstClassLoader.java:88) at org.apache.seatunnel.engine.common.loader.SeaTunnelBaseClassLoader.loadClass(SeaTunnelBaseClassLoader.java:47) at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:526) at org.apache.seatunnel.connectors.seatunnel.file.sink.BaseMultipleTableFileSink.getSaveModeHandler(BaseMultipleTableFileSink.java:131) at org.apache.seatunnel.engine.server.master.JobMaster.handleSaveMode(JobMaster.java:372) at org.apache.seatunnel.engine.server.master.JobMaster.handleSaveMode(JobMaster.java:383) at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184) at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197) at java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:179) at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197) at java.base/java.util.Iterator.forEachRemaining(Iterator.java:133) at java.base/java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1939) at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509) at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499) at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151) at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174) at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:596) at org.apache.seatunnel.engine.server.master.JobMaster.init(JobMaster.java:242) at org.apache.seatunnel.engine.server.CoordinatorService.lambda$submitJob$4(CoordinatorService.java:499) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:572) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) at java.base/java.lang.Thread.run(Thread.java:1583) Caused by: java.lang.ClassNotFoundException: org.apache.seatunnel.api.kerberos.KerberosConfig at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:445) at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:593) at org.apache.seatunnel.engine.common.loader.SeaTunnelBaseClassLoader.loadClassWithoutExceptionHandling(SeaTunnelBaseClassLoader.java:56) at org.apache.seatunnel.engine.common.loader.SeaTunnelChildFirstClassLoader.loadClassWithoutExceptionHandling(SeaTunnelChildFirstClassLoader.java:88) at org.apache.seatunnel.engine.common.loader.SeaTunnelBaseClassLoader.loadClass(SeaTunnelBaseClassLoader.java:47) at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:526) ... 35 more at org.apache.seatunnel.engine.server.CoordinatorService.lambda$submitJob$4(CoordinatorService.java:506) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:572) at java.util.concurrent.FutureTask.run(FutureTask.java:317) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) at java.lang.Thread.run(Thread.java:1583) ``` ### Zeta or Flink or Spark Version Zeta ### Java or Scala Version JDK-11 ### Screenshots  ### Are you willing to submit PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
