NookVoive opened a new issue, #8267: URL: https://github.com/apache/seatunnel/issues/8267
### Search before asking - [X] I had searched in the [issues](https://github.com/apache/seatunnel/issues?q=is%3Aissue+label%3A%22bug%22) and found no similar issues. ### What happened When I submit a job to the cluster using seatunnel.sh, due to the inconsistency between the client's SEATUNNEL_HOME and the cluster's SEATUNNEL_HOME, the cluster will load the connector plugins based on the client's SEATUNNEL_HOME path when running the task, which leads to the plugins failing to load properly. Is it possible to modify the loading of required connector plugins for the task to be based on relative paths instead of absolute paths? 当我使用seatunnel.sh向集群提交任务时，由于客户端的SEATUNNEL_HOME与集群的SEATUNNEL_HOME不一致，集群运行任务时会按照客户端的SEATUNNEL_HOME路径加载连接器插件，而这会导致插件无法正常加载。 加载任务所需连接器插件时是否可以修改为基于相对路径而不是绝对路径？ I saw the following display in the cluster's runtime log: ``` Job SeaTunnel_Job (918757460737523713) needed jar urls [file:/opt/apache-seatunnel-2.3.8/connectors/connector-jdbc-2.3.8.jar] Create classloader for job 918757460737523713 with jars [file:/opt/apache-seatunnel-2.3.8/connectors/connector-jdbc-2.3.8.jar] [localhost]:5801 [seatunnel] [5.1] submit job 918757460737523713 error com.hazelcast.nio.serialization.HazelcastSerializationException: java. lang.ClassNotFoundException: org.apache.seatunnel.connectors.seatunnel.jdbc.source.JdbcSource ``` In it, I saw the JAR path he indicated is the absolute path on my client, and this path does not exist on the cluster, so of course, there will be a failure to load! 其中我看到他所提示的jar路径是我客户端的绝对路径，而这个路径在集群上是不存在的，当然会出现无法加载的情况! ### SeaTunnel Version 2.3.8 ### SeaTunnel Config ```conf env { parallelism = 1 job.mode = "BATCH" } source { jdbc { url = "jdbc:sap://xxxxxx:30015" driver = "com.sap.db.jdbc.Driver" connection_check_timeout_sec = 100 user = "xxxxxx" password = "xxxxxx" use_select_count = true table_list = [ {table_path = "DM_JT.mytable_1"} ,{table_path = "DM_JT.mytable_2"} ] } } sink { jdbc { url = "jdbc:sap://xxxxxx:30015" driver = "com.sap.db.jdbc.Driver" user = "xxxxxx" password = "xxxxxx" generate_sink_sql = true schema_save_mode = "ERROR_WHEN_SCHEMA_NOT_EXIST" data_save_mode = "DROP_DATA" database = "${database_name}" table = "${table_name}" } } ``` ### Running Command ```shell ./bin/seatunnel.sh -c /config/hana_to_hana.conf ``` ### Error Exception ```log 2024-12-10 14:54:07,855 INFO [.p.t.AuthenticationMessageTask] [hz.main.priority-generic-operation.thread-0] - [localhost]:5801 [seatunnel] [5.1] Received auth from Connection[id=3, /172.17.0.3:5801->/10.88.2.99:39736, qualifier=null, endpo int=[10.88.2.99]:39736, remoteUuid=5b4b9ad0-f4dd-4337-95ca-97897e59ff5b, alive=true, connectionType=JVM, planeIndex=-1], successfully authenticated, clientUuid: 5b4b9ad0-f4dd-4337-95ca-97897e59ff5b, client name: hz.client_1, client version: 5.1 2024-12-10 14:54:10,345 INFO [o.a.s.e.s.m.JobMaster ] [seatunnel-coordinator-service-1] - in batch mode, the 'checkpoint.interval' configuration of env is missing, so checkpoint will be disabled 2024-12-10 14:54:10,346 INFO [o.a.s.e.s.m.JobMaster ] [seatunnel-coordinator-service-1] - Init JobMaster for Job SeaTunnel_Job (918757460737523713) 2024-12-10 14:54:10,346 INFO [o.a.s.e.s.m.JobMaster ] [seatunnel-coordinator-service-1] - Job SeaTunnel_Job (918757460737523713) needed jar urls [file:/opt/apache-seatunnel-2.3.8/connectors/connector-jdbc-2.3.8.jar] 2024-12-10 14:54:10,348 INFO [.c.c.DefaultClassLoaderService] [seatunnel-coordinator-service-1] - Create classloader for job 918757460737523713 with jars [file:/opt/apache-seatunnel-2.3.8/connectors/connector-jdbc-2.3.8.jar] 2024-12-10 14:54:10,402 ERROR [o.a.s.e.s.CoordinatorService ] [seatunnel-coordinator-service-1] - [localhost]:5801 [seatunnel] [5.1] submit job 918757460737523713 error com.hazelcast.nio.serialization.HazelcastSerializationException: java. lang.ClassNotFoundException: org.apache.seatunnel.connectors.seatunnel.jdbc.source.JdbcSource at com.hazelcast.internal.serialization.impl.defaultserializers.JavaDefaultSerializers$JavaSerializer.read(JavaDefaultSerializers.java:96) at com.hazelcast.internal.serialization.impl.defaultserializers.JavaDefaultSerializers$JavaSerializer.read(JavaDefaultSerializers.java:85) at com.hazelcast.internal.serialization.impl.StreamSerializerAdapter.read(StreamSerializerAdapter.java:44) at com.hazelcast.internal.serialization.impl.AbstractSerializationService.readObject(AbstractSerializationService.java:349) at com.hazelcast.internal.serialization.impl.ByteArrayObjectDataInput.readObject(ByteArrayObjectDataInput.java:600) at org.apache.seatunnel.engine.core.dag.logical.LogicalVertex.readData(LogicalVertex.java:99) at com.hazelcast.internal.serialization.impl.DataSerializableSerializer.readInternal(DataSerializableSerializer.java:160) at com.hazelcast.internal.serialization.impl.DataSerializableSerializer.read(DataSerializableSerializer.java:106) at com.hazelcast.internal.serialization.impl.DataSerializableSerializer.read(DataSerializableSerializer.java:51) at com.hazelcast.internal.serialization.impl.StreamSerializerAdapter.read(StreamSerializerAdapter.java:44) at com.hazelcast.internal.serialization.impl.AbstractSerializationService.readObject(AbstractSerializationService.java:349) at com.hazelcast.internal.serialization.impl.ByteArrayObjectDataInput.readObject(ByteArrayObjectDataInput.java:600) at org.apache.seatunnel.engine.core.dag.logical.LogicalDag.readData(LogicalDag.java:165) at com.hazelcast.internal.serialization.impl.DataSerializableSerializer.readInternal(DataSerializableSerializer.java:160) at com.hazelcast.internal.serialization.impl.DataSerializableSerializer.read(DataSerializableSerializer.java:106) at com.hazelcast.internal.serialization.impl.DataSerializableSerializer.read(DataSerializableSerializer.java:51) at com.hazelcast.internal.serialization.impl.StreamSerializerAdapter.read(StreamSerializerAdapter.java:44) at com.hazelcast.internal.serialization.impl.AbstractSerializationService.toObject(AbstractSerializationService.java:268) at com.hazelcast.jet.impl.execution.init.CustomClassLoadedObject.deserializeWithCustomClassLoader(CustomClassLoadedObject.java:66) at org.apache.seatunnel.engine.server.master.JobMaster.init(JobMaster.java:227) at org.apache.seatunnel.engine.server.CoordinatorService.lambda$submitJob$4(CoordinatorService.java:511) at org.apache.seatunnel.api.tracing.MDCRunnable.run(MDCRunnable.java:39) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) Caused by: java.lang.ClassNotFoundException: org.apache.seatunnel.connectors.seatunnel.jdbc.source.JdbcSource at java.net.URLClassLoader.findClass(URLClassLoader.java:387) at java.lang.ClassLoader.loadClass(ClassLoader.java:418) at org.apache.seatunnel.engine.common.loader.SeaTunnelBaseClassLoader.loadClassWithoutExceptionHandling(SeaTunnelBaseClassLoader.java:56) at org.apache.seatunnel.engine.common.loader.SeaTunnelChildFirstClassLoader.loadClassWithoutExceptionHandling(SeaTunnelChildFirstClassLoader.java:88) at org.apache.seatunnel.engine.common.loader.SeaTunnelBaseClassLoader.loadClass(SeaTunnelBaseClassLoader.java:47) at java.lang.ClassLoader.loadClass(ClassLoader.java:351) ``` ### Zeta or Flink or Spark Version _No response_ ### Java or Scala Version JDK 1.8 ### Screenshots _No response_ ### Are you willing to submit PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
