SEPURI-SAI-KRISHNA opened a new issue, #13114: URL: https://github.com/apache/seatunnel/issues/13114
### Search before asking - [X] I had searched in the [issues](https://github.com/apache/seatunnel/issues?q=is%3Aissue+label%3A%22Bug%22) and found no similar issues. ### What happened Since #9650 a connector can keep its own dependency jars in `${SEATUNNEL_HOME}/plugins/connector-xxx/` ([connector-isolated-dependency.md](https://github.com/apache/seatunnel/blob/dev/docs/en/connectors/connector-isolated-dependency.md)). On Flink this works for sources but not for sinks. The Flink source processor collects its jars with `getPluginJarAndDependencyPaths`. Both Flink sink processors still call `getPluginJarPaths`, which returns only the connector jar: - `seatunnel-flink-starter-common/.../AbstractSinkExecuteProcessor.java` line 106 (Flink 1.15 to 1.20) - `seatunnel-flink-13-starter/.../SinkExecuteProcessor.java` line 109 #9650 also made `FlinkExecution` skip the `plugins/connector-*` directories when it adds the common plugin jars, so nothing else adds them. A sink that needs a jar from its connector directory fails on Flink with a class not found error. The same job works on Zeta, and on Spark, because `SparkStarter` adds sink jars with `getPluginJarAndDependencyPaths`. I found this with a JMS sink I am building for #13078, which keeps the broker client jars in `plugins/connector-jms/`. Its E2E sink test failed on Flink 1.13.6, 1.15.3, 1.18.0 and 1.20.1 and passed on Zeta and on Spark 2.4.6 and 3.3.0. With the sink processors changed to `getPluginJarAndDependencyPaths`, it passed on all four Flink versions. The same can be seen on `dev` without any connector: with `connectors/connector-demo.jar` and `plugins/connector-demo/demo-client.jar` in a test `SEATUNNEL_HOME`, the Flink sink processor's jar list holds only `connector-demo.jar`. ### SeaTunnel Version dev (941276923f). 2.3.12, 2.3.13 and 3.0.0 have the same code. ### SeaTunnel Config ```conf Any Flink job whose sink needs a jar that is only in ${SEATUNNEL_HOME}/plugins/connector-xxx/ ``` ### Running Command ```shell bin/start-seatunnel-flink-15-connector-v2.sh --config job.conf ``` ### Error Exception ```log JMS-01 Could not load connection factory class org.apache.activemq.artemis.jms.client.ActiveMQConnectionFactory (...). Put the JMS provider's client jars in ${SEATUNNEL_HOME}/plugins/connector-jms/ ``` ### Zeta or Flink or Spark Version Flink 1.13.6, 1.15.3, 1.18.0, 1.20.1 ### Java or Scala Version Built with JDK 17; the jobs ran in the SeaTunnel E2E Flink images. ### Screenshots _No response_ ### Are you willing to submit PR? - [X] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
