lianghuan-xatu commented on code in PR #5542:
URL: https://github.com/apache/seatunnel/pull/5542#discussion_r1365388322
##########
seatunnel-engine/seatunnel-engine-server/src/main/java/org/apache/seatunnel/engine/server/job/JobImmutableInformationEnv.java:
##########
@@ -62,6 +71,20 @@ public Long getJobId() {
return jobId;
}
+ @Override
+ protected LogicalDag getLogicalDag() {
+ ImmutablePair<List<Action>, Set<URL>> immutablePair =
getJobConfigParser().parse();
+ actions.addAll(immutablePair.getLeft());
+ jarUrls.addAll(commonPluginJars);
+ jarUrls.addAll(immutablePair.getRight());
+ actions.forEach(
+ action -> {
+ addCommonPluginJarsToAction(
+ action, new HashSet<>(commonPluginJars),
Collections.emptySet());
+ });
+ return getLogicalDagGenerator().generate();
+ }
Review Comment:
In the previous parsing logic, we need to pass the local CommonPluginJar
through method parameters to the MultiTableJobParser, in order to set the URLs
of all current CommonPluginJars in the construction Action. But what we need to
set is the path to the server that will be returned after uploading to the
server. If the URLs of commonPlugin are set to Action in advance, it will cause
the CommonPluginJarUrl in the Action to be indistinguishable from the JarUrl of
the connector. we must upload CommonPluginJar and ConnectorPluginJar
separately. Because the MultiTableJobParser is in the Core module, the upload
process cannot be done simultaneously when building Actions. Instead, the
logical relationship diagrams of all Actions can be constructed and the
CommonPluginJar and ConnectorPluginJar can be uploaded recursively. So we need
to first recurse each Action and upload the ConnectorPlugin, then set the
storage path of the ConnectorPluginJar in the server to the Action. After all
the
ConnectorPluginJar paths in all upstream and downstream Action chains are
replaced, add the path of CommonPluginJar on the server to each Action.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
