Hi, Xuyang. Thanks a lot for your response! > Does that means we will support multi DMLs, multi DQLs, mixed DMLs & DQLs in one sql script?
According to the doc[1], application mode only supports one job in ha mode[2]. If users submit multiple jobs, dispatcher throws a DuplicateJobSubmissionException to notify users. In non-ha mode, the application mode doesn't have job number limitation. The SQL driver runs statement one by one and it is similar to submitting job to a session cluster. But just as the doc says, when any of multiple running jobs in Application Mode (submitted for example using executeAsync()) gets cancelled, all jobs will be stopped and the JobManager will shut down. [1] https://nightlies.apache.org/flink/flink-docs-master/docs/deployment/overview/#application-mode [2] https://github.com/apache/flink/blob/master/flink-clients/src/main/java/org/apache/flink/client/deployment/application/ApplicationDispatcherBootstrap.java#L218 Best, Shengkai Xuyang <xyzhong...@163.com> 于2024年10月31日周四 17:10写道： > Hi, Shengkai. > > Thanks for driving this great work. LGTM overall, I just have one > question. > > IIUC, application mode supports to run multi-execute in a single `main` > function[1]. Does that means > > we will support multi DMLs, multi DQLs, mixed DMLs & DQLs in one sql > script? If yes, can you explain > > a little about how do they work? > > > > > [1] > https://nightlies.apache.org/flink/flink-docs-master/docs/deployment/overview/#application-mode > > > > > -- > > Best！ > Xuyang > > > > > > 在 2024-10-31 10:18:13，"Ron Liu" <ron9....@gmail.com> 写道： > >Hi, Shengkai > > > >Thanks for your quick response. It looks good to me. > > > >Best > >Ron > > > >Shengkai Fang <fskm...@gmail.com> 于2024年10月31日周四 10:08写道： > > > >> Hi, Ron! > >> > >> > I noticed that you say this FLIP focuses on supporting deploy sql > >> scripts to the application cluster, does it mean that it only supports > >> non-interactive gateway mode? > >> > >> Yes. This FLIP only supports to deploy a script in non-interactive mode. > >> > >> > Whether all SQL commands such as DDL & DML & SELECT are supported. > >> > >> We supports all SQL commands and the execution results are visible in > the > >> JM log. But application cluster has some limitations that only one job > is > >> allowed to run in the dedicated cluster. > >> > >> > How to dynamically download the JAR specified by the user when > >> submitting the sql script, and whether it is possible to specify a local > >> jar? > >> > >> This is a good question. I think it's totally up to the deployment api. > For > >> example, kubernetes deployment provides the option > >> `kubernetes-artifacts-local-upload-enabled`[1] to upload the artifact to > >> the DFS but yarn deployment doesn't support to ship the artifacts to > DFS in > >> application mode. If runtime API can provide unified interface, I think > we > >> can use the unified API to upload local artifacts. Alternatively, we can > >> provide a special service that allows sql-gateway to support pulling > jar. > >> You can read the future work for more details. > >> > >> [1] > >> > >> > https://nightlies.apache.org/flink/flink-docs-master/docs/deployment/config/#kubernetes-artifacts-local-upload-enabled > >> > >> > >> Shengkai Fang <fskm...@gmail.com> 于2024年10月31日周四 09:30写道： > >> > >> > Hi, Feng! > >> > > >> > > if only clusterID is available, it may not be very convenient to > >> connect > >> > to this application later on. > >> > > >> > If FLIP-479 is accepted, I think we can just adapt the sql-gateway > >> > behaviour to the behaviour that FLIP-479 mentioned. > >> > > >> > > >> > Best, > >> > Shengkai > >> > > >> > > >> >
