[
https://issues.apache.org/jira/browse/FLINK-27009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17562729#comment-17562729
]
Biao Geng commented on FLINK-27009:
-----------------------------------
hi [~gyfora], sorry for the late reply. As Yang said, I had previously created
a
[PR|https://github.com/bgeng777/flink-kubernetes-operator/tree/sql-runner-demo/flink-kubernetes-sql-runner]
to create a sql runner jar in the k8s operator side. The implementation is
pretty basic: the sql runner jar just parses sql statements in the yaml and
then call stableEnvironment.executeSql(statements). In my PR, I plan to make it
bundled in the flink k8s operator image, so users can directly use it and of
course we can add some utils to make submit SQL script easier. But after some
tests, I meet a problem: Our flink kubernetes operator is based on Java11 and
as a result, the `sql-runner.jar` should be compiled in Java11 but our flink
image can be built on Java8 which leads to incompatibility issues. I am not
sure if your implementation will meet such problem.
So after some investigation, I prefer to do something in the upstream flink
project to support running SQL scripts in Application Mode first so that we can
use flink SQL client as the entry class directly. Then in our k8s operator, we
can support sql scripts easier. It is somehow simliar to what we have done for
YARN Application Mode support of Python jobs.
I have created a [draft of Support submitting SQL scripts using Application
Mode|https://docs.google.com/document/d/1C-o3wAgSE3TxxqaMxwIH7_Pt5w-v2PrSb6KU1vGNuE0/edit#heading=h.8rcq4dx7kcjy]
for the upstream change but I do agree that it can take some uncertain time to
discuss this upstream change. If we do not want this jira to be blocked by
that, we can discuss how to improve the sql runner solution. Hope my previous
experience can be helpful.
> Support SQL job submission in flink kubernetes opeartor
> -------------------------------------------------------
>
> Key: FLINK-27009
> URL: https://issues.apache.org/jira/browse/FLINK-27009
> Project: Flink
> Issue Type: New Feature
> Components: Kubernetes Operator
> Reporter: Biao Geng
> Assignee: Gyula Fora
> Priority: Major
>
> Currently, the flink kubernetes opeartor is for jar job using application or
> session cluster. For SQL job, there is no out of box solution in the
> operator.
> One simple and short-term solution is to wrap the SQL script into a jar job
> using table API with limitation.
> The long-term solution may work with
> [FLINK-26541|https://issues.apache.org/jira/browse/FLINK-26541] to achieve
> the full support.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
