MartijnVisser commented on code in PR #29934:
URL: https://github.com/apache/flink/pull/29934#discussion_r4601303799


##########
docs/content/docs/concepts/sql-table-concepts/compiled_plan.md:
##########
@@ -0,0 +1,326 @@
+---
+title: "Compiled Plans"
+weight: 5
+type: docs
+---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+  http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied.  See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+# Compiled Plans
+
+The planner translates a Table API or SQL program into a pipeline of operators 
every time you
+submit it. A compiled plan is the result of this translation, written to a 
JSON file. It contains
+the execution nodes of the pipeline with their IDs and versions, the edges 
between them, the tables
+and functions they use, and the values of the options that the nodes depend 
on. When you run a
+plan, Flink doesn't plan the query again, so the job gets the same operators 
with the same IDs
+every time, including on a newer Flink version within the limits in
+[Flink Versions](#flink-versions).
+
+Use a compiled plan when you upgrade a job to a newer Flink version and need 
it to keep its state,
+see [Upgrading the Flink Version]({{< ref "docs/ops/upgrading" 
>}}#upgrading-the-flink-version), and to set the state
+TTL of each stateful operator, see
+[Configure Operator-level State TTL]({{< ref 
"docs/concepts/sql-table-concepts/overview" 
>}}#configure-operator-level-state-ttl).
+
+## Compile a Plan
+
+The `COMPILE PLAN` statement writes the plan of an `INSERT` statement or a 
statement set
+to a JSON file.
+
+- Run a `COMPILE PLAN` statement
+
+{{< tabs "compile-plan" >}}
+{{< tab "Java" >}}
+```java
+TableEnvironment tableEnv = 
TableEnvironment.create(EnvironmentSettings.inStreamingMode());
+tableEnv.executeSql(
+    "CREATE TABLE orders (order_id BIGINT, order_line_id BIGINT, buyer_id 
BIGINT, ...)");
+tableEnv.executeSql(
+    "CREATE TABLE line_orders (order_line_id BIGINT, order_status TINYINT, 
...)");
+tableEnv.executeSql(
+    "CREATE TABLE enriched_orders (order_id BIGINT, order_line_id BIGINT, 
order_status TINYINT, ...)");
+
+// CompilePlan#writeToFile only supports a local file path, if you need to 
write to remote filesystem,
+// please use tableEnv.executeSql("COMPILE PLAN 'hdfs://path/to/plan.json' FOR 
...")
+CompiledPlan compiledPlan = 
+    tableEnv.compilePlanSql(
+        "INSERT INTO enriched_orders \n" 
+       + "SELECT a.order_id, a.order_line_id, b.order_status, ... \n" 
+       + "FROM orders a JOIN line_orders b ON a.order_line_id = 
b.order_line_id");
+
+compiledPlan.writeToFile("/path/to/plan.json");
+
+```
+
+{{< /tab >}}
+{{< tab "Scala" >}}
+```scala
+val tableEnv = TableEnvironment.create(EnvironmentSettings.inStreamingMode())
+tableEnv.executeSql(
+    "CREATE TABLE orders (order_id BIGINT, order_line_id BIGINT, buyer_id 
BIGINT, ...)")
+tableEnv.executeSql(
+    "CREATE TABLE line_orders (order_line_id BIGINT, order_status TINYINT, 
...)")
+tableEnv.executeSql(
+    "CREATE TABLE enriched_orders (order_id BIGINT, order_line_id BIGINT, 
order_status TINYINT, ...)")
+
+val compiledPlan = 
+    tableEnv.compilePlanSql(
+       """
+        |INSERT INTO enriched_orders
+        |SELECT a.order_id, a.order_line_id, b.order_status, ...
+        |FROM orders a JOIN line_orders b ON a.order_line_id = b.order_line_id
+        |""".stripMargin)
+// CompilePlan#writeToFile only supports a local file path, if you need to 
write to remote filesystem,
+// please use tableEnv.executeSql("COMPILE PLAN 'hdfs://path/to/plan.json' FOR 
...")
+compiledPlan.writeToFile("/path/to/plan.json")
+```
+{{< /tab >}}
+{{< tab "SQL CLI" >}}
+
+```sql
+Flink SQL> CREATE TABLE orders (order_id BIGINT, order_line_id BIGINT, 
buyer_id BIGINT, ...);
+[INFO] Execute statement succeeded.
+
+Flink SQL> CREATE TABLE line_orders (order_line_id BIGINT, order_status 
TINYINT, ...);
+[INFO] Execute statement succeeded.
+
+Flink SQL> CREATE TABLE enriched_orders (order_id BIGINT, order_line_id 
BIGINT, order_status TINYINT, ...);
+[INFO] Execute statement succeeded.
+
+Flink SQL> COMPILE PLAN 'file:///path/to/plan.json' FOR INSERT INTO 
enriched_orders
+> SELECT a.order_id, a.order_line_id, b.order_status, ...
+> FROM orders a JOIN line_orders b ON a.order_line_id = b.order_line_id;
+[INFO] Execute statement succeeded.
+```
+
+{{< /tab >}}
+{{< /tabs >}}
+
+
+- SQL Syntax
+
+    ```sql
+    COMPILE PLAN <plan_file_path> [IF NOT EXISTS] FOR 
<insert_statement>|<statement_set>;
+    
+    statement_set:
+        STATEMENT SET
+        BEGIN
+        insert_statement;
+        ...
+        insert_statement;
+        END;
+    
+    insert_statement:
+        <insert_from_select>|<insert_from_values>
+    ```
+    This will generate a JSON file at `/path/to/plan.json`.
+
+{{< hint info >}}
+`COMPILE PLAN` statement supports writing the plan to a remote 
[filesystem]({{< ref "docs/deployment/filesystems/overview" >}}) scheme like 
`hdfs://` or `s3://`. 
+Please be sure that the target path has set up the write access.
+{{< /hint >}}
+
+If the file already exists, `COMPILE PLAN` fails:
+
+```text
+org.apache.flink.table.api.TableException: Cannot overwrite the plan file 
'file:///path/to/plan.json'. Either manually remove the file or, if you're 
debugging your job, set the option 'table.plan.force-recompile' to true.
+```
+
+With `IF NOT EXISTS`, `COMPILE PLAN` keeps the existing file and doesn't 
compile the statement.
+`CompiledPlan#writeToFile(path, true)` also keeps an existing file. With
+`table.plan.force-recompile` = `true`, `COMPILE PLAN` overwrites the file. 
Only use this while you
+develop a job: a new compilation can assign other operator IDs than the ones 
in a savepoint.
+
+The SQL Client or SQL Gateway that runs the statement reads and writes the 
plan file, so it needs
+access to the path. With a remote SQL Gateway, a local path is a path on the 
machine of the
+Gateway.
+
+## Execute a Plan
+
+`EXECUTE PLAN` statement will deserialize the specified file back to execution 
plan of the current table program and then submit the job.
+
+- Run an `EXECUTE PLAN` statement
+{{< tabs "execute-plan" >}}
+{{< tab "Java" >}}
+```java
+TableEnvironment tableEnv = 
TableEnvironment.create(EnvironmentSettings.inStreamingMode());
+tableEnv.executeSql(
+    "CREATE TABLE orders (order_id BIGINT, order_line_id BIGINT, buyer_id 
BIGINT, ...)");
+tableEnv.executeSql(
+    "CREATE TABLE line_orders (order_line_id BIGINT, order_status TINYINT, 
...)");
+tableEnv.executeSql(
+    "CREATE TABLE enriched_orders (order_id BIGINT, order_line_id BIGINT, 
order_status TINYINT, ...)");
+
+// PlanReference#fromFile only supports a local file path, if you need to read 
from remote filesystem,
+// please use tableEnv.executeSql("EXECUTE PLAN 
'hdfs://path/to/plan.json'").await();
+tableEnv.loadPlan(PlanReference.fromFile("/path/to/plan.json")).execute().await();
+
+```
+
+{{< /tab >}}
+{{< tab "Scala" >}}
+```scala
+val tableEnv = TableEnvironment.create(EnvironmentSettings.inStreamingMode())
+tableEnv.executeSql(
+    "CREATE TABLE orders (order_id BIGINT, order_line_id BIGINT, buyer_id 
BIGINT, ...)")
+tableEnv.executeSql(
+    "CREATE TABLE line_orders (order_line_id BIGINT, order_status TINYINT, 
...)")
+tableEnv.executeSql(
+    "CREATE TABLE enriched_orders (order_id BIGINT, order_line_id BIGINT, 
order_status TINYINT, ...)")
+
+// PlanReference#fromFile only supports a local file path, if you need to read 
from remote filesystem,
+// please use tableEnv.executeSql("EXECUTE PLAN 
'hdfs://path/to/plan.json'").await()
+tableEnv.loadPlan(PlanReference.fromFile("/path/to/plan.json")).execute().await()
+```
+{{< /tab >}}
+{{< tab "SQL CLI" >}}
+
+```sql
+Flink SQL> CREATE TABLE orders (order_id BIGINT, order_line_id BIGINT, 
buyer_id BIGINT, ...);
+[INFO] Execute statement succeeded.
+
+Flink SQL> CREATE TABLE line_orders (order_line_id BIGINT, order_status 
TINYINT, ...);
+[INFO] Execute statement succeeded.
+
+Flink SQL> CREATE TABLE enriched_orders (order_id BIGINT, order_line_id 
BIGINT, order_status TINYINT, ...);
+[INFO] Execute statement succeeded.
+
+Flink SQL> EXECUTE PLAN 'file:///path/to/plan.json';
+[INFO] Submitting SQL update statement to the cluster...
+[INFO] SQL update statement has been successfully submitted to the cluster:
+Job ID: 79fbe3fa497e4689165dd81b1d225ea8
+```
+
+{{< /tab >}}
+{{< /tabs >}}
+
+- SQL Syntax
+
+    ```sql
+    EXECUTE PLAN <plan_file_path>;
+    ```
+    This will deserialize the JSON file and submit an insert statement job.
+
+To run a plan that your Table API program ships as a resource in its JAR, read 
the resource
+and use `PlanReference.fromJsonString`, as in the Java example in
+[Upgrading the Flink Version]({{< ref "docs/ops/upgrading" 
>}}#upgrading-the-flink-version).
+`PlanReference.fromResource` fails for a file inside a JAR.
+
+### Tables and Functions
+
+With the default `table.plan.compile.catalog-objects` = `ALL`, a plan stores 
each catalog table
+with its schema and options. `EXECUTE PLAN` then also works in a new session 
in which the tables
+haven't been created. If a table is in the catalog, a restore takes the 
options that its connector
+allows to change from there, such as the `path` of the filesystem connector. 
See
+`table.plan.restore.catalog-objects`.
+
+Temporary tables and functions are stored only by their name. Create them 
again before you run
+the plan, otherwise `EXECUTE PLAN` fails:
+
+```text
+org.apache.flink.table.api.TableException: Cannot resolve table 
'default_catalog.default_database.orders' and the persisted plan does not 
include all required catalog table metadata. Make sure a registered catalog 
contains the table when restoring or the table is available as a temporary 
table. Otherwise regenerate the plan with 'table.plan.compile.catalog-objects' 
!= 'IDENTIFIER' and make sure the table was not compiled as a temporary table.
+```
+
+The plan assumes that its tables and functions don't change. If a table in the 
catalog has another
+schema than in the plan, `EXECUTE PLAN` fails with `The schema of table '...' 
from the persisted plan
+does not match the schema loaded from the catalog`. A change of the signature 
of a function isn't
+supported either.
+
+### Options
+
+For each node, the plan stores the values of the options that the node uses in 
its
+`configuration` entry. For example, the sink of the plan above:
+
+```json
+"configuration" : {
+  "table.exec.sink.keyed-shuffle" : "AUTO",
+  "table.exec.sink.not-null-enforcer" : "ERROR",
+  "table.exec.sink.rowtime-inserter" : "ENABLED",
+  "table.exec.sink.type-length-enforcer" : "IGNORE",
+  "table.exec.sink.upsert-materialize" : "AUTO"
+}
+```
+
+The job uses these values, not the ones of the session. With
+`SET 'table.exec.sink.not-null-enforcer' = 'DROP'` before `EXECUTE PLAN`, the 
sink still fails on a
+NULL value in a `NOT NULL` column. To use another value, compile a new plan, 
see
+[Changing the Query]({{< ref "docs/ops/upgrading" >}}#changing-the-query) for 
what that means for the state of a job.
+
+## Compile and Execute the Plan in One Statement
+
+`COMPILE AND EXECUTE PLAN` compiles the statement, writes the plan to the 
file, and runs it. If the
+file already exists, it runs the plan from the file and ignores the statement. 
Running the same
+statement again, including on a newer Flink version, runs the plan that was 
compiled first. If you
+change the statement, the change is ignored without a warning. Compile a 
changed statement to a new
+file.
+
+The statement is still validated, so unlike `EXECUTE PLAN`, it needs its 
tables in the session.
+
+- SQL Syntax
+
+    ```sql
+    COMPILE AND EXECUTE PLAN <plan_file_path> FOR 
<insert_statement>|<statement_set>;
+    ```
+
+## What Can't Be Compiled
+
+A plan can't contain:
+
+* a table created from a `DataStream`, for example with 
`StreamTableEnvironment#fromDataStream`.
+* legacy table sources and sinks (`TableSource`, `TableSink`).
+* table aggregate functions.
+* process table functions, in Flink versions before 2.1.
+
+`COMPILE PLAN` fails for these. In the Table API, `compilePlan()` succeeds, 
and the error comes when
+you write the plan, for example with `writeToFile` or `asJsonString`. For a 
table from a
+`DataStream`, the error is:
+
+```text
+Cannot serialize the table as it's an external inline table. This might be 
caused by a usage of StreamTableEnvironment#fromDataStream or 
TableResult#collect, which are not supported in compiled plans.
+```
+
+For a table aggregate function, the error is the following, although it's not 
a bug: table
+aggregate functions aren't supported in a plan.
+
+```text
+Can not serialize ExecNode with id: 6. Missing type, this is a bug, please 
file a ticket.
+```
+
+Writing a plan doesn't check that it can be loaded again. Run the plan once 
with `EXECUTE PLAN`,
+for example on a test cluster, before you rely on it for an upgrade.
+
+## Flink Versions
+
+The plan stores the Flink version that compiled it, and the type of each node 
includes its
+version, such as `stream-exec-join_1`. A newer Flink version runs the plan as 
long as it still
+supports these node versions, otherwise `EXECUTE PLAN` fails to load the plan. 
Flink guarantees
+that a plan runs after an upgrade by one minor version, for example from 2.2 
to 2.3. Over more
+versions, this isn't guaranteed, although the tests of Flink restore plans 
that were compiled with
+Flink 1.19 and later.

Review Comment:
   Rewrote it.



##########
docs/content/docs/concepts/sql-table-concepts/overview.md:
##########
@@ -199,104 +199,7 @@ Window-based operations (like [Window Join]({{< ref 
"docs/sql/reference/queries/
 
 **Generate a Compiled Plan**
 
-The setup process begins by generating a JSON file using the `COMPILE PLAN` 
statement, 
-which represents the serialized execution plan of the current table program. 
-{{< hint info >}}
-Currently, `COMPILE PLAN` statement does not support `SELECT... FROM...` 
queries. 
-{{< /hint >}}
-
-- Run a `COMPILE PLAN` statement
-
-{{< tabs "compile-plan" >}}
-{{< tab "Java" >}}
-```java
-TableEnvironment tableEnv = 
TableEnvironment.create(EnvironmentSettings.inStreamingMode());
-tableEnv.executeSql(
-    "CREATE TABLE orders (order_id BIGINT, order_line_id BIGINT, buyer_id 
BIGINT, ...)");
-tableEnv.executeSql(
-    "CREATE TABLE line_orders (order_line_id BIGINT, order_status TINYINT, 
...)");
-tableEnv.executeSql(
-    "CREATE TABLE enriched_orders (order_id BIGINT, order_line_id BIGINT, 
order_status TINYINT, ...)");
-
-// CompilePlan#writeToFile only supports a local file path, if you need to 
write to remote filesystem,
-// please use tableEnv.executeSql("COMPILE PLAN 'hdfs://path/to/plan.json' FOR 
...")
-CompiledPlan compiledPlan = 
-    tableEnv.compilePlanSql(
-        "INSERT INTO enriched_orders \n" 
-       + "SELECT a.order_id, a.order_line_id, b.order_status, ... \n" 
-       + "FROM orders a JOIN line_orders b ON a.order_line_id = 
b.order_line_id");
-
-compiledPlan.writeToFile("/path/to/plan.json");
-
-```
-
-{{< /tab >}}
-{{< tab "Scala" >}}
-```scala
-val tableEnv = TableEnvironment.create(EnvironmentSettings.inStreamingMode())
-tableEnv.executeSql(
-    "CREATE TABLE orders (order_id BIGINT, order_line_id BIGINT, buyer_id 
BIGINT, ...)")
-tableEnv.executeSql(
-    "CREATE TABLE line_orders (order_line_id BIGINT, order_status TINYINT, 
...)")
-tableEnv.executeSql(
-    "CREATE TABLE enriched_orders (order_id BIGINT, order_line_id BIGINT, 
order_status TINYINT, ...)")
-
-val compiledPlan = 
-    tableEnv.compilePlanSql(
-       """
-        |INSERT INTO enriched_orders
-        |SELECT a.order_id, a.order_line_id, b.order_status, ...
-        |FROM orders a JOIN line_orders b ON a.order_line_id = b.order_line_id
-        |""".stripMargin)
-// CompilePlan#writeToFile only supports a local file path, if you need to 
write to remote filesystem,
-// please use tableEnv.executeSql("COMPILE PLAN 'hdfs://path/to/plan.json' FOR 
...")
-compiledPlan.writeToFile("/path/to/plan.json")
-```
-{{< /tab >}}
-{{< tab "SQL CLI" >}}
-
-```sql
-Flink SQL> CREATE TABLE orders (order_id BIGINT, order_line_id BIGINT, 
buyer_id BIGINT, ...);
-[INFO] Execute statement succeeded.
-
-Flink SQL> CREATE TABLE line_orders (order_line_id BIGINT, order_status 
TINYINT, ...);
-[INFO] Execute statement succeeded.
-
-Flink SQL> CREATE TABLE enriched_orders (order_id BIGINT, order_line_id 
BIGINT, order_status TINYINT, ...);
-[INFO] Execute statement succeeded.
-
-Flink SQL> COMPILE PLAN 'file:///path/to/plan.json' FOR INSERT INTO 
enriched_orders
-> SELECT a.order_id, a.order_line_id, b.order_status, ...
-> FROM orders a JOIN line_orders b ON a.order_line_id = b.order_line_id;
-[INFO] Execute statement succeeded.
-```
-
-{{< /tab >}}
-{{< /tabs >}}
-
-
-- SQL Syntax
-
-    ```sql
-    COMPILE PLAN [IF NOT EXISTS] <plan_file_path> FOR 
<insert_statement>|<statement_set>;
-    
-    statement_set:
-        EXECUTE STATEMENT SET
-        BEGIN
-        insert_statement;
-        ...
-        insert_statement;
-        END;
-    
-    insert_statement:
-        <insert_from_select>|<insert_from_values>
-    ```
-    This will generate a JSON file at `/path/to/plan.json`.
-
-{{< hint info >}}
-`COMPILE PLAN` statement supports writing the plan to a remote 
[filesystem]({{< ref "docs/deployment/filesystems/overview" >}}) scheme like 
`hdfs://` or `s3://`. 
-Please be sure that the target path has set up the write access.
-{{< /hint >}}
+Compile the statement to a plan file as described in [Compile a Plan]({{< ref 
"docs/concepts/sql-table-concepts/compiled_plan" >}}#compile-a-plan).

Review Comment:
   Added a short `COMPILE PLAN` example.



##########
docs/content/docs/concepts/sql-table-concepts/overview.md:
##########
@@ -335,71 +238,9 @@ Conceptually, the TTL of downstream stateful operator 
should be greater than or
 
 **Execute the Compiled Plan**
 
-`EXECUTE PLAN` statement will deserialize the specified file back to execution 
plan of the current table program and then submit the job.
+Run the plan as described in [Execute a Plan]({{< ref 
"docs/concepts/sql-table-concepts/compiled_plan" >}}#execute-a-plan).

Review Comment:
   Added the `EXECUTE PLAN` statement.



##########
docs/content/docs/concepts/sql-table-concepts/overview.md:
##########
@@ -567,36 +408,15 @@ Now change the value of left state to `"3000 ms"` and 
right state to `"9000 ms"`
 Table programs that are executed in streaming mode are intended as *standing 
queries* which means they
 are defined once and are continuously evaluated as static end-to-end pipelines.
 
-In case of stateful pipelines, any change to both the query or Flink's planner 
might lead to a completely
-different execution plan. This makes stateful upgrades and the evolution of 
table programs challenging
-at the moment. The community is working on improving those shortcomings.
-
-For example, by adding a filter predicate, the optimizer might decide to 
reorder joins or change the
-schema of an intermediate operator. This prevents restoring from a savepoint 
due to either changed
-topology or different column layout within the state of an operator.
-
-The query implementer must ensure that the optimized plans before and after 
the change are compatible.
-Use the `EXPLAIN` command in SQL or `table.explain()` in Table API to [get 
insights]({{< ref "docs/dev/table/common" >}}#explaining-a-table).
-
-Since new optimizer rules are continuously added, and operators become more 
efficient and specialized,
-also the upgrade to a newer Flink version could lead to incompatible plans.
-
-{{< hint warning >}}
-Currently, the framework cannot guarantee that state can be mapped from a 
savepoint to a new table
-operator topology.
-
-In other words: Savepoints are only supported if both the query and the Flink 
version remain constant.
-{{< /hint >}}
-
-Since the community rejects contributions that modify the optimized plan and 
the operator topology

Review Comment:
   Brought back the `EXPLAIN` hint, the patch release statement and the 
historical data advice. Not the "rejects contributions that modify the 
optimized plan" line: 2.2.1 changed a plan for a bug fix (FLINK-38916).



##########
docs/content/docs/ops/upgrading.md:
##########
@@ -176,16 +176,210 @@ When upgrading an application by changing its topology, 
a few things need to be
 
 ### Table API & SQL
 
-Due to the declarative nature of Table API & SQL programs, the underlying 
operator topology and state
-representation are mostly determined and optimized by the table planner.
+For Table API & SQL programs, the table planner decides the operator topology 
and how each operator
+stores its state. What you can change while keeping the state depends on what 
changes: the Flink
+version, the query, or a [process table function (PTF)]({{< ref 
"docs/dev/table/functions/ptfs" >}})
+that the query calls.
+
+#### Upgrading the Flink Version
+
+A savepoint contains the state of each operator, but not the operators 
themselves. A DataStream
+program builds the same operators again from its code when it restarts. A SQL 
script or a Table API
+program contains only the query: the planner, which is part of Flink and not 
of your JAR, turns it
+into operators again on every start. A newer Flink version can plan the same 
query differently, for
+example because of new optimizer rules or a new version of an operator, and 
the state in the
+savepoint then doesn't match the new operators.
+
+A [compiled plan]({{< ref "docs/concepts/sql-table-concepts/compiled_plan" >}})
+stores the result of the planning: the execution nodes of the pipeline with 
their IDs and versions.
+A newer Flink version runs the plan as it is, as long as it still supports the 
versions of the
+execution nodes in the plan, so that the operators find their state in the 
savepoint.
+`COMPILE AND EXECUTE PLAN` compiles the plan only if the file doesn't exist 
yet, and otherwise runs
+the plan from the file. Run the same statement on every Flink version:
+
+{{< tabs "upgrade-with-compiled-plan" >}}
+{{< tab "SQL CLI" >}}
+```sql
+-- 1. When you first deploy the job: create the tables and register the 
functions
+--    that the query uses, then run
+COMPILE AND EXECUTE PLAN 's3://my-bucket/plans/my-job.json' FOR
+INSERT INTO sink
+SELECT * FROM MyPtf(input => TABLE src PARTITION BY user_id, uid => 'my-ptf');
+
+-- 2. Before the upgrade, stop the job with a savepoint. SHOW JOBS lists the 
job ID,
+--    and STOP JOB prints the path of the savepoint.
+SET 'execution.checkpointing.savepoint-dir' = 's3://my-bucket/savepoints';
+STOP JOB '<job_id>' WITH SAVEPOINT;
+
+-- 3. On the new Flink version: create the same tables and register the same 
functions,
+--    set the savepoint from step 2, and run the same statement.
+SET 'execution.state-recovery.path' = 
's3://my-bucket/savepoints/savepoint-...';
+COMPILE AND EXECUTE PLAN 's3://my-bucket/plans/my-job.json' FOR
+INSERT INTO sink
+SELECT * FROM MyPtf(input => TABLE src PARTITION BY user_id, uid => 'my-ptf');
+-- The setting applies to every later job of the session.
+RESET 'execution.state-recovery.path';
+```
+{{< /tab >}}
+{{< tab "Java" >}}
+```java
+TableEnvironment tEnv = 
TableEnvironment.create(EnvironmentSettings.inStreamingMode());
+tEnv.executeSql("CREATE TABLE src (...) WITH (...)");
+tEnv.executeSql("CREATE TABLE sink (...) WITH (...)");
+tEnv.createTemporarySystemFunction("MyPtf", MyPtf.class);
+
+tEnv.executeSql(
+        "COMPILE AND EXECUTE PLAN 's3://my-bucket/plans/my-job.json' FOR "
+                + "INSERT INTO sink "
+                + "SELECT * FROM MyPtf(input => TABLE src PARTITION BY 
user_id, uid => 'my-ptf')");

Review Comment:
   Added under Changing the Query. I ran it in the SQL Client on master: 
`Totals` kept its state, `Alerts` started empty.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to