luca-p-castelli opened a new pull request, #28054: URL: https://github.com/apache/flink/pull/28054
## What is the purpose of the change Today `COMPILE PLAN` requires a file path and writes the compiled `ExecNodeGraph` JSON to disk: ```sql COMPILE PLAN '/path/to/plan.json' FOR INSERT INTO sink SELECT * FROM src; ``` This requires the caller to share a filesystem with the executor. For tooling built on the SQL Gateway (CLIs, IDEs, CI systems) this adds filesystem-coordination complexity, and for remote SQL Gateway deployments it does not work at all — the file lands on the gateway host, not the caller's host. This PR makes the file path optional. When omitted, the compiled plan is returned inline as a single-row, single-column `STRING` result set with `ResultKind.SUCCESS_WITH_CONTENT` — the same result-set pattern already used by `EXPLAIN`: ```sql COMPILE PLAN FOR INSERT INTO sink SELECT * FROM src; -- returns plan JSON via fetchResults ``` Fully backward compatible: the existing `COMPILE PLAN '<path>' [IF NOT EXISTS] FOR <dml>` syntax and behavior are unchanged. This is an internal-only change at the API level — both `SqlCompilePlan` and `CompilePlanOperation` are annotated `@Internal`; no new public API surface, no new REST endpoints. Discussion thread: https://lists.apache.org/thread/j2z9obdyr9k0078jj1cd6tjj5nvbdk86 ## Brief change log - **Parser** (`parserImpls.ftl`): wrap the file-path branch in `LOOKAHEAD(2)` so the grammar accepts both `COMPILE PLAN '<path>' [IF NOT EXISTS] FOR <dml>` and `COMPILE PLAN FOR <dml>`. - **AST** (`SqlCompilePlan`): `planFile` is now `@Nullable`; `unparse` emits the path and `IF NOT EXISTS` only when present. - **Operation** (`CompilePlanOperation`): `filePath` is now `@Nullable`; constructor adds a `Preconditions.checkArgument` rejecting `IF NOT EXISTS` when no file path is specified. - **API** (`TableEnvironmentImpl`): when the file path is null, return the plan via `TableResultUtils.buildStringArrayResult("result", new String[]{compiledPlan.asJsonString()})` — mirroring how `EXPLAIN` returns its single `STRING` column. Extracted a shared `compileOperationToPlan` helper used by both the file-based and inline paths. - **Tests**: parser round-trip tests in `FlinkSqlParserImplTest` for inline `INSERT` and inline `STATEMENT SET`; integration tests in `CompiledPlanITCase` (`testCompilePlanInline`, `testCompilePlanInlineWithStatementSet`) verifying `ResultKind.SUCCESS_WITH_CONTENT`, the presence of `flinkVersion`, and the expected sink identifiers in the returned JSON. ## Verifying this change This change added tests and can be verified as follows: - `FlinkSqlParserImplTest#compilePlan` — extended with parse/unparse round-trip cases for both inline forms (single `INSERT`, `STATEMENT SET`). - `CompiledPlanITCase#testCompilePlanInline` — verifies single-`INSERT` inline form: result kind is `SUCCESS_WITH_CONTENT`, returned JSON contains `flinkVersion`, source-scan, and sink exec nodes. - `CompiledPlanITCase#testCompilePlanInlineWithStatementSet` — verifies `STATEMENT SET` inline form: returned JSON contains both sink identifiers (`sinkA`, `sinkB`). - All existing file-based `COMPILE PLAN` tests pass unchanged. ## Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): **no** - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: **no** (both `SqlCompilePlan` and `CompilePlanOperation` are `@Internal`) - The serializers: **no** - The runtime per-record code paths (performance sensitive): **no** - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: **no** - The S3 file system connector: **no** ## Documentation - Does this pull request introduce a new feature? **yes** (extends existing `COMPILE PLAN` syntax with an optional file path) - If yes, how is the feature documented? **JavaDocs** (`SqlCompilePlan` and `CompilePlanOperation` javadocs updated to document the optional-path semantics). Happy to extend the SQL reference documentation in a follow-up commit on this PR if reviewers prefer that to land here. --- ##### Was generative AI tooling used to co-author this PR? - [X] Yes — Claude Code (Claude Opus 4.7) Generated-by: Claude Code (Claude Opus 4.7) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
