raminqaf opened a new pull request, #29844:
URL: https://github.com/apache/flink/pull/29844
## What is the purpose of the change
The "Adding a new SQL built-in function" section of `AGENTS.md` listed six
steps and named few files. It didn't say when a function needs code generation,
and it left out steps every function PR needs, such as the PyFlink method and
its completeness tests. FLIP-618 (FLINK-41336) adds about 25 functions, so this
guide lands first.
| Topic | Before |
After
|
|---------------------|------------------------------------------------|----------------------------------------------------------------------------------------------------------|
| How a function runs | "the planner wires up codegen automatically" |
`runtimeClass` by default; scalar codegen only for shared work, operator
context or lazy arguments |
| PyFlink | not mentioned |
where the method goes, the snake_case name the completeness tests check,
`test_expression.py`, the rst |
| Tests | "Add tests in flink-table-planner and runtime" |
each behavior once at the cheapest level, ITCase cases only in the existing
ITCase of the category |
| Docs | "Document in docs/" | a
row in `sql_functions.yml` and `_zh`, only what a user can observe, results
from running the SQL |
The implementation-path rule comes from `BuiltInFunctionDefinitions` on
master: 69 functions use `runtimeClass`, 18 scalar functions and 10 aggregates
use `runtimeProvided`, and the other 172 have no runtime of their own and are
translated by the planner, mostly to Calcite operators. Scalar codegen earns
its place where a runtime class can't do the job: `PARSE_JSON` parses an input
once for all calls on it, `CURRENT_WATERMARK` reads the operator context, and
`COALESCE` evaluates an argument only when needed. Precomputing a literal
argument is no reason for codegen, since a runtime class can do it in its
constructor. No built-in aggregate uses `runtimeClass`; all of them go through
`AggFunctionFactory`.
The guide names an example for each step instead of one function to copy. No
existing function follows every step, mostly because they repeat the validation
errors of their type strategy tests in the ITCase.
## Brief change log
- `AGENTS.md`: the section is an eight-step checklist: check other engines,
define, choose how it runs, implement, Table API, PyFlink, test, document. It
also covers naming, `ValidationException` vs `TableRuntimeException`, no
logging per record, and short Javadoc.
- `flink-table-runtime/AGENTS.md`: how a runtime class reads a literal
argument. Aggregates are constructed by `AggFunctionFactory`, not by
`specialize()`.
- `flink-table-planner/AGENTS.md`: where a `runtimeProvided` scalar or
aggregate is wired in, how to share work in generated code, and where function
tests go.
- `[hotfix][python]`: lists 16 PyFlink expressions that were missing from
`expressions.rst`, such as `parse_json`, `uuid_v4` and `lead`. No test compares
that list with the code.
The docs step describes today's flow with `sql_functions.yml`. The
per-function pages proposed in FLINK-41315 will update it when they land.
## Verifying this change
This change is documentation only. Every class, method and count in the
guide was checked against master. The PyFlink docs build, `sphinx-autogen`
followed by `make html` with `SPHINXOPTS="-a -W"`, passes and renders all 16
new pages.
## Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): no
- The public API, i.e., is any changed class annotated with
`@Public(Evolving)`: no
- The serializers: no
- The runtime per-record code paths (performance sensitive): no
- Anything that affects deployment or recovery: JobManager (and its
components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no
- The S3 file system connector: no
## Documentation
- Does this pull request introduce a new feature? no
- If yes, how is the feature documented? not applicable
---
##### Was generative AI tooling used to co-author this PR?
- [X] Yes (please specify the tool below)
Generated-by: Claude Code 2.1.321 (Claude Opus 5.5)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
