twalthr opened a new pull request, #29269: URL: https://github.com/apache/flink/pull/29269
## What is the purpose of the change This pull request adds `ValueView<T>` to Process Table Functions (PTFs), as proposed in [FLIP-565](https://cwiki.apache.org/confluence/spaces/FLINK/pages/406620840/FLIP-565+Improve+ProcessTableFunctions+for+late+data+handling+and+state+access). By default, a PTF value state entry is *eager*: the full `Row`/POJO is read from the state backend when `eval()` is called and written back when it returns (a Read-Modify-Write cycle), even if the function never reads or changes it. `ListView` and `MapView` already avoid this for collections. `ValueView<T>` provides the same benefit for a single value: it deserializes the value only on `getValue()` and serializes only on `setValue()`/`clear()`, which is preferable when state is accessed conditionally. ## Brief change log - Add `ValueView<T>` (`@PublicEvolving`) in `flink-table-common`, modeled after `ListView`/`MapView`, with `getValue()`, `setValue()`, `isEmpty()`, `clear()` and a `newValueViewDataType()` factory. Null values are not supported; `setValue(null)` equals `clear()`. - Add the state-backed `StateValueView` / `KeyedStateValueView` that delegates to `ValueState` via `ExternalSerializer` (internal storage, external exposure). - Handle `@DataTypeHint` on `ValueView<T>` in `DataTypeExtractor` as a direct definition of the value type `T` (works for atomic and composite value types such as `ValueView<Row>`). - Rework `ProcessTableRunner` to hold a single `StateHandle[]` (state kind, state handle, and — only for eager value state — hash function, equaliser and per-invocation scratch), replacing the previous parallel arrays. Moving view state to/from the function is now cheap; the eager value-state Read-Modify-Write path stays explicit. The operator and code generator are updated accordingly. - Reject `ValueView` in accumulators of aggregating functions with a `ValidationException`. - Document the difference between eager value state and views, and add `ValueView` examples (JavaDoc + `ptfs.md`, including the Chinese docs). ## Verifying this change This change added tests and can be verified as follows: - `DataTypeExtractorTest` — reflective, annotated-atomic, and `ValueView<Row>` (composite hint) extraction. - `DataViewUtilsTest` — rejection of `ValueView` in aggregate accumulators (field and whole-accumulator). - `ProcessTableFunctionTestHarnessTest` — end-to-end runtime behavior: value view state, initial state, lazy access, `setValue(null)` clear, and a `ValueView<Row>` value. - `ProcessTableFunctionSemanticTests` — new `process-value-state` and `process-complex-value-state` programs (the latter uses `ValueView<Tuple2<Instant, List<Integer>>>`), exercising the full planner path including hash/equaliser generation over the composite value type. ## Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): no - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: yes (new `@PublicEvolving` `ValueView`; `ProcessTableFunction` JavaDoc) - The serializers: no (reuses `ExternalSerializer`; `ValueView` state uses a standard `ValueStateDescriptor`) - The runtime per-record code paths (performance sensitive): yes (PTF state handling in `ProcessTableRunner`) - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no - The S3 file system connector: no ## Documentation - Does this pull request introduce a new feature? yes - If yes, how is the feature documented? docs and JavaDocs --- ##### Was generative AI tooling used to co-author this PR? - [X] Yes (please specify the tool below) Generated-by: Claude Code (Claude Opus 4.8) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
