autophagy commented on code in PR #29957:
URL: https://github.com/apache/flink/pull/29957#discussion_r4642565772


##########
docs/content/release-notes/flink-2.4.md:
##########
@@ -0,0 +1,359 @@
+---
+title: "Release Notes - Flink 2.4"
+---
+
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+  http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied.  See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+# Release notes - Flink 2.4
+
+These release notes discuss important aspects, such as configuration, behavior 
or dependencies,
+that changed between Flink 2.3 and Flink 2.4. Please read these notes 
carefully if you are
+planning to upgrade your Flink version to 2.4.
+
+### Table SQL / API
+
+#### Upgrade Calcite to 1.41.0
+
+##### [FLINK-39859](https://issues.apache.org/jira/browse/FLINK-39859)
+
+Flink 2.4 upgrades Apache Calcite, the foundation of the SQL parser and 
optimizer, to version
+1.41.0. This incorporates the fixes and improvements from the intervening 
Calcite releases.
+Applications that depend directly on Calcite classes or extend Flink's planner 
should check
+their dependencies and custom planner code against the upgraded version.
+
+#### Common subexpression elimination in Calc code generation
+
+##### [FLINK-39268](https://issues.apache.org/jira/browse/FLINK-39268)
+
+Flink 2.4 avoids redundant evaluation of common subexpressions within a Calc 
operator, which
+combines filtering and projection. Repeated deterministic expressions can 
reuse an already
+computed result instead of being evaluated again for the same input row. This 
applies to both
+batch and streaming queries and can reduce the cost of repeated built-in or 
user-defined
+function calls.
+
+Reuse respects conditional evaluation, such as short-circuiting in `AND`, 
`OR`, and `CASE`.
+UDF authors should ensure that `isDeterministic()` correctly describes their 
function:
+functions with side effects or results that vary for identical inputs must 
return `false`.
+
+#### Built-in UDF metrics
+
+##### [FLINK-38071](https://issues.apache.org/jira/browse/FLINK-38071) 
(FLIP-485)
+
+SQL and Table API users can now monitor UDF execution through two built-in 
metrics:
+`udfProcessingTime`, a histogram of sampled invocation durations, and 
`udfExceptionCount`,
+a counter of exceptions from the function. Metrics are registered on the 
executing operator
+under `udf.<udf_name>`. For asynchronous functions, processing time includes 
the time from
+dispatch until completion.
+
+The feature is opt-in:
+
+- `table.exec.udf-metric-enabled` (default: `false`) enables metric collection.
+- `table.exec.udf-metric.sample-interval` (default: `100`) controls how often 
invocation
+  durations are measured. Exceptions are counted on every invocation.
+
+The metrics cover directly invoked synchronous and asynchronous scalar and 
table functions.
+Python functions, aggregate functions, and Process Table Functions are not 
covered. Enabling
+the metrics adds overhead; use sampling to control the cost.
+
+#### Early firing for interval joins
+
+##### [FLINK-36953](https://issues.apache.org/jira/browse/FLINK-36953) 
(FLIP-497)
+
+Outer interval joins can emit unmatched rows before their join interval 
closes. The new
+`EARLY_FIRE` hint emits a speculative, null-padded result after a configured 
delay. If a
+matching row arrives later within the interval, Flink corrects the speculative 
result with
+update-before and update-after records.
+
+```sql
+SELECT /*+ EARLY_FIRE('delay'='5s') */ o.id, s.ship_time
+FROM Orders o
+LEFT OUTER JOIN Shipments s
+ON o.id = s.order_id
+AND o.order_time BETWEEN s.ship_time - INTERVAL '1' HOUR AND s.ship_time;
+```
+
+The optional `time-mode` selects `rowtime` or `proctime`; it defaults to the 
time mode of the
+interval join. Each unmatched outer row fires at most once. The feature is 
disabled unless
+the hint is present and requires downstream operators and sinks that accept 
updates.
+Existing interval joins without the hint retain their behavior.
+
+#### FROM_CHANGELOG and TO_CHANGELOG enhancements
+
+##### [FLINK-39258](https://issues.apache.org/jira/browse/FLINK-39258) 
(FLIP-564)
+
+Flink 2.4 adds partitioned processing and additional controls to the changelog 
conversion
+PTFs introduced in Flink 2.3:
+
+- `PARTITION BY` support to co-locate related changes, with corresponding 
convenience methods
+  on partitioned tables in the Table API.
+- Partitioned `FROM_CHANGELOG` calls can use the existing PTF `ORDER BY` 
support to reorder
+  CDC events per key using a watermarked time attribute. Late events are 
dropped by the
+  sorting operation.
+- `error_handling => 'FAIL' | 'SKIP'` for `FROM_CHANGELOG` to control how NULL 
or unmapped
+  operation codes are handled. The default is `FAIL`.
+- `produces_full_deletes` for `TO_CHANGELOG` to choose between full-row 
deletes and deletes
+  containing only identifying keys. The default is `true`.
+
+These additions make it easier to consume custom CDC streams and write 
changelog events to
+append-only sinks. Users must still provide correctly ordered, complete change 
events for
+the selected changelog mode; the functions do not reconstruct arbitrary 
missing before images.
+See the [Changelog Conversion 
documentation](https://nightlies.apache.org/flink/flink-docs-release-2.4/docs/sql/reference/queries/changelog/)
+for supported mappings and input requirements.
+
+#### Process Table Function state access and broadcast support
+
+##### [FLINK-39254](https://issues.apache.org/jira/browse/FLINK-39254) 
(FLIP-565)
+
+Flink 2.4 adds lazy value-state access and broadcast inputs to Process Table 
Functions:
+
+- `ValueView` provides lazy value-state access alongside `ListView` and 
`MapView`. A function
+  can read or update state only when needed, avoiding unnecessary 
serialization work.
+- `BROADCAST_SEMANTIC_TABLE` lets a PTF receive a rule or control table on 
every parallel
+  instance. State declared with `@StateHint(StateKind.BROADCAST)` is writable 
from the
+  broadcast input and readable while processing the regular inputs.
+- `NOTIFY_STATEFUL_SETS` lets a broadcast update trigger re-evaluation of 
existing stateful
+  partitions. This is opt-in because visiting all stateful partitions can be 
expensive.
+
+#### Process Table Function test harness
+
+##### [FLINK-39357](https://issues.apache.org/jira/browse/FLINK-39357) 
(FLIP-567)
+
+The new `ProcessTableFunctionTestHarness` lets developers test PTF business 
logic without
+starting a Flink cluster. Tests can configure scalar and table arguments, 
partitioning, and
+initial state, feed rows into individual inputs, and inspect the resulting 
output.
+
+The harness also supports advancing watermarks and the system clock, 
inspecting timers,

Review Comment:
   Yes, @weiqingy is right. State TTL and system clock stuff was deliberately 
removed from the final implementation after discussion in some PRs. Changelog 
support didn't make the cut for 2.4.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to