Aaron He created FLINK-41356:
--------------------------------
Summary: Support early fire in the async state interval join
Key: FLINK-41356
URL: https://issues.apache.org/jira/browse/FLINK-41356
Project: Flink
Issue Type: Sub-task
Components: Table SQL / Planner, Table SQL / Runtime
Reporter: Aaron He
h2. Motivation
FLINK-41355 adds async state versions of the SQL interval join, selected by
table.exec.async-state.enabled, but keeps the synchronous functions for a join
with the EARLY_FIRE hint (FLIP-497), because only they implement early fire. A
job with an early-fire interval join therefore still runs that operator on
synchronous state on a disaggregated state backend such as ForSt.
h2. Proposal
Port the interval-join early fire of FLINK-40171 and FLINK-40172 to
AsyncStateTimeIntervalJoin and its row-time and processing-time subclasses, and
drop the planner fallback, so table.exec.async-state.enabled selects the async
functions with or without the hint.
* State: the same early-fire state as the synchronous functions, with the same
names, types and registration conditions: IntervalJoinLeftFired /
IntervalJoinRightFired (MapState<Long, List<Boolean>>) and, for processing-time
early fire on a row-time join, IntervalJoinLeftEarlyFireSchedule /
IntervalJoinRightEarlyFireSchedule. A savepoint of the synchronous early-fire
join restores into the async functions.
* Logic: the same as the synchronous functions: speculative outer pads at the
early-fire time, retraction when a later match arrives, no second pad at
expiry; inner joins ignore the hint; processing-time early fire on a row-time
join uses processing-time timers told apart by the timer's time domain.
* Cleanup: the fired bits of expired buckets are deleted together with the
buckets, using the cleanup of FLINK-41355 (one MapState#asyncPutAll of null
values; asyncClear above 65,536 entries on full expiry).
* No new configuration option, no JSON plan change and no ExecNode version
change.
* Docs: the interval join entry of the disaggregated state page no longer
excludes the EARLY_FIRE hint.
h2. Dependency
Stacked on FLINK-41355, which depends on FLINK-41091 and FLINK-41095.
h2. Tests
* The early-fire harness cases of RowTimeIntervalJoinTest and
ProcTimeIntervalJoinTest run against both the synchronous and the async
functions.
* The committed early-fire restore programs (interval-join-early-fire,
interval-join-proc-time-early-fire) restore their synchronous savepoints with
async state on.
* The planner test expects the async operator for an EARLY_FIRE join with async
state on.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
