mobanihan opened a new pull request, #29728:
URL: https://github.com/apache/flink/pull/29728
## What is the purpose of the change
A pattern that ends in `notFollowedBy(...).within(...)` emits a match only
from its Pending state. A partial match still inside a looping or optional
stage before the NOT pattern reaches the Pending state only when a next event
arrives, so when the window expired first, the partial match was discarded as a
timeout: elements were missing from the match (FLINK-31040), or with `greedy()`
the match was lost (FLINK-31044). This pull request closes such a stage when
its window expires and releases the partial match as a match.
The first commit fixes a separate, existing defect that the second would
otherwise make worse: the ignore state of an optional repeat of a relaxed
`times(n, m)` has no stop edge, so a forbidden event arriving while an attempt
waits there does not cancel it. I kept it under FLINK-31040 because the fix
depends on it; I am happy to move it to its own ticket if you prefer.
## Brief change log
- `NFACompiler`: the ignore states of the optional repeats of a relaxed
`times(n, m)` get the stop edge of a following `notFollowedBy`, as the ignore
state of a relaxed `oneOrMore()` already does.
- `NFACompiler`: the PROCEED edges that mean "this stage may end here" are
marked, and the factory hands the NFA, for each state, the Pending state those
edges reach.
- `NFA.advanceTime`: a partial match that times out in such a state moves
into that Pending state with the same events and timestamps, and is released,
or kept Pending when only the stage's own window expired.
- After a greedy stage the Pending state's edges are guarded by the
stage's condition, so time closes the stage into an unguarded copy of the
Pending state, scoped to the stage that installed the guard. The copy is
created after every other state, so no existing state name changes.
## Verifying this change
This change added tests and can be verified as follows:
- `NotPatternITCase`: 15 new tests covering the reported patterns
(FLINK-31040, FLINK-31044 with and without greedy), `times(n, m)` and group
patterns before a trailing NOT pattern, a stage closed by its own window, the
forbidden event inside an optional repeat, and a greedy stage that must not be
closed across later stages.
- `CEPOperatorTest`: a keyed pattern emits the released match on the
watermark.
- Without the first commit's change, its two tests fail. Without the
second commit's change, 13 of its 14 tests fail; the 14th,
`testStageClosedByTimeBeforeAGreedyLoopIsDroppedByAnEventOfTheLoop`, guards
against a wrong match the change could introduce, and fails when the unguarded
copy is not scoped to its stage.
- The whole `flink-cep` suite passes after each commit, and
`spotless:check` and `checkstyle:check` are clean.
- Savepoints taken before the change restore into it (checked on a
backport of the change to 1.20).
## Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): no
- The public API, i.e., is any changed class annotated with
`@Public(Evolving)`: no
- The serializers: no (the serialized NFA state format is unchanged)
- The runtime per-record code paths (performance sensitive): no (the
change runs when partial matches time out)
- Anything that affects deployment or recovery: JobManager (and its
components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no, but see the note
below
- The S3 file system connector: no
A savepoint taken by this version while a pattern with a `greedy()` stage
before a trailing NOT pattern holds a partial match cannot be restored into an
earlier version (the earlier NFA has no state of that name). This is in the
JIRA release note.
## Documentation
- Does this pull request introduce a new feature? no
- If yes, how is the feature documented? not applicable
---
##### Was generative AI tooling used to co-author this PR?
- [X] Yes (please specify the tool below)
Generated-by: Claude Code (Claude Fable 5.1, Claude Opus 5.5)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
