rkhachatryan commented on PR #29954: URL: https://github.com/apache/flink/pull/29954#issuecomment-7051299304
> My main concern is that we are potentially significantly increasing number of "urgent" mails that can overtake the queue, do we know how bounded is this new source? @Izeren good point. Two parts: 1. Starvation of the input: urgent mails arriving while a batch is consumed are served before the rest of the batch, so a sustained stream of them could extend it indefinitely. That was already the case in FLINK-35051, but making operator events urgent makes it easier to hit. I'll push a commit that caps the urgent mails served per batch round at `pending urgent + batch size` (computed at `createBatch`). In normal operation urgent mails will still overtake the batch, but the round always ends and input gets its turn. 2. Beyond that, operator events are bounded by the coordinator's send rate and are cheap to apply. Starvation requires a sustained event rate above what the task can process, and then the backlog grows unboundedly regardless of the policy (as it did before FLINK-35051, when events were normal mails). For reference, how the variants compare (UC = unaligned checkpoint): | | pre-FLINK-35051 | FLINK-35051 | this PR, uncapped | this PR, with the cap | |---|---|---|---|---| | **UC trigger waits behind…** | | | | | | rest of the current batch | yes | no | no | no (yes only once the budget is used up) | | normal mails queued before it | yes | no | no | no | | urgent mails queued before it | n/a | yes, order not guaranteed | yes, FIFO | yes, FIFO | | one default-action step (if it arrives mid-batch) | yes | no | no | only once the budget is used up | | **Correctness** | | | | | | operator events vs. UC trigger (lost splits) | OK | broken | OK | OK | | FIFO among urgent mails | n/a | no | yes | yes | | stale gate mail vs. next UC barrier (FLINK-41547) | affected | affected | affected | affected | | **Control mails** | front of mailbox (LIFO) | urgent, order not guaranteed | urgent, FIFO, behind pending urgent mails | same | | **Sustained urgent overload** | | | | | | input | gets a turn every round | stalls (hard to reach) | stalls | gets a turn every round | | normal mails while default action is suspended | progress | starve | starve | starve | | urgent backlog | grows | grows | grows | grows | The one-step wait is the price for input getting a turn under overload. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
