[
https://issues.apache.org/jira/browse/FLINK-40667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Fan updated FLINK-40667:
----------------------------
Description:
Ownership of the source buffer is transferred to the filtering handler too
early: `recover()` transfers it via `retainBuffer()` as an argument expression,
so the extra reference exists before `segmentSerializerFor` (arg5) evaluates
and before `ChannelStateFilteringHandler.filterAndRewrite`'s two early throws
run — any of which orphans it, so refCnt never reaches 0 and `closeInternal()`
later `free()`s a segment still held by a live `NetworkBuffer`. The inner
`GateFilterHandler` already owns the true transfer boundary
(`sourceBufferOwnershipTransferred`, flipped at `vc.setNextBuffer`), which is
where FLINK-40345 added the recycle-on-throw guard.
Fix: transfer ownership only at that boundary — pass the base buffer down and
`retainBuffer()` inside the inner handler right before `vc.setNextBuffer`, so
every pre-hand-off throw runs while the caller still owns the buffer and its
`finally` recycles it.
was:Siblings of FLINK-40345 (which patched only the inner handler):
`SpillingWithFilteringHandler.recover()` retains the buffer before
`segmentSerializerFor` can throw `IOException`, and
`ChannelStateFilteringHandler.filterAndRewrite`'s two early throws fire before
the inner guard. On the exception the retained buffer is never released, so
`closeInternal()` later `free()`s a segment still held by a live
`NetworkBuffer`. Fix: obtain the serializer before retaining, and move
`retainBuffer()` past the early throws.
> Buffer ref-count leaks in CDR recovery filtering end in free() on a live
> segment
> --------------------------------------------------------------------------------
>
> Key: FLINK-40667
> URL: https://issues.apache.org/jira/browse/FLINK-40667
> Project: Flink
> Issue Type: Sub-task
> Components: Runtime / Checkpointing
> Reporter: Rui Fan
> Assignee: Rui Fan
> Priority: Major
>
> Ownership of the source buffer is transferred to the filtering handler too
> early: `recover()` transfers it via `retainBuffer()` as an argument
> expression, so the extra reference exists before `segmentSerializerFor`
> (arg5) evaluates and before `ChannelStateFilteringHandler.filterAndRewrite`'s
> two early throws run — any of which orphans it, so refCnt never reaches 0 and
> `closeInternal()` later `free()`s a segment still held by a live
> `NetworkBuffer`. The inner `GateFilterHandler` already owns the true transfer
> boundary (`sourceBufferOwnershipTransferred`, flipped at `vc.setNextBuffer`),
> which is where FLINK-40345 added the recycle-on-throw guard.
> Fix: transfer ownership only at that boundary — pass the base buffer down and
> `retainBuffer()` inside the inner handler right before `vc.setNextBuffer`, so
> every pre-hand-off throw runs while the caller still owns the buffer and its
> `finally` recycles it.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
