Github user StefanRRichter commented on the issue:
https://github.com/apache/flink/pull/4863
@aljoscha from the top of my head, I think some concerns are invalid. After
being added to to the checkpoint store, the incremental handles should be
complete and self contained - all placeholder handles are replaced by the real
handles. What is, in fact, build up incrementally is the reference count for
each sst-file handle when we re-register on recovery. So if we do not register
a handle, it does not account for the reference counting, which might be
correct if we assume the handle is "dead". However, we should not call
`discardState()` on an unregistered handle because if they are unregistered,
they are assumed to still own their handles and will wipe all of them without
considering other references in the registry. So for most parts, we can simply
rely on the remaining handles building up a reasonable count, except for
handles that only still occurred in the broken incremental checkpoints. Those
are leaking because we lost any way to properly delete them. But I think this ki
nd of leak always existed in the implementation, because we cannot call
dispose if we cannot retrieve the checkpoint.
To sum up: it should work, there is a leak to be aware of, but that problem
always existed.
---
