[
https://issues.apache.org/jira/browse/FLINK-21419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17301484#comment-17301484
]
Xintong Song commented on FLINK-21419:
--------------------------------------
Backgrounds:
- GC cleaner was fist introduced in FLINK-13985, as a safe net incase the
segment is not explicitly freed.
- Then in FLINK-14894 and FLINK-15758, it was discovered that trying to free
the underlying memory when there are potential references to it could be
dangerous. That's where we switched to completely relying on GC.
- The problem of solely relying on GC is that, memory is not released timely
and it creates more GC pressure due to triggering GC when not being able to
reserve memory. FLINK-20663
bq. The fact does not changed.
Actually, it did changed a bit.
- {{MemorySegment}} is never thread safe, thus it is the users' responsibility
to make sure they are not accessed after being freed. (By "users" I mean
operators/tasks, because currently we do not support user codes / UDFs to use
managed memory).
- One of the most dangerous potential sources of access-after-freed is that,
one could try to access the memory from a {{ByteBuffer}} wrapped from the
segment. It can easily get overlooked that a wrapped {{ByteBuffer}} is
different from a normal java {{ByteBuffer}} and can be released when the
segment is freed. Thus, we have forbidden wrapping buffers from unsafe memory
segments.
- That is to say, while we are not 100% preventing access-after-freed, the
chances are significantly reduced. We do not provide further protections
against such unsafe accesses, for the chance of such issues is small and the
price (performance wise) for the protection is high.
- You may take a look at the long discussion in FLINK-20663 for more details..
> Remove GC cleaner mechanism for unsafe memory segments
> ------------------------------------------------------
>
> Key: FLINK-21419
> URL: https://issues.apache.org/jira/browse/FLINK-21419
> Project: Flink
> Issue Type: Sub-task
> Components: Runtime / Coordination
> Reporter: Xintong Song
> Assignee: Nicholas Jiang
> Priority: Major
> Labels: pull-request-available
> Fix For: 1.13.0
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
