[
https://issues.apache.org/jira/browse/FLINK-16686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17288794#comment-17288794
]
Dong Lin commented on FLINK-16686:
----------------------------------
Discussed with [~yunta] about this bug. This bug is hard to fix partly because
a proper solution requires changes to RocksDB and Kryo. It will take more time
to agree on a coordinated solution across Flink, RocksDB and Kryo. Given that
we have a workaround solution and the impact of this bug seems low (based on
the number of comments in this thread), we have not initiated serious
discussion in the Flink community to tackle this bug.
The workaround solution is what Andrey described above, i.e. write a custom
serializer.
> [State TTL] Make user class loader available in native RocksDB compaction
> thread
> --------------------------------------------------------------------------------
>
> Key: FLINK-16686
> URL: https://issues.apache.org/jira/browse/FLINK-16686
> Project: Flink
> Issue Type: Bug
> Components: Runtime / State Backends
> Affects Versions: 1.8.0
> Reporter: Andrey Zagrebin
> Priority: Major
>
> The issue is initially reported
> [here|https://stackoverflow.com/questions/60745711/flink-kryo-serializer-because-chill-serializer-couldnt-be-found].
> The problem is that the java code of Flink compaction filter is called from
> RocksDB native C++ code. It is called in the context of the native compaction
> thread. RocksDB has utilities to create java Thread context for the Flink
> java callback. Presumably, the Java thread context class loader is not set at
> all and if it is queried then it produces NullPointerException.
> The provided report enabled a list state with TTL. The compaction filter has
> to deserialise elements to check expiration. The deserialiser relies on Kryo
> which queries the thread context class loader which is expected to be the
> user class loader of the task but turns out to be null.
> We should investigate how to pass the user class loader to the compaction
> thread of the list state with TTL.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
