[
https://issues.apache.org/jira/browse/FLINK-9169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16463869#comment-16463869
]
ASF GitHub Bot commented on FLINK-9169:
---------------------------------------
Github user StefanRRichter commented on a diff in the pull request:
https://github.com/apache/flink/pull/5950#discussion_r186082804
--- Diff:
flink-core/src/main/java/org/apache/flink/api/common/typeutils/TypeSerializerSerializationUtil.java
---
@@ -102,17 +99,24 @@
*
* @return the deserialized serializer.
*/
- public static <T> TypeSerializer<T> tryReadSerializer(DataInputView in,
ClassLoader userCodeClassLoader, boolean useDummyPlaceholder) {
+ public static <T> TypeSerializer<T> tryReadSerializer(
+ DataInputView in,
+ ClassLoader userCodeClassLoader,
+ boolean useDummyPlaceholder) throws IOException {
+
final
TypeSerializerSerializationUtil.TypeSerializerSerializationProxy<T> proxy =
- new
TypeSerializerSerializationUtil.TypeSerializerSerializationProxy<>(userCodeClassLoader,
useDummyPlaceholder);
+ new
TypeSerializerSerializationUtil.TypeSerializerSerializationProxy<>(userCodeClassLoader);
try {
proxy.read(in);
return proxy.getTypeSerializer();
- } catch (IOException e) {
- LOG.warn("Deserialization of serializer errored;
replacing with null.", e);
-
- return null;
+ } catch (UnloadableTypeSerializerException e) {
--- End diff --
I would let this bubble up one more level, remove the flag here and only
catch `UnloadableTypeSerializerException ` in the case where this method is
called with `true`.
> NPE when restoring from old savepoint and state serializer could not be
> deserialized
> ------------------------------------------------------------------------------------
>
> Key: FLINK-9169
> URL: https://issues.apache.org/jira/browse/FLINK-9169
> Project: Flink
> Issue Type: Bug
> Components: State Backends, Checkpointing
> Affects Versions: 1.5.0, 1.4.2
> Reporter: Till Rohrmann
> Assignee: Tzu-Li (Gordon) Tai
> Priority: Blocker
> Fix For: 1.5.0
>
>
> A user reported to have observed the following exception when restoring a
> Flink job from a 1.3 savepoint with Flink 1.4.
> {code}
> 2018-04-02 21:44:18,146 INFO org.apache.flink.runtime.taskmanager.Task
> - ApplyAMUpdate (13/160) (7248adb0b85f4458ae4144963d65
> 6fa6) switched from RUNNING to FAILED.
> java.lang.IllegalStateException: Could not initialize keyed state backend.
> at
> org.apache.flink.streaming.api.operators.AbstractStreamOperator.initKeyedState(AbstractStreamOperator.java:293)
> at
> org.apache.flink.streaming.api.operators.AbstractStreamOperator.initializeState(AbstractStreamOperator.java:225)
> at
> org.apache.flink.streaming.runtime.tasks.StreamTask.initializeOperators(StreamTask.java:692)
> at
> org.apache.flink.streaming.runtime.tasks.StreamTask.initializeState(StreamTask.java:679)
> at
> org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:253)
> at org.apache.flink.runtime.taskmanager.Task.run(Task.java:718)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.lang.NullPointerException
> at
> org.apache.flink.util.Preconditions.checkNotNull(Preconditions.java:58)
> at
> org.apache.flink.runtime.state.RegisteredKeyedBackendStateMetaInfo.<init>(RegisteredKeyedBackendStateMetaInfo.java:53)
> at
> org.apache.flink.contrib.streaming.state.RocksDBKeyedStateBackend$RocksDBFullRestoreOperation.restoreKVStateMetaData(RocksDBKeyedStateB
> ackend.java:1216)
> at
> org.apache.flink.contrib.streaming.state.RocksDBKeyedStateBackend$RocksDBFullRestoreOperation.restoreKeyGroupsInStateHandle(RocksDBKeye
> dStateBackend.java:1153)
> at
> org.apache.flink.contrib.streaming.state.RocksDBKeyedStateBackend$RocksDBFullRestoreOperation.doRestore(RocksDBKeyedStateBackend.java:1
> 139)
> at
> org.apache.flink.contrib.streaming.state.RocksDBKeyedStateBackend.restore(RocksDBKeyedStateBackend.java:1034)
> at
> org.apache.flink.streaming.runtime.tasks.StreamTask.createKeyedStateBackend(StreamTask.java:773)
> at
> org.apache.flink.streaming.api.operators.AbstractStreamOperator.initKeyedState(AbstractStreamOperator.java:283)
> ... 6 more
> {code}
> Looking at the {{KeyedBackendStateMetaInfoSnapshotReaderWriters}}, we create
> {{RegisteredKeyedBackendStateMetaInfo.Snapshot}} where the
> {{stateSerializer}} can be {{null}}. This is not the problem, however, in
> {{RocksDBKeyedStateBackend#restoreKVStateMetaData}} we create a
> {{RegisteredKeyedBackendStateMetaInfo}} from the deserialized {{Snapshot}}
> where we null check the state serializer. This will then fail with an
> indescriptive NPE.
> I think the same should happen when resuming with Flink 1.5 from a 1.4
> savepoint.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
