[jira] [Commented] (FLINK-13117) Serialize Issue when restoring from savepoint
[ https://issues.apache.org/jira/browse/FLINK-13117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16890811#comment-16890811 ] Jark Wu commented on FLINK-13117: - Note that the pull request [#9179|https://github.com/apache/flink/pull/9179] should be under FLINK-13337. > Serialize Issue when restoring from savepoint > - > > Key: FLINK-13117 > URL: https://issues.apache.org/jira/browse/FLINK-13117 > Project: Flink > Issue Type: Bug >Reporter: dmgkeke >Priority: Major > Labels: pull-request-available > Time Spent: 20m > Remaining Estimate: 0h > > Hi, I have one question. > My application should be maintained without loss of data. > So I use savepoint to deploy a new application. > > Normally there is no problem in restarting. > However, when the schema of some case classes is changed, a serialize error > occurs at restart. > > How can I resolve this? > Flink version is 1.8.0 > Here is the error log. > Thank you. > > org.apache.flink.util.StateMigrationException: The new state serializer > cannot be incompatible. > at > org.apache.flink.contrib.streaming.state.RocksDBKeyedStateBackend.updateRestoredStateMetaInfo(RocksDBKeyedStateBackend.java:527) > at > org.apache.flink.contrib.streaming.state.RocksDBKeyedStateBackend.tryRegisterKvStateInformation(RocksDBKeyedStateBackend.java:475) > at > org.apache.flink.contrib.streaming.state.RocksDBKeyedStateBackend.createInternalState(RocksDBKeyedStateBackend.java:613) > at > org.apache.flink.runtime.state.KeyedStateFactory.createInternalState(KeyedStateFactory.java:47) > at > org.apache.flink.runtime.state.ttl.TtlStateFactory.createStateAndWrapWithTtlIfEnabled(TtlStateFactory.java:72) > at > org.apache.flink.runtime.state.AbstractKeyedStateBackend.getOrCreateKeyedState(AbstractKeyedStateBackend.java:286) > at > org.apache.flink.streaming.api.operators.AbstractStreamOperator.getOrCreateKeyedState(AbstractStreamOperator.java:568) > at > org.apache.flink.streaming.runtime.operators.windowing.WindowOperator.open(WindowOperator.java:240) > at > org.apache.flink.streaming.runtime.tasks.StreamTask.openAllOperators(StreamTask.java:424) > at > org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:290) > at org.apache.flink.runtime.taskmanager.Task.run(Task.java:711) > at java.lang.Thread.run(Thread.java:748) > -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (FLINK-13117) Serialize Issue when restoring from savepoint
[ https://issues.apache.org/jira/browse/FLINK-13117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16881795#comment-16881795 ] dmgkeke commented on FLINK-13117: - [~yunta] Thank you for your reply. I had already read the schema evolution document. But I can't change all kinds of my classes to POJO or Avro. So I found another solution. I changed the serializer(FieldSerializer => CompatibleFieldSerializer) through the ExecutionConfig.addDefaultKryoSerializer function with trait. I will end the issue. Thank you again. > Serialize Issue when restoring from savepoint > - > > Key: FLINK-13117 > URL: https://issues.apache.org/jira/browse/FLINK-13117 > Project: Flink > Issue Type: Bug >Reporter: dmgkeke >Priority: Major > > Hi, I have one question. > My application should be maintained without loss of data. > So I use savepoint to deploy a new application. > > Normally there is no problem in restarting. > However, when the schema of some case classes is changed, a serialize error > occurs at restart. > > How can I resolve this? > Flink version is 1.8.0 > Here is the error log. > Thank you. > > org.apache.flink.util.StateMigrationException: The new state serializer > cannot be incompatible. > at > org.apache.flink.contrib.streaming.state.RocksDBKeyedStateBackend.updateRestoredStateMetaInfo(RocksDBKeyedStateBackend.java:527) > at > org.apache.flink.contrib.streaming.state.RocksDBKeyedStateBackend.tryRegisterKvStateInformation(RocksDBKeyedStateBackend.java:475) > at > org.apache.flink.contrib.streaming.state.RocksDBKeyedStateBackend.createInternalState(RocksDBKeyedStateBackend.java:613) > at > org.apache.flink.runtime.state.KeyedStateFactory.createInternalState(KeyedStateFactory.java:47) > at > org.apache.flink.runtime.state.ttl.TtlStateFactory.createStateAndWrapWithTtlIfEnabled(TtlStateFactory.java:72) > at > org.apache.flink.runtime.state.AbstractKeyedStateBackend.getOrCreateKeyedState(AbstractKeyedStateBackend.java:286) > at > org.apache.flink.streaming.api.operators.AbstractStreamOperator.getOrCreateKeyedState(AbstractStreamOperator.java:568) > at > org.apache.flink.streaming.runtime.operators.windowing.WindowOperator.open(WindowOperator.java:240) > at > org.apache.flink.streaming.runtime.tasks.StreamTask.openAllOperators(StreamTask.java:424) > at > org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:290) > at org.apache.flink.runtime.taskmanager.Task.run(Task.java:711) > at java.lang.Thread.run(Thread.java:748) > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (FLINK-13117) Serialize Issue when restoring from savepoint
[ https://issues.apache.org/jira/browse/FLINK-13117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16879588#comment-16879588 ] Yun Tang commented on FLINK-13117: -- [~sch.oh] You could refer to [state schema_evolution|https://ci.apache.org/projects/flink/flink-docs-release-1.8/dev/stream/state/schema_evolution.html] docs to know how to migrate your state when schema changed. Please note there is no guarantee that any kind of schema change could be migrated well, you'd better follow the instruction in this doc to evolve your classes. Last but not least, I don't think this is a bug with your current provided information. IMO, [user mail list|https://flink.apache.org/community.html#mailing-lists] should be a better place to ask for help. > Serialize Issue when restoring from savepoint > - > > Key: FLINK-13117 > URL: https://issues.apache.org/jira/browse/FLINK-13117 > Project: Flink > Issue Type: Bug >Reporter: dmgkeke >Priority: Major > > Hi, I have one question. > My application should be maintained without loss of data. > So I use savepoint to deploy a new application. > > Normally there is no problem in restarting. > However, when the schema of some case classes is changed, a serialize error > occurs at restart. > > How can I resolve this? > Flink version is 1.8.0 > Here is the error log. > Thank you. > > org.apache.flink.util.StateMigrationException: The new state serializer > cannot be incompatible. > at > org.apache.flink.contrib.streaming.state.RocksDBKeyedStateBackend.updateRestoredStateMetaInfo(RocksDBKeyedStateBackend.java:527) > at > org.apache.flink.contrib.streaming.state.RocksDBKeyedStateBackend.tryRegisterKvStateInformation(RocksDBKeyedStateBackend.java:475) > at > org.apache.flink.contrib.streaming.state.RocksDBKeyedStateBackend.createInternalState(RocksDBKeyedStateBackend.java:613) > at > org.apache.flink.runtime.state.KeyedStateFactory.createInternalState(KeyedStateFactory.java:47) > at > org.apache.flink.runtime.state.ttl.TtlStateFactory.createStateAndWrapWithTtlIfEnabled(TtlStateFactory.java:72) > at > org.apache.flink.runtime.state.AbstractKeyedStateBackend.getOrCreateKeyedState(AbstractKeyedStateBackend.java:286) > at > org.apache.flink.streaming.api.operators.AbstractStreamOperator.getOrCreateKeyedState(AbstractStreamOperator.java:568) > at > org.apache.flink.streaming.runtime.operators.windowing.WindowOperator.open(WindowOperator.java:240) > at > org.apache.flink.streaming.runtime.tasks.StreamTask.openAllOperators(StreamTask.java:424) > at > org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:290) > at org.apache.flink.runtime.taskmanager.Task.run(Task.java:711) > at java.lang.Thread.run(Thread.java:748) > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
