[
https://issues.apache.org/jira/browse/FLINK-34561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gyula Fora closed FLINK-34561.
------------------------------
Resolution: Not A Problem
This works as expected. CRD and the operator is backward compatible, not
forward compatible. As you see in the docs:
```
This means that if you have Flink resources deployed (and Flink applications
running), you can still safely upgrade to newer versions of the operator and
CRD without any problems.
```
We do not currently provide any guarantee for downgrading the operator as it
may populate new status fields that are not compatible with previous operator
versions.
However newer operator versions are much more flexible on JSON deserialisation.
So I would assume that going back to 1.6 from 1.7 would not be an issue however
we do not have explicit guarantee there.
We recommend upgrading to the latest prod release: 1.7.0
> Downgrading flink-kubernetes-operator causes failure
> ----------------------------------------------------
>
> Key: FLINK-34561
> URL: https://issues.apache.org/jira/browse/FLINK-34561
> Project: Flink
> Issue Type: Bug
> Components: Kubernetes Operator
> Reporter: Avi Sanwal
> Priority: Major
>
> Hi,
> We are currently using flink-kubernetes-operator 1.4 version (with v1beta1
> CRD) to manage flink deployments.
> We tried an upgrade to version 1.6 version, but when we try to rollback to
> 1.4, noticed that flink-operator fails to come up with this execption:
> {noformat}
> Exception in thread "main" io.javaoperatorsdk.operator.OperatorException:
> Error starting operator
> at io.javaoperatorsdk.operator.Operator.start(Operator.java:125)
> at
> org.apache.flink.kubernetes.operator.FlinkOperator.run(FlinkOperator.java:215)
> at
> org.apache.flink.kubernetes.operator.FlinkOperator.main(FlinkOperator.java:229)
> Caused by: io.javaoperatorsdk.operator.OperatorException:
> java.lang.IllegalStateException:
> com.fasterxml.jackson.databind.JsonMappingException: Could not deserialize
> spec, this indicates a bug... (through reference chain:
> org.apache.flink.kubernetes.operator.api.FlinkDeployment["status"]->org.apache.flink.kubernetes.operator.api.status.FlinkDeploymentStatus["lifecycleState"])
> at
> io.javaoperatorsdk.operator.api.config.ExecutorServiceManager.lambda$executeAndWaitForAllToComplete$2(ExecutorServiceManager.java:107)
> at java.base/java.util.ArrayList.forEach(Unknown Source)
> at
> io.javaoperatorsdk.operator.api.config.ExecutorServiceManager.executeAndWaitForAllToComplete(ExecutorServiceManager.java:102)
> at
> io.javaoperatorsdk.operator.api.config.ExecutorServiceManager.boundedExecuteAndWaitForAllToComplete(ExecutorServiceManager.java:82)
> at
> io.javaoperatorsdk.operator.ControllerManager.start(ControllerManager.java:36)
> at io.javaoperatorsdk.operator.Operator.start(Operator.java:120)
> ... 2 more
> Caused by: java.lang.IllegalStateException:
> com.fasterxml.jackson.databind.JsonMappingException: Could not deserialize
> spec, this indicates a bug... (through reference chain:
> org.apache.flink.kubernetes.operator.api.FlinkDeployment["status"]->org.apache.flink.kubernetes.operator.api.status.FlinkDeploymentStatus["lifecycleState"])
> at
> io.javaoperatorsdk.operator.api.config.ConfigurationService$1.clone(ConfigurationService.java:112)
> at java.base/java.util.Optional.map(Unknown Source)
> at
> io.javaoperatorsdk.operator.processing.event.source.informer.InformerManager.get(InformerManager.java:175)
> at
> io.javaoperatorsdk.operator.processing.event.source.informer.ManagedInformerEventSource.get(ManagedInformerEventSource.java:110)
> at
> io.javaoperatorsdk.operator.processing.event.EventProcessor.submitReconciliationExecution(EventProcessor.java:128)
> at
> io.javaoperatorsdk.operator.processing.event.EventProcessor.handleMarkedEventForResource(EventProcessor.java:120)
> at
> io.javaoperatorsdk.operator.processing.event.EventProcessor.handleAlreadyMarkedEvents(EventProcessor.java:376)
> at
> io.javaoperatorsdk.operator.processing.event.EventProcessor.start(EventProcessor.java:371)
> at
> io.javaoperatorsdk.operator.processing.Controller.start(Controller.java:336)
> at
> io.javaoperatorsdk.operator.ControllerManager.lambda$start$0(ControllerManager.java:37)
> at
> io.javaoperatorsdk.operator.api.config.ExecutorServiceManager.lambda$executeAndWaitForAllToComplete$0(ExecutorServiceManager.java:96)
> at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
> at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown
> Source)
> at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown
> Source)
> at java.base/java.lang.Thread.run(Unknown Source)
> Caused by: com.fasterxml.jackson.databind.JsonMappingException: Could not
> deserialize spec, this indicates a bug... (through reference chain:
> org.apache.flink.kubernetes.operator.api.FlinkDeployment["status"]->org.apache.flink.kubernetes.operator.api.status.FlinkDeploymentStatus["lifecycleState"])
> at
> com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingException.java:402)
> at
> com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingException.java:361)
> at
> com.fasterxml.jackson.databind.ser.std.StdSerializer.wrapAndThrow(StdSerializer.java:316)
> at
> com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:782)
> at
> com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:178)
> at
> com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:733)
> at
> io.fabric8.kubernetes.model.jackson.BeanPropertyWriterDelegate.serializeAsField(BeanPropertyWriterDelegate.java:66)
> at
> com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:774)
> at
> com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:178)
> at
> com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSerializerProvider.java:480)
> at
> com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(DefaultSerializerProvider.java:319)
> at
> com.fasterxml.jackson.databind.ObjectMapper._writeValueAndClose(ObjectMapper.java:4624)
> at
> com.fasterxml.jackson.databind.ObjectMapper.writeValueAsString(ObjectMapper.java:3869)
> at
> io.javaoperatorsdk.operator.api.config.ConfigurationService$1.clone(ConfigurationService.java:110)
> ... 14 more
> Caused by: java.lang.RuntimeException: Could not deserialize spec, this
> indicates a bug...
> at
> org.apache.flink.kubernetes.operator.api.utils.SpecUtils.deserializeSpecWithMeta(SpecUtils.java:64)
> at
> org.apache.flink.kubernetes.operator.api.status.ReconciliationStatus.deserializeLastReconciledSpecWithMeta(ReconciliationStatus.java:71)
> at
> org.apache.flink.kubernetes.operator.api.status.ReconciliationStatus.deserializeLastReconciledSpec(ReconciliationStatus.java:59)
> at
> org.apache.flink.kubernetes.operator.api.status.CommonStatus.getLifecycleState(CommonStatus.java:75)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown
> Source)
> at java.base/java.lang.reflect.Method.invoke(Unknown Source)
> at
> com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:689)
> at
> io.fabric8.kubernetes.model.jackson.BeanPropertyWriterDelegate.serializeAsField(BeanPropertyWriterDelegate.java:66)
> at
> com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:774)
> ... 24 more
> Caused by: com.fasterxml.jackson.databind.exc.UnrecognizedPropertyException:
> Unrecognized field "ephemeralStorage" (class
> org.apache.flink.kubernetes.operator.api.spec.Resource), not marked as
> ignorable (2 known properties: "cpu", "memory"])
> at [Source: UNKNOWN; byte offset: #UNKNOWN] (through reference chain:
> org.apache.flink.kubernetes.operator.api.spec.FlinkDeploymentSpec["jobManager"]->org.apache.flink.kubernetes.operator.api.spec.JobManagerSpec["resource"]->org.apache.flink.kubernetes.operator.api.spec.Resource["ephemeralStorage"])
> at
> com.fasterxml.jackson.databind.exc.UnrecognizedPropertyException.from(UnrecognizedPropertyException.java:61)
> at
> com.fasterxml.jackson.databind.DeserializationContext.handleUnknownProperty(DeserializationContext.java:1132)
> at
> com.fasterxml.jackson.databind.deser.std.StdDeserializer.handleUnknownProperty(StdDeserializer.java:2202)
> at
> com.fasterxml.jackson.databind.deser.BeanDeserializerBase.handleUnknownProperty(BeanDeserializerBase.java:1705)
> at
> com.fasterxml.jackson.databind.deser.BeanDeserializerBase.handleUnknownVanilla(BeanDeserializerBase.java:1683)
> at
> com.fasterxml.jackson.databind.deser.BeanDeserializer.vanillaDeserialize(BeanDeserializer.java:320)
> at
> com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:177)
> at
> com.fasterxml.jackson.databind.deser.impl.MethodProperty.deserializeAndSet(MethodProperty.java:129)
> at
> com.fasterxml.jackson.databind.deser.BeanDeserializer.vanillaDeserialize(BeanDeserializer.java:314)
> at
> com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:177)
> at
> com.fasterxml.jackson.databind.deser.impl.MethodProperty.deserializeAndSet(MethodProperty.java:129)
> at
> com.fasterxml.jackson.databind.deser.BeanDeserializer.vanillaDeserialize(BeanDeserializer.java:314)
> at
> com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:177)
> at
> com.fasterxml.jackson.databind.deser.DefaultDeserializationContext.readRootValue(DefaultDeserializationContext.java:323)
> at
> com.fasterxml.jackson.databind.ObjectMapper._readValue(ObjectMapper.java:4706)
> at
> com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:2879)
> at
> com.fasterxml.jackson.databind.ObjectMapper.treeToValue(ObjectMapper.java:3343)
> at
> org.apache.flink.kubernetes.operator.api.utils.SpecUtils.deserializeSpecWithMeta(SpecUtils.java:60)
> ... 34 more
> {noformat}
> Looks like some FlinkDeployments were created by 1.6 version and later 1.4
> was able to handle those.
> But as per
> [docs|https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-main/docs/operations/compatibility/],
> the FlinkDeployment created by 1.6 should have been backward compatible with
> 1.4.
> {quote}Starting from v1beta1 (operator version 1.0.0) we aim to provide
> backward compatibility for the already deployed Flink custom resources
> (FlinkDeployment, FlinkSessionJob).
> {quote}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
