Shanthoosh Venkataraman created SAMZA-1640:
----------------------------------------------

             Summary: JobModel Json deserialization error in standalone:
                 Key: SAMZA-1640
                 URL: https://issues.apache.org/jira/browse/SAMZA-1640
             Project: Samza
          Issue Type: Bug
            Reporter: Shanthoosh Venkataraman
            Assignee: Shanthoosh Venkataraman


*Problem:*

JobModel deserialization in ZkJobCoordinator throws the following exception in 
some scenarios.
{code:java}
org.codehaus.jackson.map.exc.UnrecognizedPropertyException: Unrecognized field 
"task-partition-mappings" (Class org.apache.samza.job.model.JobModel), not 
marked as ignorable at [Source: java.io.StringReader@1f7030a6; line: 1, column: 
315] (through reference chain: 
org.apache.samza.job.model.JobModel["task-partition-mappings"]) at 
org.codehaus.jackson.map.exc.UnrecognizedPropertyException.from(UnrecognizedPropertyException.java:53)
 at 
org.codehaus.jackson.map.deser.StdDeserializationContext.unknownFieldException(StdDeserializationContext.java:267)
 at 
org.codehaus.jackson.map.deser.std.StdDeserializer.reportUnknownProperty(StdDeserializer.java:673)
 at 
org.codehaus.jackson.map.deser.std.StdDeserializer.handleUnknownProperty(StdDeserializer.java:659)
 at 
org.codehaus.jackson.map.deser.BeanDeserializer.handleUnknownProperty(BeanDeserializer.java:1365)
 at 
org.codehaus.jackson.map.deser.BeanDeserializer.deserialize(BeanDeserializer.java:652)
 at 
org.codehaus.jackson.map.deser.BeanDeserializer._deserializeUsingPropertyBased(BeanDeserializer.java:923)
 at 
org.codehaus.jackson.map.deser.BeanDeserializer.deserializeFromObjectUsingNonDefault(BeanDeserializer.java:739)
 at 
org.codehaus.jackson.map.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:683)
 at 
org.codehaus.jackson.map.deser.BeanDeserializer.deserialize(BeanDeserializer.java:580)
 at 
org.codehaus.jackson.map.ObjectMapper._readMapAndClose(ObjectMapper.java:2732) 
at org.codehaus.jackson.map.ObjectMapper.readValue(ObjectMapper.java:1863) at 
org.apache.samza.serializers.model.TestSamzaObjectMapper.testContainerModelCompatible(TestSamzaObjectMapper.java:82)
 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) 
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 at java.lang.reflect.Method.invoke(Method.java:498) at 
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
 at 
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
 at 
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
 at 
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
 at 
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) 
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325) at 
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
 at 
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
 at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290) at 
org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71) at 
org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288) at 
org.junit.runners.ParentRunner.access$000(ParentRunner.java:58) at 
org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268) at 
org.junit.runners.ParentRunner.run(ParentRunner.java:363) at 
org.junit.runner.JUnitCore.run(JUnitCore.java:137) at 
com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
 at 
com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:51)
 at 
com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:237)
 at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)
{code}

*Reason:*

We started storing the field 'task-partition-mappings' in JobModel after 
SAMZA-1555.

Above exception occurs when we try to deserialize the new JobModel format with 
the old JobModel deserilization code(vice-versa).

Potentially can occur during rolling bounce upgrade/canary scenarios in 
standalone.

*Fix:* 
A. Exclude task-partition-mappings field in deserialization/serialization path. 
`task-partition-mappings` is derived out of the containers field in JobModel. 
It doesn't add value to store it in JobModel.

B. Fix the test testContainerModelCompatible in TestSamzaObjectMapper which was 
supposed to validate this scenario.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Reply via email to