[
https://issues.apache.org/jira/browse/FLINK-21896?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17307129#comment-17307129
]
godfrey he edited comment on FLINK-21896 at 3/23/21, 2:39 PM:
--------------------------------------------------------------
The reason is the serialization results for an object without
{{serialVersionUID}} are different between jdk8 and jdk11. The following code
can reproduce the difference:
{code:java}
StructuredType structuredType =
StructuredType.newBuilder(MapView.class)
.attributes(
Collections.singletonList(
new StructuredType.StructuredAttribute(
"map",
new MapType(new IntType(), new
BigIntType(false)))))
.build();
System.out.println(EncodingUtils.encodeObjectToString(structuredType));
{code}
the result of jdk8 is
{code:java}
rO0ABXNyADNvcmcuYXBhY2hlLmZsaW5rLnRhYmxlLnR5cGVzLmxvZ2ljYWwuU3RydWN0dXJlZFR5cGUt-WvI46_69wIABVoADmlzSW5zdGFudGlhYmxlTAAKYXR0cmlidXRlc3QAEExqYXZhL3V0aWwvTGlzdDtMAAtjb21
......
{code}
the result of jdk11 is
{code:java}
rO0ABXNyADNvcmcuYXBhY2hlLmZsaW5rLnRhYmxlLnR5cGVzLmxvZ2ljYWwuU3RydWN0dXJlZFR5cGWW-4qkzgeWwgIABVoADmlzSW5zdGFudGlhYmxlTAAKYXR0cmlidXRlc3QAEExqYXZhL3V0aWwvTGlzdDtMAAtjb21......
{code}
Because StructuredType has no serialVersionUID.
Why there are only two cases will occur such errors? because the two cases have
distinct aggregate with local aggregate, which will produce RawType (RawType
will encode the inner serializer object (which contains StructuredType) into
string as the value of {{getSerializerString}}, see
RawType#getSerializerString), the full structure of RawType in failed cases is:
{code:java}
StructuredType structuredType =
StructuredType.newBuilder(MapView.class)
.attributes(
Collections.singletonList(
new StructuredType.StructuredAttribute(
"map",
new MapType(new IntType(), new
BigIntType(false)))))
.build();
new RawType<>(
MapView.class,
ExternalSerializer.of(
new FieldsDataType(
structuredType,
Collections.singletonList(
new KeyValueDataType(
new MapType(
new IntType(),
new
BigIntType(false)),
new AtomicDataType(new
IntType()),
new AtomicDataType(
new
BigIntType(false)))))))
{code}
The solution is we add serialVersionUID for StructuredType.
And I suggest we should add serialVersionUID for all logical types.
was (Author: godfreyhe):
The reason is the serialization results for an object without
{{serialVersionUID}} are different between jdk8 and jdk11. The following code
can reproduce the difference:
{code:java}
StructuredType structuredType =
StructuredType.newBuilder(MapView.class)
.attributes(
Collections.singletonList(
new StructuredType.StructuredAttribute(
"map",
new MapType(new IntType(), new
BigIntType(false)))))
.build();
System.out.println(EncodingUtils.encodeObjectToString(structuredType));
{code}
the result of jdk8 is
{code:java}
rO0ABXNyADNvcmcuYXBhY2hlLmZsaW5rLnRhYmxlLnR5cGVzLmxvZ2ljYWwuU3RydWN0dXJlZFR5cGUt-WvI46_69wIABVoADmlzSW5zdGFudGlhYmxlTAAKYXR0cmlidXRlc3QAEExqYXZhL3V0aWwvTGlzdDtMAAtjb21
......
{code}
the result of jdk11 is
{code:java}
rO0ABXNyADNvcmcuYXBhY2hlLmZsaW5rLnRhYmxlLnR5cGVzLmxvZ2ljYWwuU3RydWN0dXJlZFR5cGWW-4qkzgeWwgIABVoADmlzSW5zdGFudGlhYmxlTAAKYXR0cmlidXRlc3QAEExqYXZhL3V0aWwvTGlzdDtMAAtjb21......
{code}
Because StructuredType has no serialVersionUID.
Why there are only two cases will occur such errors? because the two cases have
distinct aggregate with local aggregate, which will produce RawType (RawType
will encode the inner serializer object into string as the value of
{{getSerializerString}}), the full structure of RawType in failed cases is:
{code:java}
StructuredType structuredType =
StructuredType.newBuilder(MapView.class)
.attributes(
Collections.singletonList(
new StructuredType.StructuredAttribute(
"map",
new MapType(new IntType(), new
BigIntType(false)))))
.build();
new RawType<>(
MapView.class,
ExternalSerializer.of(
new FieldsDataType(
structuredType,
Collections.singletonList(
new KeyValueDataType(
new MapType(
new IntType(),
new
BigIntType(false)),
new AtomicDataType(new
IntType()),
new AtomicDataType(
new
BigIntType(false)))))))
{code}
The solution is we add serialVersionUID for StructuredType.
And I suggest we should add serialVersionUID for all logical types.
> GroupAggregateJsonPlanTest.testDistinctAggCalls fail
> ----------------------------------------------------
>
> Key: FLINK-21896
> URL: https://issues.apache.org/jira/browse/FLINK-21896
> Project: Flink
> Issue Type: Bug
> Components: Table SQL / Planner
> Affects Versions: 1.13.0
> Reporter: Guowei Ma
> Assignee: godfrey he
> Priority: Major
> Labels: test-stability
>
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=15083&view=logs&j=f66801b3-5d8b-58b4-03aa-cc67e0663d23&t=1abe556e-1530-599d-b2c7-b8c00d549e53&l=6364
> {code:java}
> at
> org.apache.flink.table.planner.plan.nodes.exec.stream.GroupAggregateJsonPlanTest.testDistinctAggCalls(GroupAggregateJsonPlanTest.java:148)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.base/java.lang.reflect.Method.invoke(Method.java:566)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at
> org.junit.rules.ExpectedException$ExpectedExceptionStatement.evaluate(ExpectedException.java:239)
> at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
> at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:48)
> at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> at org.junit.runners.Suite.runChild(Suite.java:128)
> at org.junit.runners.Suite.runChild(Suite.java:27)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
