[
https://issues.apache.org/jira/browse/FLINK-14875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jing Zhang updated FLINK-14875:
-------------------------------
Description:
If source contains data which has immutable collection, the exception will be
thrown out:
{code:java}
Caused by: com.esotericsoftware.kryo.KryoException:
java.lang.UnsupportedOperationException
Serialization trace:
logTags_ (com.aliyun.openservices.log.common.Logs$LogGroup)
mLogGroup (com.aliyun.openservices.log.common.LogGroupData)
at
com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:125)
at
com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:528)
at com.esotericsoftware.kryo.Kryo.readObjectOrNull(Kryo.java:730)
at
com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:113)
at
com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:528)
at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:761)
at
org.apache.flink.api.java.typeutils.runtime.kryo.KryoSerializer.deserialize(KryoSerializer.java:315)
at
org.apache.flink.api.common.typeutils.base.ListSerializer.deserialize(ListSerializer.java:138)
at
org.apache.flink.api.common.typeutils.base.ListSerializer.deserialize(ListSerializer.java:47)
at
org.apache.flink.util.InstantiationUtil.deserializeFromByteArray(InstantiationUtil.java:463)
at
org.apache.flink.table.dataformat.BinaryRow.getGeneric(BinaryRow.java:440)
at BaseRowSerializerProjection$52.apply(Unknown Source)
at BaseRowSerializerProjection$52.apply(Unknown Source)
at
org.apache.flink.table.typeutils.BaseRowSerializer.baseRowToBinary(BaseRowSerializer.java:250)
at
org.apache.flink.table.typeutils.BaseRowSerializer.serializeToPages(BaseRowSerializer.java:285)
at
org.apache.flink.table.typeutils.BaseRowSerializer.serializeToPages(BaseRowSerializer.java:55)
at
org.apache.flink.table.runtime.sort.BinaryInMemorySortBuffer.write(BinaryInMemorySortBuffer.java:190)
at
org.apache.flink.table.runtime.sort.BinaryExternalSorter.write(BinaryExternalSorter.java:540)
... 10 more
Caused by: java.lang.UnsupportedOperationException
at
java.util.Collections$UnmodifiableCollection.add(Collections.java:1055)
at
com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:109)
at
com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:22)
at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:679)
at
com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:106)
... 27 more
{code}
the exception could also appears in a simple ITCase in attachments.
I find similar problems in [How to set Unmodifiable collection serializer of
Kryo in Spark
code|https://stackoverflow.com/questions/46818293/how-to-set-unmodifiable-collection-serializer-of-kryo-in-spark-code],
is there any way to set unmodifiable collection serializer of Kryo in at
present?
was:
If source contains data which has immutable collection, the exception will be
thrown out:
{code:java}
Caused by: com.esotericsoftware.kryo.KryoException:
java.lang.UnsupportedOperationException
Serialization trace:
logTags_ (com.aliyun.openservices.log.common.Logs$LogGroup)
mLogGroup (com.aliyun.openservices.log.common.LogGroupData)
at
com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:125)
at
com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:528)
at com.esotericsoftware.kryo.Kryo.readObjectOrNull(Kryo.java:730)
at
com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:113)
at
com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:528)
at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:761)
at
org.apache.flink.api.java.typeutils.runtime.kryo.KryoSerializer.deserialize(KryoSerializer.java:315)
at
org.apache.flink.api.common.typeutils.base.ListSerializer.deserialize(ListSerializer.java:138)
at
org.apache.flink.api.common.typeutils.base.ListSerializer.deserialize(ListSerializer.java:47)
at
org.apache.flink.util.InstantiationUtil.deserializeFromByteArray(InstantiationUtil.java:463)
at
org.apache.flink.table.dataformat.BinaryRow.getGeneric(BinaryRow.java:440)
at BaseRowSerializerProjection$52.apply(Unknown Source)
at BaseRowSerializerProjection$52.apply(Unknown Source)
at
org.apache.flink.table.typeutils.BaseRowSerializer.baseRowToBinary(BaseRowSerializer.java:250)
at
org.apache.flink.table.typeutils.BaseRowSerializer.serializeToPages(BaseRowSerializer.java:285)
at
org.apache.flink.table.typeutils.BaseRowSerializer.serializeToPages(BaseRowSerializer.java:55)
at
org.apache.flink.table.runtime.sort.BinaryInMemorySortBuffer.write(BinaryInMemorySortBuffer.java:190)
at
org.apache.flink.table.runtime.sort.BinaryExternalSorter.write(BinaryExternalSorter.java:540)
... 10 more
Caused by: java.lang.UnsupportedOperationException
at
java.util.Collections$UnmodifiableCollection.add(Collections.java:1055)
at
com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:109)
at
com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:22)
at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:679)
at
com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:106)
... 27 more
{code}
the exception could also appears in a simple ITCase in attachments.
I find similar problems in [How to set Unmodifiable collection serializer of
Kryo in Spark
code|https://stackoverflow.com/questions/46818293/how-to-set-unmodifiable-collection-serializer-of-kryo-in-spark-code],
however there is no way to set unmodifiable collection serializer of Kryo in
Flink at present. Maybe we could take it into consideration.
> Blink planner should pass the right ExecutionConfig to the creation of
> serializer
> ---------------------------------------------------------------------------------
>
> Key: FLINK-14875
> URL: https://issues.apache.org/jira/browse/FLINK-14875
> Project: Flink
> Issue Type: Bug
> Components: API / Type Serialization System
> Reporter: Jing Zhang
> Priority: Major
> Attachments: ImmutableCollectionKryoDeserializerITCase.java
>
>
> If source contains data which has immutable collection, the exception will be
> thrown out:
> {code:java}
> Caused by: com.esotericsoftware.kryo.KryoException:
> java.lang.UnsupportedOperationException
> Serialization trace:
> logTags_ (com.aliyun.openservices.log.common.Logs$LogGroup)
> mLogGroup (com.aliyun.openservices.log.common.LogGroupData)
> at
> com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:125)
> at
> com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:528)
> at com.esotericsoftware.kryo.Kryo.readObjectOrNull(Kryo.java:730)
> at
> com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:113)
> at
> com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:528)
> at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:761)
> at
> org.apache.flink.api.java.typeutils.runtime.kryo.KryoSerializer.deserialize(KryoSerializer.java:315)
> at
> org.apache.flink.api.common.typeutils.base.ListSerializer.deserialize(ListSerializer.java:138)
> at
> org.apache.flink.api.common.typeutils.base.ListSerializer.deserialize(ListSerializer.java:47)
> at
> org.apache.flink.util.InstantiationUtil.deserializeFromByteArray(InstantiationUtil.java:463)
> at
> org.apache.flink.table.dataformat.BinaryRow.getGeneric(BinaryRow.java:440)
> at BaseRowSerializerProjection$52.apply(Unknown Source)
> at BaseRowSerializerProjection$52.apply(Unknown Source)
> at
> org.apache.flink.table.typeutils.BaseRowSerializer.baseRowToBinary(BaseRowSerializer.java:250)
> at
> org.apache.flink.table.typeutils.BaseRowSerializer.serializeToPages(BaseRowSerializer.java:285)
> at
> org.apache.flink.table.typeutils.BaseRowSerializer.serializeToPages(BaseRowSerializer.java:55)
> at
> org.apache.flink.table.runtime.sort.BinaryInMemorySortBuffer.write(BinaryInMemorySortBuffer.java:190)
> at
> org.apache.flink.table.runtime.sort.BinaryExternalSorter.write(BinaryExternalSorter.java:540)
> ... 10 more
> Caused by: java.lang.UnsupportedOperationException
> at
> java.util.Collections$UnmodifiableCollection.add(Collections.java:1055)
> at
> com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:109)
> at
> com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:22)
> at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:679)
> at
> com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:106)
> ... 27 more
> {code}
> the exception could also appears in a simple ITCase in attachments.
> I find similar problems in [How to set Unmodifiable collection serializer of
> Kryo in Spark
> code|https://stackoverflow.com/questions/46818293/how-to-set-unmodifiable-collection-serializer-of-kryo-in-spark-code],
> is there any way to set unmodifiable collection serializer of Kryo in at
> present?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
