[
https://issues.apache.org/jira/browse/HUDI-5936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HUDI-5936:
---------------------------------
Labels: pull-request-available (was: )
> Potential serialization issue when FileStatus is not serializable
> -----------------------------------------------------------------
>
> Key: HUDI-5936
> URL: https://issues.apache.org/jira/browse/HUDI-5936
> Project: Apache Hudi
> Issue Type: Bug
> Reporter: Shawn Chang
> Assignee: Shawn Chang
> Priority: Major
> Labels: pull-request-available
>
> Hadoop3's FileStatus is serializable and won't have this issue. However, when
> users run Hudi on older Hadoop or customized FileSystem implementation whose
> FileStatus is not serializable then it's possible to run into serialization
> issue.
>
> Exception:
> {code:java}
> com.esotericsoftware.kryo.KryoException:
> java.util.ConcurrentModificationException Caused by:
> java.util.ConcurrentModificationException at
> java.util.Vector$Itr.checkForComodification(Vector.java:1212 ) at
> java.util.Vector$Itr.next(Vector.java:1165 ) at
> com.esotericsoftware.kryo.serializers.CollectionSerializer.write(CollectionSerializer.java:99
> ) at
> com.esotericsoftware.kryo.serializers.CollectionSerializer.write(CollectionSerializer.java:40
> ) at com.esotericsoftware.kryo.Kryo.writeObject(Kryo.java:575 ) at
> com.esotericsoftware.kryo.serializers.ObjectField.write(ObjectField.java:79 )
> {code}
>
> The LOC that causes this issue:
> https://github.com/apache/hudi/blob/master/hudi-common/src/main/java/org/apache/hudi/metadata/FileSystemBackedTableMetadata.java#L109
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
