Shawn Chang created HUDI-5936:
---------------------------------
Summary: Potential serialization issue when FileStatus is not
serializable
Key: HUDI-5936
URL: https://issues.apache.org/jira/browse/HUDI-5936
Project: Apache Hudi
Issue Type: Bug
Reporter: Shawn Chang
Assignee: Shawn Chang
Hadoop3's FileStatus is serializable and won't have this issue. However, when
users run Hudi on older Hadoop or customized FileSystem implementation whose
FileStatus is not serializable then it's possible to run into serialization
issue.
Exception:
{code:java}
com.esotericsoftware.kryo.KryoException:
java.util.ConcurrentModificationException Caused by:
java.util.ConcurrentModificationException at
java.util.Vector$Itr.checkForComodification(Vector.java:1212 ) at
java.util.Vector$Itr.next(Vector.java:1165 ) at
com.esotericsoftware.kryo.serializers.CollectionSerializer.write(CollectionSerializer.java:99
) at
com.esotericsoftware.kryo.serializers.CollectionSerializer.write(CollectionSerializer.java:40
) at com.esotericsoftware.kryo.Kryo.writeObject(Kryo.java:575 ) at
com.esotericsoftware.kryo.serializers.ObjectField.write(ObjectField.java:79 )
{code}
The LOC that causes this issue:
https://github.com/apache/hudi/blob/master/hudi-common/src/main/java/org/apache/hudi/metadata/FileSystemBackedTableMetadata.java#L109
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
