JingsongLi commented on PR #6575:
URL: https://github.com/apache/paimon/pull/6575#issuecomment-4637974651
I don't think this `readObject` gives the intended default for old
serialized `HadoopFileIO` instances.
The code sets `atomicRenameEnabled = true` before calling
`in.defaultReadObject()`. For streams written by older versions, the new
primitive boolean field is absent, and Java deserialization applies the default
value for that missing field. That means `defaultReadObject()` can overwrite
the earlier `true` with `false`, so old serialized objects would deserialize
with atomic rename disabled.
Could we detect the missing field explicitly, e.g. via
`readFields().defaulted("atomicRenameEnabled")`, or use a nullable/boxed
representation plus defaulting after deserialization? Otherwise upgrade
compatibility appears to regress the old HDFS atomic overwrite behavior.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
