luoyuxia commented on code in PR #3500:
URL: https://github.com/apache/fluss/pull/3500#discussion_r3502930456
##########
fluss-lake/fluss-lake-paimon/src/main/java/org/apache/fluss/lake/paimon/source/PaimonSplitSerializer.java:
##########
@@ -55,11 +65,20 @@ public PaimonSplit deserialize(int version, byte[]
serialized) throws IOExceptio
DataSplit dataSplit;
try {
dataSplit = InstantiationUtil.deserializeObject(in,
getClass().getClassLoader());
-
+ DataInputStream dis = new DataInputStream(in);
+ boolean isBucketUnAware = dis.readBoolean();
if (version == VERSION_1) {
- DataInputStream dis = new DataInputStream(in);
- boolean isBucketUnAware = dis.readBoolean();
- return new PaimonSplit(dataSplit, isBucketUnAware);
+ // VERSION_1 payloads do not contain partition values. Keep
them readable, but
+ // don't try to reconstruct partition names from DataSplit
because non-string
+ // partition columns require type information that is not
present in the payload.
+ return new PaimonSplit(dataSplit, isBucketUnAware,
Collections.emptyList());
Review Comment:
The PR returns new PaimonSplit(dataSplit, isBucketUnAware,
Collections.emptyList()) for old VERSION_1 payloads. However, old payloads
still contain DataSplit.partition(). For the previously supported string
partition case, the old PaimonSplit.partition() recovered the value via
BinaryRow.getString(i). After this change, deserializing the same v1 bytes
changes ["A"] into [].
This breaks the expected compatibility contract of
SimpleVersionedSerializer. It is reasonable that non-string v1 payloads cannot
be reconstructed type-safely, but old string partition splits should not lose
their existing behavior.
Suggested fix: in the VERSION_1 branch, preserve the old logic by reading
dataSplit.partition() with getString(i) when fields exist. Add a v1
compatibility test that serializes old-format bytes and verifies a string
partition still round-trips.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
