jw-itq commented on PR #9854:
URL: https://github.com/apache/seatunnel/pull/9854#issuecomment-3290174896
> Could you share the core logic? I found all new method is private scope.
@Hisoka-X thanks. I'm not sure if this issue needs to be resolved, but I
think it would be helpful for someone like me who needs to upgrade to ensure
correct backward compatibility during checkpoint restore.
I think we can also close this PR, is there a unified way to handle it in
the future.
### Core Logic for Backward Compatibility in SeaTunnel TableSchema
Deserialization
### Background
In SeaTunnel version 2.3.12, the
[TableSchema](file:///Users/shiwanming/job/package-swm/seatunnel/seatunnel-api/src/main/java/org/apache/seatunnel/api/table/catalog/TableSchema.java#L46-L231)
class was refactored: the
[columns](file:///Users/shiwanming/job/package-swm/seatunnel/seatunnel-api/src/main/java/org/apache/seatunnel/api/table/catalog/AbstractSchema.java#L34-L34)
field was moved from the
[TableSchema](file:///Users/shiwanming/job/package-swm/seatunnel/seatunnel-api/src/main/java/org/apache/seatunnel/api/table/catalog/TableSchema.java#L46-L231)
class into its parent class
[AbstractSchema](file:///Users/shiwanming/job/package-swm/seatunnel/seatunnel-api/src/main/java/org/apache/seatunnel/api/table/catalog/AbstractSchema.java#L31-L85).
This change introduced a backward compatibility issue — data serialized with
pre-2.3.12 versions cannot be correctly deserialized in the new version.
### Core Logic Steps
#### Step 1: Custom Deserialization Method
Implement a private
[readObject](file:///Users/shiwanming/job/package-swm/seatunnel/seatunnel-api/src/main/java/org/apache/seatunnel/api/table/catalog/TableSchema.java#L108-L108)
method to customize the deserialization process — a standard extension point
provided by Java’s serialization mechanism:
```java
private void readObject(ObjectInputStream stream) throws IOException,
ClassNotFoundException
```
#### Step 2: Read Serialized Fields
Use `ObjectInputStream.GetField` to read all fields from the serialized
data. This approach supports reading data serialized by any version:
```java
ObjectInputStream.GetField fields = stream.readFields();
```
#### Step 3: Version Detection
Attempt to read the
[columns](file:///Users/shiwanming/job/package-swm/seatunnel/seatunnel-api/src/main/java/org/apache/seatunnel/api/table/catalog/AbstractSchema.java#L34-L34)
field — which existed only in the old version — to determine the data version:
```java
List<Column> columns = null;
try {
columns = (List<Column>) fields.get("columns", null);
} catch (IllegalArgumentException e) {
columns = null; // Field does not exist in newer versions
}
```
- If
[columns](file:///Users/shiwanming/job/package-swm/seatunnel/seatunnel-api/src/main/java/org/apache/seatunnel/api/table/catalog/AbstractSchema.java#L34-L34)
is not null → data is from an old version.
- If
[columns](file:///Users/shiwanming/job/package-swm/seatunnel/seatunnel-api/src/main/java/org/apache/seatunnel/api/table/catalog/AbstractSchema.java#L34-L34)
is null → data is from a new version.
#### Step 4: Compatibility Handling
If old-version data is detected, invoke the
[initializeParentFields](file:///Users/shiwanming/job/package-swm/seatunnel/seatunnel-api/src/main/java/org/apache/seatunnel/api/table/catalog/TableSchema.java#L124-L139)
method to populate the
[columns](file:///Users/shiwanming/job/package-swm/seatunnel/seatunnel-api/src/main/java/org/apache/seatunnel/api/table/catalog/AbstractSchema.java#L34-L34)
data into the parent class
[AbstractSchema](file:///Users/shiwanming/job/package-swm/seatunnel/seatunnel-api/src/main/java/org/apache/seatunnel/api/table/catalog/AbstractSchema.java#L31-L85):
```java
if (columns != null) {
initializeParentFields(columns);
}
```
#### Step 5: Initialize Parent Class Fields
Use reflection to assign the legacy
[columns](file:///Users/shiwanming/job/package-swm/seatunnel/seatunnel-api/src/main/java/org/apache/seatunnel/api/table/catalog/AbstractSchema.java#L34-L34)
data into the corresponding fields of the parent class
[AbstractSchema](file:///Users/shiwanming/job/package-swm/seatunnel/seatunnel-api/src/main/java/org/apache/seatunnel/api/table/catalog/AbstractSchema.java#L31-L85):
```java
private void initializeParentFields(List<Column> columns) {
// Set the 'columns' field in the parent class
java.lang.reflect.Field columnsField =
AbstractSchema.class.getDeclaredField("columns");
columnsField.setAccessible(true);
setFinalField(columnsField, this, columns);
// Set the 'columnNames' field in the parent class
List<String> columnNames =
columns.stream().map(Column::getName).collect(Collectors.toList());
java.lang.reflect.Field columnNamesField =
AbstractSchema.class.getDeclaredField("columnNames");
columnNamesField.setAccessible(true);
setFinalField(columnNamesField, this, columnNames);
}
```
#### Step 6: Handle Final Fields
Since the
[columns](file:///Users/shiwanming/job/package-swm/seatunnel/seatunnel-api/src/main/java/org/apache/seatunnel/api/table/catalog/AbstractSchema.java#L34-L34)
and
[columnNames](file:///Users/shiwanming/job/package-swm/seatunnel/seatunnel-api/src/main/java/org/apache/seatunnel/api/table/catalog/AbstractSchema.java#L37-L37)
fields in
[AbstractSchema](file:///Users/shiwanming/job/package-swm/seatunnel/seatunnel-api/src/main/java/org/apache/seatunnel/api/table/catalog/AbstractSchema.java#L31-L85)
are declared as `final`, special handling is required:
```java
private void setFinalField(java.lang.reflect.Field field, Object target,
Object value)
```
This method supports multiple JDK versions:
1. **JDK 8–11**: Remove the `FINAL` modifier via reflection.
2. **JDK 17+**: Use the `Unsafe` API.
3. **Fallback**: Direct field access (if other methods fail).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
