buvb commented on issue #2070:
URL: https://github.com/apache/fluss/issues/2070#issuecomment-3663412917
Hi @loserwang1024,
Thanks for reaching out! I'd be happy to collaborate.
For schema version compatibility during tiered data processing, my current
implementation handles it in FlussRecordAsPaimonRow:
1. **Paimon schema wider than Fluss record** (Paimon already synced, old
Fluss records still being consumed):
- Padding NULL for missing business fields via isNullAt() returning true
2. **Fluss record wider than Paimon schema** (Fluss updated, Paimon not yet
synced):
- Using Math.min(internalRow.getFieldCount(), businessFieldCount) to
ignore extra fields
- This prevents crashes during the transition period
The key insight is that during tiering, the Fluss record and Paimon schema
may temporarily be out of sync. The implementation gracefully handles both
directions of mismatch.
You can check the details in PR #2189.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
