Yukang-Lian opened a new pull request, #66899:
URL: https://github.com/apache/doris/pull/66899
### What problem does this PR solve?
Issue Number: N/A
Related PR: #65810
Problem Summary:
Row Binlog supports fixed-column partial updates, but did not materialize
correct rows for flexible partial updates (`UPDATE_FLEXIBLE_COLUMNS`). Flexible
updates use a per-row skip bitmap and can merge or filter rows before writing,
so Row Binlog must reconstruct complete rows and keep its operation/LSN
sidecars aligned with the surviving rows.
This PR:
- Reuses the existing flexible partial-update aggregation and historical-row
read plans to materialize full AFTER rows.
- Preserves full BEFORE rows when historical values are enabled.
- Produces the existing APPEND / UPDATE / DELETE Row Binlog operations
without changing the on-disk format or query protocol.
- Keeps Row Binlog LSNs aligned when duplicate keys are merged or
delete-then-insert rows are filtered.
- Uses a read-only MOW key probe so Row Binlog history lookup does not
modify the base tablet delete bitmap.
- Deep-copies the input batch before materialization so Row Binlog does not
mutate the Base Rowset writer's input.
The implementation is shared by local and Cloud storage. It does not change
FE syntax, Table Stream metadata, Stream Offset handling, or protobuf
definitions.
### Release note
Support Row Binlog for flexible partial updates.
### Check List (For Author)
- Test
- [x] Regression test
- [x] Unit Test
- [x] Manual test
- ASAN BE build and an isolated ASAN Cloud cluster.
- `row_binlog_p0`: 13/13 passed.
- `unique_with_mow_p0/flexible`: 25/25 passed.
- `unique_with_mow_p0/partial_update`: 47/49 passed; two existing S3
Broker Load cases remained PENDING because this local environment has no
regression S3 credentials. Neither entered the changed Row Binlog path.
- Behavior changed:
- [ ] No.
- [x] Yes. Row Binlog now supports `UPDATE_FLEXIBLE_COLUMNS` and emits
complete rows using the existing Row Binlog operation semantics.
- Does this need documentation?
- [x] No. This adds support for an existing Row Binlog and
partial-update combination without introducing syntax, properties, or metadata.
- [ ] Yes.
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
