Caideyipi opened a new pull request, #18441:
URL: https://github.com/apache/iotdb/pull/18441
## Description
### Region-level downgrade ordering
- Add the hidden common option
`pipe_realtime_region_level_downgrading_enabled`, disabled by default.
- When enabled, serialize downgraded TsFiles at DataRegion level. A later
TsFile cannot pass until the previous TsFile and all tablets generated from it
have been committed.
- Leave downgrade only after all downgraded TsFiles complete, and stop
forwarding realtime tablets through `USING_BOTH` while region-level downgrade
is active. This preserves ordering for equal-timestamp data with
`serialize-by-region` and `realtime-first=false`.
### Bounded latest-TsFile tail
- Retain a bounded realtime tail only for the latest TsFile so flushes
cannot continuously chase and prolong downgrade.
- Promote the previous tail to TsFile transfer when a newer epoch arrives,
while preserving events queued before downgrade.
### TsFile lifecycle
- Track generated-tablet completion for both processor conversion paths.
- Keep the TsFile progress index until the final generated tablet commits.
- Restrict raw-tablet commit hooks to progress-reporting or TsFile-derived
events.
### Verification
- DataNode test compilation.
- Node Commons compilation with both English and Chinese locales.
- DataNode and Node Commons Spotless checks.
- `PipeRealtimeExtractTest`: 9 tests passed.
- `git diff --check`.
<hr>
This PR has:
- [x] been self-reviewed.
- [x] concurrent read and write
- [x] added comments explaining the intent of non-obvious logic.
- [x] added or updated unit tests to cover the new code paths.
<hr>
##### Key changed/added classes (or packages if there are too many classes)
in this PR
- `PipeRealtimeDataRegionHybridSource`
- `PipeTsFileInsertionEvent`
- `PipeRawTabletInsertionEvent`
- `PipeEventCollector`
- `PipeRealtimeExtractTest`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
