HaHaJeff commented on code in PR #269:
URL: https://github.com/apache/paimon-cpp/pull/269#discussion_r3913813989
##########
src/paimon/core/operation/commit/realtime_commit_properties.cpp:
##########
@@ -285,9 +280,9 @@ Result<std::map<std::string, std::string>>
RealtimeCommitProperties::Build(
}
auto offset_iter = merged_offsets.find(partition_bucket);
int64_t previous_end_offset = offset_iter == merged_offsets.end() ? 0
: offset_iter->second;
- if (offset_range.begin != previous_end_offset) {
+ if (offset_range.begin < previous_end_offset) {
Review Comment:
Could this skip an earlier prepared segment?\n\nFor example, if P0 contains
offset 10 and P1 contains offset 30, committing only P1 advances the watermark
to 31. P0 may then be reclaimed and later treated as already committed,
although its files were never published.\n\nI think row offsets can remain
sparse, but commit progress should still be contiguous.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
