numinnex opened a new issue, #4726: URL: https://github.com/apache/iggy/issues/4726
### Bug description On restart the server logs `recovered offset ahead of partition data; clamping` for consumer offsets that turn out to be correct. The warning comes from `seed_recovered_offset` (`core/partitions/src/partition_storage.rs`), which also stores `current_offset` into the in-memory consumer offset. `load_partition` (`core/server/src/partition_helpers.rs`) runs these steps in this order: 1. Recovers segments up to the WAL checkpoint. 2. Calls `restore_partition_offsets`. 3. Calls `configure_consumer_offsets(current_offset)`. 4. Replays the WAL tail in `open_partition_persistence`. So the check runs before messages that exist only in the WAL tail are loaded. The replay then re-applies the tail's `StoreConsumerOffset` entries and restores the right value. The offset file is never rewritten, so nothing is actually rewound, but the warning reads as if consumers were moved back. An external test saw the warning with `current_offset=0` on every restart. Offsets were correct afterwards, and a restarted consumer resumed without duplicates. ### Open question We could not explain why `current_offset` came out as exactly 0. The full log line, including its `offset_space_ceiling` field, would settle it. ### Affected area / component Server: partition recovery ### Versions master @ 3c265dd89 ### Proposed fix 1. Seed consumer offsets unclamped before the replay. The replay writes into those maps, so they must exist by then. 2. After the replay and the re-anchor, run a separate clamp pass that warns only when an offset is still ahead of the data. 3. Reword the message so it does not read like a rewind. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
