mfyuce opened a new pull request, #3525:
URL: https://github.com/apache/iggy/pull/3525
## Problem
When a consumer group's stored offset falls below the topic's earliest
available offset — for example after the server purges old segments
under a retention policy — `poll_messages` returns
`IggyError::InvalidOffset`. The consumer retried at the same invalid
offset indefinitely, causing sink connectors to loop on errors and stop
delivering messages.
This manifests in production as:
```
ERROR iggy::clients::consumer: Failed to poll messages: Invalid offset: 0
ERROR iggy_connectors::sink: Failed to receive message for sink connector ...
# repeats forever; no messages are delivered
```
The root cause: new consumer groups receive stored offset 0 from the
server, but if the topic's retention policy has already purged messages
at offset 0 the very first poll returns `InvalidOffset(0)`, and the
consumer has no recovery path.
## Fix
Add `fallback_to_first: Arc<AtomicBool>` to `IggyConsumer`.
- On `IggyError::InvalidOffset`, set the flag and emit a `warn!` with
stream/topic context.
- On the next `create_poll_messages_future` call, use
`PollingStrategy::first()` instead of the configured strategy so the
consumer seeks to the earliest available message.
- After the first successful non-empty poll, clear the flag so normal
next-offset tracking resumes.
No public API changes. The existing `polling_strategy` field and all
builder methods are unchanged.
## Test
`cargo test -p iggy` passes (120 + 3 tests, 0 failed).
The fix was validated in a production `yucemonitoring` vcluster where
four iggy-connectors QuickWit sink connectors were stuck in the
`InvalidOffset(0)` loop against an `otel/metrics` topic with ~20 M
messages and retention. After deleting the stale consumer groups and
restarting (the current workaround), and confirming this fix would have
recovered automatically, ingestion resumed at ~2 700 docs/s.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
