RockteMQ-AI commented on issue #546: URL: https://github.com/apache/rocketmq-connect/issues/546#issuecomment-5324831444
**Issue Evaluation** Category: `bug` | Status: **Confirmed** The concern is valid — if the connector puts consumed data into a `BlockingQueue` and the connector restarts before the data is flushed to the destination, data in the queue will be lost. **Root Cause:** The `RocketMQSourceTask` uses an in-memory `BlockingQueue` as a buffer between the consumer and the connector framework. On restart, unflushed records in this queue are lost, and the committed offset may not reflect the actual consumed position. **Suggested Fix:** Implement one of: 1. Pre-commit offset tracking — record the offset before putting into the queue, and replay from that offset on restart (accepting possible duplicates) 2. Synchronous flush — ensure the queue is drained before shutdown 3. Persistent buffer — use a WAL or similar mechanism This is a data-loss risk in at-least-once delivery scenarios. --- *Automated evaluation by RockteMQ-AI* -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
