papinifrancesco opened a new issue, #2640: URL: https://github.com/apache/activemq/issues/2640
Found while working on #2630 (the first case is also described in [this comment](https://github.com/apache/activemq/issues/2630#issuecomment-5959474384)). The JDBC store keeps a durable subscription's ack position as a single id (`ACTIVEMQ_ACKS.LAST_ACKED_ID`). When the broker expires a message for an offline durable subscription, it acks it like a consumer would (`Topic.messageExpired()`, `STANDARD_ACK_TYPE`), so `LAST_ACKED_ID` is set to that message's id, wherever it is relative to the subscription's real position. KahaDB tracks acks per message and has neither of these problems. **1. Pending messages are lost** (the position moves forward): 1. offline durable subscription; 2. send A with no TTL, then B with a 500 ms TTL; 3. B expires and the expiry task acks it: `LAST_ACKED_ID` moves to B's id; 4. restart the broker and reconnect the subscription: A is never delivered. **2. Consumed messages are delivered again** (the position moves back). The expiry browse reads the topic from its first row, including rows the subscription has already acked, which stay in the table until the cleanup deletes them. 1. the durable subscription consumes A (1 s TTL) and then B (no TTL), and goes offline; 2. A expires and the expiry task acks it again: `LAST_ACKED_ID` moves back to A's id, and the topic's `Expired` statistic counts the consumed message on every run; 3. restart the broker and reconnect the subscription: B is delivered a second time. Both cases also happen through a topic browse (e.g. over JMX) instead of the expiry task. **Tests.** Both cases, through the expiry task and through a browse, are H2 tests in `JDBCDurableSubExpirationOrderTest` on [this branch](https://github.com/papinifrancesco/activemq/tree/jdbc-topic-recover-expired-full). On current `main` these four tests fail; on that branch they pass. **Related: expiry through the browse barely progresses on JDBC.** The cleanup deletes the acked rows of non-prioritized messages only on every 10th pass (`priorityIterator++ % 10`, priority 0), and until then each expiry run re-expires the same first page. Expiry therefore advances one `maxExpirePageSize` page per 10 cleanup periods (50 minutes by default). In a 20-minute H2 run (50,000 expired messages, page 400, expiry every 5 s, cleanup every 6 s), `main` expired 6,993 messages, and its `Expired` statistic counted 91,880. **Possible fixes.** - Expire only messages after the subscription's `LAST_ACKED_ID`, stopping at the first one that has not expired. The branch above does this with a JDBC `recoverExpired()` (the earlier version of #2634). With it, all four tests pass, and the run above expired all 50,000 messages in about 11 minutes, each counted once. - Or send the broker's expiry acks as `EXPIRED_ACK_TYPE`, and have the JDBC store never move `LAST_ACKED_ID` backwards, nor past a pending message. This changes the ack the KahaDB path receives too. **Disclosure:** found and reproduced with an AI assistant (Claude Code, Anthropic). The results come from local H2 test runs. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected] For further information, visit: https://activemq.apache.org/contact
