allthingssecurity opened a new pull request, #28464: URL: https://github.com/apache/camel/pull/28464
# Description [CAMEL-25946](https://issues.apache.org/jira/browse/CAMEL-25946) A change stream ends with an `invalidate` event when its collection is dropped or renamed, or its database dropped, and the server closes the cursor. The changeStreams consumer routes that event and takes its resume token. Its next cursor uses `resumeAfter(token)`, which MongoDB rejects for the token of an invalidate event: error 260 InvalidResumeToken, "Attempting to resume a change stream using 'resumeAfter' is not allowed from an invalidate notification." (MongoDB manual, "Resume a Change Stream"). So: - after a drop or rename, the consumer fails to open a cursor every `cursorRegenerationDelay`, each time with a WARN and a stack trace, while the route shows as Started. A collection created again under the same name is never consumed; - with `changeStreamTokenRepository` or a resume strategy, the stored invalidate token makes the next start of the route fail with `FailedToStartRouteException`. Changes: - `MongoDbChangeStreamsThread.initializeCursor`: when `resumeAfter(token)` fails with InvalidResumeToken (260), the consumer opens the change stream with `startAfter(token)` (MongoDB 4.2). That starts a new change stream after the invalidate event, and the consumer logs it at INFO, once for each change stream it opens with the invalidate token (C14). That is once per route start, or per cursor regeneration until the first event of the new stream replaces the token, which is rare. - A token that is invalid for another reason fails with `startAfter` too. That failure carries the first error as a suppressed exception, and the consumer tries again after the delay, as before. - The token of a normal event is still used with `resumeAfter`. - The change stream is now built in a small `watch()` helper, because `startAfter` needs a fresh `ChangeStreamIterable`. The server allows only one resume option per change stream. - No option, generated file or upgrade guide change: nothing for users to migrate. Servers older than 4.2 have no `startAfter`, so there the consumer keeps retrying as today. - Backport: camel-4.23.x, 4.22.x, 4.18.x and 4.14.x open the change stream the same way. I read the code on those branches but did not run the tests there. The cursor regeneration loop on main changed in CAMEL-25679, so a cherry-pick should be checked against the older loop. Tests: - New `MongoDbChangeStreamsInvalidateTest`, with the mocked client of `MongoDbConsumerCursorRegenerationTest`. Its fake change stream answers like a 4.2+ server on which the collection was dropped and created again (comment on the fake): - no resume option: the insert, drop and invalidate events, then a closed cursor; - `resumeAfter` the invalidate token: `MongoCommandException` 260; - `startAfter` it: the insert into the new collection; - both options together: error 50865, "Do not specify both 'resumeAfter' and 'startAfter' in a $changeStream stage", as `document_source_change_stream.cpp` of the server (r4.2.0 to r8.0.0). The consumer never sets both. - Fail on main: - `consumerGoesOnAfterTheCollectionIsDropped`: `Expected: <4> but was: <3>`. With the fix it also checks that the change stream after that resumes with `resumeAfter` the new event's token (F3). - `consumerStartsFromAStoredTokenOfAnInvalidateEvent`: `FailedToStartRouteException ... error 260 (InvalidResumeToken)`. With the fix the repository holds the token of the new event (awaited: the consumer stores it after the exchange is done). - camel-mongodb unit tests: 24 run, 0 failures. The formatter and import-sort plugins ran. I built offline with `-Dasm-version=9.11`. The ITs were not run locally; CI runs them. # Target - [x] I checked that the commit is targeting the correct branch (Camel 4 uses the `main` branch) # Tracking - [x] If this is a large change, bug fix, or code improvement, I checked there is a [JIRA issue](https://issues.apache.org/jira/browse/CAMEL) filed for the change (usually before you start working on it): [CAMEL-25946](https://issues.apache.org/jira/browse/CAMEL-25946). # Apache Camel coding standards and style - [x] I checked that each commit in the pull request has a meaningful subject line and body. - [ ] I have run `mvn clean install -DskipTests` locally from root folder and I have committed all auto-generated changes. (I built camel-mongodb and ran its tests, including the formatter and import-sort plugins. No generated file changes. I did not run the full root build.) # AI-assisted contributions - [x] If this PR includes AI-generated code, commits have proper co-authorship attribution (e.g., `Co-authored-by` trailers) and the PR description identifies the AI tool used. This PR was prepared with Claude Code (Claude Opus 5.5). The commit carries a `Co-Authored-By` trailer. _Claude Code on behalf of allthingssecurity_ 🤖 Generated with [Claude Code](https://claude.com/claude-code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
