[
https://issues.apache.org/jira/browse/CAMEL-27195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Claus Ibsen resolved CAMEL-27195.
---------------------------------
Resolution: Won't Fix
Not changed in Camel: the change streams consumer keeps the token repository as
a bean, as for the regular component (and as camel-kafka does with
offsetRepository).
The Kamelet {{mongodb-changes-stream-source}} can pass it through as an
optional parameter instead, for example {{changeStreamTokenRepository:
'{{?tokenRepository}}'}}. A user who wants to resume after a restart declares a
{{StateRepository}} bean, such as a {{FileStateRepository}}, and sets the
parameter to {{#myRepository}}. As the value starts with the placeholder, the
parameter is dropped when it is not set.
_Claude Code on behalf of Claus Ibsen_
> camel-mongodb - change streams consumer: resume after restart without a bean
> ----------------------------------------------------------------------------
>
> Key: CAMEL-27195
> URL: https://issues.apache.org/jira/browse/CAMEL-27195
> Project: Camel
> Issue Type: Improvement
> Components: camel-mongodb
> Reporter: Claus Ibsen
> Priority: Minor
>
> h3. Background
> A {{changeStreams}} consumer resumes after a restart only when
> {{changeStreamTokenRepository}} is set. It takes a {{StateRepository<String,
> String>}} object, such as a {{FileStateRepository}}. After each exchange, the
> resume token is stored under {{<routeId>/<collection>}}, and the next start
> resumes after it. Without a repository, the changes made while the route was
> stopped are never consumed.
> The other option, {{changeStreamToken}}, is a fixed token: it is used on
> every start and wins over the repository, so each restart consumes the same
> changes again. It is not a way to keep the place.
> h3. The problem
> The repository can only be given as a bean. That works in a Java or YAML
> route, e.g.:
> {code}
> mongodb:mongo?...&consumerType=changeStreams&changeStreamTokenRepository=#class:org.apache.camel.support.processor.state.FileStateRepository#fileStateRepository('/var/data/tokens.dat')
> {code}
> but it cannot be offered as a plain, optional option where only endpoint
> parameters can be set. The Kamelet {{mongodb-changes-stream-source}} cannot
> offer resume after restart because of this: an optional placeholder drops a
> parameter only when the value starts with the placeholder, so the
> {{#class:...('{{?file}}')}} form is never dropped, and a Kamelet that always
> creates the bean forces every user to have a file.
> h3. Suggested improvement
> Add a simple built-in way to keep the tokens, set from the endpoint URI, for
> example one of:
> * {{changeStreamTokenFile=<path>}}: the consumer creates and owns a
> {{FileStateRepository}} on that file (like {{offsetStorageFileName}} in
> camel-debezium);
> * {{persistentChangeStreamToken=true}} with an optional
> {{changeStreamTokenDb}} / {{changeStreamTokenCollection}}: store the token in
> a MongoDB collection, like {{persistentTailTracking}} does for the tailable
> consumer.
> {{changeStreamTokenRepository}}, when set, would still win.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)