[ 
https://issues.apache.org/jira/browse/CAMEL-27195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Claus Ibsen resolved CAMEL-27195.
---------------------------------
    Resolution: Won't Fix

Not changed in Camel: the change streams consumer keeps the token repository as 
a bean, as for the regular component (and as camel-kafka does with 
offsetRepository).

The Kamelet {{mongodb-changes-stream-source}} can pass it through as an 
optional parameter instead, for example {{changeStreamTokenRepository: 
'{{?tokenRepository}}'}}. A user who wants to resume after a restart declares a 
{{StateRepository}} bean, such as a {{FileStateRepository}}, and sets the 
parameter to {{#myRepository}}. As the value starts with the placeholder, the 
parameter is dropped when it is not set.

_Claude Code on behalf of Claus Ibsen_

> camel-mongodb - change streams consumer: resume after restart without a bean
> ----------------------------------------------------------------------------
>
>                 Key: CAMEL-27195
>                 URL: https://issues.apache.org/jira/browse/CAMEL-27195
>             Project: Camel
>          Issue Type: Improvement
>          Components: camel-mongodb
>            Reporter: Claus Ibsen
>            Priority: Minor
>
> h3. Background
> A {{changeStreams}} consumer resumes after a restart only when 
> {{changeStreamTokenRepository}} is set. It takes a {{StateRepository<String, 
> String>}} object, such as a {{FileStateRepository}}. After each exchange, the 
> resume token is stored under {{<routeId>/<collection>}}, and the next start 
> resumes after it. Without a repository, the changes made while the route was 
> stopped are never consumed.
> The other option, {{changeStreamToken}}, is a fixed token: it is used on 
> every start and wins over the repository, so each restart consumes the same 
> changes again. It is not a way to keep the place.
> h3. The problem
> The repository can only be given as a bean. That works in a Java or YAML 
> route, e.g.:
> {code}
> mongodb:mongo?...&consumerType=changeStreams&changeStreamTokenRepository=#class:org.apache.camel.support.processor.state.FileStateRepository#fileStateRepository('/var/data/tokens.dat')
> {code}
> but it cannot be offered as a plain, optional option where only endpoint 
> parameters can be set. The Kamelet {{mongodb-changes-stream-source}} cannot 
> offer resume after restart because of this: an optional placeholder drops a 
> parameter only when the value starts with the placeholder, so the 
> {{#class:...('{{?file}}')}} form is never dropped, and a Kamelet that always 
> creates the bean forces every user to have a file.
> h3. Suggested improvement
> Add a simple built-in way to keep the tokens, set from the endpoint URI, for 
> example one of:
> * {{changeStreamTokenFile=<path>}}: the consumer creates and owns a 
> {{FileStateRepository}} on that file (like {{offsetStorageFileName}} in 
> camel-debezium);
> * {{persistentChangeStreamToken=true}} with an optional 
> {{changeStreamTokenDb}} / {{changeStreamTokenCollection}}: store the token in 
> a MongoDB collection, like {{persistentTailTracking}} does for the tailable 
> consumer.
> {{changeStreamTokenRepository}}, when set, would still win.



--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

Reply via email to