[
https://issues.apache.org/jira/browse/CAMEL-24525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on CAMEL-24525 started by Andrea Cosentino.
------------------------------------------------
> camel-google-firestore: the realtime change buffer is unbounded
> ---------------------------------------------------------------
>
> Key: CAMEL-24525
> URL: https://issues.apache.org/jira/browse/CAMEL-24525
> Project: Camel
> Issue Type: Improvement
> Components: camel-google
> Reporter: Andrea Cosentino
> Assignee: Andrea Cosentino
> Priority: Minor
>
> The Firestore consumer buffers the document changes reported by the realtime
> snapshot listener in an unbounded queue, and drains it on the scheduled poll:
> {code:java}
> private final Queue<Exchange> pendingExchanges = new
> ConcurrentLinkedQueue<>();
> {code}
> The listener callback runs on a Firestore client thread and is never
> throttled by the route. Whenever changes arrive faster than the poll consumes
> them the queue grows without limit, and every element is a full Exchange
> holding the document data. The queue also keeps growing while the route is
> suspended, or when the consumer is configured with a long poll delay, so a
> burst of writes on a busy collection can exhaust the heap.
> Add a {{maxPendingChanges}} consumer option that bounds the buffer:
> * the default stays unbounded, so existing routes are unaffected;
> * when a limit is configured and reached, the oldest buffered change is
> discarded - the route then keeps up with the current state of the collection
> - and a warning is logged;
> * discarded exchanges are released back to the exchange pool instead of being
> dropped on the floor, and so is anything still buffered when the consumer
> stops.
> Blocking the listener callback is deliberately not offered as a policy: that
> thread belongs to the Firestore gRPC client, and stalling it would stall the
> watch stream itself.
> This is the item left open in CAMEL-24347, whose other findings are already
> fixed.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
