Here is a commit on my personal fork of the camel github repo that adds a setting to disable that cache. I tested it with the reproducer and it appears to be working well:
https://github.com/apache/camel/compare/main...artnaseef:camel:asn/disable-error-handler-cache-setting Art On Mon, Jun 3, 2024 at 12:46 PM Arthur Naseef <artnas...@apache.org> wrote: > Out Of Memory (OOM) occurs when using the Recipient List with a large > number of dynamic URLs. For example: > > .recipientList(simple("http:// > {{downstream-server}}/employee/${header.emplId}")) > > with a large number of values for ${header.emplId} leads to the OOM. > > REPRODUCER: > ============= > https://github.com/artnaseef/camel-recipient-list-oom-reproducer > > - See the README.md for instructions to reproduce and detect the problem > > DETAILS > ======= > The MulticastProcessor, which RecipientListProcessor extends, has the > following "unlimited" cache: > > private final ConcurrentMap<Processor, Processor> errorHandlers = new > ConcurrentHashMap<>(); > > Entries are added to this map for every unique processor created - every > unique URL generates a unique processor. The entries themselves are > wrapped processor instances for error handling IIUC (to support the custom > error handling used by multicast and recipient-list). Entries are only > removed from this map on shutdown. Ironically, there is an LRUCache for > the processors themselves, with a default maximum size of 1000, so the > wrapped processors may get recreated even though the error handler remains > in the map indefinitely. > > IMPACT VERSIONS: > ================ > Appears to impact versions >= 3.10.0 > > COMMIT: 0d9227ff16fb00e047fdd087740c87cce01bb545 > ======= > It appears this commit introduced the use of the errorHandlers "unlimited" > cache for recipient lists. > > FOLLOW-UP > ========== > I have ideas and questions for implemeting a fix: > - IDEA 1: We can use an LRUCache for this data structure as well. > - Does it make more sense to remove the entries from errorHandlers > when the related Processor entry is removed from it's LRUCache? > - IDEA 2: setting on recipient list to disable the errorHandler cache > (for dyamic urls with little chance of duplicates, this could be the best) > > Art >
