Simon Rasmussen created CAMEL-23602:
---------------------------------------
Summary: threads EIP does not respect maxQueueSize when using
virtual threads
Key: CAMEL-23602
URL: https://issues.apache.org/jira/browse/CAMEL-23602
Project: Camel
Issue Type: Bug
Components: came-core
Affects Versions: 4.20.0
Reporter: Simon Rasmussen
We are looking into switching from platform threads to virtual threads. We have
multiple routes, consuming from SQS queues. To achieve high throughput today,
we use threads EIP like this:
{code:java}
from(someSqsQueue?&waitTimeSeconds=10&maxMessagesPerPoll=10&extendMessageVisibility=true).threads().maxQueueSize(50).bean(service);{code}
as you can see, we use the threads EIP to do the actual processing. This way,
we have a single consumer thread, using long poll, to fetch up to 10 messages
at a time, and offload them to a pool of threads to get processed. The max
queue size is set relatively low, because if it gets full, then
extendMessageVisibility kicks in, and we don’t want to do this across a too
large set of messages. It also has the benefit that the polling thread will
pick up processing, effectively backing off from pulling in more load.
Now, if we switch to using virtual threads, we observe that the max queue size
is ignored, and we keep pulling in messages unlimited – isn’t this a bug in
threads EIP when combined with virtual threads?
I can use a custom executor service with threads to overcome this, but is that
really the best way?
Our motivation for switching to virtual threads is due to having too many
platform threads when the number of consumers using this pattern increases
within an application.
see also:
https://camel.zulipchat.com/#narrow/channel/257298-camel/topic/Unbounded.20threads.20EIP.20when.20using.20virtual.20threads.3F/with/596960264
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
