Hello. I wanted to post a quick follow-up of my efforts to tune my app to
get better performance with a recipient list. Previously, I was only able
to get 1,000,000 messages through my example in five minutes. After some
playing around, I got all million messages completed in about 45 seconds!
I am routing through a Kafka broker, and my dynamic router EIP component
that is now utilizing a recipient list. Here is what I did:
1. Create a custom pool the way the documentation suggests:
@Bean
public ExecutorService customPool() {
DefaultThreadPoolFactory threadPoolFactory = new DefaultThreadPoolFactory();
threadPoolFactory.setCamelContext(camelContext);
DefaultExecutorServiceManager executorServiceManager = new
DefaultExecutorServiceManager(camelContext);
executorServiceManager.setThreadPoolFactory(threadPoolFactory);
return executorServiceManager.newCachedThreadPool(this,
"DynamicRouterSpringBootThreadFactory");
}
2. Utilizing better parallelism to generate and send messages:
Flux.range(1, limit)
.flatMap(n -> Mono.just(n)
.map(Object::toString)
.subscribeOn(Schedulers.boundedElastic())
.doOnNext(strN ->
producerTemplate.sendBodyAndHeaders("direct:command", strN,
Map.of("command", PROCESS_NUMBER_COMMAND,
"number", strN))))
.subscribe();
I am much more satisfied with these results. If any of you have other
suggestions that I should try, please let me know. I will see how it
affects the results, and I will reply to the thread with whatever I find.
When this is all complete, I will submit my changes, which include a spring
boot examples project that demonstrates the dynamic router eip component
utilized in a more real-life scenario where messages are routed to multiple
(separate) JVMs, and stood up in a docker-compose stack where it can all be
controlled and monitored via swagger endpoints.
Thanks,
Steve
On Sat, Dec 16, 2023 at 12:21 PM Steve973 <steve...@gmail.com> wrote:
> Hello. Are there examples somewhere that show how to make a recipient
> list as fast as possible? I have already tried to provide different types
> of thread pools from the DefaultExecutorServiceManager. If I am sending a
> lot of small messages, what options should I use to make the recipient list
> route as fast as possible?
>
>
>
