This is an automated email from the ASF dual-hosted git repository. davsclaus pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/camel.git
commit 056f45cba16b30d5c7495731dbd0ab4d7020cb41 Author: Claus Ibsen <[email protected]> AuthorDate: Thu Aug 19 14:07:50 2021 +0200 Polish and cleanup documentation --- .../modules/ROOT/pages/threading-model.adoc | 298 +++++++++++---------- 1 file changed, 151 insertions(+), 147 deletions(-) diff --git a/docs/user-manual/modules/ROOT/pages/threading-model.adoc b/docs/user-manual/modules/ROOT/pages/threading-model.adoc index 27727e7..2ecad7e 100644 --- a/docs/user-manual/modules/ROOT/pages/threading-model.adoc +++ b/docs/user-manual/modules/ROOT/pages/threading-model.adoc @@ -1,44 +1,41 @@ [[ThreadingModel-ThreadingModel]] = Threading Model -*Since Camel 2.3* +The threading model in Camel is based on a pluggable reactive routing engine, +and thread pools from the JDK concurrency API. -The threading model in Camel is based on leveraging the JDK concurrency -API which provides thread pools, named `ExecutorService`. - -Camel leverages thread pools in the following places: +This page focuses on thread pools. +Camel leverages thread pools in several places such as: * several xref:{eip-vc}:eips:enterprise-integration-patterns.adoc[EIP] patterns supports using thread pools for concurrency * xref:components::seda-component.adoc[SEDA] component for asynchronous connectivity -* xref:async.adoc[Threads DSL] in the Camel route -* And some component provide thread pools by nature such as -xref:components::jms-component.adoc[JMS], xref:components::jetty-component.adoc[Jetty] +* xref:{eip-vc}:eips:threads-eip.adoc[Threads] EIP in Camel routes +* Some components use thread pools out of the box, such as +xref:components::jms-component.adoc[JMS] or xref:components::jetty-component.adoc[Jetty] -[[ThreadingModel-Threadpoolprofiles]] == Thread pool profiles -By default when a thread pool is to be created then its based on the -default thread pool profile which is: +By default, when a thread pool is to be created by Camel, then the pool configuration +is based upon a profile, the _default thread pool profile_ -[source,xml] --------------------------------------------------------------------------------------------------------- - <threadPoolProfile id="defaultThreadPoolProfile" defaultProfile="true" - poolSize="10" maxPoolSize="20" maxQueueSize="1000" allowCoreThreadTimeOut="true" - rejectedPolicy="CallerRuns"/> --------------------------------------------------------------------------------------------------------- +The default profile is pre-configured out of the box with the following settings: -NOTE: From Camel 3.4 onwards the default thread pool profile has `allowCoreThreadTimeOut=true` while -it was `allowCoreThreadTimeOut=false` in older Camel versions. +[width="100%",cols="25%,25%,50%",options="header",] +|=== +| Option | Default | Description +| *poolSize* | `10` | Sets the default core pool size (threads to keep minimum in pool) +| *keepAliveTime* | `60` | Sets the default keep alive time (in seconds) for inactive threads +| *maxPoolSize* | `20` | Sets the default maximum pool size +| *maxQueueSize* | `1000` | Sets the default maximum number of tasks in the work queue. Use -1 for an unbounded queue. +| *allowCoreThreadTimeOut* | `true` | Sets default whether to allow core threads to timeout +| *rejectedPolicy* | `CallerRuns` | Sets the default handler for tasks which cannot be executed by the thread pool. Has four options: +`Abort, CallerRuns, Discard, DiscardOldest` which corresponds to the same four options provided out of the box in the JDK. +|=== What that means is that for example when you use xref:{eip-vc}:eips:multicast-eip.adoc[Multicast] with `parallelProcessing=true` enabled, -then it would create a thread pool based on the profile above. The -`rejectedPolicy` has four options: -`Abort, CallerRuns, Discard, DiscardOldest` which corresponds to the -same four options provided out of the box in the JDK. -*Notice:* option allowCoreThreadTimeOut is a new option from *Camel -2.15* onwards. +then it would create a thread pool based on the profile above. You can define as many thread pool profiles as you like. But there must only *one* default profile. A custom thread pool profile will inherit @@ -46,151 +43,139 @@ from the default profile. Which means that any option you do not explicit define will fallback and use the option from the default profile. -You can use `-1` in maxQueueSize to indicate a unbounded queue. +=== Configuring default thread pool profile + +In Spring XML you can configure thread pool profile with `threadPoolProfile` as shown: + +[source,xml] +---- +<threadPoolProfile id="defaultThreadPoolProfile" + defaultProfile="true" + poolSize="5" + maxPoolSize="10"/> +---- + +And in Java DSL -In Java DSL you can configure the default thread pool profile from the -`ExecutorServiceStrategy`/`ExecutorServiceManager` which you access from -`CamelContext`. +[source,java] +---- +ThreadPoolProfile profile = camelContext.getExecutorServiceManager().getDefaultThreadPoolProfile(); +profile.setPoolSize(5); +profile.setMaxPoolSize(10); +---- -[[ThreadingModel-Usingthreadpoolprofiles]] -== Using thread pool profiles +And with camel-main, Spring Boot or Quarkus you can configure this in the `application.properties|yaml` file: + +[source,properties] +---- +## configure default thread pool profile +camel.threadpool.pool-size = 5 +camel.threadpool.max-pool-size = 5 +---- + + +=== Using thread pool profiles Suppose you want to use a custom thread pool profile for a Multicast EIP pattern in a Camel route you can do it using the `executorServiceRef` -attribute as shown: +attribute as shown in Spring XML: [source,xml] --------------------------------------------------------------------------- -<camelContext ...> - ... - <threadPoolProfile id="fooProfile" +<camelContext> + + <threadPoolProfile id="fooProfile" poolSize="20" maxPoolSize="50" maxQueueSize="-1"/> - ... <route> - ... <multicast strategyRef="myStrategy" executorServiceRef="fooProfile"> ... </multicast> - ... - <route> + </route> </camelContext> --------------------------------------------------------------------------- What Camel will do at runtime is to lookup in the -xref:registry.adoc[Registry] for a `ExecutorService` with the id = -fooProfile. If none found it will fallback and see if there is a -`ThreadPoolProfile` defined with that id. And in this example there is -and so the profile is used for creating a new `ExecutorService` which is +xref:registry.adoc[Registry] for a `ExecutorService` with the id `fooProfile`. +If none found it will fallback and see if there is a +`ThreadPoolProfile` defined with that id. In this example there is a profile, +so the profile is used as base settings for creating a new `ExecutorService` which is handed back to the xref:{eip-vc}:eips:multicast-eip.adoc[Multicast] EIP to use in the -route. - -[[ThreadingModel-Createcustomthreadpool]] -== Create custom thread pool - -You can also use the <threadPool/> tag in Spring XML to create a -specific thread pool (eg `ExecutorService`). Notice that any options you -do not explicit define, will have Camel to use the default thread pool -profile as fallback. For example if you omit setting the `maxQueueSize` -then Camel will fallback and use the value from the default thread pool -profiles, which by default is 1000. - -[[ThreadingModel-Management]] -== Management - -All the thread pools that Camel creates are managed and thus you can see -them in JConsole under the `threadpools` category. - -[[ThreadingModel-ExecutorServiceStrategy]] -== ExecutorServiceStrategy - -*Since Camel 2.3 to 2.8.x* + - Camel provides a pluggable strategy to hook in your own thread pool -provider, for example from a WorkManager in a J2EE server etc. + - See the `org.apache.camel.spi.ExecutorServiceStrategy` interface which -you should implement and hook into the WorkManager. - -See -xref:advanced-configuration-of-camelcontext-using-spring.adoc[Advanced -configuration of CamelContext using Spring] for how to configure it. - -You can configure it on the `CamelContext` from Java DSL using the -getter/setter. +Camel route. -[[ThreadingModel-ExecutorServiceManager]] -== ExecutorServiceManager +In Java DSL you can use `ThreadPoolProfileBuilder` to create a profile and then register the profile: -*Since Camel 2.9* +[source,java] +---- +ThreadPoolProfileBuilder builder = new ThreadPoolProfileBuilder("fooProfile"); +builder.poolSize(20).maxPoolSize(50).maxQueueSize(-1); -In camel 2.9.0 the `ExecutorServiceManager` replaces the -`ExecutorServiceStrategy`. It is renamed to manager as is not only -provides a strategy for thread pool creation but also keeps track of -thread pools and thread pool profiles. +camelContext.getExecutorServiceManager().registerThreadPoolProfile(builder.build()); +---- -To hook in custom thread pool providers (e.g. for J2EE servers) a -`ThreadPoolFactory` interface can be implemented. The implementation can -be set in the `ExecutorServiceManager`. The Factory interface is much -simpler then the former `ExecutorServiceStrategy` and makes the job of -integrators much easier. +== Creating custom thread pools -See -xref:advanced-configuration-of-camelcontext-using-spring.adoc[Advanced -configuration of CamelContext using Spring] for how to configure it. +You can also use the `<threadPool>` tag in Spring XML to create a +specific thread pool (`ExecutorService`). Notice that any options you +do not explicit define, will have Camel to use the default thread pool +profile as fallback. For example if you omit setting the `maxQueueSize` +then Camel will fallback and use the value from the default thread pool +profiles, which by default is `1000`. -[[ThreadingModel-Customizingthreadnames]] == Customizing thread names -On the `ExecutorServiceStrategy`/`ExecutorServiceManager` you can +On the `ExecutorServiceManager` you can configure the thread name pattern using the `setThreadNamePattern` method, which defines the thread names used when a thread pool creates a thread. -The default pattern is for: - -* *Camel 2.9.x or older:* -`Camel ($\{camelId}) thread #$\{counter} - $\{name}` -* *Camel 2.10 onwards:* `Camel (#camelId#) thread ##counter# - #name#` +The default pattern is: -Notice we renamed the tokens from Camel 2.10 onwards to not clash with -tokens by the xref:using-propertyplaceholder.adoc[Property Placeholder]. +[source,text] +---- +Camel (#camelId#) thread ##counter# - #name# +---- In the pattern you can use the following placeholders -* $\{camelId} *Camel 2.6:* is the xref:camelcontext.adoc[CamelContext] -name -* $\{counter} is a unique incrementing counter. -* $\{name} is the regular thread name. -* $\{longName} is the long thread name which can includes endpoint -parameters etc. +* `\#camelId#` - The xref:camelcontext.adoc[CamelContext] name +* `\#counter#` An unique incrementing counter +* `\#name#` - The thread name +* `\#longName#` - The long thread name which can includes endpoint parameters etc. -Notice the pattern name has changed from Camel 2.10 onwards, use #name# -instead. - -In Camel 2.11 onwards its easier to set the thread name pattern on the -CamelContext using the threadNamePattern attribute in the XML files as -shown below: +In Spring XML the pattern can be set with `threadNamePattern` attribute as shown: [source,xml] --------------------------------------------------------------------------------------------------------------- - <camelContext xmlns="http://camel.apache.org/schema/spring"; threadNamePattern="Riding the thread #counter#"> - <route> - <from uri="seda:start"/> - <to uri="log:result"/> - <to uri="mock:result"/> - </route> - </camelContext> --------------------------------------------------------------------------------------------------------------- +---- +<camelContext threadNamePattern="Riding the thread #counter#"> + <route> + <from uri="seda:start"/> + <to uri="log:result"/> + <to uri="mock:result"/> + </route> +</camelContext> +---- -[[ThreadingModel-Componentdevelopers]] -== Component developers +In Java DSL you can set the pattern as follows: -If you develop your own Camel component and are in need of a thread -pool, then its advised to use the -`ExecutorServiceStrategy`/`ExecutorServiceManager` to create the thread -pool you need. +[source,java] +---- +CamelContext camel = ... +camel.getExecutorServiceManager().setThreadNamePattern("Riding the thread #counter#") +---- + +And with camel-main, Spring Boot or Quarkus you can configure this in the `application.properties|yaml` file: -[[ThreadingModel-Shutdown]] -== Shutdown +[source,properties] +---- +## camel-main or quarkus +camel.main.thread-name-pattern = Riding the thread #counter# + +## spring boot +camel.springboot.thread-name-pattern = Riding the thread #counter# +---- + +== Shutting down thread pools All thread pools created by Camel will be properly shutdown when `CamelContext` shutdowns which ensures no leaks in the pools in case you @@ -200,13 +185,13 @@ The `ExecutorServiceManager` has APIs for shutting down thread pools graceful and aggressively. Its encourage to use this API for creating and shutting down thread pools. -From *Camel 2.11* onwards Camel the graceful -`shutdownGraceful(executorService)` method from `ExecutorServiceManager` +The graceful `shutdownGraceful(executorService)` method from `ExecutorServiceManager` will shutdown graceful at first, until a timeout value is hit. After that it shutdown aggressively, again using the timeout value to wait for the operation to complete. This means you can wait at most 2 x timeout -for shutting down the thread pool. + - The timeout value is by default `10000` millis. You can configure a +for shutting down the thread pool. + +The timeout value is by default `10000` millis. You can configure a custom value on the `ExecutorServiceManager` if needed. During shutdown Camel will log every 2nd second at INFO level progress of shutting down the thread pool. For example in case a shutdown takes a while, then @@ -218,23 +203,42 @@ thread pool is as follows: [width="100%",cols="25%,75%",options="header"] |=== |Method |Description +|shutdown |Marks the thread pool as shutdown +(like calling the `ExecutorService.shutdown()` method). +|shutdownNow |Forces the thread pool to shut down now +(like calling the `ExecutorService.shutdownNow()` method). +|shutdownGraceful |Marks the thread pool as shutdown, and graceful shutdown +the pool, by waiting for tasks to complete. A default timeout value of +10 sec is used, before shutdown becomes aggressive using `shutdownNow`, +forcing threads to shut down quicker. +|shutdownGraceful(timeout) |As shutdownGraceful but with custom timeout value +|awaitTermination |To wait graceful for the termination of a thread pool (eg +to wait for its tasks to complete). Will wait until all tasks are completed or timed out. +|=== -|shutdown |Marks the thread pool as shutdown, eg just as calling -`ExecutorService.shutdown()` method +== JMX Management -|shutdownNow |Forces the thread pool to shutdown now, eg just as calling -`ExecutorService.shutdownNow()` method +All the thread pools that Camel creates are managed and thus you can see +them in JXM under the `threadpools` tree. -|shutdownGraceful |*Camel 2.11:* Marks the thread pool as shutdown, and graceful shutdown -the pool, by waiting for tasks to complete. A default timeout value of -10 sec is used, before - shutdown becomes aggressive using `shutdownNow`, to force threads to -shutdown. +NOTE: This requires to enabled JMX by including `camel-management` JAR in the classpath. -|shutdownGraceful(timeout) |*Camel 2.11:* As above but with custom timeout value +== Component developers -|awaitTermination |*Camel 2.11:* To wait graceful for the termination of a thread pool (eg -to wait for its tasks to complete). Will wait until all tasks is -completed or a timeout value is hit. -|=== +If you develop your own Camel component and are in need of a thread +pool, then its advised to use the +`ExecutorServiceStrategy`/`ExecutorServiceManager` to create the thread +pool you need. + +=== ExecutorServiceStrategy + +Camel provides a pluggable strategy to hook in your own thread pool +provider, for example from a WorkManager in a J2EE server etc. + +See the `org.apache.camel.spi.ExecutorServiceStrategy` interface which +you should implement and hook into the WorkManager. + +=== ExecutorServiceManager +To hook in custom thread pool providers (e.g. for J2EE servers) a +`ThreadPoolFactory` interface can be implemented. The implementation can +be set in the `ExecutorServiceManager`.
