[
https://issues.apache.org/jira/browse/CXF-8951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Ködderitzsch updated CXF-8951:
-----------------------------------
Description:
Concurrent usage of Webclients causes massively more threads to be created in
cxf-3.6.x/4.x than before.
Supposedly this results from introducing a new conduit implementation based on
the "new" java http client compared to HttpUrlConnection before.
It seems that a new http client is created per requesting thread - and each
http client in turn creates an own pool of selector/worker threads.
To demonstate I've attached a test project with several methods to test
different scenarios.
Each tests launches a simple JAX-RS Server, configures/uses WebClients in
different configuration and submits 1000 requests via up to 100 parrallel
requestor threads.
The number of live threads in the JVM instance if printed after each request.
Baseline is the number of threads used by the server instance + the 100
requesting threads.
singleClientInstanceWithNewConduit -> reuses a threadsafe client instance,
thread count rises to 700+ live threads
singleClientInstanceWithOldConduit -> reuses a threadsafe client instance,
stays at about 200 threads
clientPerRequestWithOldConduit -> creates a client instance per request (no
closing!), keeps below 200 threads
clientPerRequestWithNewConduit -> creates a client instance per request (no
closing!), creates a runaway thread leak (!)
clientPerRequestWithNewConduit is additionally interesting because current
documentation is not clear about whether client instances
are required to be resource managed or not.
It appears that closing clients was not required up until cxf-3.6.0 and
documentation actively encourages creating new "lightweight" client instances
on the fly (see Thread safety in
[https://cxf.apache.org/docs/jax-rs-client-api.html]).
However, cxf-3.6+ implementation (and comments in CXF-8885) seem to suggest
that clients are in fact *not lightweight* (anymore?) but need to be closed,
when no longer used.
But then in turn WebClient/Client does not even implement
Closeable/Autoclosable. So, it's all quite muddy - and there doesn't seem to be
a real consensus on the idiomatic usage of Webclients.
It would be very nice if you could bring some clarity on this as well.
was:
Concurrent usage of Webclients causes massively more threads to be created in
cxf-3.6.x/4.x than before.
Supposedly this results from introducing a new conduit implementation based on
the "new" java http client compared to HttpUrlConnection before.
It seems that a new http client is created per requesting thread - and each
http client in turn creates an own pool of selector/worker threads.
To demonstate I've attached a test project with several methods to test
different scenarios.
Each tests launches a simple JAX-RS Server, configures/uses WebClients in
different configuration and submits 1000 requests via up to 100 parrallel
requestor threads.
The number of live threads in the JVM instance if printed after each request.
Baseline is the number of threads used by the server instance + the 100
requesting threads.
singleClientInstanceWithNewConduit -> reuses a threadsafe client instance,
thread count rises to 700+ live threads
singleClientInstanceWithOldConduit -> reuses a threadsafe client instance,
stays add about 200 threads
clientPerRequestWithOldConduit -> creates a client instance per request (no
closing!), keeps below 200 threads
clientPerRequestWithNewConduit -> creates a client instance per request (no
closing!), creates a runaway thread leak (!)
clientPerRequestWithNewConduit is additionally interesting because current
documentation is not clear about whether client instances
are required to be resource managed or not.
It appears that closing clients was not required up until cxf-3.6.0 and
documentation actively encourages creating new "lightweight" client instances
on the fly (see Thread safety in
[https://cxf.apache.org/docs/jax-rs-client-api.html]).
However, cxf-3.6+ implementation (and comments in CXF-8885) seem to suggest
that clients are in fact *not lightweight* (anymore?) but need to be closed,
when no longer used.
But then in turn WebClient/Client does not even implement
Closeable/Autoclosable. So, it's all quite muddy - and there doesn't seem to be
a real consensus on the idiomatic usage of Webclients.
It would be very nice if you could bring some clarity on this as well.
> Concurrent WebClient usage cause massive thread overhead
> --------------------------------------------------------
>
> Key: CXF-8951
> URL: https://issues.apache.org/jira/browse/CXF-8951
> Project: CXF
> Issue Type: Bug
> Components: Core, JAX-RS
> Affects Versions: 3.6.2, 4.0.3
> Reporter: Lars Ködderitzsch
> Priority: Major
> Attachments: cxf-connector-threads.zip
>
>
> Concurrent usage of Webclients causes massively more threads to be created in
> cxf-3.6.x/4.x than before.
> Supposedly this results from introducing a new conduit implementation based
> on the "new" java http client compared to HttpUrlConnection before.
> It seems that a new http client is created per requesting thread - and each
> http client in turn creates an own pool of selector/worker threads.
> To demonstate I've attached a test project with several methods to test
> different scenarios.
> Each tests launches a simple JAX-RS Server, configures/uses WebClients in
> different configuration and submits 1000 requests via up to 100 parrallel
> requestor threads.
> The number of live threads in the JVM instance if printed after each request.
> Baseline is the number of threads used by the server instance + the 100
> requesting threads.
>
> singleClientInstanceWithNewConduit -> reuses a threadsafe client instance,
> thread count rises to 700+ live threads
> singleClientInstanceWithOldConduit -> reuses a threadsafe client instance,
> stays at about 200 threads
> clientPerRequestWithOldConduit -> creates a client instance per request
> (no closing!), keeps below 200 threads
> clientPerRequestWithNewConduit -> creates a client instance per request
> (no closing!), creates a runaway thread leak (!)
> clientPerRequestWithNewConduit is additionally interesting because current
> documentation is not clear about whether client instances
> are required to be resource managed or not.
> It appears that closing clients was not required up until cxf-3.6.0 and
> documentation actively encourages creating new "lightweight" client instances
> on the fly (see Thread safety in
> [https://cxf.apache.org/docs/jax-rs-client-api.html]).
> However, cxf-3.6+ implementation (and comments in CXF-8885) seem to suggest
> that clients are in fact *not lightweight* (anymore?) but need to be closed,
> when no longer used.
> But then in turn WebClient/Client does not even implement
> Closeable/Autoclosable. So, it's all quite muddy - and there doesn't seem to
> be a real consensus on the idiomatic usage of Webclients.
> It would be very nice if you could bring some clarity on this as well.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
