Joey Frazee created NIFI-3978:
---------------------------------
Summary: Site-to-Site HTTP client tests fails on machines with too
many cores
Key: NIFI-3978
URL: https://issues.apache.org/jira/browse/NIFI-3978
Project: Apache NiFi
Issue Type: Bug
Reporter: Joey Frazee
Assignee: Joey Frazee
Priority: Minor
The S2S http client tests will fail on machines with many cores because the
[thread pool
limits|https://github.com/apache/nifi/blob/master/nifi-commons/nifi-site-to-site-client/src/test/java/org/apache/nifi/remote/client/http/TestHttpClient.java#L435]
are too low (see error below).
Looks like the reason for this is that Jetty will use acceptors + selectors +
request=1 threads, but calculates both the number of acceptors and selectors
based on the number of hyper threads available:
* The default number of acceptor tasks is the minimum of 1 and the number of
available CPUs divided by 8
* The default number of selectors is equal to half of the number of processors
available to the JVM
So suppose you have a 12 core 2 HT machine. Then you'll use 12 * 2 / 8 + 12 * 2
/ 2 + 1 = 16 threads and all is well with the current limit of 20. But if you
have a 16 core machine, you'll use 16 * 2 / 8 + 16 * 2 / 2 + 1 = 21 and go over
the limit.
We should bump this up to 50 which should accommodate builds on 2 x 16 core.
{code}
Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0 sec <<<
FAILURE! - in org.apache.nifi.remote.client.http.TestHttpClient
org.apache.nifi.remote.client.http.TestHttpClient Time elapsed: 0 sec <<<
ERROR!
java.lang.IllegalStateException: Insufficient threads: max=20 <
needed(acceptors=4 + selectors=16 + request=1)
at org.eclipse.jetty.server.Server.doStart(Server.java:414)
at
org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at
org.apache.nifi.remote.client.http.TestHttpClient.setup(TestHttpClient.java:501)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
