[
https://issues.apache.org/jira/browse/NIFI-3978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16024930#comment-16024930
]
ASF GitHub Bot commented on NIFI-3978:
--------------------------------------
GitHub user jfrazee opened a pull request:
https://github.com/apache/nifi/pull/1858
NIFI-3978 Increase threadpool size for S2S HTTP tests
Increases threadpool limits for S2S HTTP tests so they'll pass on many core
machines, as discussed in
[NIFI-3978](https://issues.apache.org/jira/browse/NIFI-3978).
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/jfrazee/nifi NIFI-3978
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/nifi/pull/1858.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1858
----
commit bea2ddaf10f88832eadefbbd036b25bfabe47c72
Author: Joey Frazee <[email protected]>
Date: 2017-05-25T15:32:06Z
NIFI-3978 Increase threadpool size for S2S HTTP tests
----
> Site-to-Site HTTP client tests fails on machines with too many cores
> --------------------------------------------------------------------
>
> Key: NIFI-3978
> URL: https://issues.apache.org/jira/browse/NIFI-3978
> Project: Apache NiFi
> Issue Type: Bug
> Reporter: Joey Frazee
> Assignee: Joey Frazee
> Priority: Minor
>
> The S2S http client tests will fail on machines with many cores because the
> [thread pool
> limits|https://github.com/apache/nifi/blob/master/nifi-commons/nifi-site-to-site-client/src/test/java/org/apache/nifi/remote/client/http/TestHttpClient.java#L435]
> are too low (see error below).
> Looks like the reason for this is that Jetty will use acceptors + selectors +
> request=1 threads, but calculates both the number of acceptors and selectors
> based on the number of hyper threads available:
> * The default number of acceptor tasks is the minimum of 1 and the number of
> available CPUs divided by 8
> * The default number of selectors is equal to half of the number of
> processors available to the JVM
> So suppose you have a 12 core 2 HT machine. Then you'll use 12 * 2 / 8 + 12 *
> 2 / 2 + 1 = 16 threads and all is well with the current limit of 20. But if
> you have a 16 core machine, you'll use 16 * 2 / 8 + 16 * 2 / 2 + 1 = 21 and
> go over the limit.
> We should bump this up to 50 which should accommodate builds on 2 x 16 core.
> {code}
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0 sec <<<
> FAILURE! - in org.apache.nifi.remote.client.http.TestHttpClient
> org.apache.nifi.remote.client.http.TestHttpClient Time elapsed: 0 sec <<<
> ERROR!
> java.lang.IllegalStateException: Insufficient threads: max=20 <
> needed(acceptors=4 + selectors=16 + request=1)
> at org.eclipse.jetty.server.Server.doStart(Server.java:414)
> at
> org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
> at
> org.apache.nifi.remote.client.http.TestHttpClient.setup(TestHttpClient.java:501)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
