[
https://issues.apache.org/jira/browse/HTTPCORE-530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16535687#comment-16535687
]
Oleg Kalnichevski commented on HTTPCORE-530:
--------------------------------------------
[~lethal] Michael, classes that represent HTTP messages are not thread safe by
design. They are simple value objects and are not meant to be used
concurrently. Making {{HeaderGroup}} use {{CopyOnWriteArrayList}} internally
would be conceptually wrong.
This problem has never been reported by someone else and I have never seen it
in my testing of HttpClient. For now I am leaning toward suspecting a bug in
your application code before suspecting a defect in the framework.
Were you able to reproduce the problem with the code you have posted? Could you
please please post the _exact_ exception stack trace?
Oleg
> NullPointerException in HttpCore >= 4.4.5
> -----------------------------------------
>
> Key: HTTPCORE-530
> URL: https://issues.apache.org/jira/browse/HTTPCORE-530
> Project: HttpComponents HttpCore
> Issue Type: Bug
> Components: HttpCore
> Affects Versions: 4.4.9
> Reporter: Michael Leith
> Priority: Major
> Labels: bugfix
>
> h2. Details
> We're getting a NullPointerException exception in versions >= 4.4.5. The
> exception occurs ~20 times per 10m requests in DefaultConnectionReuseStrategy
> when calling request.headerIterator. The change that introduced it was
> swapping to checking the request for keepAlive instead of just the response
> [1] in version 4.4.5. From looking at the source in github, this change is
> still in version 5.0, so the NullPointerException may be in versions > 4.4.9,
> but that's the latest version I have tested on so far.
> From testing the source of the exception is multiple threads accessing the
> HeaderGroup, with one thread calling setHeader & the other calling
> iterator(name). The HeaderGroup is backed by a thread-unsafe ArrayList, which
> is the source of the exception. I have a fix for this, is there somewhere I
> can submit a pull request?
> The fix is to swap to a thread safe CopyOnWriteArrayList. This will have a
> performance impact, but it should be minor since the list is small (16
> indices) and headers don't appear to be modified frequently.
> [1]
> [https://github.com/apache/httpcomponents-core/blob/ff381e315a165047e6c2db9b1e749b47cad31405/httpcore5/src/main/java/org/apache/hc/core5/http/impl/DefaultConnectionReuseStrategy.java#L84]
> h2. Client details
> Here's a simplified version of our client. From comparing it to the examples
> provided in the docs there isn't anything that appears to be thread-unsafe
> aside from the shared HttpClientContext, which I've confirmed is not the root
> cause by making it thread local.
> // single threaded
> // Isn't necessarily thread safe, I've tried moving it into the "post"
> method to see if it is the cause, but the exceptions persist.
> private final HttpClientContext context = HttpClientContext.adapt(new
> BasicHttpContext());
> final RegistryBuilder<ConnectionSocketFactory> registryBuilder =
> RegistryBuilder.<ConnectionSocketFactory>create().register("https",
> sslSocketFactory.get());.
> final PoolingHttpClientConnectionManager connectionManager = new
> PoolingHttpClientConnectionManager(registryBuilder.build())
> connectionManager.setMaxTotal(connectionPoolSize);
> connectionManager.setDefaultMaxPerRoute(maxConnectionsPerRoute);
> connectionManager.setValidateAfterInactivity(duration);
> final CloseableHttpClient client = HttpClients.custom()
> .setConnectionManager(connectionManager)
> .setKeepAliveStrategy((response, context) -> keepAliveDuration.toMillis())
> .setUserAgent(userAgent)
> .build();
> // removing this does not fix the NullPointerException
> executorService.scheduleAtFixedRate(
> () ->
> connectionManager.closeIdleConnections(idleConnectionExpiry.toMillis(),
> TimeUnit.MILLISECONDS),
> Duration.ofSeconds(30).toMillis(),
> Duration.ofSeconds(30).toMillis(),
> TimeUnit.MILLISECONDS);
> // Called from multiple threads
> Result post(uri, config, entity, hostName) {
> final HttpPost req = new HttpPost(uri);
> req.setConfig(config);
> req.setEntity(entity);
> req.addHeader(HttpHeaders.HOST, hostName);
> try (CloseableHttpResponse response = client.execute(req, context))
> { // handle response }
> catch(...)
> { // handle exception }
> }
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
