[
https://issues.apache.org/jira/browse/NIFI-1932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dave Torok updated NIFI-1932:
-----------------------------
Description:
We have run into HTTP connection pool race conditions when using the
GetDynamoDB processors under continuous load. This issue is that the pool is
set to the max concurrent threads, and it is likely that the HTTP connection is
not returned to the pool in time for the next processor thread to grab it.
AbstractAWSProcessor.java (lines 175-177)
protected ClientConfiguration createConfiguration(final ProcessContext
context) {
final ClientConfiguration config = new ClientConfiguration();
config.setMaxConnections(context.getMaxConcurrentTasks());
We have locally fixed it by adding extra connections, but don't know whether 1,
2, 5, 10 is sufficient to avoid the issue completely:
FIX:
config.setMaxConnections(context.getMaxConcurrentTasks() + SOMEINT);
was:
We have run into HTTP connection pool race conditions when using the
GetDynamoDB processors under continuous load. This issue is that the pool is
set to the max processors, and it is likely that the HTTP connection is not
returned to the pool in time for the next processor thread to grab it.
AbstractAWSProcessor.java (lines 175-177)
protected ClientConfiguration createConfiguration(final ProcessContext
context) {
final ClientConfiguration config = new ClientConfiguration();
config.setMaxConnections(context.getMaxConcurrentTasks());
We have locally fixed it by adding extra connections, but don't know whether 1,
2, 5, 10 is sufficient to avoid the issue completely:
FIX:
config.setMaxConnections(context.getMaxConcurrentTasks() + SOMEINT);
> AWS Processors Connection Pool issues
> -------------------------------------
>
> Key: NIFI-1932
> URL: https://issues.apache.org/jira/browse/NIFI-1932
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
> Affects Versions: 1.0.0, 0.7.0, 0.6.1
> Reporter: Dave Torok
> Priority: Minor
>
> We have run into HTTP connection pool race conditions when using the
> GetDynamoDB processors under continuous load. This issue is that the pool is
> set to the max concurrent threads, and it is likely that the HTTP connection
> is not returned to the pool in time for the next processor thread to grab it.
> AbstractAWSProcessor.java (lines 175-177)
> protected ClientConfiguration createConfiguration(final ProcessContext
> context) {
> final ClientConfiguration config = new ClientConfiguration();
> config.setMaxConnections(context.getMaxConcurrentTasks());
> We have locally fixed it by adding extra connections, but don't know whether
> 1, 2, 5, 10 is sufficient to avoid the issue completely:
> FIX:
> config.setMaxConnections(context.getMaxConcurrentTasks() + SOMEINT);
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
