[
https://issues.apache.org/jira/browse/ASTERIXDB-2945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17397170#comment-17397170
]
Ingo Müller commented on ASTERIXDB-2945:
----------------------------------------
I have tried patch set 3 and it seems to solve my problem fully: in the 96-vCPU
instance with 48 I/O devices, I see about 4800% CPU utilization during query
execution, and the queries run only marginally longer than if run against local
files.
> Ensure S3 connection pool is large enough
> -----------------------------------------
>
> Key: ASTERIXDB-2945
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-2945
> Project: Apache AsterixDB
> Issue Type: Bug
> Components: EXT - External data
> Affects Versions: 0.9.7
> Reporter: Wail Y. Alkowaileet
> Assignee: Wail Y. Alkowaileet
> Priority: Major
> Fix For: 0.9.7
>
>
> This bug has been found by [~ingomueller.net].
> S3Client has a default value for the connection pool (15 concurrent
> connections). When the number of partitions is greater than 15 (say 96),
> threads will wait for a certain time for an available connection. If no
> connection were made available, a Timeout exception will be thrown. Even if a
> thread can wait, the default size of the connection pool could hinder the
> query performance.
> To fix that, we need to make sure that the connection pool size equals to at
> least the number of partitions in the cluster.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
