[
https://issues.apache.org/jira/browse/IMPALA-2607?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tim Armstrong reassigned IMPALA-2607:
-------------------------------------
Assignee: (was: Sailesh Mukil)
> S3: need to determine the right default for fs.s3a.connection.maximum
> ---------------------------------------------------------------------
>
> Key: IMPALA-2607
> URL: https://issues.apache.org/jira/browse/IMPALA-2607
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
> Affects Versions: Impala 2.3.0
> Reporter: Dan Hecht
> Priority: Major
> Labels: s3, usability
>
> When Impala is scanning parquet files off S3, it appears we can get into a
> deadlock if fs.s3a.connection.maximum is not set high enough, because we can
> spread the connection use across files but unless we have a connection for
> all selected columns of a file, we can't make progress scanning that file.
> So, we need to figure out the right default for fs.s3a.connection.maximum for
> Impala, and also determine whether we need to put an upper bound on scanner
> concurrency given the number of connections.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
