[
https://issues.apache.org/jira/browse/IMPALA-8428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16831935#comment-16831935
]
ASF subversion and git services commented on IMPALA-8428:
---------------------------------------------------------
Commit ab416d42232caf1d3eea5bf3d715c4d229eae8bb in impala's branch
refs/heads/master from Sahil Takiar
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=ab416d4 ]
IMPALA-8428: Bump CDH_BUILD_NUMBER to 1055188.
Brings in HADOOP-14747 (S3AInputStream to implement CanUnbuffer) which
is necessary for IMPALA-8428. The file handle cache calls `unbuffer()`
on a file handle before returning it to the cache. The call to
`unbuffer()` releases any resources that the file handle is holding onto
(in the S3A case the underlying S3ObjectInputStream is closed). Without
this fix an impalad would quickly crash as every cached file handle
would be holding onto a HTTP connection.
Testing:
* Ran an exhaustive build
Change-Id: I44b2169afff9af41d6f31312feb74ee362ecacf5
Reviewed-on: http://gerrit.cloudera.org:8080/13212
Reviewed-by: Impala Public Jenkins <impala-public-jenk...@cloudera.com>
Tested-by: Impala Public Jenkins <impala-public-jenk...@cloudera.com>
> Add support for caching file handles on s3
> ------------------------------------------
>
> Key: IMPALA-8428
> URL: https://issues.apache.org/jira/browse/IMPALA-8428
> Project: IMPALA
> Issue Type: Improvement
> Components: Backend
> Affects Versions: Impala 3.3.0
> Reporter: Joe McDonnell
> Assignee: Sahil Takiar
> Priority: Critical
>
> The file handle cache is currently disabled for S3, as the S3 connector
> needed to implement proper unbuffer support. Now that
> https://issues.apache.org/jira/browse/HADOOP-14747 is fixed, Impala should
> provide an option to cache S3 file handles.
> This is particularly important for data caching, as accessing the data cache
> happens after obtaining a file handle. If getting a file handle is slow, the
> caching will be less effective.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
