[
https://issues.apache.org/jira/browse/IMPALA-10202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17206209#comment-17206209
]
ASF subversion and git services commented on IMPALA-10202:
----------------------------------------------------------
Commit d09294a4a6da142bf1bed7cf9b769c07d6d7baa0 in impala's branch
refs/heads/master from Sahil Takiar
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=d09294a ]
IMPALA-10202: Enable file handle cache for ABFS files
Like IMPALA-8428, but for ABFS, instead of S3A. Adds support for adding
ABFS file handles to the file handle cache. Support for ABFSInputStream
unbuffer operations was added in HADOOP-16859.
Ran a full table scan of a 1GB store_sales table on ABFS, made
sure the file handles were cached (validated via the runtime
profile); did this multiple times, against several different
copies of the store_sales table, in order to increase the number
of file handles cached by an impalad.
Tested:
* Tested against a ABFS storage account I have access to
Change-Id: I64f12f832980f4e0207af78368402dd09e370fc3
Reviewed-on: http://gerrit.cloudera.org:8080/16532
Reviewed-by: Impala Public Jenkins <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> Enable file handle cache for ABFS files
> ---------------------------------------
>
> Key: IMPALA-10202
> URL: https://issues.apache.org/jira/browse/IMPALA-10202
> Project: IMPALA
> Issue Type: Improvement
> Reporter: Sahil Takiar
> Assignee: Sahil Takiar
> Priority: Major
>
> We should enable the file handle cache for ABFS, we have already seen it
> benefit jobs that read data from S3A.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
