[
https://issues.apache.org/jira/browse/HADOOP-12876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15501716#comment-15501716
]
Aaron Fabbri edited comment on HADOOP-12876 at 9/18/16 10:00 PM:
-----------------------------------------------------------------
FYI we're working on on something similar for S3. The umbrella JIRA is
HADOOP-13345 and the in-memory implementation is HADOOP-13452. Once this work
is complete, it should be easy to refactor so you can access the
LocalMetadataStore from ADLS client.
was (Author: fabbri):
FYI we're working on on something similar for S3. The umbrella JIRA is
HADOOP-13448 and the in-memory implementation is HADOOP-13452. Once this work
is complete, it should be easy to refactor so you can access the
LocalMetadataStore from ADLS client.
> [Azure Data Lake] Support for process level FileStatus cache to optimize
> GetFileStatus frequent opeations
> ---------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-12876
> URL: https://issues.apache.org/jira/browse/HADOOP-12876
> Project: Hadoop Common
> Issue Type: Improvement
> Components: fs, fs/azure, tools
> Reporter: Vishwajeet Dusane
> Assignee: Vishwajeet Dusane
>
> Add support to cache GetFileStatus and ListStatus response locally for
> limited period of time. Local cache for limited period of time would optimize
> number of calls for GetFileStatus operation.
> One of the example where local limited period cache would be useful -
> terasort ListStatus on input directory follows with GetFileStatus operation
> on each file within directory. For 2048 input files in a directory would save
> 2048 GetFileStatus calls during start up (Using the ListStatus response to
> cache FileStatus instances).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
