[jira] [Updated] (HADOOP-12876) [Azure Data Lake] Support for process level FileStatus cache to optimize GetFileStatus frequent operations
[ https://issues.apache.org/jira/browse/HADOOP-12876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-12876: Issue Type: Sub-task (was: Improvement) Parent: HADOOP-14764 > [Azure Data Lake] Support for process level FileStatus cache to optimize > GetFileStatus frequent operations > -- > > Key: HADOOP-12876 > URL: https://issues.apache.org/jira/browse/HADOOP-12876 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs, fs/adl, tools >Reporter: Vishwajeet Dusane >Assignee: Vishwajeet Dusane > > Add support to cache GetFileStatus and ListStatus response locally for > limited period of time. Local cache for limited period of time would optimize > number of calls for GetFileStatus operation. > One of the example where local limited period cache would be useful - > terasort ListStatus on input directory follows with GetFileStatus operation > on each file within directory. For 2048 input files in a directory would save > 2048 GetFileStatus calls during start up (Using the ListStatus response to > cache FileStatus instances). -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-12876) [Azure Data Lake] Support for process level FileStatus cache to optimize GetFileStatus frequent operations
[ https://issues.apache.org/jira/browse/HADOOP-12876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-12876: Summary: [Azure Data Lake] Support for process level FileStatus cache to optimize GetFileStatus frequent operations (was: [Azure Data Lake] Support for process level FileStatus cache to optimize GetFileStatus frequent opeations) > [Azure Data Lake] Support for process level FileStatus cache to optimize > GetFileStatus frequent operations > -- > > Key: HADOOP-12876 > URL: https://issues.apache.org/jira/browse/HADOOP-12876 > Project: Hadoop Common > Issue Type: Improvement > Components: fs, fs/adl, tools >Reporter: Vishwajeet Dusane >Assignee: Vishwajeet Dusane > > Add support to cache GetFileStatus and ListStatus response locally for > limited period of time. Local cache for limited period of time would optimize > number of calls for GetFileStatus operation. > One of the example where local limited period cache would be useful - > terasort ListStatus on input directory follows with GetFileStatus operation > on each file within directory. For 2048 input files in a directory would save > 2048 GetFileStatus calls during start up (Using the ListStatus response to > cache FileStatus instances). -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
