[
https://issues.apache.org/jira/browse/HADOOP-6732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13050084#comment-13050084
]
Daryn Sharp commented on HADOOP-6732:
-------------------------------------
The PathData changes to FsShell have reduced the RPC load on the NN, but at the
expense of a few more objects per path. Adding iterators will negate the
additional memory pressure.
> Improve FsShell's heap consumption by switching to listStatus that returns an
> iterator
> --------------------------------------------------------------------------------------
>
> Key: HADOOP-6732
> URL: https://issues.apache.org/jira/browse/HADOOP-6732
> Project: Hadoop Common
> Issue Type: Improvement
> Reporter: Hairong Kuang
> Assignee: Daryn Sharp
>
> When listing a large directory from the command line using the default heap
> configuration, FsShell often runs out of memory. This is because all stats of
> the entries under the directory need to be in memory before printing them.
> The new API listStatus that returns an iterator of FileStatus, which
> implemented in HDFS-1091, no longer requires that all entries are fetched
> first. Thus switching to this new API will greatly improve the use of heap
> space.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
