Nathan Roberts created HDFS-5788:
------------------------------------
Summary: listLocatedStatus response can be very large
Key: HDFS-5788
URL: https://issues.apache.org/jira/browse/HDFS-5788
Project: Hadoop HDFS
Issue Type: Improvement
Components: namenode
Affects Versions: 2.2.0, 0.23.10, 3.0.0
Reporter: Nathan Roberts
Assignee: Nathan Roberts
Currently we limit the size of listStatus requests to a default of 1000
entries. This works fine except in the case of listLocatedStatus where the
location information can be quite large. As an example, a directory with 7000
entries, 4 blocks each, 3 way replication - a listLocatedStatus response is
over 1MB. This can chew up very large amounts of memory in the NN if lots of
clients try to do this simultaneously.
Seems like it would be better if we also considered the amount of location
information being returned when deciding how many files to return.
Patch will follow shortly.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
