If I recall correctly, ext3 has O(n) performance for lookup of a directory entry. So, having thousands of files in a directory is bad for performance. Additionally, there's a max of 31998 files in a directory, so you have to split into subdirs eventually.
-Todd On Mon, Oct 11, 2010 at 8:15 PM, Thanh Do <than...@cs.wisc.edu> wrote: > Hi all, > > can anyone explain to me while do HDFS has the policy > to store 64 block files in a single sub directory? > and if the number of block files increase, > it just simply creates another subdir and put the block files there. > > Thanks > Thanh > -- Todd Lipcon Software Engineer, Cloudera
