On Fri, Jun 10, 2011 at 8:42 AM, Pedro Costa <psdc1...@gmail.com> wrote: > But, how can I say that a 1KB file will only use 1KB of disc space, if > a block is configured has 64MB? In my view, if a 1KB use a block of > 64MB, the file will occupy 64MB in the disc.
A block of HDFS is the unit of distribution and replication, not the unit of storage. HDFS uses the underlying file systems for physical storage. -- Philip > > How can you disassociate a 64MB data block from HDFS of a disk block? > > On Fri, Jun 10, 2011 at 5:01 PM, Marcos Ortiz <mlor...@uci.cu> wrote: >> On 06/10/2011 10:35 AM, Pedro Costa wrote: >> >> Hi, >> >> If I define HDFS to use blocks of 64 MB, and I store in HDFS a 1KB >> file, this file will ocupy 64MB in the HDFS? >> >> Thanks, >> >> HDFS is not very efficient storing small files, because each file is stored >> in a block (of 64 MB in your case), and the block metadata >> is held in memory by the NN. But you should know that this 1KB file only >> will use 1KB of disc space. >> >> For small files, you can use Hadoop archives. >> Regards >> >> -- >> Marcos Luís Ortíz Valmaseda >> Software Engineer (UCI) >> http://marcosluis2186.posterous.com >> http://twitter.com/marcosluis2186 >> >> >
