2007/12/6, Jean Christophe Andr? <jean-christophe.andre at auf.org>:
> Nguyen Vu Hung a ?crit :
> > find . -type f -name *.png -exec ls -l {} \; | awk '{print $5}' > L2.txt
> > and sum up all the size outputed to L2.txt, and I get 165 486 070
> > This is the actual size of the current directory in byte, is this right?
> >
> No: this is the total size of your data. It will never be the same as
> the actual disk space whatever which filesystem you choose. Except if
> the indicated data size is rounded up to the block size which would be
> plain wrong.
>
I did mean "total size of my data". Just a typo. The command I've run
is what I've intented.
> > And du -sk . gives: 333 520 ( in kilobyte ).
> > and is x2 times bigger than the first size I've computed.
> >
> Twice the size is quite uncommon, except with files having exactly half
> the size of a filesystem block, I can hardly see how this can occur...
> What filesystem are you files stored on?
>
The fs is ext3.
It is twice and it is real :D
As I said in OP, all the files in this folder are small, dumped with
the same API. So it may be the case. I just think of a fictitious
situation: all bytes are 2KB and the block size is 4KB
I still don't know what is the different between df's ouput and the
size that ls -l | awk '{print $5}' reports.
> > My question: How do I optimize the disk for storing this kind of data (
> > i.e, the number of file is large, while the file size is small ). Is there
> > anything to do with filesystem's block size? What file system is the best
> > choice for this kind of data, and has fast I/O?
> >
> Yes it's related to the filesystem blocksize but not only. Because there
> is some filesystems that optimize underused block space by reusing it
> for something else like file fragments or symbolic link path.
>
> With ext3, you should take a look at "man mke2fs", especialy the -T
> option, and "man mke2fs.conf". I've got back a few gigabytes more space
> on a 300 GB hard disk simply by tuning this correctly (using "-T
> largefile" for huge size files, like ISO images).
>
It is not the case. Our files are all small.
I also found -i interesting:
-i bytes-per-inode
Specify the bytes/inode ratio. mke2fs creates an
inode for every bytes-per-inode bytes of space on the disk. The
larger the bytes-per-inode ratio,
the fewer inodes will be created. This value generally
shouldn\u2019t be smaller than the blocksize of the filesystem, since
then too many inodes will be
made. Be warned that is not possible to expand the
number of inodes on a filesystem after it is created, so be careful
deciding the correct value for
this parameter.
--
Best Regards,
Nguyen Hung Vu
vuhung16plus{[email protected]
An inquisitive look at Harajuku
http://www.flickr.com/photos/vuhung/sets/72157600109218238/
