[Lustre-discuss] Small files

2013-07-03 Thread Nikolay Kvetsinski
Hello again guys, I constantly read about that Lustre is not very good with small files. However, there is no definition of small file in Lustre point of view. Would you be able to draw some borders on what is considered to be a small file , for example is a 10 MB considered a small file, and

Re: [Lustre-discuss] Small files

2013-07-03 Thread Nikolay Kvetsinski
Thanks mate, what you say makes sense. Anyway I`m dealing with different file sizes, varying from KBs to GBs. Luckily some of the files are grouped in folders, for example in folder /fs/X all files will be 9MB, which I`ll not stripe. In /fs/Y all files will be GBs, which I`ll stripe among all

Re: [Lustre-discuss] Small files

2013-07-03 Thread White, Cliff
Worth noting – unless your IO requirements are quite strict, in most cases you won't need a large number of striping policies. The 'best' stripe for any large IO task is usually dependent on the particular hardware/network/workload involved. If you are saturating part of the system, such as

Re: [Lustre-discuss] Small files

2013-07-03 Thread Dilger, Andreas
On 2013/03/07 5:51 AM, Nikolay Kvetsinski nkvecin...@gmail.com wrote: Thanks mate, what you say makes sense. Anyway I`m dealing with different file sizes, varying from KBs to GBs. Luckily some of the files are grouped in folders, for example in folder /fs/X all files will be 9MB, which I`ll not