Hi Stephen, Can you try mounting ext4 with the nodelalloc option? I've seen the same improvement due to delayed allocation butbeen a little nervous about that option (especially in the NN where we currently follow what the kernel people call an antipattern for image rotation).
-Todd On Fri, Apr 23, 2010 at 6:12 AM, stephen mulcahy <[email protected]>wrote: > Andrew Klochkov wrote: > >> Hi, >> >> Just curious - did you try ext3? Can it be faster then ext4? Hadoop wiki >> suggests ext3 as it's used mostly for hadoop clusters: >> >> http://wiki.apache.org/hadoop/DiskSetup >> > > For completeness, I rebuilt one more time with ext3 > > mkfs.ext3 -T largefile4 DEV > (mounted with noatime) > gives me a cluster which runs TeraSort in about 22.5 minutes > > So ext4 looks like the winner, from a performance perspective, at least for > running the TeraSort on my cluster with it's specific configuration. > > -stephen > > -- > Stephen Mulcahy, DI2, Digital Enterprise Research Institute, > NUI Galway, IDA Business Park, Lower Dangan, Galway, Ireland > http://di2.deri.ie http://webstar.deri.ie http://sindice.com > -- Todd Lipcon Software Engineer, Cloudera
