On Thu, Oct 8, 2009 at 9:15 PM, Stas Oskin <[email protected]> wrote: > Hi. > > I head about this option before, but never actually tried it. > > There is also another option, called "relatime", which described as being > more compatible then noatime. > Can anyone comment on this? > > Regards. > > 2009/10/8 Edward Capriolo <[email protected]> > >> On Thu, Oct 8, 2009 at 4:00 PM, Jason Venner <[email protected]> >> wrote: >> > noatime is absolutely essential, I forget to mention it, because it is >> > automatic now for me. >> > >> > I have a fun story about atime, I have some Solaris machines with ZFS >> file >> > systems, and I was doing a find on a 6 level hashed directory tree with >> > 250000 leaf nodes. >> > >> > The find on a cold idle file system was running slowly, and the machine >> was >> > writing at 5-10MB/sec, solaris lets you toggle atime at runtime, >> > when I turned it off, the writes went to 0, and the find drastically >> speeded >> > up. >> > >> > This is very representative of a datanode with many blocks. >> > >> > >> > >> > On Thu, Oct 8, 2009 at 12:43 PM, Tom Wheeler <[email protected]> wrote: >> > >> >> I've used XFS on Silicon Graphics machines and JFS on AIX systems -- >> >> both were quite fast and extremely reliable, though this long predates >> >> my use of Hadoop. >> >> >> >> To your question, I recently came across a blog that compares >> >> performance of several Linux filesystems: >> >> >> >> http://log.amitshah.net/2009/04/re-comparing-file-systems.html >> >> >> >> I'd consider his results anecdotal unless the tests reflect the actual >> >> workload of a datanode, but since he's made the code available, you >> >> could probably adapt it yourself to get a better measure. >> >> >> >> On Thu, Oct 8, 2009 at 2:26 PM, Stas Oskin <[email protected]> >> wrote: >> >> > Hi. >> >> > >> >> > Thanks for the info. >> >> > >> >> > What about JFS, any idea how well it compares to XFS? >> >> > >> >> > From what I read, JFS is considered more stable then XFS, but less >> >> > performing, so I wonder if this true. >> >> > >> >> > Also, Ext4 is around the corner and was recently accepted into kernel, >> so >> >> I >> >> > wonder if anyone knows about this one. >> >> >> >> -- >> >> Tom Wheeler >> >> http://www.tomwheeler.com/ >> >> >> > >> > >> > >> > -- >> > Pro Hadoop, a book to guide you from beginner to hadoop mastery, >> > http://www.amazon.com/dp/1430219424?tag=jewlerymall >> > www.prohadoopbook.com a community for Hadoop Professionals >> > >> >> The good news is its not like you are stuck into the file system you >> pick. Assuming you use the normal replication level 3, you can pull >> out a datanode, format it's disk with any FS you want and then stick >> it back into the cluster. Hadoop should not care after all. Not >> suggesting this...but you could theoretically run each node with a >> different file system, look at the performance and say "THIS is the >> one for me" >> >
Relatime is like a noatime that gets updated only periodically, not every read. Hadoop does not use atime so there is no benefit to relative. Go 'noatime nodiratime' although I think nodiratime is a subset of noatime. FYI almost nothing really uses noatime, I heard mutt does and older versions of vim might have. But I turned if and never had an issue.
