Unfortunately, I'm not aware of any comprehensive documentation on how HDFS adheres to POSIX and how it differs. To some degree, it's possible to look at the FileSystem specification and corresponding contract tests as a source of information, but it's not structured to answer that question directly. I filed HADOOP-13104 to track improving that, but I haven't been able to work on it yet.
http://hadoop.apache.org/docs/r2.7.2/hadoop-project-dist/hadoop-common/filesystem/index.html https://issues.apache.org/jira/browse/HADOOP-13104 There is also some discussion of POSIX specifically related to permission handling in the HDFS permissions guide. http://hadoop.apache.org/docs/r2.7.2/hadoop-project-dist/hadoop-hdfs/HdfsPermissionsGuide.html --Chris Nauroth From: Gagan Brahmi <[email protected]<mailto:[email protected]>> Date: Friday, June 3, 2016 at 9:18 AM To: Deepak Goel <[email protected]<mailto:[email protected]>> Cc: user <[email protected]<mailto:[email protected]>> Subject: Re: Hadoop/HDFS/POSIX AFAIK the major difference include the simplified Coherency model<https://hadoop.apache.org/docs/r2.7.1/hadoop-project-dist/hadoop-hdfs/HdfsDesign.html#Simple_Coherency_Model> in HDFS. The other one is the file permissions. The read (r), write (w) & execute (x) works the same way for directories in HDFS like a POSIX file system. But for the files, execute permission is ignored in HDFS since you can't really execute a file in HDFS. HTH Regards, Gagan Brahmi On Fri, Jun 3, 2016 at 1:30 AM, Deepak Goel <[email protected]<mailto:[email protected]>> wrote: Hey Namaskara~Nalama~Guten Tag~Bonjour The Hadoop document gives a pointer that HDFS relaxes some POSIX requirements to maintain high throughput rate. https://hadoop.apache.org/docs/r1.2.1/hdfs_design.html But it does not give exactly what are the things which it relaxes? Any idea which programs should I look into to find the things which HDFS relaxes? Thanks Deepak -- Keigu Deepak 73500 12833 www.simtree.net<http://www.simtree.net>, [email protected]<mailto:[email protected]> [email protected]<mailto:[email protected]> LinkedIn: www.linkedin.com/in/deicool<http://www.linkedin.com/in/deicool> Skype: thumsupdeicool Google talk: deicool Blog: http://loveandfearless.wordpress.com Facebook: http://www.facebook.com/deicool "Contribute to the world, environment and more : http://www.gridrepublic.org "
