[ http://issues.apache.org/jira/browse/HADOOP-438?page=comments#action_12434015 ] Konstantin Shvachko commented on HADOOP-438: --------------------------------------------
I just wanted to mention that there are file systems out there without path restrictions. http://en.wikipedia.org/wiki/Comparison_of_file_systems#Limits > DFS pathname limitation. > ------------------------ > > Key: HADOOP-438 > URL: http://issues.apache.org/jira/browse/HADOOP-438 > Project: Hadoop > Issue Type: Bug > Components: dfs > Affects Versions: 0.5.0, 0.4.0, 0.3.2, 0.3.1, 0.3.0, 0.2.1, 0.2.0, 0.1.1, > 0.1.0 > Reporter: Konstantin Shvachko > > I was trying to create a deep hierarchy of directories using DFS mkdirs(). > When the path to the leaf directory became long (~20000) DFS was still able > to create > directories with these names, but UTF8 started truncating long strings > resulting in > incorrect logging of namespace edits. That later crashed the namenode during > restart, > when it was trying to reproduce file creation logged in the edits file with > truncated names. > UTF8 is deprecated now so we will have to replace it with Text. > With UTF8 we should enforce a pathname limit of 0xffff/3 = 21845 > With Text it is going to be larger. Not sure what the exact number is. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
