DFS pathname limitation.
------------------------
Key: HADOOP-438
URL: http://issues.apache.org/jira/browse/HADOOP-438
Project: Hadoop
Issue Type: Bug
Components: dfs
Affects Versions: 0.5.0, 0.4.0, 0.3.2, 0.3.1, 0.3.0, 0.2.1, 0.2.0, 0.1.1,
0.1.0
Reporter: Konstantin Shvachko
Fix For: 0.6.0
I was trying to create a deep hierarchy of directories using DFS mkdirs().
When the path to the leaf directory became long (~20000) DFS was still able to
create
directories with these names, but UTF8 started truncating long strings
resulting in
incorrect logging of namespace edits. That later crashed the namenode during
restart,
when it was trying to reproduce file creation logged in the edits file with
truncated names.
UTF8 is deprecated now so we will have to replace it with Text.
With UTF8 we should enforce a pathname limit of 0xffff/3 = 21845
With Text it is going to be larger. Not sure what the exact number is.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
