[
https://issues.apache.org/jira/browse/HDFS-2718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13199367#comment-13199367
]
Todd Lipcon commented on HDFS-2718:
-----------------------------------
No major issues, but this is fairly critical code and committing without a
committer +1 worries me. Here's my feedback on the patch:
- Rather than adjusting the code to handle the invalid edit log case of first
OP_ADD having multiple blocks, we should fix the CreateEditLogs tool to create
a realistic edit log. That would allow us to remove code from the loader, as
well as avoid having to add the new INodeFileUnderConstruction constructor.
- The {{updateFile}} call takes a number of parameters which aren't used -- in
particular permissions, replication, and preferredBlockSize. Having these
parameters present implies that they'll also be used to update the file, which
is not the case.
- I think the construction of filenames in TestEditLog may be buggy -- I think
you mean to do {{"/filename" + (startIndex + i)}} since otherwise it will do
two string-appends, rather than a string append of an addition.
> Optimize OP_ADD in edits loading
> --------------------------------
>
> Key: HDFS-2718
> URL: https://issues.apache.org/jira/browse/HDFS-2718
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: name-node
> Affects Versions: 0.22.0, 0.24.0, 1.0.0
> Reporter: Konstantin Shvachko
> Assignee: Konstantin Shvachko
> Fix For: 0.24.0, 0.23.1, 0.22.1
>
> Attachments: editsLoader-0.22.patch, editsLoader-0.22.patch,
> editsLoader-0.22.patch, editsLoader-trunk.patch, editsLoader-trunk.patch,
> editsLoader-trunk.patch, editsLoader-trunk.patch
>
>
> During loading the edits journal FSEditLog.loadEditRecords() processes OP_ADD
> inefficiently. It first removes the existing INodeFile from the directory
> tree, then adds it back as a regular INodeFile, and then replaces it with
> INodeFileUnderConstruction if files is not closed. This slows down edits
> loading. OP_ADD should be done in one shot and retain previously existing
> data.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
