[
https://issues.apache.org/jira/browse/HDFS-6871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14103984#comment-14103984
]
Daryn Sharp commented on HDFS-6871:
-----------------------------------
My only concern which I don't have cycles to investigate is a subtle change.
Currently, by the time the overwritten file appears to exist in the namesystem,
the former deleted file's blocks are removed from the block map, corrupt
replicas, replication queues, etc. Any block updates from DNs are ignored
after the create releases the lock.
This patch allows blocks to appear to exist after the create releases the lock.
As long as everything updating the blocks ensures that its block collection
isn't orphaned (removed from inodemap) we are probably ok. We need to ensure
there is zero chance that an edit can be generated for a deleted block.
Persist blocks comes to mind. And that the block manager, replication manager,
etc do not become fouled by expecting deleted blocks to exist.
I'll trust those currently reviewing will ensure all cases are covered.
> Improve NameNode performance when creating file
> -------------------------------------------------
>
> Key: HDFS-6871
> URL: https://issues.apache.org/jira/browse/HDFS-6871
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: namenode, performance
> Reporter: Yi Liu
> Assignee: Yi Liu
> Priority: Critical
> Fix For: 2.6.0
>
> Attachments: HDFS-6871.001.patch, HDFS-6871.002.patch,
> HDFS-6871.003.patch
>
>
> Creating file with overwrite flag will cause NN fall into flush edit logs and
> block other requests if the file exists.
> When we create a file with overwrite flag (default is true) in HDFS, NN will
> remove original file if it exists. In FSNamesystem#startFileInternal, NN
> already holds the write lock, it calls {{deleteInt}} if the file exists,
> there is logSync in {{deleteInt}}. So in this case, logSync is under write
> lock, it will heavily affect the NN performance.
> We should ignore the force logSync in {{deleteInt}} in this case.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
