[
https://issues.apache.org/jira/browse/HDFS-2500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13137656#comment-13137656
]
Todd Lipcon commented on HDFS-2500:
-----------------------------------
failed tests are due to TestDfsOverAvroRpc timing out (already a JIRA filed for
this). Will commit momentarily. Thanks for the review, Eli.
> Avoid file system operations in BPOfferService thread while processing deletes
> ------------------------------------------------------------------------------
>
> Key: HDFS-2500
> URL: https://issues.apache.org/jira/browse/HDFS-2500
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: data-node
> Affects Versions: 0.23.0
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Attachments: hdfs-2500-1.patch, hdfs-2500.txt
>
>
> While running a workload with concurrent writes and deletes, I saw a lot of
> NotReplicatedYetExceptions being thrown due to late arrivals of blockReceived
> reports from the DN. Looking at the DN logs, I found that the blockReceived
> message was being delayed as much as 15 seconds because the OfferService
> thread was blocked on file system operations processing deletes. We
> previously moved the deletions to another thread, but it still accesses the
> file system to determine the block length in the main thread. On a heavily
> loaded system this can take a long time.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
