[
https://issues.apache.org/jira/browse/HDFS-9945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kihwal Lee updated HDFS-9945:
-----------------------------
Hadoop Flags: Reviewed
Release Note: This new dfsadmin command, evictWriters, stops active block
writing activities on a data node. The affected writes will continue without
the node after a write pipeline recovery. This is useful when data node
decommissioning is blocked by slow writers. If issued against a
non-decommissioing data node, all current writers will be stopped, but new
write requests will continue to be served.
> Datanode command for evicting writers
> -------------------------------------
>
> Key: HDFS-9945
> URL: https://issues.apache.org/jira/browse/HDFS-9945
> Project: Hadoop HDFS
> Issue Type: New Feature
> Components: datanode
> Reporter: Kihwal Lee
> Assignee: Kihwal Lee
> Fix For: 2.8.0
>
> Attachments: HDFS-9945.patch, HDFS-9945.v2.patch
>
>
> It will be useful if there is a command to evict writers from a datanode.
> When a set of datanodes are being decommissioned, they can get blocked by
> slow writers at the end. It was rare in the old days since mapred jobs
> didn't last too long, but with many different types of apps running on
> today's YARN cluster, we are often see very long tail in datanode
> decommissioning.
> I propose a new dfsadmin command, {{evictWriters}}, to be added. I initially
> thought about having namenode automatically telling datanodes on
> decommissioning, but realized that having a command is more flexible. E.g.
> users can choose not to do this at all, choose when to evict writers, or
> whether to try multiple times for whatever reasons.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
