[
https://issues.apache.org/jira/browse/HDDS-1881?focusedWorklogId=303054&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-303054
]
ASF GitHub Bot logged work on HDDS-1881:
----------------------------------------
Author: ASF GitHub Bot
Created on: 28/Aug/19 17:14
Start Date: 28/Aug/19 17:14
Worklog Time Spent: 10m
Work Description: anuengineer commented on pull request #1196: HDDS-1881.
Design doc: decommissioning in Ozone
URL: https://github.com/apache/hadoop/pull/1196#discussion_r309754264
##########
File path: hadoop-hdds/docs/content/design/decommissioning.md
##########
@@ -0,0 +1,622 @@
+---
+title: Decommissioning in Ozone
+summary: Formal process to shut down machines in a safe way after the required
replications.
+date: 2019-07-31
+jira: HDDS-1881
+status: current
+author: Anu Engineer, Marton Elek, Stephen O'Donnell
+---
+
+# Abstract
+
+The goal of decommissioning is to turn off a selected set of machines without
data loss. It may or may not require to move the existing replicas of the
containers to other nodes.
+
+There are two main classes of the decommissioning:
+
+ * __Maintenance mode__: where the node is expected to be back after a while.
It may not require replication of containers if enough replicas are available
from other nodes (as we expect to have the current replicas after the restart.)
+
+ * __Decommissioning__: where the node won't be started again. All the data
should be replicated according to the current replication rules.
+
+Goals:
+
+ * Decommissioning can be canceled any time
+ * The progress of the decommissioning should be trackable
+ * The nodes under decommissioning / maintenance mode should not been used for
new pipelines / containers
+ * The state of the datanodes should be persisted / replicated by the SCM (in
HDFS the decommissioning info exclude/include lists are replicated manually by
the admin). If datanode is marked for decommissioning this state be available
after SCM and/or Datanode restarts.
+ * We need to support validations before decommissioing (but the violations
can be ignored by the admin).
+ * The administrator should be notified when a node can be turned off.
+ * The maintenance mode can be time constrained: if the node marked for
maintenance for 1 week and the node is not up after one week, the containers
should be considered as lost (DEAD node) and should be replicated.
+
+# Introduction
+
+Ozone is a highly available file system that relies on commodity hardware. In
other words, Ozone is designed to handle failures of these nodes all the time.
+
+The Storage Container Manager(SCM) is designed to monitor the node health and
replicate blocks and containers as needed.
+
+At times, Operators of the cluster can help the SCM by giving it hints. When
removing a datanode, the operator can provide a hint. That is, a planned
failure of the node is coming up, and SCM can make sure it reaches a safe state
to handle this planned failure.
+
+Some times, this failure is transient; that is, the operator is taking down
this node temporarily. In that case, we can live with lower replica counts by
being optimistic.
+
+Both of these operations, __Maintenance__, and __Decommissioning__ are similar
from the Replication point of view. In both cases, and the user instructs us on
how to handle an upcoming failure.
+
+Today, SCM (*Replication Manager* component inside SCM) understands only one
form of failure handling. This paper extends Replica Manager failure modes to
allow users to request which failure handling model to be adopted(Optimistic or
Pessimistic).
+
+Based on physical realities, there are two responses to any perceived failure,
to heal the system by taking corrective actions or ignore the failure since the
actions in the future will heal the system automatically.
+
+## User Experiences (Decommissioning vs Maintenance mode)
+
+From the user's point of view, there are two kinds of planned failures that
the user would like to communicate to Ozone.
+
+The first kind is when a 'real' failure is going to happen in the future. This
'real' failure is the act of decommissioning. We denote this as "decommission"
throughout this paper. The response that the user wants is SCM/Ozone to make
replicas to deal with the planned failure.
+
+The second kind is when the failure is 'transient.' The user knows that this
failure is temporary and cluster in most cases can safely ignore this issue.
However, if the transient failures are going to cause a failure of
availability; then the user would like the Ozone to take appropriate actions to
address it. An example of this case, is if the user put 3 data nodes into
maintenance mode and switched them off.
Review comment:
Let me edit the source and send it to you, I am just adding comments. You
don't have to fix these.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 303054)
Time Spent: 42h 40m (was: 42.5h)
> Design doc: decommissioning in Ozone
> ------------------------------------
>
> Key: HDDS-1881
> URL: https://issues.apache.org/jira/browse/HDDS-1881
> Project: Hadoop Distributed Data Store
> Issue Type: Sub-task
> Reporter: Elek, Marton
> Assignee: Elek, Marton
> Priority: Major
> Labels: design, pull-request-available
> Time Spent: 42h 40m
> Remaining Estimate: 0h
>
> Design doc can be attached to the documentation. In this jira the design doc
> will be attached and merged to the documentation page.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
