GlenGeng commented on a change in pull request #2050: URL: https://github.com/apache/ozone/pull/2050#discussion_r595698722
########## File path: hadoop-hdds/docs/content/feature/SCM-HA.md ########## @@ -0,0 +1,162 @@ +--- +title: "SCM High Availability" +weight: 1 +menu: + main: + parent: Features +summary: HA setup for Storage Container Manager to avoid any single point of failure. +--- +<!--- + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. +--> + +Ozone has two leader nodes (*Ozone Manager* for key space management and *Storage Container Management* for block space management) and storage nodes (Datanode). Data is replicated between Datanodes with the help of RAFT consensus algorithm. + +<div class="alert alert-warning" role="alert"> +Please note that SCM-HA is not ready for production in secure environments. Security work is in progress and will be finished soon. +</div> + +To avoid any single point of failure the leader nodes also should have a HA setup. + +Both Ozone Manager and Storage Container Manager supports HA. In this mode the internal state is replicated via RAFT (with Apache Ratis) + +This document explains the HA setup of Storage Container Manager (SCM), please check [this page]({{< ref "OM-HA" >}}) for HA setup of Ozone Manager (OM). While they can be setup for HA independently, a reliable, full HA setup requires enabling HA for both services. + +## Configuration + +HA mode of Storage Container Manager can be enabled with the following settings in `ozone-site.xml`: + +```XML +<property> + <name>ozone.scm.ratis.enable</name> + <value>true</value> +</property> +``` +One Ozone configuration (`ozone-site.xml`) can support multiple SCM HA node set, multiple Ozone clusters. To select between the available SCM nodes a logical name is required for each of the clusters which can be resolved to the IP addresses (and domain names) of the Storage Container Managers. + +This logical name is called `serviceId` and can be configured in the `ozone-site.xml` + +Most of the time you need to set only the values of your current cluster: + + ```XML +<property> + <name>ozone.scm.service.ids</name> + <value>cluster1</value> +</property> +``` + +For each of the defined `serviceId` a logical configuration name should be defined for each of the servers + +```XML +<property> + <name>ozone.scm.nodes.cluster1</name> + <value>scm1,scm2,scm3</value> +</property> +``` + +The defined prefixes can be used to define the address of each of the SCM services: + +```XML +<property> + <name>ozone.scm.address.cluster1.scm1</name> + <value>host1</value> +</property> +<property> + <name>ozone.scm.address.cluster1.scm1</name> + <value>host2</value> +</property> +<property> + <name>ozone.scm.address.cluster1.scm1</name> + <value>host3</value> +</property> +``` Review comment: the primordial node is an optional configuration, which is used in k8s. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
