ChenSammi commented on code in PR #8528: URL: https://github.com/apache/ozone/pull/8528#discussion_r2128138829
########## hadoop-hdds/docs/content/feature/Topology.md: ########## @@ -23,86 +23,190 @@ summary: Configuration for rack-awarness for improved read/write limitations under the License. --> -Ozone can use topology related information (for example rack placement) to optimize read and write pipelines. To get full rack-aware cluster, Ozone requires three different configuration. - - 1. The topology information should be configured by Ozone. - 2. Topology related information should be used when Ozone chooses 3 different datanodes for a specific pipeline/container. (WRITE) - 3. When Ozone reads a Key it should prefer to read from the closest node. - -<div class="alert alert-warning" role="alert"> - -Ozone uses RAFT replication for Open containers (write), and an async replication for closed, immutable containers (cold data). As RAFT requires low-latency network, topology awareness placement is available only for closed containers. See the [page about Containers]({{< ref "concept/Containers.md">}}) about more information related to Open vs Closed containers. - -</div> - -## Topology hierarchy - -Topology hierarchy can be configured with using `net.topology.node.switch.mapping.impl` configuration key. This configuration should define an implementation of the `org.apache.hadoop.net.CachedDNSToSwitchMapping`. As this is a Hadoop class, the configuration is exactly the same as the Hadoop Configuration - -### Static list - -Static list can be configured with the help of ```TableMapping```: - -```XML +Apache Ozone uses topology information (e.g., rack placement) to optimize data access and improve resilience. A fully rack-aware cluster needs: + +1. Configured network topology. +2. Topology-aware DataNode selection for container replica placement (write path). +3. Prioritized reads from topologically closest DataNodes (read path). + +## Applicability to Container Types + +Ozone's topology-aware placement strategies vary by container replication type and state: + +* **RATIS Replicated Containers:** Ozone uses RAFT replication for Open containers (write), and an async replication for closed, immutable containers (cold data). As RAFT requires low-latency network, topology awareness placement is available only for closed containers. See the [page about Containers](concept/Containers.md) about more information related to Open vs Closed containers. +* **Erasure Coded (EC) Containers:** EC demands topology awareness from the initial write. For an EC key, OM allocates a block group of `$d+p$` distinct DataNodes selected by SCM's `ECPipelineProvider` to ensure rack diversity and fault tolerance. This topology-aware selection is integral to the EC write path for new blocks. \[2] + +## Configuring Topology Hierarchy + +Ozone determines DataNode network locations (e.g., racks) using Hadoop's rack awareness, configured via `net.topology.node.switch.mapping.impl` in `core-site.xml`. This key specifies a `org.apache.hadoop.net.CachedDNSToSwitchMapping` implementation. \[1] Review Comment: Ozone doesn't mandatory requires core-site.xml. So maybe "ozone-site.xml" is more appropriate here. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@ozone.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@ozone.apache.org For additional commands, e-mail: issues-h...@ozone.apache.org