adoroszlai opened a new pull request, #4539:
URL: https://github.com/apache/ozone/pull/4539
## What changes were proposed in this pull request?
The default topology contains a single rack. With rack-aware container
placement policy (HDDS-8300), overreplication is considered a misreplication,
since more replicas are in a single rack than desired. Yet misreplication
cannot be resolved since there is no other rack.
This can be reproduced by configuring rack-awareness for integration tests:
```
diff --git hadoop-ozone/integration-test/src/test/resources/ozone-site.xml
hadoop-ozone/integration-test/src/test/resources/ozone-site.xml
index 0c5ae1fa88..87a85dc1f1 100644
--- hadoop-ozone/integration-test/src/test/resources/ozone-site.xml
+++ hadoop-ozone/integration-test/src/test/resources/ozone-site.xml
@@ -31,4 +31,9 @@
<value>4</value>
</property>
+ <property>
+ <name>ozone.scm.container.placement.impl</name>
+
<value>org.apache.hadoop.hdds.scm.container.placement.algorithms.SCMContainerPlacementRackAware</value>
+ </property>
+
</configuration>
```
and running:
```
$ mvn -am -pl :ozone-integration-test
-Dtest='TestDecommissionAndMaintenance#testContainerIsReplicatedWhenAllNodesGotoMaintenance'
clean test
...
at
org.apache.hadoop.ozone.scm.node.TestDecommissionAndMaintenance.waitForContainerReplicas(TestDecommissionAndMaintenance.java:757)
at
org.apache.hadoop.ozone.scm.node.TestDecommissionAndMaintenance.testContainerIsReplicatedWhenAllNodesGotoMaintenance(TestDecommissionAndMaintenance.java:431)
...
[ERROR] Errors:
[ERROR]
TestDecommissionAndMaintenance.testContainerIsReplicatedWhenAllNodesGotoMaintenance:431->waitForContainerReplicas:757
ยป Timeout
```
Container placement should be considered valid (not misreplicated) if there
is only a single rack. This will let the overreplication logic take care of
the extra replicas instead of the misreplication one.
https://issues.apache.org/jira/browse/HDDS-8383
## How was this patch tested?
The same integration test passed. Also ran unit tests related to topology
and placement policy.
https://github.com/adoroszlai/hadoop-ozone/actions/runs/4617349558
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
