adoroszlai opened a new pull request, #5610:
URL: https://github.com/apache/ozone/pull/5610
## What changes were proposed in this pull request?
Client's node (place in the topology) is looked up via
`SCMNodeManager.getNodesByAddress`. But `SCMNodeManager` only knows about
datanodes. If the request comes from a host which does not run Ozone Datanode,
then read will not be topology-aware. While read cannot be host-local in this
case, it still could benefit from being rack-local.
Examples where such non-datanode hosts may occur:
* edge nodes (client-only)
* dedicated metadata nodes (OM, SCM, etc.)
* Docker/Kubernetes containers running only one service
https://issues.apache.org/jira/browse/HDDS-9674
## How was this patch tested?
Added unit test.
Manually tested on `ozone-topology` environment:
* added `s3g` container in `docker-compose.yaml` with address `10.5.0.73`
* added `scm` (in `rack1`) and `s3g` (in `rack2`) in topology
`network-config`
* changed `test.sh` to run `basic/basic.robot` and `s3/objectputget.robot`
* enabled debug logging for `KeyManagerImpl`
OM log shows datanodes in the same rack were sorted to head of list for both
`scm` and `s3g`:
```
DEBUG om.KeyManagerImpl: Sorted datanodes
[8fa9bb3e-3ff4-4f9b-a71e-0ffffcd70be5(ozone-topology_datanode_5_1.ozone-topology_net/10.5.0.8),
04478345-c6ef-4c2b-a688-8819babd5db7(ozone-topology_datanode_2_1.ozone-topology_net/10.5.0.5),
a8a8f57f-a0dc-41e5-9bcc-f7278f0ba7c3(ozone-topology_datanode_4_1.ozone-topology_net/10.5.0.7)]
for client 10.5.0.71, result:
[04478345-c6ef-4c2b-a688-8819babd5db7(ozone-topology_datanode_2_1.ozone-topology_net/10.5.0.5),
a8a8f57f-a0dc-41e5-9bcc-f7278f0ba7c3(ozone-topology_datanode_4_1.ozone-topology_net/10.5.0.7),
8fa9bb3e-3ff4-4f9b-a71e-0ffffcd70be5(ozone-topology_datanode_5_1.ozone-topology_net/10.5.0.8)]
DEBUG om.KeyManagerImpl: Sorted datanodes
[8fa9bb3e-3ff4-4f9b-a71e-0ffffcd70be5(ozone-topology_datanode_5_1.ozone-topology_net/10.5.0.8),
04478345-c6ef-4c2b-a688-8819babd5db7(ozone-topology_datanode_2_1.ozone-topology_net/10.5.0.5),
a8a8f57f-a0dc-41e5-9bcc-f7278f0ba7c3(ozone-topology_datanode_4_1.ozone-topology_net/10.5.0.7)]
for client 10.5.0.73, result:
[a8a8f57f-a0dc-41e5-9bcc-f7278f0ba7c3(ozone-topology_datanode_4_1.ozone-topology_net/10.5.0.7),
8fa9bb3e-3ff4-4f9b-a71e-0ffffcd70be5(ozone-topology_datanode_5_1.ozone-topology_net/10.5.0.8),
04478345-c6ef-4c2b-a688-8819babd5db7(ozone-topology_datanode_2_1.ozone-topology_net/10.5.0.5)]
```
CI:
https://github.com/adoroszlai/ozone/actions/runs/6882001098
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
